Data processing method and related device

Abstract

The embodiment of the invention discloses a data processing method and a related device, in order to test the influence of intervention items such as advertisements on contents, a plurality of contents are divided into two groups, intervention items are not applied to the first group of contents, intervention items are applied to the second group of contents, and a random allocation principle is met as much as possible in the division process; the covariables of the first group of contents are similar to the covariables of the second group of contents, and no selection deviation exists as far as possible. Compared with the content, the object of the consumption content has the advantages that the total sample amount is large enough, the influence of selection deviation can be reduced, the object of the consumption content is used as an individual of an experimental group and a control group, and the selection deviation which cannot be evaluated cannot be introduced. And obtaining an object control group consuming the first group of contents and an object experiment group consuming the second group of contents, and determining the influence degree of the intervention item on the contents according to the object control group and the object experiment group. Therefore, similar experiment testing is carried out through scientific scheme design, and the application scene of verification intervention items is expanded.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and related devices. Background Art

[0002] As Internet applications penetrate into every aspect of life, people pay more and more attention to user choices. The revision of the application page, whether to add advertisements to the video, and other intervention items to the original content are generally implemented according to the choices of the majority of users. For example, if more users choose not to add advertisements to the video than to add advertisements to the video, then the final choice is not to add advertisements to the video.

[0003] In related technologies, AB experiments (or random experiments) are generally conducted to verify and evaluate the impact of an intervention item on the original content. Usually, a part of the users are used as the experimental group through random diversion, so that they can experience the content with the added intervention item, and another part of the users are used as the control group, so that they can experience the original content, or the content without the added intervention item. The difference in benefits (such as financial benefits, non-financial benefits such as usage time, etc.) between the two groups is compared to evaluate the impact of the intervention item on the content.

[0004] However, some intervention items cannot be verified through AB experiments, such as verifying the impact of smoking cessation on lung cancer, which limits the application scenarios of verifying intervention items. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a data processing method and related devices for expanding the application scenarios of verification intervention items.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a data processing method, the method comprising:

[0008] access to multiple content and intervention items;

[0009] Dividing the plurality of the contents into a first group of contents and a second group of contents to which the intervention item is applied, wherein the degree of similarity between the covariates of the first group of contents and the covariates of the second group of contents is greater than a first preset threshold, and the covariates are independent variables that affect the corresponding group of contents and are not the intervention item during the process of testing the contents;

[0010] obtaining a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content;

[0011] The degree of influence of the intervention item on the content is determined based on the subject control group and the subject experimental group.

[0012] On the other hand, an embodiment of the present application provides a data processing device, the device comprising: an acquisition unit, a division unit and a determination unit;

[0013] The acquisition unit is used to acquire multiple contents and intervention items;

[0014] The division unit is used to divide the plurality of the contents into a first group of contents and a second group of contents to which the intervention item is applied, wherein the degree of similarity between the covariates of the first group of contents and the covariates of the second group of contents is greater than a first preset threshold, and the covariates are independent variables that affect the corresponding group of contents and are not the intervention item during the process of testing the contents;

[0015] The acquisition unit is further used to acquire a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content;

[0016] The determination unit is used to determine the degree of influence of the intervention item on the content based on the object control group and the object experimental group.

[0017] On the other hand, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0018] The memory is used to store a computer program and transmit the computer program to the processor;

[0019] The processor is configured to execute the method described in the above aspects according to the instructions in the computer program.

[0020] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the above aspects.

[0021] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method described in the above aspects.

[0022] It can be seen from the above technical solution that in order to test the degree of influence of the intervention item on the content, the intervention item and multiple contents to be tested are obtained, and the multiple contents are divided into two groups, namely the first group of contents and the second group of contents, wherein the second group of contents is subjected to the intervention item, and the first group of contents is not subjected to the intervention item, and the random allocation principle is satisfied as much as possible during the division process, so that the covariates of the first group of contents and the covariates of the second group of contents are similar, so that the first group of contents and the second group of contents have no selection bias as much as possible. Compared with the content, the total sample size of the objects consuming the content is not only large enough to reduce the influence of the selection bias, but also the objects consuming the content are used as individuals in the experimental group and the control group, which will not introduce selection bias that cannot be evaluated. Based on this, the control group of objects consuming the first group of contents and the experimental group of objects consuming the second group of contents are obtained, and the degree of influence of the intervention item on the content is determined according to the control group and the experimental group. Therefore, the content is used as the carrier of the intervention item, and multiple contents are divided into two groups of contents based on the benchmark of similar covariates. The situation of the objects consuming the two groups of contents is compared to obtain the degree of influence of the intervention item on the content. Using the data that can be collected to construct experimental and control groups similar to those in AB experiments is equivalent to a "quasi-experimental" test, so that the analytical framework of causal inference can be used to evaluate the effects through observation, providing high-quality data for causal inference. This not only avoids the situation where causal inference cannot be performed or the reliability of causal inference results is low due to data quality, but also expands the application scenarios of verifying intervention items. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of an application scenario of the data processing method provided in an embodiment of the present application;

[0025] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of multiple content divisions provided in an embodiment of the present application;

[0027] Figure 4 A method based on the embodiment of the present application is provided Figure 3 A schematic diagram of grouping objects that consume content;

[0028] Figure 5 A method based on the embodiment of the present application is provided Figure 4Schematic diagram of calculating propensity scores;

[0029] Figure 6 A method based on the embodiment of the present application is provided Figure 5 Schematic diagram for achieving object matching;

[0030] Figure 7 A method based on the embodiment of the present application is provided Figure 6 Schematic diagram to determine the extent of impact;

[0031] Figure 8 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0032] Fig. 9 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0033] Fig.10 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0035] Taking the interference item as an example, ideally, the tester wants to compare the impact of the strategy and the non-strategy on each user, and calculate the strategy gain (positive or negative) for each user. But in reality, a user can only have one state: hit the strategy or miss the strategy. Therefore, an alternative method that the tester can use is to find two sufficiently similar users, check the impact of the strategy on one user and the non-strategy on the other user, compare the difference in benefits between the two, and then evaluate the effectiveness of the strategy. The benefits can be financial benefits or non-financial benefits such as usage time.

[0036] In related technologies, the impact of a strategy is measured by comparing the revenue of the experimental group and the control group through AB experiments. In this case, the strategy is applied with users as the basic unit, and users are the basic carriers of strategy implementation. As long as the grouping of users during the experiment is random, that is, it can be assumed that the two groups of users are indifferent, then the impact of the strategy can be evaluated by comparing the revenue of the two groups of users after the experiment.

[0037] However, in real life, there are certain factors that make it impossible to conduct AB experiments, and thus it is impossible to directly evaluate the impact of strategies through experiments, which limits the application scenarios of verifying intervention items. For example, to verify the impact of smoking cessation on lung cancer, since it is impossible to find two groups of users, one group of users is forced to not quit smoking. In other words, when the basic carrier of the implementation of the intervention item is the user, the application scenarios of the verification intervention item will be limited.

[0038] Based on this, the embodiment of the present application provides a data processing method and related devices, which use content as a carrier of intervention items, divide multiple contents into two groups of content based on similar benchmarks of covariates, compare the situations of the objects consuming the two groups of content, and obtain the degree of influence of the intervention items on the content. Use the collectible data to construct an experimental group and a control group similar to those in an AB experiment, which is equivalent to a "quasi-experimental" test, so as to use the analysis framework of causal inference to evaluate the effect through "observation", and provide high-quality data for causal inference, which not only avoids the situation where causal inference cannot be performed due to data quality or the reliability of causal inference results is low, but also expands the application scenarios of verifying intervention items.

[0039] The data processing method provided in this application can be applied to computer devices with data processing capabilities, such as terminal devices and servers. Among them, the terminal device can specifically be a desktop computer, a laptop computer, a smart phone, a tablet computer, an Internet of Things device, a portable wearable device and an aircraft, etc. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle-mounted device, etc. The smart vehicle-mounted device can be a vehicle-mounted navigation terminal and a vehicle-mounted computer, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc., but it is not limited to this; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. Terminal devices and servers can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0040] In order to facilitate understanding of the data processing method provided in the embodiment of the present application, the application scenario of the data processing method is exemplarily introduced below by taking the execution subject of the data processing method as a server as an example.

[0041] See also Figure 1 , which is a schematic diagram of an application scenario of the data processing method provided in the embodiment of the present application. Figure 1As shown, the application scenario includes a terminal device 110 and a server 120, and the terminal device 110 and the server 120 can communicate through the network. Among them, a content test client is running on the terminal device 110, and the content can be video, text, music, etc. The users facing the content test client can be operation managers responsible for operating the content, etc., and the content test client can trigger the generation of a test suitable for testing the degree of influence of various intervention items on the content. The server 120 can be a server corresponding to the content test client, which is used to execute the data processing method provided in the embodiment of the present application, so as not to affect the experience of the object while adding suitable intervention items. The following is an example of testing the degree of influence on the video after adding advertisements to the video.

[0042] In actual applications, a user can trigger a video test operation for a type through a content test client running on the terminal device 110. After the terminal device 110 detects that the user has triggered the operation, it will generate a test instruction for the type of video and send the test instruction to the server 120.

[0043] After receiving the test instruction sent by the terminal device 110, the server 120 can first obtain multiple videos of this type and the advertisements that are to be added. Based on dividing the multiple videos into two groups, the videos in the first group of videos do not have advertisements added, and the videos in the second group of videos have advertisements added, and the random allocation principle is met as much as possible during the division process, so that the covariates of the first group of videos are similar to the covariates of the second group of videos, such as Figure 1 As shown, based on the features of the videos, multiple videos are divided into multiple categories, 50% of the videos are randomly selected from each type and assigned to the first group of videos, and the remaining 50% of the videos are assigned to the second group of videos, so that there is no selection bias between the first group of videos and the second group of videos as much as possible.

[0044] The server 120 delivers the first group of videos without advertisements and the second group of videos with advertisements to users for viewing, and collects relevant data. Compared with the content, the total sample size of the subjects consuming the content is not only large enough to reduce the impact of selection bias, but also the subjects consuming the content are used as individuals in the experimental group and the control group, which will not introduce selection bias that cannot be evaluated. Based on this, the server 120 obtains the control group of subjects consuming the first group of videos and the experimental group of subjects consuming the second group of videos. Figure 1 As shown, the server 120 uses the users who watch the first group of videos as the control group and the users who watch the second group of videos as the experimental group. The degree of influence of the intervention item on the content is determined based on the control group and the experimental group.

[0045] Therefore, we use content as the carrier of intervention items, divide multiple contents into two groups based on the benchmark of similar covariates, and compare the situations of the subjects who consume the two groups of content to obtain the degree of influence of the intervention items on the content. We use the collected data to construct the experimental group and control group similar to those in the AB experiment, which is equivalent to a "quasi-experimental" test, so as to use the analytical framework of causal inference to evaluate the effect through observation, and provide high-quality data for causal inference, which not only avoids the situation where causal inference cannot be performed due to data quality or the reliability of causal inference results is low, but also expands the application scenarios of verifying intervention items.

[0046] The data processing method provided in the embodiment of the present application can be executed by the server. However, in other embodiments of the present application, the terminal device can also have similar functions as the server, thereby executing the data processing method provided in the embodiment of the present application, or the terminal device and the server can jointly execute the data processing method provided in the embodiment of the present application, which is not limited in the present embodiment.

[0047] The data processing method provided by the present application is described in detail below through a method embodiment.

[0048] See also Figure 2 , which is a flow chart of the data processing method provided in the embodiment of the present application. For the convenience of description, the following embodiment is still introduced by taking the execution subject of the data processing method as a server as an example. Figure 2 As shown, the data processing method includes S201-S204.

[0049] S201: Acquire multiple contents and intervention items.

[0050] In order to test the impact of intervention items on content, obtain intervention items and multiple contents to be tested. Intervention items refer to variables to be tested that participate in content and achieve the effect of influencing content, such as strategies and plans. Content refers to the carrier to be tested, such as videos, texts, web pages, etc.

[0051] As a possible implementation, in order to avoid the impact of different contents on subsequent tests, the obtained multiple contents may be made to be of the same type, so as to reduce the impact of the type on the test results.

[0052] S202: Divide a plurality of contents into a first group of contents and a second group of contents to which an intervention item is applied.

[0053] Divide multiple contents into two groups, one of which is subject to the intervention item and the other is not, so as to compare the differences between the two groups in subsequent tests, and thus clarify the extent of the impact of the intervention item on the content. In order to ensure that only the intervention item has an impact, a "randomization mechanism" is required in the grouping process, so that the two groups of content are as consistent as possible, that is, there is no systematic selection bias, then the differences between the two subsequent groups can be attributed to the intervention item, and thus the size and significance of the intervention effect (that is, the impact of the intervention item on the content) can be inferred.

[0054] If there is selection bias, for example, videos with high popularity are divided into one group, and videos with low popularity are divided into another group. It is unknown whether the difference between the two groups is due to the influence of the intervention item or the influence of the hot factor. After analysis, the different covariates of the two groups of content will produce selection bias, thus affecting the intervention effect. Among them, the covariate refers to the independent variable that affects the content of the corresponding group and is not an intervention item during the content testing process. In other words, the covariate is not an intervention item. The covariate of the first group of content affects the first group of content, and the covariate of the second group of content affects the second group of content. Ensuring that the covariates of the two groups of content are relatively similar can minimize the difference between the two groups of content caused by the covariate. If the content is a video, the covariate can be the characteristics of the video, such as the popularity of the video, the content positioning of the video, etc. For example, if you want to study the effect of temperature on precipitation, but variables such as altitude, longitude and latitude, and local humidity will also affect precipitation. Then, temperature is the intervention item, and altitude, longitude and latitude, and local humidity are covariates.

[0055] Based on this, in the process of grouping, the covariate distributions of the two groups of content can be made similar, or even the covariates of the two groups of content can be made consistent, that is, the degree of similarity between the covariates of the first group of content and the covariates of the second group of content is greater than a first preset threshold, so as to meet the principle of random allocation as much as possible and ensure that there is no selection bias between the two groups of content as much as possible.

[0056] The embodiment of the present application does not specifically limit the size and content covariates of the first preset threshold, and those skilled in the art can set them according to actual needs.

[0057] S203: Obtain a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content.

[0058] As can be seen from the above, in an AB experiment, since the intervention item acts on the object, the experimental group and the control group can be constructed based on whether the object hits the intervention item, but this method will limit the application scenario of verifying the intervention item. Based on this, the embodiment of the present application no longer acts on the object with the intervention item, but on the content consumed by the object. At this time, the carrier (i.e., content) of the intervention item or the association between the carrier and the object can be used to indirectly use the object (i.e., the object consuming the content) to construct an experimental group and a control group similar to those in an AB experiment.

[0059] For example, taking adding advertisements to a certain type of video as an example, the intervention item refers to adding advertisements, and the content refers to the video. If the video itself is used to construct an experimental group and a control group similar to an AB experiment, the video with added advertisements is the sample in the experimental group, and the video without added advertisements is the sample in the control group. If the user who watched the video is used to construct an experimental group and a control group similar to an AB experiment, any user who has watched a video with advertisements can be used as a sample in the experimental group, and any user who has not watched a video with advertisements can be used as a sample in the control group.

[0060] After research, it was found that different construction methods, or selecting different samples as samples in the experimental group or control group, have different effects on the evaluation of intervention items. In order to reduce selection bias, the experimental group and the control group satisfy the conditional independence assumption (CIA), that is, under the condition of given variables, other variables are independent. Among them, the CIA assumption is an important assumption of causal inference, which requires the existence of a "randomization mechanism", that is, the distribution between the samples included in the control group and the experimental group is random, and there is no systematic selection bias. If there is a randomization mechanism, the difference between the experimental group and the control group can be attributed to the intervention measures, so that the size and significance of the intervention effect can be inferred.

[0061] Based on this, the embodiment of the present application proposes to construct an experimental group and a control group similar to those in an AB experiment based on the objects consuming the content, that is, to construct an object control group based on the objects consuming the first group of content, and to construct an object experimental group based on the objects consuming the second group of content.

[0062] First, the sample size of the subjects who consume content should be large enough to be used as the sample of the experimental and control groups, which can reduce the impact of selection bias. In the Internet scenario, the subjects who consume content are often counted in millions, tens of millions, or even hundreds of millions. If the number of content is only in the order of thousands, tens of thousands, or hundreds of thousands, the sample size may be insufficient when the content is used as the sample of the experimental and control groups. In addition, considering the lack of randomness in causal inference, it is often necessary to select samples with similar covariate distributions from the experimental and control groups for comparison, which will further reduce the number of available samples. Taking all factors into consideration, when the number of content is much smaller than the number of subjects, subjects can be preferentially selected as samples of the experimental and control groups.

[0063] Secondly, using the subjects who consume content as samples for the experimental and control groups will not introduce selection bias that cannot be evaluated. Using content as samples for the experimental and control groups may introduce selection bias that cannot be evaluated. Continuing with the example of adding advertisements to a certain type of video, each playback (video view, vv) of this type of video is used as a sample in the experimental and control groups. The playback volume of the video with added advertisements is the experimental group, and the playback volume of the video without added advertisements is the control group. If the advertisement causes the playback volume of the video to decrease, or in other words, the intervention item has a negative impact on the content, resulting in a decrease in the sample size, the selection bias of the sample brought about by this is difficult to measure and evaluate. If users who watch the video are used as samples, if the user feels that the experience has deteriorated due to the addition of advertisements, and will no longer watch the video, this change is a change that needs to be measured for the evaluation effect of the intervention item, rather than a change in the nature of the sample itself, and will not introduce bias that cannot be evaluated.

[0064] Based on this, after the first and second groups of content are released, the test is conducted to observe which subjects consume the first and second groups of content, and a control group of subjects is constructed based on the subjects who consume the first group of content, and an experimental group of subjects is constructed based on the subjects who consume the second group of content.

[0065] S204: Determine the degree of influence of the intervention item on the content based on the control group and the experimental group.

[0066] The consumption of the contents included in the first group by the subjects in the control group and the consumption of the contents included in the second group by the subjects in the experimental group can be compared. The difference between the two groups is caused by the intervention item, so the degree of influence of the intervention item on the content can be determined.

[0067] It should be noted that in the specific implementation of this application, the object consumption content is equal to the user information related to the user and other related data. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the user's separate permission or consent, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0068] It can be seen from the above technical solution that in order to test the degree of influence of the intervention item on the content, the intervention item and multiple contents to be tested are obtained, and the multiple contents are divided into two groups, namely the first group of contents and the second group of contents, wherein the second group of contents is subjected to the intervention item, and the first group of contents is not subjected to the intervention item, and the random allocation principle is satisfied as much as possible during the division process, so that the covariates of the first group of contents and the covariates of the second group of contents are similar, so that the first group of contents and the second group of contents have no selection bias as much as possible. Compared with the content, the total sample size of the objects consuming the content is not only large enough to reduce the influence of the selection bias, but also the objects consuming the content are used as individuals in the experimental group and the control group, which will not introduce selection bias that cannot be evaluated. Based on this, the control group of objects consuming the first group of contents and the experimental group of objects consuming the second group of contents are obtained, and the degree of influence of the intervention item on the content is determined according to the control group and the experimental group. Therefore, the content is used as the carrier of the intervention item, and multiple contents are divided into two groups of contents based on the benchmark of similar covariates. The situation of the objects consuming the two groups of contents is compared to obtain the degree of influence of the intervention item on the content. Using the data that can be collected to construct experimental and control groups similar to those in AB experiments is equivalent to a "quasi-experimental" test, so that the analytical framework of causal inference can be used to evaluate the effects through observation, providing high-quality data for causal inference. This not only avoids the situation where causal inference cannot be performed or the reliability of causal inference results is low due to data quality, but also expands the application scenarios of verifying intervention items.

[0069] As a possible implementation method, an embodiment of the present application provides a specific implementation method of S202, namely, a specific implementation method of dividing multiple contents into a first group of contents and a second group of contents with intervention items applied. Specifically, if the covariate is a feature, the multiple contents can be divided into multiple categories according to the features included in the content; a first preset number of contents are extracted from the contents of each category to obtain a first group of contents, and a second preset number of contents are extracted from the contents of each category to obtain a second group of contents.

[0070] Among them, features describe the specific performance of content in a certain aspect, such as funny content, emotional content, etc. Features can also be called attributes. Generally, they are expressed in the form of a column in the data set. One feature represents a column of data.

[0071] As a possible implementation, after determining the features, a classification result for the content can be obtained based on a trained classification model, and then based on the classification result, a portion of the content (i.e., a second preset number of content) is randomly selected from each type to form a second group of content, and intervention items are added thereto, and the remaining content (i.e., a first preset number of content) forms a first group of content without adding intervention items. It should be noted that the category distribution of the content included in the first group of content is consistent with the category distribution of the content included in the second group of content, so that the similarity between the covariates of the first group of content and the covariates of the second group of content is greater than the first preset threshold, thereby reducing selection bias.

[0072] See also Figure 3 , which is a schematic diagram of a plurality of content divisions provided in an embodiment of the present application. Figure 3 In the above example, each content has n features, and multiple contents are input into the classification model respectively. Based on the classification model, the contents can be divided into m categories, 50% of the contents in each category are divided into the first group of contents, and the remaining 50% of the contents are divided into the second group of contents, and intervention items are added to the second group of contents including each content. Wherein, m and n are both integers greater than 0.

[0073] The embodiments of the present application do not specifically limit the classification model, and those skilled in the art can set it according to actual needs. For example, if a decision tree is used as a classification model, each leaf node corresponds to a type of content, and a certain proportion of content can be randomly extracted from each leaf node (i.e., a second preset number of content is obtained) as the content of the added intervention item, and no intervention item is added to the remaining content. For another example, if a clustering model is used as a classification model, each cluster corresponds to a type of content, and a certain proportion of content can be randomly extracted from each cluster (i.e., a second preset number of content is obtained) as the content of the added intervention item, and no intervention item is added to the remaining content.

[0074] The embodiment of the present application does not specifically limit the size of the extraction ratio, and those skilled in the art can set it according to actual conditions, for example, 50%-50%, 60%-40%, 70%-30%, etc. It should be noted that the extraction ratio can be changed to increase the second preset number, thereby increasing the content of the intervention items and correspondingly obtaining more benefits. Based on this, the size of the second preset number can be increased as much as possible while ensuring that it does not affect the subsequent evaluation of the intervention items, thereby increasing the benefits.

[0075] Therefore, the contents are divided into multiple categories based on the characteristics of the contents, and multiple contents are divided into a first group of contents and a second group of contents based on the criterion of consistent category distribution, so that the category distribution of the first group of contents is consistent with the category distribution of the second group of contents, and the principle of random allocation is met as much as possible, so that the covariates of the first group of contents are similar to the covariates of the second group of contents, and there is no selection bias between the first group of contents and the second group of contents as much as possible.

[0076] As a possible implementation method, after research, it was found that some objects consume both the first group of content and the second group of content. For example, some users watch both videos with ads and videos without ads. If these users are also taken into account, it may be difficult to eliminate the network effect they bring. Based on this, this type of object can be treated as invalid data.

[0077] Specifically, a set of objects consuming content is obtained, the object set including multiple objects; the multiple objects are divided into an object control group, an object experimental group and an invalid object group according to the content consumed by the objects; wherein the objects included in the object control group are objects that only consume the first group of content, the objects included in the object experimental group are objects that only consume the second group of content, and the objects included in the invalid object group are objects that consume both the first group of content and the second group of content.

[0078] Continuing with the example of adding advertisements to videos, users can be divided into three categories: (1) users who have only watched videos with advertisements are added to the experimental group; (2) users who have only watched videos without advertisements are added to the control group; (3) users who have watched both videos with advertisements and videos without advertisements are added to the invalid group.

[0079] See also Figure 4 , which is a diagram based on the embodiment of the present application. Figure 3 A diagram showing how to group objects that consume content. Figure 4 In the example, the subjects who viewed the second group of content are subject 1, subject 2, subject 3, subject 4 and subject 5, and the subjects who viewed the first group of content are subject 5, subject 6 and subject x. The subject experimental group includes subject 1, subject 2 and subject 3, and the subject control group includes subject 6 and subject x. The invalid subject group includes subject 4 and subject 5. Wherein, x is an integer greater than 6.

[0080] Therefore, the subjects consuming content are divided more finely into a control group, an experimental group and an invalid group. The invalid group that brings network effects is removed and will not be analyzed in the future, so as to improve the accuracy of the subsequent evaluation of the intervention items.

[0081] As a possible implementation, the present application embodiment provides a specific implementation of S204, that is, a specific implementation of determining the degree of influence of the intervention item on the content based on the object control group and the object experimental group.

[0082] After determining the control group and the experimental group, subsequent causal inference analysis can be performed based on the observational data related to the control group and the experimental group, so as to clarify the magnitude and significance of the subsequent intervention on the content. After analysis, it was found that although the two groups of content were made as consistent as possible during the grouping process to reduce systematic selection bias, the content consumed by the subjects is not random. Taking video as an example, users watching videos with added advertisements is not necessarily random. For example, it may be related to covariates such as user preferences, causing users to watch some videos based on their preferences. Therefore, there will be selection bias.

[0083] As mentioned above, if there is a randomization mechanism, then the differences between the experimental group and the control group can be attributed to the intervention measures, and the size and significance of the intervention effect can be inferred. If the CIA assumption does not hold, that is, when there is selection bias in the observed data, the observed data needs to be fully analyzed and processed to minimize the impact of selection bias and improve the reliability of causal inference. This process involves the selection of covariates. By selecting appropriate covariates and appropriate processing, the covariates of the experimental group and the control group are made similar, and then the size and significance of the intervention effect can be measured on this basis.

[0084] Based on this, the embodiments of the present application make the preferences of the objects included in the object control group and the object experimental group similar based on the similarity between the objects, so as to reduce the selection bias. Specifically, the similarity between the objects included in the object control group and the objects included in the object experimental group is determined; the first target object and the second target object are determined according to the similarity; the first target object is removed from the object control group to obtain an updated object control group, and the second target object is removed from the object experimental group to obtain an updated object experimental group. Thus, the degree of influence of the intervention item on the content can be determined according to the updated object control group and the updated object experimental group.

[0085] Wherein, the first target object is an object in the object control group, and the similarity between the first target object and the objects included in the object experimental group is less than or equal to the second preset threshold, or in other words, the first target object is not similar to the objects in the object experimental group, then the first target object may have its own unique preferences, which may lead to selection bias. Similarly, the second target object is an object in the object experimental group, and the similarity between the first target object and the objects included in the object control group is less than or equal to the second preset threshold, then the second target object may have its own unique preferences, which may lead to selection bias.

[0086] The embodiment of the present application does not specifically limit the size of the second preset threshold, and those skilled in the art can set it according to actual needs.

[0087] Therefore, based on the similarity between the objects, the objects with their own unique preferences are eliminated from the experimental group and the control group, so that the objects included in the experimental group and the control group have a higher similarity, avoiding the influence of the objects' preferences on the size and significance of the impact of the intervention items on the content in the subsequent determination.

[0088] The embodiment of the present application does not specifically limit the specific implementation method of determining the similarity between the objects included in the control group and the objects included in the experimental group. For example, models such as Propensity Score Matching (PSM), Difference-in-Differences (DID), and Synthetic Control Method (SCM) can be used to deal with the selection bias in the observed data. PSM is used as an example for illustration.

[0089] The selection bias of observed data processed based on PSM is mainly divided into calculating propensity scores and matching based on propensity scores. The following are the explanations of each.

[0090] (1) Calculate the propensity score.

[0091] According to the characteristics of the objects included in the object control group, the probability of the objects included in the object control group being assigned to the target group is determined, and according to the characteristics of the objects included in the object experimental group, the probability of the objects included in the object experimental group being assigned to the target group is determined, and the target group is the object control group or the object experimental group; according to the probability that the objects included in the object control group are assigned to the target group and the probability that the objects included in the object experimental group are assigned to the target group, the similarity between the objects included in the object control group and the objects included in the object experimental group is determined.

[0092] Among them, the characteristics of the object belong to the covariate of the object. The covariate is a factor that affects the effect of the intervention. To reduce the selection bias, it is necessary to select covariates related to the intervention measures, such as the age, gender, and region of the object. It is worth mentioning that since the object has clearly defined characteristics, such as gender, age, content preference, etc., there are rich feature labels that can be used as covariates to evaluate and adjust the selection bias in the observed data. This is one of the beneficial effects of using the objects of content consumption as samples to construct the experimental group and the control group of objects.

[0093] Based on the covariates, the probability of each sample (the subjects included in the control group and the subjects included in the experimental group) being assigned to the target group is calculated, which can be called the propensity score. The basic principle of calculating the propensity score is to replace multiple covariates with one score to balance the distribution of covariates between the control group and the experimental group. The confounding factors in non-randomized studies are treated similarly to randomized balanced treatment to reduce selection bias.

[0094] The present application embodiment does not specifically limit the method of calculating the propensity score, such as logistic regression, decision tree, random forest, support vector machine (SVM), etc. It should be noted that the following three factors can be referred to when selecting the method of calculating the propensity score.

[0095] Factor 1: Data type and distribution. If the data is binary, you can choose logistic regression or support vector machine. If the data is multi-classification, you can choose decision tree or random forest.

[0096] Factor 2: Model assumptions and preconditions. Logistic regression assumes that samples are independent and identically distributed, and needs to meet some preconditions, such as linear separability. Decision trees and random forests do not need to meet the linear separability condition, but need to pay attention to the overfitting problem.

[0097] Factor 3: Model performance and stability. Logistic regression and support vector machines perform better when processing high-dimensional data, while decision trees and random forests perform better when processing nonlinear data.

[0098] See also Figure 5 , which is a diagram based on the embodiment of the present application. Figure 4 Schematic diagram of calculating propensity scores. Figure 5 In, based on Figure 4 After obtaining the subject experimental group and the subject control group in a manner, the propensity score of each subject included in the subject experimental group and the subject control group is calculated based on the characteristics of the subject.

[0099] Thus, based on the propensity score, the similarity between objects belonging to two different groups can be obtained.

[0100] (2) Matching based on propensity score.

[0101] Specifically, samples with similar propensity scores in the experimental group and the control group are matched, and samples that are not successfully matched are the first target objects or the second target objects, so that the influence of covariates can be controlled by eliminating the first target objects and / or the second target objects.

[0102] The embodiments of the present application do not specifically limit the matching method, for example, caliper matching (Caliper Matching), CEM matching (Coarsened Exact Matching), nearest neighbor matching (Nearest Neighbor Matching), etc.

[0103] Therefore, determining the impact of the intervention item on the content based on the updated subject experimental group and the updated subject control group can reduce the selection bias and the influence of confounding variables so as to make a more reasonable comparison between the updated subject experimental group and the updated subject control group.

[0104] As a possible implementation method, when choosing a method for calculating the propensity score and matching the propensity score, the accuracy of the result needs to be considered comprehensively. Specifically, after obtaining the updated object experimental group and the updated object control group based on the propensity score, it is also necessary to evaluate whether the covariates of the updated object experimental group and the updated object control group are balanced, that is, whether the selection bias of the observed data has been basically eliminated or is small enough.

[0105] Based on this, the embodiment of the present application provides a specific implementation method for determining the degree of influence of the intervention item on the content based on the update object control group and the update object experimental group. Specifically, the first distribution of the covariates of the update object control group and the second distribution of the covariates of the update object experimental group can be determined; if the first distribution is similar to the second distribution, it means that the covariates of the update object experimental group and the update object control group are relatively balanced, and whether the selection bias of the observation data has been basically eliminated or is small enough, the degree of influence of the intervention item on the content can be determined based on the update object control group and the update object experimental group.

[0106] The first distribution refers to the distribution of the covariates in the updated object control group, and the second distribution refers to the distribution of the covariates in the updated object experimental group. If the two distributions are similar, by comparing the distribution of the propensity scores in the updated object experimental group, the updated object control group, the object experimental group and the object control group (the latter is before and after matching), and the quantile-quantile plot (QQ-Plot) of the characteristics before and after matching, it can be concluded that the distribution of the propensity scores of the updated object experimental group and the updated object control group is closer, and the distribution of the variables is also closer.

[0107] The embodiments of the present application do not specifically limit the method of evaluating and obtaining the updated object control group and the updated object experimental group. For example, a standard mean difference (SMD) test, prognostic scores (Prognosticscores), a kernel density plot (Kernel Density Plot), etc. can be used.

[0108] Taking the SMD test as an example, it mainly calculates the mean difference between the experimental group and the control group of the updated objects and standardizes it into the form of standard deviation for comparison. The specific formula is as follows:

[0109] SMD = (mean of the experimental group of updated subjects - mean of the control group of updated subjects) / standard deviation.

[0110] SMD is a quantitative indicator, which refers to the distance between the two groups' means in terms of the standard deviation of the covariate distribution. It can be used to test whether there is a significant difference between the updated object experimental group and the updated object control group, and to determine the matching effect. The larger the SMD value, the greater the difference between the updated object experimental group and the updated object control group; otherwise, the smaller the difference.

[0111] Therefore, the combination of methods that can minimize the SMD of each covariate between the experimental group and the control group (i.e., the combination of the method of calculating the propensity score and the method of matching based on the propensity score) is the most appropriate choice. Therefore, when choosing between different methods, it is necessary to comprehensively consider various factors to improve the accuracy and reliability of the propensity score.

[0112] The following uses the combination of random forest and caliper matching as an example to illustrate how to deal with selection bias in observational data.

[0113] a) Data preparation. Obtain observational data related to the control group and the experimental group, determine the independent variable (i.e., the covariate of the subject) and the dependent variable, where the independent variable is a variable that may affect the dependent variable, also known as a confounding factor, such as the characteristics of the subject. The dependent variable is the difference between the control group and the experimental group, such as whether an intervention item is applied.

[0114] b) Calculate the propensity score. Use the observation data related to the control group and the experimental group to build a random forest model. Use the established random forest model to predict all samples and get the probability of each sample belonging to the experimental group (i.e., the target group). This probability is the propensity score, which represents the probability of the sample being assigned to the experimental group.

[0115] c) Select a caliper: Select a caliper (i.e., a second preset threshold), i.e., a fixed distance, for matching the objects included in the object experimental group and the objects included in the object control group. As a possible implementation, the second preset threshold can be determined based on the probability that the objects included in the object control group are assigned to the target group and the probability that the objects included in the object experimental group are assigned to the target group. In other words, the size of the caliper can be determined based on the propensity score of each object. As a possible implementation, the size of the caliper can be determined based on the distribution of the sample and the accuracy of the propensity score.

[0116] d) Matching samples: For each subject included in the experimental group, one or more samples with the closest propensity score are selected from the control group for matching. The specific matching method can be calculated using methods such as Euclidean distance and Mahalanobis distance. Objects in the control group whose distance is less than the selected caliper can be used as matching samples. If an object included in a certain experimental group does not have a corresponding object included in the control group, the object included in the experimental group can be determined as the first target object and excluded from the analysis.

[0117] e) Check the matching quality: Calculate the SMD results for each covariate. If the results are close to the selected caliper, the matching quality and effect are good, and the matched samples can be used for analysis, that is, based on the updated object experimental group and the updated object control group, determine the degree of influence of the intervention item on the content.

[0118] See also Figure 6 , which is a diagram based on the embodiment of the present application. Figure 5 Schematic diagram of implementing object matching. Figure 6 In , the distances between the objects included in the object experimental group and the objects included in the object control group are calculated respectively, such as the distance between object 1 and object 6 is distance 1, the distance between object 1 and object n is distance 2, etc., wherein the distances less than or equal to the caliper are distance 1 and distance 4, and the distances greater than the caliper are distance 2, distance 3, distance 5, and distance 6. Therefore, the objects that are successfully matched are object 1 and object 6, and object 2 and object x.

[0119] Therefore, after completing the processing of the selection bias of the observed data, if the updated object experimental group and the updated object control group are obtained using PSM, the degree of influence of the intervention item on the content can be determined based on the updated object experimental group and the updated object control group.

[0120] The embodiments of the present application do not specifically limit the method for determining the degree of influence of the intervention on the content, which may be the average treatment effect (ATE), the average treatment effect on the treated (ATT), the conditional average treatment effect (CATE), etc.

[0121] The following is an explanation using ATE as an example, that is, the embodiment of the present application also provides a specific implementation method for determining the degree of influence of the intervention item on the content based on the update object control group and the update object experimental group. Specifically, a first expected value of the evaluation index of the update object control group and a second expected value of the evaluation index of the update object experimental group are determined; if the second expected value is greater than the first expected value, the intervention item has a positive influence on the content; if the second expected value is less than the first expected value, the intervention item has a negative influence on the content.

[0122] Among them, the evaluation index refers to the embodiment of the degree of influence of the intervention item on the content. The embodiment of the present application does not specifically limit the evaluation index, and those skilled in the art can set it according to actual needs. For example, if the impact of adding advertisements to a video is tested, revenue can be used as an evaluation index. The first expected value refers to the expected value of the evaluation index of the updated object control group, and the second expected value refers to the expected value of the evaluation index of the updated object experimental group.

[0123] ATE evaluates the average effect of the intervention on the subject groups included in the updated subject experimental group and the updated subject control group, which can be expressed as the following formula:

[0124] ATE=E[Y1)]-E[Y(0)]

[0125] Where ATE represents the average treatment effect, E[Y(1)] represents the first expected value of the evaluation index Y in the updated object experimental group, and E[Y(0)] represents the second expected value of the evaluation index Y in the updated object control group.

[0126] In simple terms, ATE is the difference in expected values ​​for the evaluation indicator Y between the experimental group and the control group. If ATE is positive, that is, the second expected value is greater than the first expected value, it means that the intervention item has a positive impact on the content; if ATE is negative, that is, the second expected value is less than the first expected value, it means that the intervention item has a negative impact on the content; if ATE is zero, that is, the second expected value is equal to the first expected value, it means that the intervention item has no impact on the content.

[0127] See also Figure 7 , which is a diagram based on the embodiment of the present application. Figure 6 Schematic diagram for determining the extent of impact. Figure 7 In , we can get the evaluation index-object 1 based on object 1 and the evaluation index-object 2 based on object 2, and then get E[Y(1)] based on the average of the evaluation index-object 1 and the evaluation index-object 2. Similarly, we can get the evaluation index-object 6 based on object 6 and the evaluation index-object x based on object x, and then get E[Y(0)] based on the average of the evaluation index-object 6 and the evaluation index-object x, and then get ATE.

[0128] As a possible implementation method, in addition to using ATE to determine the degree of influence of the intervention item on the content, meta-learning (meta-learner) can also be used to determine the degree of influence of the intervention item on the content, that is, the embodiment of the present application provides a specific implementation method for determining the degree of influence of the intervention item on the content based on the updated object control group and the updated object experimental group. Specifically, the meta-learner is used to update the object control group and the updated object experimental group to predict the treatment effect, obtain an estimated value of the treatment effect, and calculate the confidence interval and significance level of the treatment effect to evaluate the reliability and significance of the intervention effect (the degree of influence of the intervention item on the content).

[0129] In general, there are many methods of causal inference that can be used. Before obtaining observation data in the early stage, we should try to improve the data quality through scientific and effective planning, such as using PSM to obtain the experimental group and the control group of updated objects, so as to improve the accuracy of causal inference results and provide strong data support for the implementation of strategies. Therefore, for scenarios where AB experiments cannot be conducted, "quasi-experimental" tests are conducted through scientific program design to provide high-quality data for causal inference, avoiding situations where causal inference cannot be conducted due to data quality or the reliability of causal inference results is low.

[0130] With respect to the data processing method described above, the present application also provides a corresponding data processing device so that the above data processing method can be applied and implemented in practice.

[0131] See also Figure 8 , which is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 8 As shown, the data processing device 800 includes: an acquisition unit 801, a division unit 802 and a determination unit 803;

[0132] The acquisition unit 801 is used to acquire multiple contents and intervention items;

[0133] The division unit 802 is used to divide the plurality of the contents into a first group of contents and a second group of contents to which the intervention item is applied, wherein the degree of similarity between the covariates of the first group of contents and the covariates of the second group of contents is greater than a first preset threshold, and the covariates are independent variables that affect the corresponding group of contents and are not the intervention item during the process of testing the contents;

[0134] The acquisition unit 801 is further configured to acquire a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content;

[0135] The determining unit 803 is used to determine the influence degree of the intervention item on the content according to the object control group and the object experimental group.

[0136] It can be seen from the above technical solution that in order to test the degree of influence of the intervention item on the content, the intervention item and multiple contents to be tested are obtained, and the multiple contents are divided into two groups, namely the first group of contents and the second group of contents, wherein the second group of contents is subjected to the intervention item, and the first group of contents is not subjected to the intervention item, and the random allocation principle is satisfied as much as possible during the division process, so that the covariates of the first group of contents and the covariates of the second group of contents are similar, so that the first group of contents and the second group of contents have no selection bias as much as possible. Compared with the content, the total sample size of the objects consuming the content is not only large enough to reduce the influence of the selection bias, but also the objects consuming the content are used as individuals in the experimental group and the control group, which will not introduce selection bias that cannot be evaluated. Based on this, the control group of objects consuming the first group of contents and the experimental group of objects consuming the second group of contents are obtained, and the degree of influence of the intervention item on the content is determined according to the control group and the experimental group. Therefore, the content is used as the carrier of the intervention item, and multiple contents are divided into two groups of contents based on the benchmark of similar covariates. The situation of the objects consuming the two groups of contents is compared to obtain the degree of influence of the intervention item on the content. Using the data that can be collected to construct experimental and control groups similar to those in AB experiments is equivalent to a "quasi-experimental" test, so that the analytical framework of causal inference can be used to evaluate the effects through observation, providing high-quality data for causal inference. This not only avoids the situation where causal inference cannot be performed or the reliability of causal inference results is low due to data quality, but also expands the application scenarios of verifying intervention items.

[0137] As a possible implementation manner, if the covariate is a feature, the division unit 802 is specifically configured to:

[0138] Classifying the plurality of contents into a plurality of categories according to the features included in the contents;

[0139] A first preset number of contents is extracted from each category of contents to obtain a first group of contents, and a second preset number of contents is extracted from each category of contents to obtain a second group of contents, wherein the category distribution of the contents included in the first group of contents is consistent with the category distribution of the contents included in the second group of contents, and the intervention item is applied to the second group of contents.

[0140] As a possible implementation, the acquisition unit 801 is further configured to acquire a set of objects that consume the content, wherein the set of objects includes a plurality of objects;

[0141] The division unit 802 is further used to divide the multiple subjects into a subject control group, a subject experimental group and an invalid subject group according to the content consumed by the subjects;

[0142] The control group includes subjects who only consume the first group of content, the experimental group includes subjects who only consume the second group of content, and the invalid group includes subjects who consume both the first and second groups of content.

[0143] As a possible implementation manner, the determining unit 803 is specifically configured to:

[0144] determining the similarity between the subjects included in the control group of subjects and the subjects included in the experimental group of subjects;

[0145] Determine a first target object and a second target object according to the similarity, wherein the first target object is an object included in the object control group, and the similarity between the first target object and the object included in the object experimental group is less than or equal to a second preset threshold, and the second target object is an object included in the object experimental group, and the similarity between the second target object and the object included in the object control group is less than or equal to the second preset threshold;

[0146] removing the first target subject from the subject control group to obtain an updated subject control group, and removing the second target subject from the subject experimental group to obtain an updated subject experimental group;

[0147] The degree of influence of the intervention item on the content is determined based on the update object control group and the update object experimental group.

[0148] As a possible implementation manner, the determining unit 803 is specifically configured to:

[0149] Determining the probability that the subjects included in the subject control group are assigned to the target group according to the characteristics of the subjects included in the subject control group, and determining the probability that the subjects included in the subject experimental group are assigned to the target group according to the characteristics of the subjects included in the subject experimental group, the target group being the subject control group or the subject experimental group;

[0150] The similarity between the objects included in the object control group and the objects included in the object experimental group is determined based on the probability that the objects included in the object control group are assigned to the target group and the probability that the objects included in the object experimental group are assigned to the target group.

[0151] As a possible implementation, the determination unit 803 is further used to determine the second preset threshold based on the probability that the objects included in the object control group are assigned to the target group and the probability that the objects included in the object experimental group are assigned to the target group.

[0152] As a possible implementation manner, the determining unit 803 is specifically configured to:

[0153] Determining a first distribution of covariates for the control group of updated subjects and a second distribution of covariates for the experimental group of updated subjects;

[0154] If the first distribution is similar to the second distribution, determining the degree of influence of the intervention item on the content according to the update object control group and the update object experimental group.

[0155] As a possible implementation manner, the determining unit 803 is specifically configured to:

[0156] Determining a first expected value of the evaluation index of the updated object control group and a second expected value of the evaluation index of the updated object experimental group;

[0157] If the second expected value is greater than the first expected value, the intervention item has a positive impact on the content; if the second expected value is less than the first expected value, the intervention item has a negative impact on the content.

[0158] The present application also provides a computer device, which is the computer device described above. The computer device may be a server or a terminal device. The data processing device described above may be built into the server or the terminal device. The computer device provided by the present application will be described from the perspective of hardware entity. Fig. 9 The following is a schematic diagram of the server structure. Fig.10 Shown is a schematic diagram of the structure of the terminal equipment.

[0159] See also Fig. 9 , which is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1400 may have relatively large differences due to different configurations or performances, and may include one or more processors 1422, such as central processing units (CPU), memory 1432, one or more application programs 1442 or storage media 1430 (such as one or more massive storage devices) for data 1444. Among them, the memory 1432 and the storage medium 1430 may be temporary storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the processor 1422 may be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the server 1400.

[0160] The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0161] The steps performed by the server in the above embodiment can be based on the Fig. 9 The server structure shown.

[0162] The CPU 1422 is used to execute the following steps:

[0163] access to multiple content and intervention items;

[0164] Dividing the plurality of the contents into a first group of contents and a second group of contents to which the intervention item is applied, wherein the degree of similarity between the covariates of the first group of contents and the covariates of the second group of contents is greater than a first preset threshold, and the covariates are independent variables that affect the corresponding group of contents and are not the intervention item during the process of testing the contents;

[0165] obtaining a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content;

[0166] The degree of influence of the intervention item on the content is determined based on the subject control group and the subject experimental group.

[0167] Optionally, the CPU 1422 may also execute the method steps of any specific implementation of the data processing method in the embodiments of the present application.

[0168] See also Fig.10 , which is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Taking the terminal device as a smart phone as an example, the smart phone includes: a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a wireless fidelity (WiFi) module 1570, a processor 1580, and a power supply 1590. Those skilled in the art can understand that Fig.10 The structure of the smartphone shown in the figure does not constitute a limitation on the smartphone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. It is understood that Fig.10The structure shown is a partial structure of the smartphone.

[0169] Combine the following Fig.10 A detailed introduction to the various components of a smartphone:

[0170] The RF circuit 1510 may be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information from the base station, it is sent to the processor 1580 for processing; in addition, the designed uplink data is sent to the base station.

[0171] The memory 1520 may be used to store software programs and modules. The processor 1580 implements various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 1520 .

[0172] The input unit 1530 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the smartphone. Specifically, the input unit 1530 may include a touch panel 1531 and other input devices 1532. The touch panel 1531, also known as a touch screen, can collect user touch operations on or near it and drive the corresponding connection device according to a pre-set program. In addition to the touch panel 1531, the input unit 1530 may also include other input devices 1532. Specifically, other input devices 1532 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0173] The display unit 1540 may be used to display information input by the user or information provided to the user and various menus of the smartphone. The display unit 1540 may include a display panel 1541, and the display panel 1541 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0174] The smartphone may also include at least one sensor 1550, such as a light sensor, a motion sensor, and other sensors. As for other sensors that may be configured in the smartphone, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., they will not be described in detail here.

[0175] The audio circuit 1560, the speaker 1561, and the microphone 1562 can provide an audio interface between the user and the smartphone. The audio circuit 1560 can transmit the received audio data to the speaker 1561 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1560 and converted into audio data, and then the audio data is output to the processor 1580 for processing, and then sent to another smartphone through the RF circuit 1510, or the audio data is output to the memory 1520 for further processing.

[0176] The processor 1580 is the control center of the smartphone, and uses various interfaces and lines to connect various parts of the entire smartphone, and executes various functions of the smartphone and processes data by running or executing software programs and / or modules stored in the memory 1520, and calling data stored in the memory 1520. Optionally, the processor 1580 may include one or more processing units.

[0177] The smart phone also includes a power supply 1590 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1580 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption management.

[0178] Although not shown, the smartphone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0179] In the embodiment of the present application, the memory 1520 included in the smart phone can store a computer program and transmit the computer program to the processor.

[0180] The processor 1580 included in the smart phone can execute the data processing method provided in the above embodiment according to the computer program.

[0181] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute the data processing method provided in the above embodiment.

[0182] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data processing method provided in various optional implementations of the above aspects.

[0183] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the above storage medium can be at least one of the following media: read-only memory (English: Read-Only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store computer programs.

[0184] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0185] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: access to multiple content and intervention items; Dividing the plurality of the contents into a first group of contents and a second group of contents to which the intervention item is applied, wherein the degree of similarity between the covariates of the first group of contents and the covariates of the second group of contents is greater than a first preset threshold, and the covariates are independent variables that affect the corresponding group of contents and are not the intervention item during the process of testing the contents; obtaining a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content; The degree of influence of the intervention item on the content is determined based on the subject control group and the subject experimental group.

2. The method according to claim 1, characterized in that: If the covariate is a feature, the step of dividing the plurality of contents into a first group of contents and a second group of contents to which the intervention item is applied comprises: Classifying the plurality of contents into a plurality of categories according to the features included in the contents; A first preset number of contents is extracted from each category of contents to obtain a first group of contents, and a second preset number of contents is extracted from each category of contents to obtain a second group of contents, wherein the category distribution of the contents included in the first group of contents is consistent with the category distribution of the contents included in the second group of contents, and the intervention item is applied to the second group of contents.

3. The method according to claim 1, characterized in that The method further comprises: Acquire an object set that consumes the content, the object set including a plurality of objects; According to the content consumed by the subjects, the plurality of subjects are divided into a subject control group, a subject experimental group and an invalid subject group; The control group includes subjects who only consume the first group of content, the experimental group includes subjects who only consume the second group of content, and the invalid group includes subjects who consume both the first and second groups of content.

4. The method according to claim 1, characterized in that: The method further comprises: determining the similarity between the subjects included in the control group of subjects and the subjects included in the experimental group of subjects; Determine a first target object and a second target object according to the similarity, wherein the first target object is an object included in the object control group, and the similarity between the first target object and the object included in the object experimental group is less than or equal to a second preset threshold, and the second target object is an object included in the object experimental group, and the similarity between the second target object and the object included in the object control group is less than or equal to the second preset threshold; removing the first target subject from the subject control group to obtain an updated subject control group, and removing the second target subject from the subject experimental group to obtain an updated subject experimental group; The determining, based on the subject control group and the subject experimental group, the degree of influence of the intervention item on the content includes: The degree of influence of the intervention item on the content is determined based on the update object control group and the update object experimental group.

5. The method according to claim 4, characterized in that Determining the similarity between the subjects included in the subject control group and the subjects included in the subject experimental group comprises: Determining the probability that the subjects included in the subject control group are assigned to the target group according to the characteristics of the subjects included in the subject control group, and determining the probability that the subjects included in the subject experimental group are assigned to the target group according to the characteristics of the subjects included in the subject experimental group, the target group being the subject control group or the subject experimental group; The similarity between the objects included in the object control group and the objects included in the object experimental group is determined based on the probability that the objects included in the object control group are assigned to the target group and the probability that the objects included in the object experimental group are assigned to the target group.

6. The method according to claim 5, characterized in that The method further comprises: The second preset threshold is determined according to the probability that the subjects included in the subject control group are assigned to the target group and the probability that the subjects included in the subject experimental group are assigned to the target group.

7. The method according to claim 4, characterized in that The determining, based on the update object control group and the update object experimental group, the degree of influence of the intervention item on the content includes: Determining a first distribution of covariates for the control group of updated subjects and a second distribution of covariates for the experimental group of updated subjects; If the first distribution is similar to the second distribution, determining the degree of influence of the intervention item on the content according to the update object control group and the update object experimental group.

8. The method according to claim 4, characterized in that The determining, based on the update object control group and the update object experimental group, the degree of influence of the intervention item on the content includes: Determining a first expected value of the evaluation index of the updated object control group and a second expected value of the evaluation index of the updated object experimental group; If the second expected value is greater than the first expected value, the intervention item has a positive impact on the content; if the second expected value is less than the first expected value, the intervention item has a negative impact on the content.

9. A data processing device, characterized in that: The device comprises: an acquisition unit, a division unit and a determination unit; The acquisition unit is used to acquire multiple contents and intervention items; The division unit is used to divide the plurality of the contents into a first group of contents and a second group of contents to which the intervention item is applied, wherein the degree of similarity between the covariates of the first group of contents and the covariates of the second group of contents is greater than a first preset threshold, and the covariates are independent variables that affect the corresponding group of contents and are not the intervention item during the process of testing the contents; The acquisition unit is further used to acquire a control group of subjects who consume the first group of content, and an experimental group of subjects who consume the second group of content; The determination unit is used to determine the influence degree of the intervention item on the content according to the object control group and the object experimental group.

10. The device according to claim 9, characterized in that If the covariate is a feature, the partitioning unit is specifically used for: Classifying the plurality of contents into a plurality of categories according to the features included in the contents; A first preset number of contents is extracted from each category of contents to obtain a first group of contents, and a second preset number of contents is extracted from each category of contents to obtain a second group of contents, wherein the category distribution of the contents included in the first group of contents is consistent with the category distribution of the contents included in the second group of contents, and the intervention item is applied to the second group of contents.

11. The device according to claim 9, characterized in that The acquisition unit is further used to acquire a set of objects that consume the content, wherein the set of objects includes a plurality of objects; The division unit is further used to divide the multiple subjects into a subject control group, a subject experimental group and an invalid subject group according to the content consumed by the subjects; The control group includes subjects who only consume the first group of content, the experimental group includes subjects who only consume the second group of content, and the invalid group includes subjects who consume both the first and second groups of content.

12. The device according to claim 9, characterized in that The determining unit is further configured to: determining the similarity between the subjects included in the control group of subjects and the subjects included in the experimental group of subjects; Determine a first target object and a second target object according to the similarity, wherein the first target object is an object included in the object control group, and the similarity between the first target object and the object included in the object experimental group is less than or equal to a second preset threshold, and the second target object is an object included in the object experimental group, and the similarity between the second target object and the object included in the object control group is less than or equal to the second preset threshold; removing the first target subject from the subject control group to obtain an updated subject control group, and removing the second target subject from the subject experimental group to obtain an updated subject experimental group; The degree of influence of the intervention item on the content is determined based on the update object control group and the update object experimental group.

13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1 to 8 according to the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 8.

15. A computer program product comprising a computer program, characterized in that When the method is executed on a computer device, the computer device is enabled to execute the method according to any one of claims 1 to 8.

Images