Sample screening method and device, computer device and storage medium

Abstract

The application relates to a sample screening method and device, computer equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: obtaining a behavior sample set of a target user; the behavior sample set comprises at least two user behavior samples, and the user behavior sample is used for representing user attention information to materials generated based on user material switching behavior; obtaining a sample generation time of the user behavior sample; the sample generation time is used for indicating a time when a material switching behavior corresponding to the user behavior sample is first located to the materials; screening out implicit negative feedback samples from the at least two user behavior samples based on the sample generation time of the user behavior sample; the implicit negative feedback sample is used for indicating a user behavior sample triggering an exit of a continuous material browsing process. Through the above method, the purpose of extracting effective implicit negative feedback samples from a large number of user behavior samples is achieved, and the diversity and richness of the samples are increased.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of Internet, in particular to a sample screening method and device, computer equipment and storage medium. BACKGROUND

[0002] In the information flow of the recommendation system, a large amount of user behaviors are recorded by the message system, and after transmission, processing, features and sample contents are formed for online learning model training parameters. Through the recommendation strategy and model, the user preferences are mined to provide personalized recommendations for users. The latest behaviors of the user browsing the recommended data feedback form new user features and samples to optimize the model parameters, and the recommendation effect optimization is completed through the repeated iteration of the model.

[0003] Among the user behavior data, there is a kind of negative feedback data. The explicit negative feedback is the negative feedback data formed by the user explicitly clicking on the report, dislike or complaint of the material. However, the user may not know whether he likes it or not, or he does not have the habit of clicking on the negative feedback, or the process of clicking on the negative feedback is complex and troublesome, and he does not have the habit of negative feedback. Therefore, it is necessary to analyze the implicit negative feedback feature information from the user behavior data to make recommendations and filtering. However, how to extract real and effective data from a large amount of implicit negative feedback data is a problem that needs to be solved by the recommendation system. SUMMARY

[0004] Embodiments of the present application provide a sample screening method, device, computer equipment and storage medium, which can extract effective implicit negative feedback samples from a large number of user behavior samples, and increase the diversity and richness of the samples required for extracting user features. The technical solution is as follows:

[0005] On the one hand, a sample screening method is provided, which comprises:

[0006] Obtaining a behavior sample set of a target user; the behavior sample set comprises at least two user behavior samples, and the user behavior sample is used to represent the user's attention information to the material based on the user's material switching behavior;

[0007] Obtaining the sample generation time of the user behavior sample; the sample generation time is used to indicate the time when the user behavior sample corresponding to the material switching behavior is first positioned to the material;

[0008] Screening an implicit negative feedback sample from the at least two user behavior samples based on the sample generation time of the user behavior sample; the implicit negative feedback sample is used to indicate the user behavior sample that triggers the exit of the continuous material browsing process; the time interval between the adjacent two material switching behaviors in the continuous material browsing process is less than a time interval threshold.

[0009] In another aspect, a sample screening device is provided, the device comprising:

[0010] a sample set obtaining module configured to obtain a behavior sample set of a target user, the behavior sample set comprising at least two user behavior samples, the user behavior samples being indicative of attention information of the user to the material based on material switching behavior of the user;

[0011] a generation time obtaining module configured to obtain a sample generation time of the user behavior sample, the sample generation time being indicative of a time when the material corresponding to the user behavior sample is first located by material switching;

[0012] a sample screening module configured to screen an implicit negative feedback sample from the at least two user behavior samples based on the sample generation time of the user behavior sample, the implicit negative feedback sample being indicative of a user behavior sample triggering an exit of continuous material browsing behavior, and a time interval between adjacent two material switching behaviors in the continuous material browsing behavior being less than a time interval threshold.

[0013] In a possible implementation, the generation time obtaining module is configured to, in a case where a plurality of material switching behaviors locate on a same target material, combine user attention information to the material generated by the plurality of material switching behaviors into one user behavior sample, and determine a time when the target material is first located by the plurality of material switching behaviors as the sample generation time of the user behavior sample, the plurality of material switching behaviors being material switching behaviors occurring in a same continuous material browsing process.

[0014] In a possible implementation, the device further comprises:

[0015] an information obtaining module configured to obtain material release times and material index identifiers corresponding to the plurality of material switching behaviors respectively, the material release times being indicative of release batches of the material, and the material index identifiers being indicative of release sequences of the material in the corresponding release batches;

[0016] a determination module configured to, in a case where the material release times and the material index identifiers corresponding to the plurality of material switching behaviors are all same, determine that the plurality of material switching behaviors locate on a same target material.

[0017] In a possible implementation, the sample screening module comprises:

[0018] an aggregation processing submodule configured to perform aggregation processing on the at least two user behavior samples through a session window to obtain at least one cache queue, and a time interval between sample generation times of adjacent two user behavior samples in a same cache queue being less than the time interval threshold.

[0019] The sample filtering submodule is used to filter out at least one implicit negative feedback sample based on the sample generation time of each user behavior sample, with each cache queue as a unit; wherein, one cache queue corresponds to one implicit negative feedback sample.

[0020] In one possible implementation, the sample screening submodule is used for:

[0021] A first tag is added to the first behavior sample in the target cache queue; the first behavior sample is the user behavior sample with the latest sample generation time in the target cache queue, and the first tag is used to indicate that the corresponding user behavior sample is an implicit negative feedback sample; the target cache queue is any one of at least one cache queue;

[0022] A second tag is added to other behavior samples in the target cache queue; the other behavior samples are user behavior samples in the target cache queue other than the first behavior sample, and the second tag is used to indicate that the corresponding user behavior sample is a non-implicit negative feedback sample;

[0023] Output the first behavior sample and other behavior samples of the target cache queue.

[0024] In one possible implementation, the number of samples that the cache queue can hold is N, where N is a positive integer;

[0025] The device further includes:

[0026] The sample addition module is used to add a target behavior sample to the cache queue when the number of user behavior samples in the cache queue is less than N; the target behavior sample is the most recently generated user behavior sample and the time interval between its generation time and that of the previous user behavior sample is less than the time interval threshold.

[0027] The sample adding module is used to add a second tag to the second behavior sample in the cache queue when the number of user behavior samples in the cache queue is greater than or equal to N, and output the second behavior sample; after outputting the second behavior sample, add the target behavior sample to the cache queue after the second behavior sample has been output; the second behavior sample is the user behavior sample with the earliest sample generation time in the cache queue.

[0028] In one possible implementation, the sample set acquisition module is used for:

[0029] Obtain source data; the source data contains user behavior samples of each user;

[0030] According to the user identifier corresponding to the user behavior sample in the source data, each user behavior sample is user-partitioned to obtain a set of user behavior samples of each user.

[0031] In another aspect, a computer device is provided, which includes a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the sample screening method described above.

[0032] In another aspect, a computer readable storage medium is provided, which stores at least one computer program, the computer program being loaded and executed by a processor to implement the sample screening method described above.

[0033] In another aspect, a computer program product is provided, which includes at least one computer program, the computer program being loaded and executed by a processor to implement the sample screening method provided in various optional implementation manners described above.

[0034] The technical solutions provided in the present application can include the following beneficial effects:

[0035] The sample screening method provided in the embodiments of the present application determines a continuous browsing process of a user based on the time interval between material switching behaviors, and determines an implicit negative feedback sample triggering an exit from the continuous material browsing process from a plurality of user behavior samples generated in the continuous browsing process based on the sample generation time of each user behavior sample, thereby achieving the purpose of extracting effective implicit negative feedback samples from a large number of user behavior samples, and increasing the diversity and richness of the samples required for extracting user features.

[0036] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0037] The accompanying drawings incorporated in the specification and forming a part of it, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0038] Figure 1 A flowchart of a sample screening method provided by an exemplary embodiment of the present application is shown;

[0039] Figure 2 A flowchart of a sample screening method provided by an exemplary embodiment of the present application is shown;

[0040] Figure 3 A schematic diagram of a sample screening process provided by an exemplary embodiment of the present application is shown;

[0041] Figure 4 Fig. 1 shows a schematic diagram of a sample screening process based on a Flink framework according to an example embodiment of the present application;

[0042] Figure 5 Fig. 2 shows a block diagram of a sample screening device according to an example embodiment of the present application;

[0043] Figure 6 Fig. 3 shows a structural block diagram of a computer device according to an example embodiment of the present application;

[0044] Figure 7 Fig. 4 shows a structural block diagram of a computer device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0045] The example embodiments will be described in detail herein with reference to the drawings. When the following description refers to accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0046] In the embodiments of the present application, whether the user continues to browse after browsing a material is regarded as an implicit feedback of the user, and the material that is not continued to browse after exceeding the allowed time interval is determined as a material that interrupts the continuous material browsing process. Based on this, the embodiments of the present application provide a scheme for obtaining user behavior samples generated based on the user's switching behavior of each material in the continuous material browsing process, and screening implicit negative feedback samples from the user behavior samples, so that when extracting user features, the user's preferences can be mined in combination with the implicit negative feedback samples, and then more personalized recommendations that meet the user's needs can be provided to the user, and the material recommendation effect is improved.

[0047] Figure 1 Fig. 1 shows a flowchart of a sample screening method according to an example embodiment of the present application. The sample screening method can be executed by a computer device, which can be implemented as a server or a terminal, as shown in Fig. 1. The sample screening method can include the following steps: Figure 1

[0048] In step 110, a behavior sample set of a target user is obtained. The behavior sample set includes at least two user behavior samples, and the user behavior sample is used to represent the user's attention information to the material based on the user's switching behavior of the material.

[0049] ​The attention information of the user to the material can be reflected in a material located based on the material switching behavior of the user, a stay time at the material, a type of the material, and the like. Therefore, the user behavior sample can include a user identifier, a refresh frequency identifier, and a ranking of the material. The refresh frequency identifier and the ranking of the material can be used to determine the material located by the material switching behavior.

[0050] In step 120, a sample generation time of the user behavior sample is obtained. The sample generation time is used to indicate a time when the material switching behavior corresponding to the user behavior sample first locates a material.

[0051] That is, in the embodiment of the present application, the attention information of the user to the material generated by the material switching behavior locating the same material in the same continuous browsing process is integrated into one user behavior sample. In a scenario where the same material is browsed multiple times, the time when the material is located for the first time is determined as the sample generation time of the corresponding user behavior sample.

[0052] Illustratively, in the material browsing scenario of video browsing, the sample generation time of one user behavior sample is the time when the user switches to a new video. When the user repeatedly watches the same video in the same continuous video browsing process, the sample generation time of the user behavior sample corresponding to the video is still the time when the user first switches to the video.

[0053] In step 130, an implicit negative feedback sample is screened from at least two user behavior samples based on the sample generation time of the user behavior sample. The implicit negative feedback sample is used to indicate a user behavior sample triggering an exit of a continuous material browsing process. A time interval between two adjacent material switching behaviors in the continuous material browsing process is less than a time interval threshold.

[0054] In the embodiment of the present application, a material browsing process implemented based on continuous material switching behaviors with a time interval less than a time interval threshold between each other is determined as one continuous material browsing process. A user behavior sample interrupting the continuous material browsing process is determined as an implicit negative feedback sample. The sample generation time of the implicit negative feedback sample can be the latest sample generation time among the user behavior samples included in the continuous material browsing process.

[0055] The implicit negative feedback sample obtained based on the sample screening method provided in the present application is used as a basis for sample classification. Therefore, when the sample classification is performed, negative samples can be screened based on the explicit negative feedback, the complete play rate threshold, and the implicit negative feedback, and are used as input content of the user feature extraction model. Therefore, the user feature extraction model can extract more user features, thereby enriching the user portrait. When the material is recommended, more personalized material recommendation can be provided for the user.

[0056] In summary, the sample screening method provided in this application determines a user's continuous browsing process based on the time interval between material switching behaviors, and determines the implicit negative feedback samples that trigger the exit from the continuous material browsing process from multiple user behavior samples generated during the continuous browsing process based on the sample generation time of each user behavior sample. This achieves the purpose of extracting effective implicit negative feedback samples from massive user behavior samples, thereby increasing the diversity and richness of the samples required for extracting user features.

[0057] Figure 2 This application illustrates a flowchart of a sample screening method provided in an exemplary embodiment. This sample screening method can be executed by a computer device, which can be implemented as a server or a terminal, such as... Figure 2 As shown, this sample screening method may include the following steps:

[0058] Step 210: Obtain the target user's behavior sample set; the behavior sample set contains at least two user behavior samples, which are used to characterize the user's attention information to materials based on the user's material switching behavior.

[0059] In one possible implementation, the process of obtaining a sample set of target user behaviors can be implemented as follows:

[0060] Obtain the source data; this source data contains user behavior samples for each user.

[0061] Based on the user identifiers corresponding to the user behavior samples in the source data, each user behavior sample is partitioned to obtain the behavior sample set of each user.

[0062] From the obtained behavior sample sets of each user, obtain the behavior sample set of the target user, which is any one of the users.

[0063] In this embodiment, the computer device can implement the data processing process in the sample screening method provided in this application based on the Flink framework. In large-scale real-time data scenarios, the Flink framework has the characteristics of high throughput and low latency, and supports stream-batch processing and state computation, ensuring data processing consistency, high reliability and high fault tolerance. It also supports a variety of window computation methods and can be used with various operation operators to perform data processing.

[0064] Different DataStreams can be connected upstream and downstream by each operation operator in the Flink framework, and common operation operators include: Map operator, Filter operator, KeyBy operator, Aggregate operator, and Process operator, etc. Among them, the Map operator is used for data cleaning, which can convert the input stream data into the required format data output; the Filter operator is used for filtering and screening, and outputs all result sets that meet the judgment condition; the KeyBy operator is used for aggregation operation, which can group events according to a certain attribute or data field as Key, and elements with the same Key are grouped together for subsequent operator grouping processing.

[0065] In the embodiment of the present application, the computer device can read the Kafka message queue of the user sample data to obtain the source data.

[0066] Before partitioning each user behavior sample based on the user identifier corresponding to each user behavior sample in the source data, the computer device can also perform data preprocessing on the source data.

[0067] The process of performing data preprocessing on the source data can include:

[0068] S2101, using the Map operator to extract the user identifier field from each user behavior sample of the source data, and outputting a two-dimensional Tuple array.

[0069] Optionally, in some business scenarios, the material identifier field can be spliced on the user identifier field for data processing. Some business scenarios can be business scenarios based on diversion material entering continuous material browsing; illustratively, in a video post-recommendation business scenario, the material identifier field of the diversion material can be spliced on the user identifier field, wherein the video post-recommendation business scenario can refer to a scenario in which a user switches to a new video for video browsing through a down-swiping operation based on a diversion video, in which case, the user identifier field and the material identifier field of the diversion material can be extracted from the user behavior sample corresponding to the diversion video and the user behavior sample corresponding to the video switched by the down-swiping operation based on the diversion video for splicing, thereby facilitating subsequent user partitioning operation. In this case, since the material identifiers of the diversion materials are different, the same user can correspond to multiple behavior sample sets, and the user behavior samples in the same behavior sample set are generated in the material browsing process entered by the same diversion material.

[0070] S2102, using the Filter operator to filter illegal format data and abnormal fields.

[0071] In the embodiment of the present application, the Filter operator can filter at least one missing data in the fields of the user identifier, the refresh frequency identifier, the material identifier, and the like, to reduce invalid samples.

[0072] In S2103, the processed source data is subjected to user partitioning by using a KeyBy operator to obtain a behavior sample set of each user.

[0073] The KeyBy operator can partition all elements by using a Hash hash value as a key, hash elements with the same key into one subtask, and isolate processing of different partitions without changing the original element data structure; in the embodiment of the present application, the KeyBy operator uses the user identifier as the key to perform user partitioning on each user behavior sample in the source data to obtain the behavior sample set of each user; in some business scenarios, the KeyBy operator can also use a splicing identifier as the key, the splicing identifier is a splicing result of the user identifier and the material identifier, and perform user partitioning on each user behavior sample in the source data. The embodiment of the present application uses the user identifier as the key to perform user grouping to describe the sample screening method provided in the present application.

[0074] In S220, a sample generation time of the user behavior sample is obtained; the sample generation time is used to indicate a time when the user behavior sample corresponds to the material switching behavior first locates to the material.

[0075] During the material browsing process, the browsing manner of the user is uncertain, for example, the user can repeatedly watch the same material, or the user returns to the browsed material to browse after brushing and browsing multiple materials, or the user browses each material in turn until exiting, and the like; in the first two browsing manners, there can be a case that the same material corresponds to multiple material switching behaviors, in which case, the user behavior sample and the sample generation time of the user behavior sample can be determined by the following manners:

[0076] In the case that multiple material switching behaviors locate to the same target material, the attention information of the user to the material generated by the multiple material switching behaviors is combined into one user behavior sample, and the time when the multiple material switching behaviors first locate to the target material is determined as the sample generation time of the user behavior sample; the multiple material switching behaviors are material switching behaviors occurring in the same continuous material browsing process.

[0077] Illustratively, if the user returns to browse the material 2 after browsing the materials with the material identifiers 1-4, and then exits after re-browsing the material 2, at this time, the user returns to re-browse the material 2 because the user is interested in the material 2, and therefore, the user behavior sample corresponding to the material 2 cannot be determined as the user behavior sample triggering the exit of the continuous material browsing process, and the user behavior sample corresponding to the material 4 is determined as the user behavior sample triggering the exit of the continuous browsing process. Therefore, in the embodiment of the present application, whether the user behavior sample triggers the exit of the continuous browsing process is determined according to the sample generation time, in the above example, the attention information of the user to the material 2 generated in the two browsing processes of the material 2 can be combined into one user behavior sample, such as combining the stay time on the material 2, and the time of the first browsing of the material 2 is determined as the sample generation time of the user behavior sample, and then, the sample generation time of the user behavior sample of the material 4 is the latest, and therefore, the user behavior sample corresponding to the material 4 is determined as the user behavior sample triggering the exit of the continuous browsing process.

[0078] Optionally, whether the multiple material switching behaviors are positioned on the same target material can be determined in the following manner:

[0079] The material issuing time and the material index identifier corresponding to each of the multiple material switching behaviors are acquired, the material issuing time is used to indicate the issuing batch of the material, and the material index identifier is used to indicate the issuing sequence of the material in the corresponding issuing batch;

[0080] In a case where the material issuing time and the material index identifier corresponding to each of the multiple material switching behaviors are all the same, it is determined that the multiple material switching behaviors are positioned on the same target material.

[0081] Illustratively, if 10 materials are issued based on one refresh operation of the user, the 10 materials have the same material issuing time, which is used to indicate that the 10 samples belong to the same issuing batch, but the index identifiers of the 10 materials are 1-10, which are used to indicate the issuing sequence of the 10 samples in the issuing batch; if the material issuing time and the material index identifier corresponding to each of the multiple material switching behaviors indicate the second material issued based on the first refresh behavior of the user, it is indicated that the multiple material switching behaviors are positioned on the same material.

[0082] In step 230, at least two user behavior samples are aggregated through the session window to obtain at least one cache queue, and the time interval between the sample generation times of two adjacent user behavior samples in the same cache queue is less than the time interval threshold.

[0083] The window is divided into a TimeWindow generated according to time and a CountWindow generated according to count, wherein the TimeWindow includes a tumbling window, a sliding window and a session window. Since the user behavior sample is generated based on the material switching behavior of the user, and the material switching behavior has uncertainty, the arrival of the user behavior sample also has uncertainty. Therefore, in the embodiment of the present application, the session window is used for window aggregation, so that the user behavior samples generated by a series of continuous material switching operations of the user can be aggregated in one window. Once the waiting time exceeds the allowed time interval and no new user behavior sample is received, it is determined that the current session window is closed, a continuous browsing process is ended, and the user behavior sample in the cache queue in the session window is output to the downstream processing.

[0084] Optionally, the computer device can implement aggregation processing of at least two user behavior samples in the session window through an Aggregate operator. The Aggregate operator can add the user behavior samples generated by the same user in a continuous period of time to the cache queue corresponding to the session window according to the order of sample generation time, and maintain the cache queue.

[0085] Optionally, the number of samples that can be accommodated by one cache queue is N, and N is a positive integer.

[0086] The maintenance process of the cache queue can be implemented as follows:

[0087] In a case where the number of user behavior samples in the cache queue is less than N, a target behavior sample is added to the cache queue. The target behavior sample is a user behavior sample that is newly generated and has a time interval between the sample generation time of the target behavior sample and the sample generation time of a previous user behavior sample less than a time interval threshold.

[0088] In a case where the number of user behavior samples in the cache queue is greater than or equal to N, a second behavior sample in the cache queue is added with a second mark, and the second behavior sample is output. After the second behavior sample is output, a target behavior sample is added to the cache queue after the second behavior sample. The second behavior sample is a user behavior sample with the earliest sample generation time in the cache queue.

[0089] That is, for the user behavior samples in the same session window, the user behavior samples are added to the cache queue corresponding to the session window in the order of sample generation time, and the maximum number of user behavior samples that the cache queue can accommodate is N; when a new user behavior sample arrives, if the cache queue is not yet full, the newly arrived user behavior sample is added to the cache queue; if the cache queue is full, when a new user behavior sample arrives, it can be determined that the user behavior sample first stored in the current cache queue is not a sample that triggers the exit of the continuous browsing process, and therefore, the user behavior sample ranked first in the cache queue can be output after being added with a second mark, and the newly arrived user behavior sample is added to the cache queue; the second mark is used to indicate that the user behavior sample is a non-explicit negative feedback sample. Through the above method, the data caching pressure of the cache queue is reduced, and the non-explicit negative feedback sample can be screened out in advance, the sample screening process is realized in stages, and the result output pressure is reduced.

[0090] In step 240, at least one implicit negative feedback sample is screened out based on the sample generation time of each user behavior sample in a unit of cache queue; one cache queue corresponds to one implicit negative feedback sample.

[0091] In the embodiment of the present application, when the session window corresponding to the cache queue is closed, one implicit negative feedback sample is screened out based on the sample generation time of each user behavior sample in the cache queue.

[0092] Since the user behavior samples in the session window arrive in the order of sample generation time, when a session window is closed, the cache queue corresponding to the session window is output to the downstream and the implicit negative feedback sample is screened out therefrom, and therefore, the process of screening out the corresponding implicit negative feedback sample from each cache queue is performed in the order of session window closure.

[0093] Taking the process of screening out the implicit negative feedback sample from one cache queue as an example, in a possible implementation manner, the process of screening out the implicit negative feedback sample can be implemented as follows:

[0094] adding a first mark to the first behavior sample in the target cache queue; the first behavior sample is the user behavior sample with the latest sample generation time in the target cache queue, and the first mark is used to indicate that the corresponding user behavior sample is an implicit negative feedback sample; the target cache queue is any one of the at least one cache queue;

[0095] adding a second mark to the other behavior samples in the target cache queue; the other behavior samples are the user behavior samples in the target cache queue except the first behavior sample, and the second mark is used to indicate that the corresponding user behavior sample is a non-explicit negative feedback sample;

[0096] The first behavior sample of the output target cache queue and other behavior samples.

[0097] Optionally, the above process can be implemented by a Process operator, which outputs all user behavior samples in the cache queue after adding corresponding labels to each user behavior sample in the cache queue; wherein, in the output user behavior samples, the user behavior sample added last to the cache queue is added with a first label, and other user behavior samples are added with a second label; the user behavior sample added with the first label is the screened implicit negative feedback sample.

[0098] Illustratively, each user behavior sample can be configured with a no-scroll field, which is marked as "no" when the second label is added, indicating that the user behavior sample is not a user behavior sample that interrupts continuous browsing behavior; correspondingly, when the first label is added, the no-scroll field is marked as "yes", indicating that the user behavior sample is a user behavior sample that terminates continuous browsing behavior.

[0099] In a possible implementation, when the cache queue corresponding to the session window is closed, the cache queue contains one user behavior sample, that is, the user browses one material in a continuous browsing process and then exits the browsing, in this case, the computer device can determine the user behavior sample as the implicit negative feedback sample in the cache queue, and output after adding the first label to the one user behavior sample contained in the cache queue.

[0100] In the sample screening method provided by the embodiments of the present application, the Aggregate operator and the Process operator can be used for sample screening, Figure 3 A schematic diagram of a sample screening process provided by an example embodiment of the present application is shown, as Figure 3 As shown, for the behavior sample set of the target user, the Aggregate operator 310 outputs the user behavior sample added first to the cache queue after adding a second label to the user behavior sample when the number of user behavior samples in the cache queue is greater than N when maintaining the cache queue in the session window, that is, outputs a single user behavior sample; after the session window is closed, the Aggregate operator 310 outputs the cache queue maintained to the downstream, and the Process operator 320 outputs all user behavior samples in the cache queue after adding corresponding labels to the received user behavior samples in the cache queue, and the sample screening result is obtained by combining the output contents of the two operators.

[0101] In summary, the sample screening method provided in this application determines a user's continuous browsing process based on the time interval between material switching behaviors, and identifies implicit negative feedback samples that trigger the exit from the continuous material browsing process from multiple user behavior samples generated during the continuous browsing process, based on the sample generation time of each user behavior sample. This achieves the goal of extracting effective implicit negative feedback samples from massive user behavior samples, thereby increasing the diversity and richness of the samples required for extracting user features. At the same time, by outputting samples in stages during the sample screening process, the data processing pressure can be alleviated and the data processing efficiency can be improved.

[0102] To ensure the stable and secure operation of the sample screening process, in this embodiment of the application, the computer device can also monitor the running status of each operator during the sample screening process based on the Metric component contained in the Flink real-time computing framework. Based on the Metric component, the data throughput, data latency, processing time of each operator, running status of each operation operator, etc. can be read and the read information can be displayed in a visual monitoring and alarm platform.

[0103] Optionally, the Metric component may include a Metric Group and a Metric Repotter. During monitoring, the computer device can acquire metric data through the Metric Group. This metric data includes monitoring data and metrics populated by basic arithmetic operations. The metrics populated by basic arithmetic operations include various counters (accumulators), time accumulation, counting the number of samples in the user session window, accumulating the count of delayed samples, accumulating the count of samples output ahead of schedule, and accumulating the time of each step, etc. The acquired metric data is uploaded using the Metric Repotter, with a time series database (InfluxDB) as the metric data storage container, and the metric data is visualized using a visualization monitoring tool (such as Grafana).

[0104] It should be noted that the above-mentioned indicator data is only illustrative. Relevant personnel can add new indicator data based on actual needs, such as sample throughput, sample count at each stage, and processing time of each operation operator, such as the processing time of the Filter operator and the time spent determining the actual read in the Aggregate operator; this application does not impose any restrictions on this.

[0105] Optionally, the Metric component can also issue anomaly alerts when abnormal data is detected, so as to remind relevant personnel to investigate the anomalies and ensure the normal operation of the sample screening process.

[0106] By monitoring the running state of each operator in the sample screening process and displaying index data in a visual manner, the sample screening process can be controlled and the stable progress of the sample screening process can be ensured.

[0107] Based on Figure 3 the embodiment shown in the figure, Figure 4 A schematic diagram of a sample screening process based on the Flink framework is shown, and the sample screening process can include: Figure 4

[0108] S401, obtaining source data.

[0109] The source data contains user behavior samples of each user; each user behavior sample has a corresponding user identifier.

[0110] S402, using a Map operator to perform format conversion on the source data.

[0111] S403, using a Filter operator to filter illegal format data and abnormal fields.

[0112] S404, using a KeyBy operator to partition the processed source data by user to obtain a set of user behavior samples for each user.

[0113] S405, using an Aggregate operator to implement aggregation processing of at least two user behavior samples of a target user.

[0114] In this process, the Aggregate operator aggregates user behavior samples belonging to one continuous browsing process in at least two behavior samples of the target user into one session window, and adds them to the cache queue corresponding to the session window according to the sample generation time.

[0115] S406, determining whether to output the user behavior sample in advance through the Aggregate operator; if yes, performing S407, and if no, performing S408.

[0116] The Aggregate operator determines whether to output the user behavior sample in advance based on the filling state of the cache queue. When the cache queue is not yet filled, it is determined that the user behavior sample will not be output in advance; when the cache queue is filled, it is determined that the user behavior sample will be output in advance.

[0117] S407, adding a second mark to the user behavior sample added to the cache queue earliest and outputting it.

[0118] The second mark is used to indicate that the user behavior sample is a non-explicit negative feedback sample.

[0119] ​S408, when the session window is closed, adding corresponding labels to the user behavior samples in the cache queue through the Process operator, and outputting all the user behavior samples in the cache queue.

[0120] When the waiting time exceeds the allowed time interval without receiving a new user behavior sample, it is determined that the current session window is closed; the Process operator adds a first label to the last user behavior sample added in the cache queue and a second label to other user behavior samples in the cache queue; the first label is used to indicate that the user behavior sample is an implicit negative feedback sample.

[0121] S409, obtaining a sample screening result.

[0122] The embodiments of the present application are based on the Flink real-time distributed stream processing computing framework, and can guarantee the real-time performance of data processing based on the characteristics of the Flink framework, ensure high throughput and high fault tolerance on the basis of low latency, and greatly improve the problems of big data processing technology.

[0123] Figure 5 A block diagram of a sample screening device provided by an example embodiment of the present application is shown, which can be used to implement all or part of the steps of the sample screening method as shown in Figure 1 or Figure 2 The sample screening device includes: Figure 5

[0124] The sample set acquisition module 510 is configured to acquire a behavior sample set of a target user; the behavior sample set includes at least two user behavior samples, and the user behavior samples are used to indicate attention information to materials based on material switching behaviors of the user;

[0125] The sample generation time acquisition module 520 is configured to acquire a sample generation time of the user behavior sample; the sample generation time is used to indicate a time when a material corresponding to the user behavior sample is first located based on material switching;

[0126] The sample screening module 530 is configured to screen an implicit negative feedback sample from the at least two user behavior samples based on the sample generation time of the user behavior sample; the implicit negative feedback sample is used to indicate a user behavior sample triggering an exit of a continuous material browsing behavior; a time interval between two adjacent material switching behaviors under the continuous material browsing behavior is less than a time interval threshold.

[0127] ​In a possible implementation, the generation time acquisition module 520 is configured to, in a case where the multiple material switching behaviors are located on a same target material, combine user attention information on materials generated by the multiple material switching behaviors into one user behavior sample, and determine a time when the multiple material switching behaviors are first located on the target material as a sample generation time of the user behavior sample; the multiple material switching behaviors are material switching behaviors occurring in a same continuous material browsing process.

[0128] In a possible implementation, the apparatus further includes:

[0129] The information acquisition module is configured to acquire material issuing times and material index identifiers corresponding to the multiple material switching behaviors respectively; the material issuing time is used to indicate an issuing batch of a material, and the material index identifier is used to indicate an issuing sequence of the material in the corresponding issuing batch;

[0130] The determination module is configured to determine that the multiple material switching behaviors are located on a same target material in a case where the material issuing times and the material index identifiers corresponding to the multiple material switching behaviors are all same.

[0131] In a possible implementation, the sample screening module 530 includes:

[0132] The aggregation processing submodule is configured to perform aggregation processing on at least two user behavior samples through a session window to obtain at least one cache queue; in a same cache queue, a time interval between sample generation times of two adjacent user behavior samples is less than the time interval threshold.

[0133] The sample screening submodule is configured to screen at least one implicit negative feedback sample based on the sample generation times of the user behavior samples in a cache queue unit; one cache queue corresponds to one implicit negative feedback sample.

[0134] In a possible implementation, the sample screening submodule is configured to,

[0135] add a first mark to a first behavior sample in a target cache queue; the first behavior sample is a user behavior sample with a latest sample generation time in the target cache queue, and the first mark is used to indicate that the corresponding user behavior sample is an implicit negative feedback sample; the target cache queue is any one of the at least one cache queue;

[0136] add a second mark to other behavior samples in the target cache queue; the other behavior samples are user behavior samples other than the first behavior sample in the target cache queue, and the second mark is used to indicate that the corresponding user behavior sample is a non-implicit negative feedback sample;

[0137] output the first behavior sample of the target cache queue and other behavior samples.

[0138] In a possible implementation, the cache queue can accommodate N samples, where N is a positive integer.

[0139] The apparatus further includes:

[0140] The sample adding module is configured to add a target behavior sample to the cache queue when the number of user behavior samples in the cache queue is less than N, the target behavior sample being a user behavior sample that is newly generated and has a time interval between a sample generation time of the target behavior sample and a sample generation time of a previous user behavior sample less than the time interval threshold.

[0141] The sample adding module is further configured to add a second mark to a second behavior sample in the cache queue and output the second behavior sample when the number of user behavior samples in the cache queue is greater than or equal to N, and add a target behavior sample to the cache queue after the second behavior sample is output, the second behavior sample being a user behavior sample with the earliest sample generation time in the cache queue.

[0142] In a possible implementation, the sample set obtaining module 510 is configured to:

[0143] obtain source data, the source data including user behavior samples of each user;

[0144] perform user partitioning on each user behavior sample based on a user identifier corresponding to the user behavior sample in the source data, to obtain a behavior sample set of each user.

[0145] In summary, the sample screening apparatus provided by the embodiments of the present application determines a continuous browsing process of a user based on a time interval between material switching behaviors, and determines an implicit negative feedback sample that triggers an exit from the continuous material browsing process from a plurality of user behavior samples generated in the continuous browsing process based on the sample generation time of each user behavior sample, thereby achieving the purpose of extracting effective implicit negative feedback samples from a large number of user behavior samples, and increasing the diversity and richness of samples required for extracting user features.

[0146] Figure 6A structural block diagram of a computer device 600 is shown according to an example embodiment of the present application. The computer device can be implemented as a server in the above-mentioned solutions of the present application. The computer device 600 includes a central processing unit (CPU) 601, a system memory 604 including a random access memory (RAM) 602 and a read-only memory (ROM) 603, and a system bus 605 connecting the system memory 604 and the central processing unit 601. The computer device 600 also includes a mass storage device 606 for storing an operating system 609, application programs 610, and other program modules 611.

[0147] Without loss of generality, the computer readable medium can include computer storage medium and communication medium. The computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The computer storage medium includes RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) flash memory or other solid state storage technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. Of course, those skilled in the art can know that the computer storage medium is not limited to the above. The system memory 604 and the mass storage device 606 can be collectively referred to as memory.

[0148] According to various embodiments of the present application, the computer device 600 can also be connected to a remote computer operating on a network through a network such as the Internet. That is, the computer device 600 can be connected to a network 608 through a network interface unit 607 connected to the system bus 605, or can be connected to other types of networks or remote computer systems (not shown) using the network interface unit 607.

[0149] The memory further includes at least one instruction, at least one program, a code set or an instruction set stored in the memory, and the central processing unit 601 implements all or part of the steps in the sample screening method shown in each of the above embodiments by executing the at least one instruction, at least one program, code set or instruction set.

[0150] Figure 7 A structural block diagram of a computer device 700 according to an example embodiment of the present application is shown. The computer device 700 can be implemented as the terminal described above, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart watch, a television, and the like. The computer device 700 can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0151] Generally, the computer device 700 includes a processor 701 and a memory 702.

[0152] In some embodiments, the computer device 700 can also optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 703 through a bus, a signal line or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707 and a power supply 708.

[0153] In some embodiments, the computer device 700 further includes one or more sensors 709. The one or more sensors 709 include, but are not limited to, an acceleration sensor 710, a gyroscope sensor 711, a pressure sensor 712, an optical sensor 713 and a proximity sensor 714.

[0154] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the computer device 700, and can include more or fewer components than shown, or combine certain components, or use different component arrangements. Figure 7 The structure shown in the above embodiments does not constitute a limitation on the computer device 700, and can include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0155] In an exemplary embodiment, a computer readable storage medium is also provided, in which at least one computer program is stored, the computer program being loaded and executed by a processor to implement all or part of the steps of the sample screening method described above. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0156] In an exemplary embodiment, a computer program product is also provided, which includes at least one computer program, the computer program being loaded and executed by a processor to implement all or part of the steps of the sample screening method described above. Figure 1 or Figure 2 all or part of the steps of the sample screening method shown in any of the embodiments.

[0157] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application encompass any and all variations or modifications of the application herein described and falling within the scope of the general inventive concept. It is to be understood that the application is not to be limited to the specific examples, methods, and procedures described herein, and that specific examples are to be considered as illustrative only. The true scope and spirit of the application is indicated by the following claims.

[0158] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is indicated by the appended claims, rather than the description.

Claims

1. A sample screening method, characterized in that, The method includes: Obtain a set of behavioral samples of the target user; the set of behavioral samples contains at least two user behavioral samples, which are used to characterize the user’s attention to materials based on the user’s material switching behavior. The sample generation time of the user behavior sample is obtained; the sample generation time is used to indicate the time when the material switching behavior corresponding to the user behavior sample is first located to the material; Based on the sample generation time of the user behavior samples, implicit negative feedback samples are selected from the at least two user behavior samples; the implicit negative feedback samples are used to indicate user behavior samples that trigger the exit from the continuous material browsing process; the time interval between two adjacent material switching behaviors in the continuous material browsing process is less than a time interval threshold; the sample generation time of the implicit negative feedback samples is the latest among the user behavior samples included in the continuous material browsing process.

2. The method according to claim 1, characterized in that, The sample generation time for obtaining user behavior samples includes: When multiple material switching behaviors locate the same target material, the user's attention information on the material generated by the multiple material switching behaviors is merged into a single user behavior sample, and the time when the target material is first located in the multiple material switching behaviors is determined as the sample generation time of the user behavior sample; the multiple material switching behaviors are material switching behaviors that occur during the same continuous material browsing process.

3. The method according to claim 2, characterized in that, The method further includes: Obtain the material issuance time and material index identifier corresponding to each of the multiple material switching behaviors; the material issuance time is used to indicate the issuance batch of the material, and the material index identifier is used to indicate the issuance order of the material in the corresponding issuance batch; If the material issuance time and material index identifier corresponding to the multiple material switching behaviors are the same, it is determined that the multiple material switching behaviors are located on the same target material.

4. The method according to any one of claims 1 to 3, characterized in that, The sample generation time based on user behavior samples, and the selection of implicit negative feedback samples from the at least two user behavior samples, include: At least two user behavior samples are aggregated through a session window to obtain at least one cache queue; in the same cache queue, the time interval between the sample generation times of two adjacent user behavior samples is less than the time interval threshold. Using cache queues as units, at least one implicit negative feedback sample is selected based on the sample generation time of each user behavior sample; where one cache queue corresponds to one implicit negative feedback sample.

5. The method according to claim 4, characterized in that, The process of selecting at least one implicit negative feedback sample based on the sample generation time of each user behavior sample, using a cache queue as a unit, includes: A first tag is added to the first behavior sample in the target cache queue; the first behavior sample is the user behavior sample with the latest sample generation time in the target cache queue, and the first tag is used to indicate that the corresponding user behavior sample is an implicit negative feedback sample; the target cache queue is any one of at least one cache queue; A second tag is added to other behavior samples in the target cache queue; the other behavior samples are user behavior samples in the target cache queue other than the first behavior sample, and the second tag is used to indicate that the corresponding user behavior sample is a non-implicit negative feedback sample; Output the first behavior sample and other behavior samples of the target cache queue.

6. The method according to claim 5, characterized in that, The number of samples that the cache queue can hold is N, where N is a positive integer; Before filtering out at least one implicit negative feedback sample based on the sample generation time of each user behavior sample, using the cache queue as a unit, the method further includes: If the number of user behavior samples in the cache queue is less than N, the target behavior sample is added to the cache queue; the target behavior sample is the most recently generated user behavior sample whose time interval between the sample generation time of the previous user behavior sample is less than the time interval threshold. If the number of user behavior samples in the cache queue is greater than or equal to N, a second tag is added to the second behavior sample in the cache queue, and the second behavior sample is output; after the second behavior sample is output, the target behavior sample is added to the cache queue after the second behavior sample has been output; the second behavior sample is the user behavior sample with the earliest sample generation time in the cache queue.

7. The method according to claim 1, characterized in that, The acquisition of the target user's behavior sample set includes: Obtain source data; the source data contains user behavior samples of each user; Based on the user identifiers corresponding to the user behavior samples in the source data, each user behavior sample is partitioned to obtain the behavior sample set of each user.

8. A sample screening device, characterized in that, The device includes: The sample set acquisition module is used to acquire a behavioral sample set of the target user; the behavioral sample set contains at least two user behavior samples, which are used to indicate the attention information on materials generated based on the user's material switching behavior. The generation time acquisition module is used to acquire the sample generation time of user behavior samples; the sample generation time is used to indicate the time when the material is first located based on the material switching corresponding to the user behavior sample; The sample filtering module is used to filter implicit negative feedback samples from at least two user behavior samples based on the sample generation time of the user behavior samples; the implicit negative feedback samples are used to indicate user behavior samples that trigger the exit from continuous material browsing behavior; the time interval between two adjacent material switching behaviors under the continuous material browsing behavior is less than a time interval threshold; the sample generation time of the implicit negative feedback sample is the latest among the user behavior samples included in the continuous material browsing process.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the sample screening method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the sample screening method as described in any one of claims 1 to 7.

Images