A user representation method and system

Abstract

Embodiments of the present specification provide a user representation method and system, the method comprising: obtaining a time sequence comprising a plurality of time blocks, each of the plurality of time blocks comprising behavior data of a user in a time period; obtaining, based on the time sequence, feature representations of the plurality of time blocks by a feature extraction network; and obtaining, based on the feature representations of the plurality of time blocks, a vector representation of the user by an encoding network.

Description

Technical Field

[0001] This specification relates to the field of computers, and in particular to a user representation method and system. Background Technology

[0002] In today's society, with the development of information technology, more and more user services are provided to users through the internet and other online platforms. The various behaviors users exhibit on these platforms reflect their habits and preferences to a certain extent. To better provide personalized services that better align with user habits and individual needs, it is necessary to analyze user behavior data to obtain user characteristics. However, limited scenario and short-term user data are insufficient to comprehensively and accurately reflect user behavior habits. A comprehensive user profile requires integrating user behavior data from as many scenarios as possible over a long period. This involves massive amounts of data, demanding high computational power (sometimes even impossible), and complex algorithms, making implementation very difficult and ensuring consistent results.

[0003] Therefore, it is desirable to provide a user representation method and system. Summary of the Invention

[0004] One embodiment of this specification provides a user representation model training method. The user representation model training method includes: acquiring a time series comprising multiple time blocks, each time block including user behavior data within a time period; obtaining feature representations of the multiple time blocks through a feature extraction network; determining masked time blocks among the multiple time blocks; obtaining an encoded representation of the masked time blocks and an overall vector representation of the time series through an encoding network based on the feature representations of the remaining time blocks in the time series excluding the masked time blocks; adjusting the parameters of the encoding network based on the similarity between the overall vector representation of the current time series and the overall vector representations of the preceding or following time series, and the similarity between the encoded representation of the masked time blocks and the corresponding feature representations of the masked time blocks.

[0005] One embodiment of this specification provides a user representation method. The user representation method includes: acquiring a time series comprising multiple time blocks, each time block including user behavior data within a time period; obtaining feature representations of the multiple time blocks based on the time series using a feature extraction network; and obtaining a vector representation of the user based on the feature representations of the multiple time blocks using an encoding network.

[0006] In some embodiments, the user representation method includes training a user representation model according to the user standard model training method described in some embodiments of this application, wherein the user representation model includes the feature extraction network and the encoding network.

[0007] One embodiment of this specification provides a user representation system, including an acquisition module, a first feature extraction module, and a user representation module; the acquisition module is used to acquire a time series including multiple time blocks, each of the multiple time blocks including user behavior data within a time period; the first feature extraction module is used to obtain feature representations of the multiple time blocks based on the time series through a feature extraction network; the user representation module is used to obtain a vector representation of the user based on the feature representations of the multiple time blocks through an encoding network.

[0008] One embodiment of this specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes the user representation method. Attached Figure Description

[0009] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0010] Figure 1 These are schematic diagrams illustrating application scenarios of the user representation system according to some embodiments of this specification;

[0011] Figure 2 These are schematic diagrams of user representation systems according to some embodiments of this specification;

[0012] Figure 3 This is an exemplary flowchart of a user representation method according to some embodiments of this specification;

[0013] Figure 4 This is an exemplary flowchart of a user representation method according to some embodiments of this specification;

[0014] Figure 5 This is an exemplary flowchart of a user representation model training method according to some embodiments of this specification;

[0015] Figure 6 This is an exemplary flowchart of a user representation model training method according to some embodiments of this specification;

[0016] Figure 7A , Figure 7B This is a schematic diagram illustrating time block serialization according to some embodiments of this specification.

[0017] Figure 8 This is an exemplary flowchart illustrating the application of a user representation model according to some embodiments of this specification. Detailed Implementation

[0018] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0019] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0020] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario of a user representation system according to some embodiments of this specification.

[0023] like Figure 1 As shown, in some embodiments, system 100 may include user terminal 110, first computing device 120, second computing device 130, network 140, and storage device 150.

[0024] User terminal 110 can provide user-related information and data. User terminal 110 may include terminals for all users on various platforms, such as mobile clients, PC clients, funds, futures, Hong Kong and US stocks, etc. In some embodiments, the user terminal 110 may be used by one or more users, including users directly using the service or other related users. In some embodiments, user terminal 110 may be one or any combination of mobile devices 110-1, tablet computers 110-2, laptop computers 110-3, desktop computers, and other devices with input and / or output functions. In some embodiments, user terminal 110 may use the service and generate user-related data through various means such as applications (e.g., clients) or web applications (e.g., WeChat mini-programs). In some embodiments, user-related data may include various user data, including user behavior data. In some embodiments, user terminal 110 may transmit user-related information and data through network 140 and other components in system 100 (e.g., first computing device 120, second computing device 130, storage device 150).

[0025] The first computing device 120 and the second computing device 130 are systems with computing and processing capabilities, and may include various computers, such as servers and personal computers, or computing platforms composed of multiple computers connected in various structures. In some embodiments, the first computing device 120 and the second computing device 130 may be implemented on a cloud platform. For example, the cloud platform may include one or a combination of private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, cross-cloud, multi-cloud, etc. In some embodiments, the first computing device 120 and the second computing device 130 may be the same device or different devices.

[0026] The first computing device 120 and the second computing device 130 may include one or more sub-processing devices (e.g., single-core or multi-core processing devices) that can execute program instructions. As an example only, the processing devices may include various common general-purpose central processing units (CPUs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits (ASICs), or other types of integrated circuits.

[0027] The first computing device 120 can process user-related information and data. In some embodiments, the first computing device 120 can execute user representation methods as shown in some embodiments of this specification to obtain user representations that can represent specific users and their behaviors, such as user vector representations. In some embodiments, the first computing device 120 may include a user representation model, which can be used to process user behavior data to obtain user representations. In some embodiments, the first computing device 120 can obtain a trained user representation model from the second computing device 130. In some embodiments, the first computing device 120 can send user-related information and data, as well as corresponding user representations, to the second computing device 130 for model updates. In some embodiments, the first computing device 120 can transmit user-related information and data through the network 140 and other components in the system 100 (e.g., user terminal 110, second computing device 130, storage device 150). In some embodiments, the first computing device 120 can directly connect to the second computing device 130 and exchange information and / or data.

[0028] The second computing device 130 can be used for model training. In some embodiments, the second computing device 130 can execute a user representation model training method as shown in some embodiments of this specification to obtain a user representation model. In some embodiments, the second computing device 130 can obtain user-related information and data from the user terminal 110 as training data for the model. In some embodiments, the second computing device 130 can obtain user representations and corresponding user information and / or data from the first computing device 120 for updating the model. In some embodiments, the first computing device 120 and the second computing device 130 may also be the same computing device.

[0029] Network 140 facilitates information and / or data exchange. In some embodiments, one or more components of user representation system 100 (e.g., user terminal 110, first computing device 120, second computing device 130, storage device 150) can exchange information and / or data with one or more components of user representation system 100 via network 140. Network 140 may include one or more of the following: public networks (e.g., the Internet), private networks (e.g., local area networks (LANs), wide area networks (WANs)), wired networks (e.g., Ethernet), wireless networks (e.g., 802.11 networks, Wi-Fi networks), cellular networks (e.g., LTE networks), Frame Relay networks, virtual private networks (VPNs), satellite networks, telephone networks, routers, hubs, server computers, etc. In some embodiments, network 140 may be any one or more of wired or wireless networks.

[0030] Storage device 150 can store information and / or data. In some embodiments, storage device 150 can store various information and / or data generated or transmitted by other components in system 100 (e.g., user terminal 110, first computing device 120, second computing device 130), such as user behavior data, user representations, user characterization models, etc. In some embodiments, storage device 150 can store data and / or instructions that can be executed or used by the first computing device 120 and / or the second computing device 130 to perform the exemplary methods described herein. In some embodiments, storage device 150 may include one or a combination of several of the following: mass storage, removable storage, volatile read-write storage, read-only storage (ROM). In some embodiments, storage device 150 can be implemented using the cloud platform described herein. For example, the cloud platform may include one or a combination of several of the following: private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, cross-cloud, multi-cloud, etc.

[0031] In some embodiments, storage device 150 may transmit data between itself and one or more components of user representation system 100 (e.g., user terminal 110, first computing device 120, second computing device 130, etc.) via network 140. In some embodiments, storage device 150 may be part of first computing device 120 and / or second computing device 130, or it may be independent and directly or indirectly connected to first computing device 120 and / or second computing device 130.

[0032] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the content of this specification. Features, structures, methods, and other features of the exemplary embodiments described herein can be combined in various ways to obtain other and / or alternative exemplary embodiments. For example, storage device 150 may be a standalone memory or memory array, or implemented via a storage server. However, these changes and modifications will not depart from the scope of this specification.

[0033] Figure 2 This is an exemplary block diagram of a user representation system 200 according to some embodiments of this specification. In some embodiments, the user representation system 200 may be deployed on a processing device, such as a first computing device 120 or a second computing device 130.

[0034] like Figure 2 As shown, in some embodiments, the user representation system 200 may include an acquisition module 210, a first feature extraction module 220, and a user representation module 230.

[0035] In some embodiments, the acquisition module 210 can be used to acquire a time series comprising multiple time blocks. In some embodiments, each of the multiple time blocks may include user behavior data within a time period.

[0036] In some embodiments, the acquisition module 210 may include a behavior data unit, a time block odd number unit, and a time series unit. The behavior data unit can be used to acquire user behavior data within a preset time length; the time block odd number unit can be used to determine the time length and movement step size of the time block; the time series unit can be used to divide the behavior data within the preset time length based on the time length and movement step size of the time block to obtain a time series including multiple time blocks.

[0037] In some embodiments, the first feature extraction module 220 can be used to obtain feature representations of multiple time blocks based on time series through a feature extraction network.

[0038] In some embodiments, feature representations of multiple time blocks can be obtained through a feature extraction network based on sparse convolution.

[0039] In some embodiments, the user representation module 230 can be used to obtain a vector representation of the user through an encoding network based on feature representations of multiple time blocks.

[0040] In some embodiments, the user representation module 230 may include a location representation unit and a vector representation unit. The location representation unit may be used to determine the location representation of each time block in a plurality of time blocks; the vector representation unit may be used to obtain the overall vector representation of the time series as the user's vector representation through an encoding network based on the location representation of each time block in the plurality of time blocks and the feature representation of the plurality of time blocks.

[0041] In some embodiments, the feature extraction network and the encoding network may be included within the user representation model, which can be trained using a user representation model training system.

[0042] In some embodiments, the user representation model training system may include some of the same modules as the user representation system, such as the acquisition module 210 and the first feature extraction module 220.

[0043] In some embodiments, the user representation model training system may further include a masked time block determination module, an overall vector determination module, and a network parameter adjustment module. The masked time block determination module can be used to determine masked time blocks among multiple time blocks; the overall vector determination module can be used to obtain the encoded representation of the masked time blocks and the overall vector representation of the time series through an encoding network based on the feature representations of the remaining time blocks in the time series excluding the masked time blocks; the network parameter adjustment module can be used to adjust the parameters of the encoding network based on the similarity between the overall vector representation of the current time series and the overall vector representations of the preceding or following time series, and the similarity between the encoded representation of the masked time blocks and the feature representations corresponding to the masked time blocks.

[0044] In some embodiments, the network parameter adjustment module can also be used to adjust the parameters of the feature extraction network based on the similarity between the overall vector representation of the current time series and the overall vector representation of the preceding or following time series, as well as the similarity between the encoded representation of the masked time block and the feature representation corresponding to the masked time block.

[0045] In some embodiments, the overall vector determination module can be used to determine the position representation of each time block in a plurality of time blocks; based on the position representation of each time block in the plurality of time blocks and the feature representation of the remaining time blocks, the encoded representation of the masked time blocks and the overall vector representation of the time series are obtained through an encoding network.

[0046] Figure 3 This is an exemplary flowchart of a user characterization method according to some embodiments of this specification.

[0047] like Figure 3 As shown, process 300 includes the following steps. In some embodiments, process 300 may be implemented by user representation system 200.

[0048] Step 310: Obtain a time series comprising multiple time blocks. Each of these time blocks includes user behavior data within a time period. In some embodiments, this step may be performed by the acquisition module 210.

[0049] A unit of time is a measure of time, such as a second, minute, hour, or day. A time block is a period of time consisting of one or more units of time. Generally, multiple units of time can be consecutive. A time block can be represented by a matrix. For example, a time block can be represented by a 2x60 matrix. Each unit of time can be 1 second, and with 60 columns per row (meaning 60 seconds per row, representing one minute), this matrix represents a 2-minute time block. Alternatively, each unit of time can be 1 minute, and with 60 columns per row (meaning 60 minutes per row, representing one hour), this matrix represents a 2-hour time block.

[0050] User behavior data refers to data related to a user's actions on a platform. For example, when a user reads news, user behavior data could include the topic of the news article, the start time, the duration, and the number of times a user clicked on a certain type of information. Similarly, when a user watches a movie, user behavior data could include the movie's theme, synopsis, cast, and viewing time. In some embodiments, a time block can include user behavior data within a specific time period. For instance, if a time block is a 2x60 matrix where one time unit is 1 second, then the time block represents two consecutive minutes. Each element in the matrix includes user behavior data for one second within those two minutes, thus the time block can include user behavior data for those two minutes. In some embodiments, a user can include all users across multiple platforms (e.g., mobile clients, PC clients, funds, futures, Hong Kong and US stocks, etc.), and user behavior data can include user behavior data across one or more platforms.

[0051] In some embodiments, user behavior data is typically recorded in short time units after a user action is triggered. Within a given timeframe, such as a day, user behavior data may not be generated in every short time unit (e.g., 1 second). Therefore, multiple short time units (e.g., 1 second) of behavior data can be merged to obtain behavior data in a relatively longer time unit. For example, the time unit of a time block can be set to a relatively long time unit (e.g., 1 minute), meaning that each time unit within the time block contains user behavior data from that longer time unit (e.g., 1 minute). For instance, a 2x60 matrix representing a 2-hour time block includes user behavior data from those 2 hours, where the time unit of the time block is set to 1 minute, thereby reducing the data length and consequently reducing the amount of memory and computation required.

[0052] A time series is a sequence of time blocks, which can be represented by a matrix. That is, a time series can contain user behavior data from all its time blocks. In some embodiments, a time series may include at least one time block (e.g., 24, 48, etc.), and its length is the number of time blocks it contains.

[0053] In some embodiments, the time series comprising multiple time blocks described herein can be obtained through various methods. For example, based on user data within a preset time length, or based on tags for the time blocks. In some embodiments, the time series can be obtained through a user terminal and / or a storage device. In some embodiments, user behavior data can be stored in the storage device in units of time blocks, and the time blocks can be identified using tags (e.g., time tags, feature tags, etc.), allowing the extraction of the time series comprising multiple time blocks based on the tags directly from the storage device.

[0054] In some embodiments, the parameters of a time block may include a time unit, time length, and movement step size. The time unit of a time block has been explained above and can be set according to actual needs or experience. The time length of a time block refers to the length of the time period it contains, and can also be represented by the number of time units contained within the time block. For example, if the time unit of a time block is 1 minute and the time length is 30, the time length of this time block is 30 minutes (e.g., 10:30-11:00). The movement step size refers to the distance the time block moves one time along the time coordinate direction, representing the length of time the time block moves along the direction of time flow. It can be represented by the number of time units contained within this moving time length. For example, if the time length of a time block is 30 minutes, its time unit is 1 minute, the movement step size is 20, and the starting time of the movement is 10:30-11:00, then moving the time block by one movement step size means that the time period contained within the time block has shifted forward by 20 minutes, i.e., 10:50-11:10. In some embodiments, if time blocks are represented by matrices, the step size can be obtained from the distance that any row in the time block moves once, for example, it can be represented by the number of rows the time block moves.

[0055] In some embodiments, user behavior data within a preset time period can be acquired. The preset time period can be a predetermined time interval, such as 1 hour, 2 hours, 1 day, or 1 week. In some embodiments, the user's raw behavior data can be log-structured data, which typically contains a large amount of redundant and invalid information, such as long strings of uids and placeholders. This redundant and invalid information needs to be removed to obtain the valid fields. Valid fields are data fields that effectively reflect user behavior and operations, such as time, topic, and keywords. User data may also include some complex data whose meaning cannot be directly obtained, such as underlying tracking data. In some embodiments, this type of data can be interpreted, i.e., converted into recognizable information, such as Chinese definitions. In some embodiments, after a series of operations such as removing redundant and invalid information and data interpretation, the user's raw behavior data can be filtered to extract valid fields and stored as structured data.

[0056] In some embodiments, the time length and movement step size of a time block within a preset time length can be determined, for example, based on the length of the preset time length, the amount of resources, etc. In some embodiments, the time length and movement step size of a time block within a preset time length can be determined based on the length of the preset time length. For example, if the preset time length is 24 hours in a day, the time length and movement step size can represent time lengths of 30 minutes and 15 minutes, respectively; if the preset time length is 1 hour, the time length and movement step size can represent time lengths of 5 minutes and 3 minutes, respectively.

[0057] In some embodiments, the duration and movement step size of a time block can be determined based on the amount of resources available. For example, when resources are plentiful, the duration and movement step size can be smaller, while when resources are limited, the duration and movement step size can be larger.

[0058] In some embodiments, behavioral data within a preset time length can be divided based on the time block length and the movement step size to obtain a time series comprising multiple time blocks. Taking a preset time length, i.e., a 24-hour day, as an example, if the time unit of a time block is 1 second, the time length is 60, and the movement step size is 60, then one time block represents a 1-minute time length, and the time series can be divided into 24*60 time blocks. If the time unit is 1 minute, the time length is 60, and the movement step size is 60, then one time block represents an hour, and the time series contains 24 time blocks. Therefore, the length of the time series representing a day's user behavior data is 24. In some embodiments, larger values ​​for the time length and movement step size can reduce the data length, thereby reducing the amount of memory and computation required.

[0059] Figure 7A , Figure 7B This is a schematic diagram illustrating time block serialization according to some embodiments of this specification.

[0060] Taking the merging of user data spanning a day as an example, when the system records user behavior data, it typically records by the second. Therefore, the longest sequence is 24 * 60 * 60 = 86400 seconds, which may exceed processing capacity. Therefore, sequences can be merged at the minute level, i.e., merged in minutes. The longest sequence is shortened to 24 * 60 = 1440 seconds, but this still requires significant computing power. In some embodiments, the longest sequence within a given time period can be reduced by further defining the duration of time blocks and the step size of the time blocks. For example... Figure 7AAs shown, in some embodiments, after minute-level merging, the time block length can be specified as 60, and the time block movement step size can be 60. The sequence composed of these consecutive time blocks can be considered serialized, i.e., it forms a time series. This time series includes a total of 24 time blocks, which are consecutive and do not overlap. In some embodiments, the movement step size can be set to be less than the time length, allowing the time blocks to overlap. Figure 7B As shown, if the time block length is set to 60 and the time block movement step is set to 30, then the time series includes 48 time blocks.

[0061] Step 320: Based on the time series, feature representations of multiple time blocks are obtained through a feature extraction network. In some embodiments, this step can be performed by the first feature extraction module 220.

[0062] The feature representation of a time block refers to the representation of user behavior data and / or behavioral data characteristics (e.g., high number of reads, long viewing time, etc.) within the time period represented by the time block. It can take various forms, such as feature vectors, text descriptions, tags, etc. In some embodiments, feature vectors can be used as the feature representation of a time block, which can be used to represent the user behavior data and / or user behavior data characteristics contained within the time block.

[0063] In some embodiments, the volume of user behavior data is large, especially user behavior data across multiple platforms. Furthermore, for long-term user behavior data, the resulting time series data has a large dimensionality (e.g., the dimension of the matrix, specifically the number of pointers), making it difficult to achieve optimal data processing efficiency and effectiveness. Therefore, the feature representation of a time block can be a low-dimensional feature vector. For example, the user behavior data within a time block can be processed to make its feature representation a one-dimensional token-level vector. In some embodiments, multiple feature vectors included within a time block can be processed and represented by a single feature vector, thereby converting a high-dimensional feature vector into a low-dimensional feature vector. As an example, a time block has a duration of 1 hour and a time unit of 1 minute. The user behavior data within each minute can be represented by a 1*3 vector (for example, the user's behavior in each minute will have a large number of fields, each field is represented by a vector, and then all fields are pooled to get a 1*3 vector representation for each minute, where 3 represents the vector length). Then the user behavior data of this time block can be represented as a 60*3 matrix. Through feature extraction, network model processing, etc., a 1*3 matrix (that is, a 1*3 vector) can be obtained based on the 60*3 matrix, that is, the 60-dimensional feature vector representation is converted into a 1-dimensional feature vector representation.

[0064] In some embodiments, the feature representation of a time block can be obtained through various means (e.g., feature extraction networks).

[0065] In some embodiments, feature representations of multiple time blocks can be obtained using a feature extraction network based on sparse convolution. For more information on obtaining feature representations of time blocks using a feature extraction network, please refer to [link to relevant documentation]. Figure 6 Step 620 will not be repeated here.

[0066] Step 330: Based on the feature representations of multiple time blocks, a vector representation of the user is obtained through an encoding network. In some embodiments, this step can be performed by the user representation module 230.

[0067] A user's vector representation is used to characterize a user profile or represent the overall characteristics of a user. A user's vector representation can, to some extent, reflect the commonalities and / or characteristics of a user's behavior over a period of time. For example, if a user clicks on content related to a particular stock on multiple platforms within a certain time period, or if the number of clicks on content related to that stock exceeds a threshold, it indicates that the user is paying close attention to that stock during that period.

[0068] In some embodiments, a user's vector representation can be obtained based on feature representations from multiple time blocks using various methods (e.g., neural networks). In some embodiments of this specification, a coding network can be used to obtain the user's vector representation based on feature representations from multiple time blocks. This coding network can include various networks or models capable of encoding, such as NNs, CNNs, and RNNs. Specifically, the coding network can include at least one Transformer Encoder layer.

[0069] In some embodiments, the positional representation of each time block in a plurality of time blocks can be determined. The positional representation of a time block refers to the position of each time block within the time series, reflecting its relative positional relationship with other time blocks. This can be represented by labels, sequence numbers, IDs, etc. In some embodiments, a separate field can be used as the positional representation of each time block in the time series. In some embodiments, a field can be added to the feature vector of each time block in the time series, with the value being the position number of that time block in the time series. For example, a field can be added to the feature vector sequence (e.g., at the head or tail) composed of the feature vectors of all time blocks that make up the time series, recording the positional relationship of the time blocks arranged in the current order. For example, if the time blocks currently constituting the time series are A, B, and C, and their corresponding positional representations according to the order A, B, and C can be 0, 1, and 2, then the content of the field added to the head can be "0, 1, 2".

[0070] In some embodiments, after determining the position representation of each time block among multiple time blocks in a time series, the overall vector representation of the time series can be obtained as the user's vector representation through an encoding network (e.g., at least one Transformer Encoder layer) based on the position representation of each time block among multiple time blocks and the feature representation of multiple time blocks.

[0071] In some embodiments, the user representation method described above can be implemented using a user representation model, that is, by using a user representation model to obtain a vector representation of the user based on a time series including multiple time blocks. This user representation model may include the feature extraction network and encoding network described in some embodiments of this specification. For instructions on training the user representation model, please refer to [link to relevant documentation]. Figure 5 This will not be elaborated upon here.

[0072] Some embodiments of this specification demonstrate that by transforming user behavior data over a period of time into a vector representation of the user, long-term behavioral data can be modeled while reducing data storage and computational load. The resulting user vector representation can be used for various user-related data processing tasks, such as user behavior prediction, user classification, and user relationship prediction. In some embodiments, data processing tasks can be implemented using various data processing methods (e.g., rule-based reasoning algorithms, processing through network models, etc.).

[0073] Figure 4 This is an exemplary flowchart illustrating the application of a user representation model according to some embodiments of this specification. In this embodiment, for ease of explanation only, the example used is processing user behavior data over a day to obtain a user's vector representation; a day can be replaced with any other suitable time length. Steps 410-450 provide an exemplary description of user behavior data acquisition and serialization, step 460 provides an exemplary description of training the user representation model, and step 470 provides an exemplary description of the application of the user's vector representation obtained through the user representation model.

[0074] In some embodiments, steps 410-450 may be performed by the acquisition module 210.

[0075] Step 410: Pull one day's user platform behavior data from the data source.

[0076] In some embodiments, user behavior data across all platforms includes user behavior data across multiple platforms such as mobile clients, PC clients, funds, futures, and Hong Kong and US stocks. In some embodiments, the user behavior log data involves complex and diverse scenarios and can be stored in different locations. In some embodiments, user behavior data can be obtained from one or more data sources, such as various file systems, such as distributed file systems like HDFS (Hadoop Distributed File System), or centralized file systems.

[0077] In some embodiments, the method may include step 420, processing user behavior data. In some embodiments, step 420 may be performed by the acquisition module 210.

[0078] In some embodiments, user behavior data can be processed, for example, structured, and then persisted, i.e., stored in a data storage space, such as HDFS.

[0079] In some embodiments, processing behavioral data may include one or more of the following: integrating user behavior from all endpoints and business lines; providing Chinese explanations for user behavior tracking points to make them readable; tracing the source of items involved in user behavior, where the source can be of various types, such as questions, information, strategies, etc.; filtering and extracting effective fields and structuring the data based on tracking point information and item information. In some embodiments, the specific criteria for filtering effective fields can be determined based on downstream applications and experimental results.

[0080] In some embodiments, since the structure of a user's original behavior data usually contains a large amount of invalid and redundant information (e.g., uid, placeholders, etc.), the user's original behavior data can be used to extract valid fields, thereby obtaining valid fields as the user's behavior data.

[0081] In some embodiments, user behavior data can be collected through event tracking. Since the underlying event tracking data is quite complex, with many events lacking information, these events can be decoded and interpreted into Chinese meanings, and then effective fields can be selected based on business experience. Event tracking refers to collecting information within specific processes of an application to track application usage, which can then be used to further optimize the product or provide operational data support.

[0082] In some embodiments, a time block may include various types of data such as placeholders, user behavior data, and other procedural data. The amount of structured data can be reduced by extracting effective fields from this data (e.g., user-related data such as user behavior data, data with practical significance, etc.). For example, the structured data for a user A's act of reading news may include: the name of the act (e.g., news reading), the trigger time of the act, the information identifier (e.g., id), the information topic, the information keywords, and the stocks associated with the information.

[0083] Step 430: Obtain a time series including multiple time blocks based on user behavior data.

[0084] In some embodiments, user behavior data (e.g., structured user behavior data) can be merged (e.g., minute-level data scan merging), and the time length and movement step size of time blocks can be defined to serialize the time blocks, thereby obtaining a time series comprising multiple time blocks. Data scan merging can be based on the desired time unit, such as hour, day, week, etc.

[0085] For information on time blocks, time length, movement step size, and time block serialization, please refer to [link / reference]. Figure 3 Step 310 in the process will not be repeated here.

[0086] Step 440: Retrieve historical (e.g., yesterday) user behavior data from the data source.

[0087] In some embodiments, while acquiring user behavior data for the current day, historical user behavior data can be obtained from a data source, such as structured user behavior data stored from the previous day. In some embodiments, historical user behavior data can be historical user behavior data within a preset time period, such as historical user behavior data within two months from the current day.

[0088] Step 450: Incrementally merge user behavior data.

[0089] In some embodiments, the user behavior data in steps 430 and 440 can be merged to update the currently stored structured user behavior data. In some embodiments, the data for the first day can be removed from the user behavior data obtained in step 440, and the data for the second day, the third day, and so on, the last day, can be shifted forward to become the data for the new first day, the second day, and so on, the second-to-last day. Today's user behavior data can then be used as the last day's data in the table, and the updated data can be stored as the latest structured user behavior data.

[0090] Step 460: Train the user representation model based on user behavior data.

[0091] In some embodiments, sampled data from multiple users can be obtained from the user behavior data over a historical period (e.g., two months) obtained in step 450. This sampled data can be used as training data to train the user representation model. Once the model is trained, a vector representation of the user can be obtained based on a time series of a certain length. In some embodiments, the user behavior data used as training data can be a data table, referred to as a user behavior data table. This data table can be structured and serialized user behavior data. In some embodiments, the user behavior data, i.e., the user behavior data table, can be a three-dimensional sparse matrix, where each dimension represents the batch_size of the training data (i.e., the number of data samples captured in one training session), the number of time blocks, and the length of the time blocks, respectively.

[0092] For more information on how to train a user representation model based on training data, please refer to [link to relevant documentation]. Figure 5 This will not be elaborated upon here.

[0093] In some embodiments, the trained user representation model can be persisted, i.e., saved to storage space, such as a file system like HDFS.

[0094] Step 470: Input the time series of user behavior data into the user representation model and output a vector representation of the user. In some embodiments, step 470 can be performed by the user representation module 230.

[0095] For each user, a corresponding vector representation can be obtained through the user representation model. In some embodiments, structured user behavior data (i.e., time series) of all users with behavioral data for the day (which can be called the daily full user data) can be input into the persistent user representation model, and the resulting representation of each user, i.e., the user's vector representation, can be output.

[0096] In some embodiments, after obtaining the user's vector representation, it can be stored in storage space, such as a file system like HDFS. In some embodiments, the vector representations of all users across the platform can be stored in storage space daily.

[0097] It should be noted that the above description of process 400 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 400 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, for structured log data, step 420 can be omitted.

[0098] Figure 5This is an exemplary flowchart illustrating a user representation model training method according to some embodiments of this specification. Figure 5 As shown, process 500 includes the following steps. In some embodiments, the steps in method 500 may be performed once or multiple times to perform one or more iterative updates to the parameters of the user representation model.

[0099] Step 510: Obtain a time series comprising multiple time blocks. Each of these time blocks includes user behavior data within a time period. In some embodiments, step 510 may be performed by the acquisition module 210.

[0100] In some embodiments, historical user behavior data can be obtained from one or more platforms (e.g., mobile clients, PC clients, funds, futures, Hong Kong and US stocks, etc.) using various methods. Then, training data for a user representation model, i.e., at least one time series data, can be obtained based on this historical user behavior data. For more information on how to obtain training data for a user representation model and how to obtain time series data, please refer to [link to relevant documentation]. Figure 3 Step 310 Figure 4 Steps 410-450 and their related descriptions will not be repeated here.

[0101] Step 520: Obtain feature representations of multiple time blocks through a feature extraction network. In some embodiments, this step can be performed by the first feature extraction module 220.

[0102] For each time block in the time series, a corresponding feature representation can be obtained through a feature extraction network. In some embodiments, the feature representation can be represented as a feature vector. The feature extraction network can be various machine learning models, such as NN, CNN (Convolutional Neural Networks), etc.

[0103] In some embodiments, considering that the data in time blocks is generally a sparse matrix (e.g., a three-dimensional sparse matrix), a feature extraction network can be used to obtain the feature representation of the time blocks based on sparse convolution. In this embodiment, the feature extraction network can skip placeholders and other redundant information in the user feature data. For example, if the user behavior within 1 minute only includes clicking Baidu once at the 2nd second, then the user behavior data includes placeholders for the 1st second and seconds 3-60. Sparse convolution is used to remove the placeholders.

[0104] In some embodiments, the feature representation of a time block can be further represented as a low-dimensional feature vector. The high-dimensional time block feature representation can be transformed into a low-dimensional feature vector through pooling operations (e.g., max pooling, average pooling, etc.) and network layer processing such as CNN. For example, it can be transformed from a 60*3 matrix into a 1*3 vector.

[0105] For more information on how to obtain feature representations of multiple time blocks using feature extraction networks, please refer to [link to relevant documentation]. Figure 6 Step 620 will not be repeated here.

[0106] Step 530: Determine the masking time block among the multiple time blocks. In some embodiments, this step may be performed by the masking time block determination module.

[0107] A masking time block refers to a time block used as a mask. Masking time blocks can be dynamic (dynamic mask) or static (static mask). Dynamic masking time blocks mean the position of the masking time block is randomized in each run of the model; static masking time blocks mean the position of the masking time block is fixed in each run of the model.

[0108] In some embodiments, the masking time block among multiple time blocks can be determined in various ways. For example, by setting the proportion of the masking time block in the total time block. Another example is through dynamic masking, etc.

[0109] In some embodiments of this specification, a masking method is used in the model to predict the feature representation of the mask and obtain the overall vector representation corresponding to the time series, which reduces the amount of data processing. By using dynamic masks, the error caused by static masks is reduced, making the prediction more random and more in line with the real environment.

[0110] Step 540: Based on the feature representations of the remaining time blocks in the time series excluding the masking time block, an encoded representation of the masking time block and the overall vector representation of the time series are obtained through an encoding network. In some embodiments, this step can be performed by an overall vector determination module.

[0111] The encoded representation of a masked time block refers to the representation of the masked time block obtained from the output of an encoding network, such as a feature vector. In some embodiments, the feature representation of the masked time block can be reconstructed by an encoding network (e.g., at least one Transformer Encoder layer) based on the feature representations of the remaining time blocks in the time series excluding the masked time block.

[0112] The overall vector representation refers to the vector representation of the entire time series, which can take various forms, such as a single vector or a label.

[0113] In some embodiments, the feature representation of the masked time block can be restored by an encoding network based on the feature representation of the remaining time blocks in the time series excluding the masked time block. That is, the encoded representation of the masked time block obtained by the encoding network is obtained. Then, the overall vector representation of the time series is obtained based on the restored feature representation of the masked time block and the feature representation of the remaining time blocks excluding the masked time block.

[0114] In some embodiments, the position representation of each time block in a plurality of time blocks can be determined; based on the position representation of each time block in the plurality of time blocks and the feature representation of the remaining time blocks, the encoded representation of the masked time blocks is obtained through an encoding network, and the overall vector representation of the time series is obtained.

[0115] Step 550: Based on the similarity between the overall vector representation of the current time series and the overall vector representations of the preceding or following time series, and the similarity between the encoded representation of the masked time block and the feature representation corresponding to the masked time block, adjust the parameters of the encoding network. In some embodiments, this step can be performed by a network parameter adjustment module.

[0116] The current time series refers to the time series currently being processed, such as a time series obtained based on user behavior data from January 2021. The preceding time series is the time series preceding the current time series, and the following time series is the time series following the current time series. In some embodiments, the preceding and following time series can be time series adjacent to the current time series or non-adjacent time series. For example, if the current time series is obtained based on user A's behavior data from May 2021, then the preceding time series could correspond to any month from April, March, February, etc., in 2021; and the following time series could correspond to any month from June, July, August, etc., in 2021.

[0117] In some embodiments, when training a user representation model, user data from several chronological time periods can be acquired for a single user. For example, user behavior data for user A in January and February 2021 can be acquired. Another example is acquiring user behavior data for user A in January and March 2021. Based on this chronological user data, several corresponding time series are obtained, and the similarity of the overall vector representations of these time series is calculated.

[0118] Similarity refers to the degree of similarity or proximity between two pieces of data, and can be represented in various forms, such as similarity, mutual information, and difference (the greater the difference, the smaller the similarity; the smaller the difference, the greater the similarity). In some embodiments, the similarity between two pieces of data can be obtained through models (e.g., mutual information estimators), similarity algorithms, etc. For example, the overall vector representation of the current (e.g., January 2021) time series and the overall vector representation of the previous time series (e.g., the time series corresponding to December 2020) or the subsequent time series (e.g., the time series corresponding to February 2021) can be obtained using the aforementioned methods; another example is obtaining the similarity between the encoded representation of a masked time block and the feature representation corresponding to the masked time block using the aforementioned methods.

[0119] In some embodiments, the parameters of the encoding network can be adjusted based on the similarity between the overall vector representation of the current time series and the overall vector representation of the preceding or following time series, as well as the similarity between the encoded representation of the masked time block and the feature representation corresponding to the masked time block, thereby updating the encoding network.

[0120] In some embodiments, in addition to adjusting the parameters of the encoding network, the parameters of the feature extraction network can also be adjusted to update the feature extraction network. In some embodiments, the feature extraction network can also be pre-trained, so its parameters do not need to be adjusted.

[0121] For more information on adjusting the parameters of the encoding network and feature extraction network, please refer to [link to relevant documentation]. Figure 6 Steps 640 and 650 and their related descriptions will not be repeated here.

[0122] In some embodiments of this specification, long-term user behavior data is serialized to obtain a time series comprising multiple time blocks. Features of these time blocks are extracted using various methods such as data pooling and sparse convolution, and high-dimensional data is transformed into low-dimensional data, reducing data volume, resource requirements, and resource consumption, thereby improving economic efficiency. The encoding network is trained jointly based on a coarse-grained task (the similarity between the overall vector representation of the current time series and the overall vector representation of adjacent time series) and a fine-grained task (the similarity between the encoded representation of the masked time blocks and the feature representation of the masked time blocks in the input network). This effectively trains the user representation model, making network training more efficient and faster, while improving training quality, resulting in a better-performing user representation model with more accurate user representations.

[0123] Figure 6This is an exemplary flowchart of training a user representation model according to some embodiments of this specification, wherein the user representation model may include a feature extraction network and an encoding network. In some embodiments, process 600 may include the following steps:

[0124] Step 610: Input the time series data into the user representation model.

[0125] For more details on obtaining training data and time series, please refer to [link / reference]. Figure 3 and Figure 4 The details and related descriptions will not be repeated here.

[0126] Step 620: Process the time series to obtain the feature representations corresponding to each time block in the time series. In some embodiments, this step can be performed by a feature extraction network.

[0127] In some embodiments, step 620 may include embedding and uniform pooling for each field in the input data. As an example only, assuming a time block is 1 hour and the time unit is 1 minute, for each time block, the user's behavior per minute will have a large number of fields. Each field can be transformed into a vector, and then all fields can be pooled to one field per minute, resulting in a 60*3 matrix.

[0128] In some embodiments, step 620 may further include: performing feature extraction processing at the time block level.

[0129] In some embodiments, feature extraction processing at the time block level may include steps 621 and 622.

[0130] Step 621: Use sparse convolution to extract features from the time blocks.

[0131] In user behavior data, since users typically don't interact every minute, the vector matrix obtained in step 620 is generally sparse. In some embodiments, sparse convolution can be used to extract features corresponding to fields in each time block, and then pooling (e.g., max pooling) is performed to obtain a low-dimensional vector. Taking the time block in step 620 as an example, a 60*3 matrix can be transformed into a 1*3 vector, i.e., a one-dimensional token level, by extracting features through sparse convolution and performing max pooling.

[0132] Step 622: Determine the masking time block.

[0133] For more details on determining the masking time block, please refer to step 530 and its related description, which will not be repeated here.

[0134] Step 630: Process the token using the time block as the token feature.

[0135] In some embodiments, the low-dimensional data (e.g., a 1*3 vector matrix) obtained in step 620 can be processed using time blocks as token features, wherein step 630 may include steps 631 and 632.

[0136] Step 631: Determine the location representation of each time block.

[0137] In some embodiments, before performing step 632, in order to avoid losing the sequential relationship between sentences, a Position Embedding module can be added to determine the position representation of each time block.

[0138] Step 632: Input the location representation and feature representation of each time block into the encoding network.

[0139] In some embodiments, after step 631, the processed time series data, i.e., the position representation and feature representation of each time block, can be input into an encoding network, such as a standard multi-layer Transformer Encoder structure, and processed.

[0140] In some embodiments, an additional token, CLS, can be added to the beginning of the entire time series before the data is input into the Transformer Encoder. This token can be used to subsequently instruct the output of the overall vector representation of the time series at the CLS bit, i.e., the user's vector representation.

[0141] Step 640: Adjust the model parameters of the user representation model based on coarse-grained and fine-grained tasks.

[0142] Coarse-grained tasks refer to adjusting the model parameters of a user representation model so that, for a given user, the overall vector representation of the current time series output by the user representation model is similar to the overall vector representation of the time series adjacent to the current time series. Fine-grained tasks refer to adjusting the model parameters of a user representation model so that the feature representation of the masked time block restored by the encoding network, i.e., the encoded representation of the masked time block, is similar to the original feature representation of the masked time block.

[0143] In some embodiments, training samples in a coarse-grained task may include positive and negative samples, and the coarse-grained task can be performed based on these samples. Positive samples may include the user feature vectors of the previous month and the current month for the same user. Negative samples may include the user feature vectors of the same user in the current month from the positive samples and the user feature vector of another user in the current month. Positive samples may be about a single user, thus requiring the machine learning model to be trained to increase the similarity between the user feature vectors of the previous month and the current month. Negative samples may be about two users, thus requiring the machine learning model to be trained to increase the difference between the user feature vectors of one user and the user feature vectors of the other user.

[0144] Step 650: Calculate the loss for the coarse-grained task and the loss for the fine-grained task, and iteratively update the model parameters based on the loss for the coarse-grained task and the loss for the fine-grained task.

[0145] In some embodiments, the loss can be represented by similarity such as mutual information between the model output and the label, and / or difference between the model output and the label. Mutual information is a measure of the correlation between two variables; the greater the correlation, the greater the value of mutual information. The difference between two variables can be measured by cross-entropy, which, in contrast to mutual information, is a measure of correlation; the smaller the difference, the smaller the value of cross-entropy.

[0146] The loss for the coarse-grained task may include terms representing the similarity and / or difference between the overall vector representation of the current time series output by the user representation model and the overall vector representations of time series adjacent to the current time series, for a given user. The loss for the fine-grained task may include terms representing the similarity and / or difference between the feature representation of the masked time block reconstructed by the user representation model (i.e., the encoded representation of the masked time block) and the original feature representation of the masked time block. In some embodiments, loss functions for both the coarse-grained and fine-grained tasks may be computed, and the sum of these two loss functions may be used as the loss function for the user representation model.

[0147] In some embodiments, the feature extraction network and / or encoding network can be updated based on the loss function of the user representation model.

[0148] Figure 8 This is an exemplary flowchart illustrating the application of a user representation model according to some embodiments of this specification.

[0149] like Figure 8As shown, in some embodiments, process 800 may include the following steps: inputting the time series of obtained user behavior data into the model; more details about this step can be found in step 610; the embedding layer performs embedding and pooling (e.g., average pooling) on ​​each field in the time block of the time series, and after pooling into low-dimensional data, it is input into the convolutional layer; the convolutional layer uses sparse convolution to extract the features of each time block of the time series, and then pools (e.g., max pooling) to a one-dimensional token feature vector representation; more details about this step can be found in step 620; then position embedding is performed, and then the data is input into an encoding network including at least one layer (e.g., such as...). Figure 8 The encoding is performed using the encode#1, encode#2, ... shown, and finally a vector representation of the entire time series output by the encoding network is obtained. This vector representation can be used as the user's vector representation. For more detailed explanation of this step, please refer to step 630.

[0150] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0151] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0152] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0153] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0154] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0155] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0156] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for training a user representation model, comprising: Acquire user behavior data within a preset time period; Determine the time length and movement step size of the time block; Based on the time length and the movement step size of the time block, the behavioral data within the preset time length is divided to obtain a time series including the multiple time blocks, and each time block includes the user's behavioral data within a time period; The feature representations of the multiple time blocks are obtained through a feature extraction network; Determine the masking time block among the plurality of time blocks; Based on the feature representations of the remaining time blocks in the time series excluding the masking time block, the encoded representation of the masking time block and the overall vector representation of the time series are obtained through an encoding network; Based on the similarity between the overall vector representation of the current time series and the overall vector representation of the preceding or following time series, as well as the similarity between the encoded representation of the masked time block and the feature representation corresponding to the masked time block, the parameters of the encoding network and the feature extraction network are adjusted.

2. The method as described in claim 1, wherein obtaining the encoded representation of the masking time block and the overall vector representation of the time series through an encoding network based on the feature representations of the remaining time blocks in the time series excluding the masking time block includes: Determine the position representation of each of the plurality of time blocks; Based on the position representation of each time block among the plurality of time blocks and the feature representation of the remaining time blocks, the encoded representation of the masked time blocks and the overall vector representation of the time series are obtained through the encoding network.

3. The method as described in claim 1, wherein obtaining the encoded representation of the masking time block and the overall vector representation of the time series through an encoding network based on the feature representations of the remaining time blocks in the time series excluding the masking time block includes: Based on the feature representations of the remaining time blocks in the time series excluding the masking time block, the encoded representation of the masking time block is obtained through the encoding network; The overall vector representation of the time series is obtained based on the encoded representation of the masked time block and the feature representation of the remaining time blocks other than the masked time block.

4. A user representation method, comprising: Acquire user behavior data within a preset time period; Determine the time length and movement step size of the time block; Based on the time length and the movement step size of the time block, the behavioral data within the preset time length is divided to obtain a time series including the multiple time blocks, and each time block includes the user's behavioral data within a time period; Based on the time series, feature representations of the multiple time blocks are obtained through a feature extraction network; Based on the feature representations of the multiple time blocks, a vector representation of the user is obtained through an encoding network; and The user representation model is trained according to any one of claims 1 to 3, wherein the user representation model includes the feature extraction network and the encoding network.

5. The method as described in claim 4, wherein obtaining the feature representations of the plurality of time blocks through the feature extraction network includes: The feature extraction network obtains the feature representations of the multiple time blocks based on sparse convolution.

6. The method of claim 5, wherein obtaining the feature representation of the plurality of time blocks based on sparse convolution includes: Features of each time block in the multiple time blocks are extracted by sparse convolution, and then max pooled to obtain a low-dimensional vector.

7. The method of claim 4, wherein obtaining the user's vector representation through an encoding network based on the feature representations of the plurality of time blocks comprises: Determine the position representation of each of the plurality of time blocks; Based on the position representation of each time block in the plurality of time blocks and the feature representation of the plurality of time blocks, the overall vector representation of the time series is obtained through the encoding network as the vector representation of the user.

8. A user representation system, comprising an acquisition module, a first feature extraction module, and a user representation module; The acquisition module is used to acquire user behavior data within a preset time length; determine the time length and movement step size of the time block; divide the behavior data within the preset time length based on the time length and movement step size of the time block to obtain a time series including the multiple time blocks, each of the multiple time blocks including the user's behavior data within a time period; The first feature extraction module is used to obtain feature representations of the multiple time blocks based on the time series through a feature extraction network; The user representation module is used to obtain the user's vector representation through an encoding network based on the feature representations of the multiple time blocks, wherein the feature extraction network and the encoding network are trained by the method described in any one of claims 1 to 3.

9. A computer-readable storage medium storing computer instructions, wherein when a computer reads the computer instructions in the storage medium, the computer performs the method as described in any one of claims 4 to 7.

Images