Person image method based on combined spatio-temporal attention mechanism and related device

Abstract

The application discloses a kind of methods for portrait based on combined space-time attention mechanism and related devices, it is related to user portrait technical field, the method comprises: obtaining the user behavior attribute data of target user, in order to facilitate capturing complex relationship and feature in user behavior attribute data, first, discrete user behavior attribute data is converted into embedded vector representation using embedding layer, obtain comprehensive embedded vector, subsequently, first and second space-time relationship features are extracted to comprehensive embedded vector using space-time attention and convolution fusion module, the global time dependence relationship, local space dependence relationship, global space dependence relationship and local time dependence relationship of user behavior are all extracted, finally, classification and clustering are carried out according to this, determine the label of target user, generate accurate portrait.The above scheme of the application can accurately and simultaneously extract local and global space-time features in data, which provides a solid foundation for further analysis and decision-making.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of user portrait, in particular to a character portrait method based on a combined spatio-temporal attention mechanism and related devices. BACKGROUND

[0002] A character portrait refers to a tagged user model abstracted from user basic information, preferences, habits, behaviors and other information. In other words, it is a label given to a user. The label is a highly refined feature identifier obtained by analyzing user information. By labeling, users can be described using some highly generalized and easily understood features, making it easier for people to understand users, which is conducive to the development of user personalized services, analysis and decision-making. Labels are divided into static labels and dynamic labels. Static labels do not change over time, such as gender and birthday. Dynamic labels change over time, such as user social relationships, interests, social networks, etc. A static character portrait label system is usually difficult to meet the rapid update and iteration of social big data. Dynamic labels can reflect changes in user behavior and status in a timely manner and can be dynamically adjusted based on real-time user data, thus providing more accurate and personalized user portraits. Therefore, dynamic knowledge graphs are usually used to generalize and expand character portrait labels. A dynamic knowledge graph contains a user's social network graph, which is updated in an active and passive manner. The active method uses a high-frequency entity and knowledge update strategy, and the passive method uses a periodic update strategy to achieve automatic evolution of character portrait labels.

[0003] In addition, user portraits involve the extraction of spatio-temporal relationships when being constructed, including the evolution over time, the change characteristics of individual user behavior and attributes, and the characteristics of other users related thereto. Research on spatio-temporal relationship extraction can be divided into three categories: RNN-based models, CNN-based models, and Attention-based models. However, these models lack the interaction between the attention mechanism and the convolution module, which may result in the inability to accurately extract local and global spatio-temporal features at the same time.

[0004] Therefore, in view of the problem that the feature extraction model in the process of constructing a user portrait cannot accurately extract local and global spatio-temporal features at the same time, there is an urgent need for an appropriate solution. SUMMARY

[0005] The purpose of the present application is to provide a character portrait method based on a combined spatio-temporal attention mechanism and related devices, which can accurately extract local and global spatio-temporal features at the same time, thereby improving the accuracy of character portraits.

[0006] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0007] In a first aspect, the present application provides a method for generating a character portrait based on a combined spatio-temporal attention mechanism, comprising the following steps:

[0008] Obtaining user behavior attribute data of a target user; the user behavior attribute data comprises a user behavior sequence and a user social relationship network graph.

[0009] Converting the user behavior attribute data into embedded vector representation by using an embedding layer to obtain a comprehensive embedded vector.

[0010] Extracting first spatio-temporal relationship features and second spatio-temporal relationship features from the comprehensive embedded vector by using a spatio-temporal attention and convolution fusion module; the first spatio-temporal relationship features comprise global temporal dependency relationships and local spatial dependency relationships; the second spatio-temporal relationship features comprise global spatial dependency relationships and local temporal dependency relationships.

[0011] Classifying and clustering according to the first spatio-temporal relationship features and the second spatio-temporal relationship features to determine the label of the target user and generate a character portrait of the target user.

[0012] Optionally, the user social relationship network graph is a directed and weighted graph, the nodes in the user social relationship network graph represent different users, the edges between the nodes represent the relationships between the users, and the weights of the edges represent the closeness of the contact between the two users.

[0013] Optionally, converting the user behavior attribute data into embedded vector representation by using an embedding layer to obtain a comprehensive embedded vector comprises the following steps:

[0014] Extracting a spatial embedded vector, a periodic time embedded vector, a feature sequence high-dimensional embedded vector, and an adaptive embedded vector according to the user behavior sequence and the user social relationship network graph by using the embedding layer.

[0015] Summing the spatial embedded vector, the periodic time embedded vector, the feature sequence high-dimensional embedded vector, and the adaptive embedded vector to obtain the comprehensive embedded vector.

[0016] Optionally, the spatio-temporal attention and convolution fusion module comprises a first feature extraction branch and a second feature extraction branch; the first feature extraction branch comprises a temporal attention submodule and a spatial convolution submodule; the second feature extraction branch comprises a spatial attention submodule and a temporal convolution submodule; extracting the first spatio-temporal relationship features and the second spatio-temporal relationship features from the comprehensive embedded vector by using the spatio-temporal attention and convolution fusion module comprises the following steps:

[0017] Inputting the comprehensive embedded vector into the temporal attention submodule to extract first temporal features containing global temporal dependency relationships.

[0018] The integrated embedding vector is input into a space attention submodule to extract a first space feature containing global space dependency.

[0019] The first time feature is input into a space convolution submodule to extract a first space-time relationship feature containing global time dependency and local space dependency.

[0020] The first space feature is input into a time convolution submodule to extract a second space-time relationship feature containing local time dependency and global space dependency.

[0021] Optionally, in the time attention submodule, the global time dependency of the user behavior sequence over time is captured through a multi-head attention mechanism; and in the space attention submodule, the global space dependency in the user social relationship network graph is captured using a self-attention mechanism.

[0022] Optionally, in the space convolution submodule, a graph convolutional neural network is used to extract the local space dependency; and in the time convolution submodule, a dilated convolution is used to extract the local time dependency.

[0023] In a second aspect, the present application provides a character portrait system based on a combined space-time attention mechanism, comprising:

[0024] A user behavior attribute data acquisition module is configured to acquire user behavior attribute data of a target user; the user behavior attribute data includes a user behavior sequence and a user social relationship network graph.

[0025] An embedded vector representation conversion module is configured to convert the user behavior attribute data into an embedded vector representation using an embedding layer to obtain an integrated embedding vector.

[0026] A space-time relationship feature extraction module is configured to extract first space-time relationship features and second space-time relationship features from the integrated embedding vector using a space-time attention and convolution fusion module; the first space-time relationship features include global time dependency and local space dependency; and the second space-time relationship features include global space dependency and local time dependency.

[0027] A user character portrait generation module is configured to classify and cluster based on the first space-time relationship features and the second space-time relationship features, determine a label of the target user, and generate a character portrait of the target user.

[0028] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the character portrait method based on the combined space-time attention mechanism.

[0029] In a fourth aspect, the present application provides a computer readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the steps of the method for generating a character portrait based on a combined spatio-temporal attention mechanism as described above.

[0030] In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method for generating a character portrait based on a combined spatio-temporal attention mechanism as described above.

[0031] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0032] The present application provides a method for generating a character portrait based on a combined spatio-temporal attention mechanism and related devices. In the method, user behavior attribute data of a target user is obtained. In order to capture complex relationships and features in the user behavior attribute data, an embedding layer is first used to convert discrete user behavior attribute data into an embedded vector representation to obtain a comprehensive embedding vector. Then, a spatio-temporal attention and convolution fusion module is used to extract first and second spatio-temporal relationship features from the comprehensive embedding vector. The global temporal dependency relationship, local spatial dependency relationship, global spatial dependency relationship, and local temporal dependency relationship of user behavior are all extracted. Finally, classification and clustering are performed based on this to determine the label of the target user and generate an accurate character portrait of the target user. The above-mentioned solution of the present application can accurately and simultaneously extract local and global spatio-temporal features in user behavior data. Specifically, by using the spatio-temporal attention and convolution fusion module, the changing trend of data at different time points and spatial dimensions can be analyzed in depth. Not only can subtle local features be captured, but also the overall global pattern can be identified. This accurate spatio-temporal feature extraction provides a solid foundation for further label analysis and decision-making, ensuring that the internal laws of the data can be more comprehensively understood, and thus the generation strategy can be optimized to improve the ability and effectiveness of character portrait generation. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0034] Figure 1 A flowchart of a method for generating a character portrait based on a combined spatio-temporal attention mechanism according to an embodiment of the present application.

[0035] Figure 2 A data flowchart of a method for generating a character portrait based on a combined spatio-temporal attention mechanism according to an embodiment of the present application.

[0036] Figure 3 A flow chart of step A3 in a method for generating a portrait of a person based on a combined spatio-temporal attention mechanism according to an embodiment of the present application.

[0037] Figure 4 A functional module diagram of a system for generating a portrait of a person based on a combined spatio-temporal attention mechanism according to an embodiment of the present application.

[0038] Figure 5 A structural schematic diagram of a computer device according to an embodiment of the present application.

[0039] Figure 6 A structural schematic diagram of a computer program product according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0041] The above purposes, features and advantages of the present application can be more apparent and easy to understand. The present application will be described in further detail below with reference to the drawings and specific embodiments.

[0042] In an exemplary embodiment, as shown in Figure 1 a method for generating a portrait of a person based on a combined spatio-temporal attention mechanism is provided, including the following steps:

[0043] A1, obtaining user behavior attribute data of a target user; the user behavior attribute data includes a user behavior sequence and a user social relationship network graph. The user behavior attribute data is usually discrete input, such as user age, criminal record, relationship with others, etc. In the present embodiment, the user social relationship network graph is a directed weighted graph, and the nodes in the user social relationship network graph represent different users, the edges between the nodes represent the relationship between the users, and the weight of the edge represents the closeness of the two users.

[0044] A2, converting the user behavior attribute data into embedded vector representation by using an embedding layer to obtain a comprehensive embedded vector. Since the user behavior attribute data is usually discrete input, in order to capture the complex relationship and features in the user behavior attribute data, it is necessary to convert it into vector representation first. In the present embodiment, step A2 specifically includes the following steps:

[0045] A21, using an embedding layer, according to the user behavior sequence and the user social relationship network graph, spatial embedding vectors, periodic time embedding vectors, feature sequence high-dimensional embedding vectors and adaptive embedding vectors are extracted.

[0046] In the conversion process of step A21, as shown in Figure 2 , by performing eigenvalue decomposition on the standard Laplacian matrix of the user social relationship network graph attribute, the spatial embedding vector X S is generated; in the time dimension, considering that there are 7 days in a week and 24 hours in a day, the one-hot encoding is used to obtain the vector representation of day-of-week and time-of-day, and the two are connected to generate the periodic time embedding vector X T ; by inputting the user behavior sequence and the user social relationship network graph into the fully connected layer, the feature sequence high-dimensional embedding vector X f is obtained; in addition, the adaptive embedding vector can uniformly capture the complex spatio-temporal correlation in the user behavior sequence and the user social relationship network graph, the first dimension T represents the time step, the second dimension N represents the node number, and the third dimension d a is the feature dimension, representing the feature representation of each time step and each spatial position node (i.e. each user in the social network graph), this flexible way fuses time and space into a unified representation to facilitate the extraction of local and global spatio-temporal features.

[0047] A22, according to the spatial embedding vector, the periodic time embedding vector, the feature sequence high-dimensional embedding vector and the adaptive embedding vector, the comprehensive embedding vector is obtained. The four embedding vectors obtained by the above step A21 are summed to obtain the comprehensive embedding vector X emb = X f + X s + X T + X a , which is the output of the embedding layer, that is, the input of the spatio-temporal attention and convolution fusion module.

[0048] A3, using the spatio-temporal attention and convolution fusion module to extract the first spatio-temporal relationship feature and the second spatio-temporal relationship feature from the comprehensive embedding vector; the first spatio-temporal relationship feature includes global time dependence relationship and local space dependence relationship; the second spatio-temporal relationship feature includes global space dependence relationship and local time dependence relationship.

[0049] The spatio-temporal attention and convolution fusion block aims to extract dynamic, nonlinear, local and global spatio-temporal dependence relationship. The comprehensive embedding vector X embAs its input, to extract the spatio-temporal relationship contained in the user behavior attribute data, it is a two-branch structure, and each branch uses attention mechanism and convolution operation to capture time or space dependence respectively. In the embodiment, the spatio-temporal attention and convolution fusion module includes a first feature extraction branch and a second feature extraction branch, as shown in Figure 2 ; the first feature extraction branch includes a time attention sub-module and a spatial convolution sub-module, and generates features mainly with local spatial dependence and global temporal dependence; the second feature extraction branch includes a spatial attention sub-module and a temporal convolution sub-module, and generates features mainly with global spatial dependence and local temporal dependence. As shown in the flow chart Figure 3 , step A3 specifically includes the following steps:

[0050] A31, input the comprehensive embedding vector into the time attention sub-module to extract the first time feature containing global temporal dependence. Specifically, in the time attention sub-module, the multi-head attention mechanism is used to capture the global temporal dependence of the user behavior sequence over time.

[0051] The comprehensive embedding vector X emb After input, the multi-head attention mechanism is used to capture the user behavior pattern over time, and when performing dot product calculation of the similarity between Query (Q) matrix and Key (K) matrix, we integrate the local context information by convolving Q and K. As shown in the following formula:

[0052] Q1=Θ*X emb ,K1=Θ*X emb ,V1=X emb W V .

[0053]

[0054] H=softmax(A1)V1。

[0055] O1=Concat(H1,H2,…,H h )W O .

[0056] In the above formula, Θ represents the convolution kernel parameter, * represents the convolution operation, W V is the weight matrix of the Value matrix, d k is the dimension of the Query matrix and the Key matrix, used to scale the dot product calculation result; the time attention score A1 is obtained by calculating the similarity between the Query matrix and the Key matrix, and then the A1 is normalized using the softmax function to determine the time attention weight; H is the output feature of each head, and the output of h heads is spliced by the concatenation operation Concat, and then a linear transformation matrix WO The final output O1 of the temporal attention submodule is obtained. This output feature O1 can effectively capture the global temporal dependency in user behavior and can be further used as the input of the spatial convolution submodule.

[0057] A32. The integrated embedded vector is input into the spatial attention submodule to extract the first spatial feature containing global spatial dependencies. Specifically, in this embodiment, the spatial attention submodule uses a self-attention mechanism to capture the dynamic global spatial dependencies in the user's social relationship network graph.

[0058] In the spatial attention submodule, considering the dynamic changes in relationships between nodes over time, and the influence of geographical neighbors (direct neighbor nodes, users directly related to the target user) and semantic neighbor nodes (also known as potential neighbors, currently not directly connected to the target user but possibly indirectly connected through several users, or sharing similar behavioral attributes with the target user), a self-attention mechanism is used to capture the dynamic global spatial dependencies in the user's social relationship network graph. Additionally, besides the Query, Key, and Value matrices used to scale the dot product, a Z matrix is ​​used to preserve the original features of user nodes. The formula is shown below:

[0059] Q2 = X emb W Q K2 = X emb W K V2 = X emb W V .

[0060] Z = X emb W Z .

[0061]

[0062] H=α·softma(A2)V2+(1-α)·Z.

[0063] O2 = Concat(H1, H2, ..., H h W o .

[0064] In the above formula, X emb W is the synthesized embedding vector output by the embedding layer in step A2. Q W K W V and W Z These represent the weight matrices of the Query matrix, Key matrix, Value matrix, and Z matrix, respectively. kThe dimension of the Query and Key matrices is used to scale the dot product calculation results. The spatial attention score A2 is obtained by calculating the similarity between the Query and Key, and then normalizing A2 using the softmax function to determine the attention weights. α is the threshold, controlling the output of the attention mechanism and the proportion of the original output. H is the output feature of each head. The outputs of h heads are concatenated using the Concat operation, and then a linear transformation matrix W is applied. O This yields the output O2 of the spatial attention submodule, which captures global spatial dependencies. This output serves as the input to the temporal convolution submodule.

[0065] A33. Input the first temporal features into the spatial convolution submodule to extract the first spatiotemporal relationship features containing global temporal dependencies and local spatial dependencies. Specifically, in this embodiment, a graph convolutional neural network is used in the spatial convolution submodule to extract local spatial dependencies.

[0066] After the temporal attention submodule extracts the global temporal dependencies of the data, its output O1 is input into the spatial convolution submodule to extract the local spatial dependencies. In the spatial convolution submodule, a graph convolutional neural network (GCN) is used to extract local spatial dependencies and leverage the topological properties of the graph structure. Because eigenvalue decomposition has high computational complexity, Chebyshev multinomial approximation is used here, as shown in the following formula:

[0067]

[0068] Where, θ k Denotes the coefficients of the polynomial. For a Chebyshev polynomial of order k, there exists a Chebyshev polynomial. The relationship, g θ This represents the kernel parameters for graph convolution operations, * G This represents a convolution operation on graph G, where O1 is the input of this submodule, and also the output of the temporal attention submodule. Since the relationships between nodes in the graph change dynamically over time, the spatial attention score A2 learned by the spatial attention module is used to adjust the above formula:

[0069]

[0070] The first branch consists of the temporal attention submodule and the spatial convolution submodule, which successively extract the global temporal dependencies of user behavior attribute data and learn the local spatial dependencies. The output of the first branch is represented as O3.

[0071] A34. Input the first spatial features into the temporal convolution submodule to extract the second spatiotemporal relationship features containing local temporal dependencies and global spatial dependencies. Specifically, in this embodiment, causal convolution and dilated convolution are used in the temporal convolution submodule to extract local temporal dependencies.

[0072] Compared to traditional recurrent neural networks (RNNs) such as LSTM and GRU, causal convolutions have the advantages of fewer parameters and simpler structure. Furthermore, due to their parallel computation capability, the training time is significantly reduced. Therefore, in the temporal convolution submodule, convolution operations are chosen to replace traditional iterative RNNs. Since the historical range covered by causal convolutions is limited, dilated convolutions, by adding a dilation factor to the causal convolution, can achieve a larger receptive field (covering a wider range of moments) with a small number of stacked layers, thus capturing a broader range of temporal dependencies. This can be expressed by the following formula:

[0073]

[0074] Where O2 represents the output of the spatial attention submodule, O 2,t-d·k w represents the value output by the spatial attention submodule at time step td·k. k The weights of the convolution kernel are represented by K, the size of the convolution kernel is represented by d, and the dilation factor is represented by d. The dilation factor controls the interval between the input time steps in the convolution operation. The output of the temporal convolution submodule is represented as O4.

[0075] To more effectively learn the spatiotemporal correlations of local and global aspects, the outputs of two branches that focus on different aspects can be concatenated using the following formula as the final output of the spatiotemporal attention and convolutional fusion module.

[0076] O = Concat(O3O4).

[0077] This approach considers both the local and global spatial relationships between users, as well as the changing user behavior patterns over time, both in the short and long term. This can improve the accuracy of user profiles and the generalization ability of the model.

[0078] A4、According to the first spatio-temporal relationship feature and the second spatio-temporal relationship feature, classification and clustering are performed to determine the label of the target user and generate a character portrait of the target user. Specifically, by integrating the user behavior features extracted from the time and space dimensions, the features can be classified and clustered to generate more accurate user portraits of different categories. Classification of user features refers to predicting which behavior category or group the user belongs to, classifying user behavior, and generating labels based on behavior features. Clustering of user features refers to dividing users with similar features into several categories to generate user group portraits and find user groups with similar behavior or social relationships. In this way, by combining the user's historical behavior, social relationships, and dynamic attributes that change over time, the user's preferences, interests, and potential behavior patterns can be identified.

[0079] As an exemplary case, when the above scheme is applied to the construction of a user portrait of criminal intent based on social network behavior and access records, user behavior attribute data related to the user's potential criminal intent can be obtained from multiple sources, including the user's social network activity data (published comments, reviews, likes, and forwarded content), the user's social relationship network data (which users frequently interact with), the user's browsing records and website access data (monitoring specific websites frequently visited by the user), etc. The multi-dimensional data obtained is converted into high-dimensional embedding representation, and after extracting spatio-temporal features through spatio-temporal blocks, the long-term and short-term behavior patterns of the user in the social network and the local (specific group) and global (extensive suspicious social group) dependence relationships of the user in the social network can be obtained. Based on the extracted spatio-temporal features, dynamic user labels such as whether the user is violent, an extremist thinker, or a potential criminal group member can be generated.

[0080] In addition, it also includes a variety of common scenarios, such as interest change analysis in intelligent recommendation. In order to better perform user personalized recommendation, the platform needs to capture the user's viewing preferences at different times and generate dynamic user portraits in combination with the user's social relationship network. In the travel mode analysis of an intelligent travel platform, in order to provide better taxi or navigation services, the platform needs to analyze the user's travel habits and make predictions based on historical data and relationships in the social network, as the user's travel needs are strongly influenced by time and geographic location.

[0081] The method for character profiling based on the combined spatio-temporal attention mechanism provided in the above embodiments of the present application can more accurately extract local and global spatio-temporal features in data. Specifically, the above scheme can in-depth analyze the change trend of data at different time points and spatial dimensions, not only capturing subtle local features such as sudden events in a short period of time or abnormal fluctuations of a certain specific user, but also identifying overall global patterns such as long-term trends or collective behavior in a large range. This accurate spatio-temporal feature extraction provides a solid foundation for further analysis and decision-making, ensuring a more comprehensive understanding of the internal laws of data, and thus optimizing strategies and improving the prediction ability and application effect of the model.

[0082] Based on the same inventive concept, the embodiments of the present application also provide a system for implementing the above-mentioned method for character profiling based on the combined spatio-temporal attention mechanism. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme described in the above method, so the specific limitations in one or more system embodiments provided below can refer to the limitations of the method for character profiling based on the combined spatio-temporal attention mechanism described above, which will not be repeated here.

[0083] In one exemplary embodiment, as shown in Figure 4 a system for character profiling based on a combined spatio-temporal attention mechanism is provided, comprising the following modules:

[0084] A user behavior attribute data acquisition module is configured to acquire user behavior attribute data of a target user; the user behavior attribute data includes a user behavior sequence and a user social relationship network graph.

[0085] An embedded vector representation conversion module is configured to convert the user behavior attribute data into an embedded vector representation using an embedding layer to obtain a comprehensive embedded vector.

[0086] A spatio-temporal relationship feature extraction module is configured to extract first spatio-temporal relationship features and second spatio-temporal relationship features from the comprehensive embedded vector using a spatio-temporal attention and convolution fusion module; the first spatio-temporal relationship features include global temporal dependency relationships and local spatial dependency relationships; the second spatio-temporal relationship features include global spatial dependency relationships and local temporal dependency relationships.

[0087] A user character profiling generation module is configured to classify and cluster based on the first spatio-temporal relationship features and the second spatio-temporal relationship features, determine a label of the target user, and generate a character profile of the target user.

[0088] Of course, Figure 4 the architecture shown is only exemplary, and when implementing different functions, one or at least two components of the system shown can be omitted according to actual needs. Figure 4 ​

[0089] In an example embodiment, a computer device is provided, which can be a server or a terminal, and an internal structure diagram thereof can be as shown in FIG. 1. Figure 5 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store user behavior attribute data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program, when executed by the processor, implements the character portrait method based on the combined spatio-temporal attention mechanism mentioned in the foregoing embodiments.

[0090] Those skilled in the art can understand that Figure 5 The structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0091] In an example embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method embodiments described above.

[0092] In an example embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program, when executed by a processor, implements the steps in the method embodiments described above.

[0093] In an example embodiment, a computer program product is provided, which includes a computer program. The computer program, when executed by a processor, implements the steps in the method embodiments described above.

[0094] In an exemplary embodiment, the computer program product provided adopts a two-layer structure, the front end is developed by using Vue, mainly provides the interface for user interaction with the system, including adding, deleting, modifying, searching and viewing user portrait label, character search, viewing task details and other functions, and the administrator can complete complex user label management and data query tasks by simple operation. The back end of the label management is developed by using the Spring framework, the database adopts Mysql, the back end of the portrait generation is developed by using Flask, and the database adopts neo4j. The architecture diagram of the computer program product is as shown in Figure 6 The computer program product provided for the embodiment has advantages in performance, scalability, user experience and data processing capability, and can meet the user analysis and management requirements in complex business scenarios.

[0095] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0096] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to a memory, database or other medium used in the embodiments provided by the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (Read-Only Memory, ROM), a magnetic tape, a floppy disk, a flash memory, an optical memory, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), a ferroelectric memory (Ferroelectric Random Access Memory, FRAM), a phase change memory (Phase Change Memory, PCM), a graphene memory, etc. The volatile memory can include a random access memory (Random Access Memory, RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0097] The database involved in each embodiment provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each embodiment provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0098] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as the combination does not result in a contradiction.

[0099] The principles and implementation modes of the present application are described by applying specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges will be changed according to the idea of the present application. In conclusion, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for portrait image based on combined spatio-temporal attention mechanism, characterized in that, The method comprises the following steps: obtaining user behavior attribute data of a target user; the user behavior attribute data comprises a user behavior sequence and a user social relationship network graph; the user social relationship network graph is a directed weighted graph, nodes in the user social relationship network graph represent different users, edges between nodes represent relationships between users, and the weight of the edge represents the closeness of the relationship between the two users; using an embedding layer to convert the user behavior attribute data into an embedded vector representation to obtain a comprehensive embedding vector; using a spatio-temporal attention and convolution fusion module to extract first spatio-temporal relationship features and second spatio-temporal relationship features from the comprehensive embedding vector; the first spatio-temporal relationship features include global temporal dependency relationships and local spatial dependency relationships; the second spatio-temporal relationship features include global spatial dependency relationships and local temporal dependency relationships; classifying and clustering according to the first spatio-temporal relationship features and the second spatio-temporal relationship features to determine the label of the target user and generate a character portrait of the target user; using an embedding layer to convert the user behavior attribute data into an embedded vector representation to obtain a comprehensive embedding vector, specifically comprising: using an embedding layer to extract a spatial embedding vector, a periodic time embedding vector, a feature sequence high-dimensional embedding vector, and an adaptive embedding vector according to the user behavior sequence and the user social relationship network graph; summing the spatial embedding vector, the periodic time embedding vector, the feature sequence high-dimensional embedding vector, and the adaptive embedding vector to obtain a comprehensive embedding vector; In the conversion process using the embedding layer, the spatial embedding vector is generated by performing eigenvalue decomposition on the standard Laplacian matrix of the user social relationship network graph attributes X S ; In the time dimension, considering that there are 7 days in a week and 24 hours in a day, the day-of-week and time-of-day vector representations are obtained using one-hot encoding, and the two are connected to generate a periodic time embedding vector X T ; The feature sequence high-dimensional embedding vector is obtained by inputting the user behavior sequence and the user social relationship network graph into the full connection layer X f ; In addition, the adaptive embedding vector is used to uniformly capture the complex spatio-temporal correlation in the user behavior sequence and the user social relationship network graph, the first dimension T represents the time step, the second dimension N represents the number of nodes, and the third dimension d a is the feature dimension, representing the feature representation of each time step and each spatial location node. the spatio-temporal attention and convolution fusion module includes a first feature extraction branch and a second feature extraction branch; the first feature extraction branch includes a temporal attention submodule and a spatial convolution submodule; the second feature extraction branch includes a spatial attention submodule and a temporal convolution submodule; the spatio-temporal attention and convolution fusion module is used to extract first spatio-temporal relationship features and second spatio-temporal relationship features from the comprehensive embedding vector, specifically comprising: inputting the comprehensive embedding vector into the temporal attention submodule to extract first temporal features containing global temporal dependency relationships; inputting the comprehensive embedding vector into the spatial attention submodule to extract first spatial features containing global spatial dependency relationships; inputting the first temporal features into the spatial convolution submodule to extract first spatio-temporal relationship features containing global temporal dependency relationships and local spatial dependency relationships; inputting the first spatial features into the temporal convolution submodule to extract second spatio-temporal relationship features containing local temporal dependency relationships and global spatial dependency relationships.

2. The method of claim 1, wherein the method is based on a combination spatio-temporal attention mechanism. In the temporal attention submodule, the global temporal dependency relationships of the user behavior sequence changing over time are captured through a multi-head attention mechanism; in the spatial attention submodule, the dynamic global spatial dependency relationships in the user social relationship network graph are captured using a self-attention mechanism.

3. The method of claim 1, wherein the method is based on a combination spatio-temporal attention mechanism. In the spatial convolution submodule, a graph convolutional neural network is used to extract local spatial dependency relationships; in the temporal convolution submodule, a dilated convolution is used to extract local temporal dependency relationships.

4. A person image system based on a combined spatio-temporal attention mechanism, characterized in that, The method comprises the following steps: The user behavior attribute data acquisition module is configured to acquire user behavior attribute data of a target user; the user behavior attribute data includes a user behavior sequence and a user social relationship network graph; The user social relationship network graph is a directed and weighted graph, nodes in the user social relationship network graph represent different users, edges between nodes represent relationships between users, and weights of the edges represent the closeness of the two users; The embedded vector representation conversion module is configured to convert the user behavior attribute data into an embedded vector representation by using an embedding layer to obtain a comprehensive embedded vector; The spatio-temporal relationship feature extraction module is configured to extract first spatio-temporal relationship features and second spatio-temporal relationship features from the comprehensive embedded vector by using a spatio-temporal attention and convolution fusion module; The first spatio-temporal relationship features include global temporal dependency relationships and local spatial dependency relationships; and the second spatio-temporal relationship features include global spatial dependency relationships and local temporal dependency relationships; The user character portrait generation module is configured to classify and cluster the first spatio-temporal relationship features and the second spatio-temporal relationship features to determine a label of the target user and generate a character portrait of the target user; The embedded vector representation conversion module is configured to convert the user behavior attribute data into an embedded vector representation by using an embedding layer to obtain a comprehensive embedded vector, and specifically includes: The embedding layer is used to extract a spatial embedded vector, a periodic time embedded vector, a feature sequence high-dimensional embedded vector, and an adaptive embedded vector according to the user behavior sequence and the user social relationship network graph; The comprehensive embedded vector is obtained by summing the spatial embedded vector, the periodic time embedded vector, the feature sequence high-dimensional embedded vector, and the adaptive embedded vector; In the conversion process using the embedding layer, the spatial embedding vector is generated by performing eigenvalue decomposition on the standard Laplacian matrix of the user social relationship network graph attributes X S In the time dimension, considering that there are 7 days in a week and 24 hours in a day, the day-of-week and time-of-day vector representations are obtained using one-hot encoding, and the two are connected to generate a periodic time embedding vector X T The feature sequence high-dimensional embedding vector is obtained by inputting the user behavior sequence and the user social relationship network graph into the full connection layer X f In addition, the adaptive embedding vector is used to uniformly capture the complex spatio-temporal correlation in the user behavior sequence and the user social relationship network graph, the first dimension T represents the time step, the second dimension N represents the number of nodes, and the third dimension d a is the feature dimension, representing the feature representation of each time step and each spatial location node The spatio-temporal attention and convolution fusion module includes a first feature extraction branch and a second feature extraction branch; the first feature extraction branch includes a time attention submodule and a spatial convolution submodule; the second feature extraction branch includes a spatial attention submodule and a time convolution submodule; the spatio-temporal attention and convolution fusion module is used to extract first spatio-temporal relationship features and second spatio-temporal relationship features from the comprehensive embedded vector, and specifically includes: The comprehensive embedded vector is input into the time attention submodule to extract first time features containing global temporal dependency relationships; The comprehensive embedded vector is input into the spatial attention submodule to extract first spatial features containing global spatial dependency relationships; The first time features are input into the spatial convolution submodule to extract first spatio-temporal relationship features containing global temporal dependency relationships and local spatial dependency relationships; The first spatial features are input into the time convolution submodule to extract second spatio-temporal relationship features containing local temporal dependency relationships and global spatial dependency relationships.

5. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the character portrait method based on the combined spatio-temporal attention mechanism according to any one of claims 1-3.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the method for portrait image based on combined spatio-temporal attention mechanism in any one of claims 1-3.

7. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the method for portrait image based on combined spatio-temporal attention mechanism in any one of claims 1-3.

Images