Multi-modal and multi-dimensional network user portraying method and device and electronic equipment

Abstract

The invention provides a multi-modal and multi-dimensional network user portraying method and device and electronic equipment, and the method comprises the steps: collecting all pieces of multi-modal posting data, user information and posting behavior characteristics, and carrying out the clustering of the posting data belonging to the same network user; obtaining cognitive characteristics of each network user based on a user cognitive characteristic extraction model, and extracting demographic characteristics; and constructing multi-dimensional user portrait frame data of the network users according to the demographic characteristics, the posting behavior characteristics and the cognitive characteristics, so as to obtain network user portrait data of the network users based on the multi-dimensional user portrait frame data. According to the method and the device, the effectiveness, the accuracy and the comprehensiveness of obtaining the multi-dimensional features of the network users can be effectively improved based on cross-platform and multi-modal data, so that the accuracy and the comprehensiveness of the generated network user portraits can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of user portrait, and in particular to a multi-modal and multi-dimensional network user portrait method, device and electronic equipment. BACKGROUND

[0002] With the vigorous development of the Internet, social media and online forums have become the main platforms for people to widely express opinions, share information and interact. Social media is not only a tool for information transmission, but also to some extent, it influences and shapes group cognition. In order to deeply mine and understand the information of network users, user portrait technology can be used to characterize network users, and then according to the network user portrait, enterprises and design teams can better understand the network user demands and the design direction of products or services, and thus better provide personalized services, product recommendations and precise marketing for network users, and also can make business decisions such as ranking statistics, regional analysis, industry trends and competitor analysis according to the network user portrait.

[0003] However, the existing network user portrait method usually acquires user data for a specific platform, takes single-dimensional post data as the portrait basis and performs portrait based on text recognition. Among them, the user data acquisition method for a specific platform may lead to the incompleteness and one-sidedness of the user portrait; the user portrait method taking single-dimensional post data as the portrait basis may lead to the loss of some portrait information of the user; and the portrait method based on text recognition is easy to miss a large number of user features. Therefore, the existing network user portrait method cannot guarantee the accuracy and comprehensiveness of the network user portrait. SUMMARY

[0004] In view of this, the embodiments of the present application provide a multi-modal and multi-dimensional network user portrait method, device and electronic equipment to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present application provides a multi-modal and multi-dimensional network user portrait method, comprising:

[0006] acquiring multi-modal post data from each network social platform respectively, and collecting user information and posting behavior characteristics of each post data in the corresponding network social platform;

[0007] According to the user information and posting behavior characteristics corresponding to each post data, the post data belonging to the same network user is clustered to obtain a cross-platform post information data set containing a plurality of post data corresponding to each network user;

[0008] The cross-platform post information dataset is input into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each of the network users, and extracts the demographic characteristics corresponding to each of the network users from the user information corresponding to each of the network users.

[0009] Based on the demographic characteristics, posting behavior characteristics, and cognitive characteristics of each network user, a multi-dimensional user profile framework data is constructed for each network user, so as to generate network user profile data for each network user based on the multi-dimensional user profile framework data.

[0010] In some embodiments of this application, the step of acquiring multimodal post data from various online social platforms and collecting user information and posting behavior characteristics of each post data in its corresponding online social platform includes:

[0011] The system collects authorized multimodal post data and corresponding user information from various online social platforms. The modalities of the post data include text, images, videos, and audio. The user information includes personal information of the user and account information on the corresponding online social platform.

[0012] Each of the aforementioned post data was cleaned and standardized.

[0013] In addition, the posting behavior characteristics corresponding to each of the authorized post data are extracted from each of the various online social platforms.

[0014] In some embodiments of this application, the step of clustering post data belonging to the same network user based on the user information and posting behavior characteristics corresponding to each post data to obtain a cross-platform post information dataset containing multiple post data corresponding to each network user includes:

[0015] Based on the account information in each of the user information, account information with at least one piece of information being the same is identified as account information belonging to the same network user, and the post data corresponding to each account information belonging to the same network user is clustered to obtain cross-platform post information data groups corresponding to each of the network users.

[0016] Based on the posting behavior characteristics corresponding to each of the aforementioned post data, the posting behavior characteristics corresponding to each of the currently unclustered post data are matched with the posting behavior characteristics corresponding to the post data in each of the cross-platform post information data groups. Based on the corresponding matching results, the currently unclustered post data are added to each of the cross-platform post information data groups to form a cross-platform post information dataset containing the cross-platform post information data groups corresponding to each of the aforementioned network users.

[0017] In some embodiments of this application, the step of inputting the cross-platform post information dataset into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each of the network users respectively, and extracts the demographic characteristics corresponding to each of the network users respectively from the user information corresponding to each of the network users, includes:

[0018] Each cross-platform post information dataset corresponding to each network user is input into a preset user cognitive feature extraction model, so that the user cognitive feature extraction model outputs the cognitive features corresponding to each network user. The user cognitive feature extraction model includes a pre-trained Transformer model.

[0019] Demographic features corresponding to each network user are extracted from the user information of each network user.

[0020] In some embodiments of this application, the step of constructing multi-dimensional user profile framework data for each of the network users based on their respective demographic characteristics, posting behavior characteristics, and cognitive characteristics, and generating network user profile data for each of the network users based on the multi-dimensional user profile framework data, includes:

[0021] Based on the demographic characteristics, posting behavior characteristics, and cognitive characteristics of each of the aforementioned network users, a multi-dimensional user profile framework data is constructed for each of the aforementioned network users.

[0022] Feature analysis is performed on the posting behavior features and cognitive features in each of the multi-dimensional user profile framework data to obtain the posting behavior summary features corresponding to the posting behavior features and the cognitive summary features corresponding to the cognitive features in each of the multi-dimensional user profile framework data. The demographic features, posting behavior summary features, and cognitive summary features in the multi-dimensional user profile framework data of each network user are respectively used as the network user profile data of each network user.

[0023] In some embodiments of this application, the demographic characteristics include: occupational category, user age, and region.

[0024] In some embodiments of this application, the characteristics of the posting behavior include: topics of interest, network platform preferences, language habits, and internet time preferences.

[0025] In some embodiments of this application, the cognitive characteristics include: personality traits, attitudinal tendencies, and emotional characteristics.

[0026] Another aspect of this application provides a multimodal and multi-dimensional network user profiling device, comprising:

[0027] The cross-platform acquisition module for multimodal data is used to acquire various post data of multimodal data from various online social platforms, and to collect user information and posting behavior characteristics of each post data in the corresponding online social platform.

[0028] The user clustering module is used to cluster post data belonging to the same network user based on the user information and posting behavior characteristics corresponding to each post data, so as to obtain a cross-platform post information dataset containing multiple post data corresponding to each network user.

[0029] The multi-dimensional feature extraction module is used to input the cross-platform post information dataset into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each of the network users, and extracts the demographic characteristics corresponding to each of the network users from the user information corresponding to each of the network users.

[0030] The user profile generation module is used to construct multi-dimensional user profile framework data for each of the network users based on their respective demographic characteristics, posting behavior characteristics, and cognitive characteristics, so as to generate network user profile data for each of the network users based on the multi-dimensional user profile framework data.

[0031] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the multimodal and multidimensional network user profiling method.

[0032] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the described multimodal and multidimensional network user profiling method.

[0033] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the described multimodal and multidimensional network user profiling method.

[0034] The multimodal and multidimensional online user profiling method provided in this application obtains multimodal post data from various online social platforms, and collects user information and posting behavior characteristics of each post data in its corresponding online social platform; based on the user information and posting behavior characteristics of each post data, the post data belonging to the same online user are clustered to obtain a cross-platform post information dataset containing multiple post data corresponding to each online user; the cross-platform post information dataset is input into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each online user, and extracts the demographic characteristics corresponding to each online user from the user information corresponding to each online user; based on the demographic characteristics of each online user, the method further extracts the demographic characteristics corresponding to each online user. Demographic characteristics, posting behavior characteristics, and cognitive characteristics are used to construct multi-dimensional user profile frameworks for each of the aforementioned online users. Based on these multi-dimensional user profile frameworks, individual online user profiles are generated for each of the aforementioned online users. This approach leverages cross-platform and multi-modal data to effectively improve the effectiveness, accuracy, and comprehensiveness of acquiring multi-dimensional characteristics of online users. This, in turn, enhances the accuracy and comprehensiveness of the generated online user profiles. Consequently, it helps enterprises and design teams better understand the needs of online users and the design direction of products or services, providing more precise services to online users. This enables better personalized services, product recommendations, and targeted marketing for online users. Furthermore, it allows for business decisions based on online user profiles, such as ranking statistics, regional analysis, industry trend analysis, and competitor analysis, thereby improving the user experience for online users and service providers.

[0035] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0036] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0038] Figure 1 This is a schematic diagram of the first process of a multimodal and multidimensional network user profiling method in one embodiment of this application.

[0039] Figure 2 This is a schematic diagram of the second process of the multimodal and multidimensional network user profiling method in one embodiment of this application.

[0040] Figure 3 This is a schematic diagram of the structure of a multimodal and multidimensional network user profiling device according to an embodiment of this application.

[0041] Figure 4 This is a schematic diagram illustrating the execution logic of the multimodal and multidimensional network user profiling method in an application example of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0043] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0044] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0045] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0046] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0047] In today's social networks, users can express their opinions by posting various opinions (i.e., post data), forming a virtual public sphere that records people's thoughts, interests, emotions, and other information. This reflects a kind of group cognition, and there is a certain correlation between the use of online social platforms and the formation of group cognition. Therefore, using user profiling technology to characterize online users can not only help businesses and design teams better understand user needs and product or service design directions, thus enabling more personalized services, product recommendations, and targeted marketing, but also facilitate business decisions such as ranking statistics, regional analysis, industry trend analysis, and competitor analysis. Furthermore, analyzing social media user profiles using user profiling technology has significant theoretical and practical implications for the study of group cognition.

[0048] Existing methods for profiling online users primarily analyze user posting and comment data, collect user click and browsing behavior data, and obtain user demographic information through surveys. These analyses comprehensively infer user interests, preferences, and behavioral patterns to form user profile characteristics, completing the profile analysis for subsequent stages. This approach mainly targets specific platforms, focuses on a single dimension, and primarily uses text recognition as its analytical method. Platform-specific user profiling can lead to incomplete and one-sided profiles. Data generated by users across different platforms and channels may be scattered and inconsistent; a lack of cross-platform and comprehensive user profiles may fail to accurately reflect the overall characteristics and behaviors of users. Single-dimensional user profiling may result in the loss of certain profile information. User profile attributes, such as interests and preferences, easily change with time and environment; without analysis from multiple attributes and dimensions, these changes may not be captured in a timely manner. Current user profiling technologies mainly focus on text content recognition, but posts on various social media platforms contain a large amount of images and videos; focusing solely on text will miss a significant amount of information, reducing the accuracy of user profiles. In summary, existing user profiling technologies that target specific platforms, focus on a single dimension, and primarily rely on text recognition for analysis still have room for improvement in terms of the sources of information acquired and the breadth of identification.

[0049] Based on this, in order to address the problems of existing online user profiling methods, such as the incompleteness and one-sidedness of user profiles due to platform-specific user data acquisition methods; the potential loss of certain user profile information due to user profiling methods based on single-dimensional post data; and the tendency to miss a large number of user features due to text recognition profiling methods, this application provides a multimodal and multidimensional online user profiling method, a multimodal and multidimensional online user profiling device, an electronic device, a computer-readable storage medium, and a computer program product for executing the multimodal and multidimensional online user profiling method, so as to improve the comprehensiveness and accuracy of user profiles.

[0050] The following examples will provide a detailed description.

[0051] Based on this, embodiments of this application provide a multimodal and multidimensional network user profiling method that can be implemented by a multimodal and multidimensional network user profiling device. See [link to relevant documentation]. Figure 1 The multimodal and multidimensional network user profiling method specifically includes the following:

[0052] Step 100: Obtain multimodal post data from each of the various online social platforms, and collect user information and posting behavior characteristics of each post data in the corresponding online social platform.

[0053] Understandably, online user profiling serves as a tool for identifying target users. By linking and converting data on online users' attributes, behaviors, and expectations to create representative virtual representations, it helps businesses and design teams better align user needs with design directions. Furthermore, the completeness and accuracy of online user profiling are crucial for businesses to achieve big data analysis, improve marketing precision and recommendation matching, ultimately aiming to enhance product and online service quality and increase business profits. In one or more embodiments of this application, online users may also be simply referred to as users.

[0054] Step 200: Based on the user information and posting behavior characteristics corresponding to each of the post data, the post data belonging to the same network user are clustered to obtain a cross-platform post information dataset containing multiple post data corresponding to each of the network users.

[0055] In other words, the cross-platform post information dataset is used to store a one-to-many correspondence between each network user's preset unique identifier (such as identity ID) and the post data.

[0056] Step 300: Input the cross-platform post information dataset into a preset user cognitive feature extraction model, so that the user cognitive feature extraction model outputs the cognitive features corresponding to each of the network users, and extracts the demographic features corresponding to each of the network users from the user information corresponding to each of the network users.

[0057] In one or more embodiments of this application, the user cognitive feature extraction model refers to an existing machine learning model that can output the cognitive features of a network user based on multiple post data of the input network user. It can be an open source model or a general deep learning model that has been trained in advance.

[0058] Step 400: Based on the demographic characteristics, posting behavior characteristics, and cognitive characteristics of each network user, construct multi-dimensional user profile framework data for each network user, and generate network user profile data for each network user based on the multi-dimensional user profile framework data.

[0059] In step 400, based on a preset multi-dimensional user profile framework, for each of the network users, their demographic characteristics, posting behavior characteristics, and cognitive characteristics are respectively filled into the multi-dimensional user profile framework framework, thereby forming multi-dimensional user profile framework data corresponding to each network user. Then, the multi-dimensional user profile framework data of the network users can be directly used as the network user profile corresponding to that network user, or some features in the multi-dimensional user profile framework data can be further analyzed, summarized, and refined to form the network user profile corresponding to that network user.

[0060] As described above, the multimodal and multidimensional network user profiling method provided in this application can effectively improve the effectiveness, accuracy, and comprehensiveness of obtaining multidimensional characteristics of network users based on cross-platform and multimodal data. This can effectively improve the accuracy and comprehensiveness of the generated network user profile, thereby helping enterprises and design teams to better understand the needs of network users and the design direction of products or services, providing more precise services to network users, and better enabling personalized services, product recommendations, and precision marketing for network users. Furthermore, it can make business decisions based on network user profiles, such as ranking statistics, regional analysis, industry trends, and competitor analysis, thereby improving the user experience for network users and service enterprises.

[0061] To further improve the comprehensiveness, effectiveness, and reliability of cross-platform multimodal data collection, and thus enhance the accuracy and comprehensiveness of online user profiling, this application provides a multimodal and multidimensional online user profiling method, see [link to relevant documentation]. Figure 2 Step 100 in the multimodal and multidimensional network user profiling method specifically includes the following:

[0062] Step 110: Collect authorized multimodal post data and corresponding user information from various online social platforms. The modalities of the post data include text, images, videos, and audio. The user information includes user personal information and account information on the corresponding online social platform.

[0063] In one or more embodiments of this application, the user's personal information may include personalized information such as the network user's occupation category, age, and location; the account information may include information such as the network user's username, email address, and phone number that can identify the user's unique account on the online social platform.

[0064] Step 120: Perform data cleaning and standardization on each of the aforementioned post data.

[0065] In one or more embodiments of this application, the modality refers to the data type, that is, the data type of the post data can be text, image, video, and audio, etc.

[0066] And, step 130: extract the posting behavior features corresponding to each of the authorized post data from each of the various online social platforms.

[0067] It should be noted that in the multimodal and multidimensional online user profiling method provided in this application, all data collected from online social platforms are publicly available data that has been authorized for collection by the online social platforms and individual users.

[0068] Specifically, in steps 110 to 130, the multimodal and multidimensional network user profiling device can collect users' personal information, account information, and corresponding post data on multiple social media platforms. The post data includes multimodal data such as text, images, videos, and audio. The raw post data needs to be cleaned, and after cleaning, all collected data needs to be standardized, including removing duplicate data, standardizing data formats, and handling missing values, to ensure the consistency of the data structure.

[0069] In other words, previous user profiling analysis techniques primarily focused on analyzing text data, thus the data acquired during the data acquisition phase was mainly text data. However, the network user profiling method provided in this application employs multimodal data analysis technology, therefore acquiring not only text data but also image data, video data, and audio data. For this data, the corresponding data formats are converted to form a dataset with a unified format.

[0070] To further improve the accuracy, effectiveness, and reliability of cross-platform user account alignment, and thus enhance the accuracy and comprehensiveness of online user profiling, this application provides a multimodal and multidimensional online user profiling method, see [link to relevant documentation]. Figure 2 Step 200 in the multimodal and multidimensional network user profiling method specifically includes the following:

[0071] Step 210: Based on the account information in each of the user information, identify account information with at least one identical piece of information as belonging to the same network user, and cluster the post data corresponding to each account information belonging to the same network user to obtain cross-platform post information data groups corresponding to each of the network users.

[0072] Step 220: Based on the posting behavior features corresponding to each of the post data, match the posting behavior features corresponding to each of the currently unclustered post data with the posting behavior features corresponding to the post data in each of the cross-platform post information data groups, and add the currently unclustered post data to each of the cross-platform post information data groups based on the matching results, so as to form a cross-platform post information dataset containing the cross-platform post information data groups corresponding to each of the network users.

[0073] Specifically, multimodal and multidimensional network user profiling devices can match and associate user accounts across different platforms, thereby identifying accounts used by the same user on different social media platforms. Cross-platform user alignment can leverage existing deep learning and natural language processing technologies to extract features from user data (such as usernames, email addresses, and phone numbers), and use existing rule-based matching algorithms and text similarity algorithms to associate accounts. Cross-platform user alignment also mines user posting behavior features, such as activity time, language habits, and content bias, to perform user account association authentication, ensuring the accuracy and reliability of cross-platform user account alignment. After completing user account alignment, a cross-platform post information dataset is built for each user, aggregating all user information and posting information for further analysis.

[0074] In user behavior analysis, association rule mining is a commonly used method. It discovers frequent data and association rules in user behavior, thereby exploring users' purchasing habits and behavioral patterns. Cluster analysis divides users into different groups, each with similar characteristics and behavioral patterns. By clustering user behavior data, cluster analysis can discover similarities and differences between users, providing a basis for personalized services and precision marketing. Besides association rule mining and cluster analysis, classification prediction is another user behavior analysis method based on big data technology. Classification prediction uses machine learning algorithms to predict future user behavior based on historical behavior data and other characteristics. Furthermore, in research on user behavior analysis methods based on big data technology, other technologies such as natural language processing, image recognition, and recommendation systems can be combined. By integrating big data technology with these technologies, the accuracy and effectiveness of user behavior analysis can be further improved.

[0075] In other words, previous user profiling analysis techniques mainly analyzed data from a single social media platform. Even when cross-platform user alignment was considered, it primarily relied on extracting account data features for alignment. This resulted in an insufficient number of accounts that could be aligned, overlooking a large number of user accounts. However, the network user profiling method provided in this application uses account alignment technology based not only on account data features but also on account behavior features. It employs big data-based behavioral analysis methods and tensor fusion network-based alignment methods to align the same user's accounts across different social media platforms, thereby increasing the accuracy of user account alignment.

[0076] To further improve the accuracy, effectiveness, and reliability of user cognitive characteristic extraction, and thus enhance the accuracy and comprehensiveness of online user profiling, this application provides a multimodal and multidimensional online user profiling method, see [link to relevant documentation]. Figure 2 Step 300 in the multimodal and multidimensional network user profiling method specifically includes the following:

[0077] Step 310: Input the cross-platform post information data group corresponding to each of the network users in the cross-platform post information dataset into the preset user cognitive feature extraction model, so that the user cognitive feature extraction model outputs the cognitive features corresponding to each of the network users respectively, wherein the user cognitive feature extraction model includes: a pre-trained Transformer model.

[0078] And, step 320: extract the demographic features corresponding to each of the network users from the user information in the user information corresponding to each of the network users.

[0079] Specifically, for cross-platform post datasets, this application uses multimodal data analysis technology to analyze user-related data. Multimodal data analysis refers to the analysis and mining of different types of data (such as text, images, audio, and video) to obtain richer and more comprehensive information and insights. Multimodal data analysis uses modality transformation technology, leveraging various data mining and data augmentation techniques to convert data types, unify data formats, and combine image and sound data to improve the accuracy of video content understanding. Multimodal user data analysis also uses sentiment analysis and emotion recognition technology, combining text content, audio tone, and image facial expressions to infer users' emotional states and emotions, and analyze their personality traits. Through multimodal user data analysis technology, all user-related data can be deeply mined and analyzed to form user profile features.

[0080] In step 320 of this application, a Transformer-based AI large model technique can be used to analyze multimodal user data. The data analysis algorithm used in this application is a large model algorithm developed for multimodal user data profiling analysis. This algorithm has constructed a dedicated training set for social media user posting data and has been specifically trained according to the needs of user profiling analysis. It can analyze multimodal content such as text, images, videos, and audio, and accurately analyze multi-dimensional user profile features.

[0081] To further improve the comprehensiveness, effectiveness, and reliability of multi-dimensional user profile construction, and thus enhance the accuracy and comprehensiveness of online user profiles, this application provides a multi-modal and multi-dimensional online user profile method, see [link to relevant documentation]. Figure 2 Step 400 in the multimodal and multidimensional network user profiling method specifically includes the following:

[0082] Step 410: Based on the demographic characteristics, posting behavior characteristics, and cognitive characteristics of each of the network users, construct multi-dimensional user profile framework data for each of the network users.

[0083] Step 420: Perform feature analysis on the posting behavior features and cognitive features in each of the multi-dimensional user profile framework data to obtain the posting behavior summary features corresponding to the posting behavior features and the cognitive summary features corresponding to the cognitive features in each of the multi-dimensional user profile framework data, so as to use the demographic features, posting behavior summary features and cognitive summary features in the multi-dimensional user profile framework data of each network user as the network user profile data of each network user.

[0084] Specifically, online user profiling is based on a multi-dimensional user profiling framework, analyzing user profiles from multiple dimensions and aspects. This multi-dimensional framework includes three categorized profile dimensions: demographic information, behavioral characteristics, and cognitive characteristics. The demographic dimension includes categories such as occupation, age, and location; users with similar demographic characteristics tend to interact more and naturally form groups. The behavioral characteristic dimension includes categories such as interests, online platform preferences, language habits, and internet usage time preferences; users with similar interests will engage in discussions and social interactions based on these shared interests. The cognitive characteristic dimension includes categories such as personality traits, attitudes, and emotional traits; users with similar cognitive characteristics are more likely to receive similar content. Then, different technical solutions are used to analyze user profile feature information for different dimensions within different multi-dimensional user profiling frameworks. For the demographic dimension, relevant information can usually be directly obtained from the personal information section of the account without further processing or analysis. For the behavioral characteristic dimension, data statistics, word frequency analysis, and other techniques are used to analyze user behavior profiles based on the user account's profile feature data. For the cognitive feature dimension, natural language processing and large-scale model analysis techniques are used to analyze various cognitive features of users to complete the user cognitive feature profile analysis. After completing the profile analysis of demographic, behavioral, and cognitive features, the results of the profile analysis of each dimension are combined to generate a multi-dimensional user profile result.

[0085] In other words, while previous user profiling analysis technologies primarily focused on demographic and behavioral dimensions, this application adds an analysis of cognitive characteristics. Cognitive characteristics characterize the personality, attitudes, and emotions of internet users, providing a more comprehensive profile of the user.

[0086] To further improve the comprehensiveness, effectiveness, and reliability of multi-dimensional user profile construction, and thus enhance the accuracy and comprehensiveness of online user profiles, this application provides a multimodal and multi-dimensional online user profile method. The demographic characteristics may include at least occupational category, user age, and location; the posting behavior characteristics may include at least topics of interest, online platform preferences, language habits, and online time preferences; and the cognitive characteristics may include at least personality traits, attitudinal tendencies, and emotional characteristics. A corresponding example of a multi-dimensional user profile framework is shown in Table 1.

[0087] Table 1 Multidimensional User Profile Framework

[0088]

[0089] From a software perspective, this application also provides an apparatus for performing all or part of the multimodal and multidimensional network user profiling method, see [link to relevant documentation]. Figure 3 The multimodal and multidimensional network user profiling device specifically includes the following components:

[0090] The cross-platform acquisition module 10 for multimodal data is used to acquire various post data of multimodal data from various online social platforms, and to collect user information and posting behavior characteristics of each post data in the corresponding online social platform.

[0091] User clustering module 20 is used to cluster post data belonging to the same network user based on the user information and posting behavior characteristics corresponding to each post data, so as to obtain a cross-platform post information dataset containing multiple post data corresponding to each network user.

[0092] The multi-dimensional feature extraction module 30 is used to input the cross-platform post information dataset into a preset user cognitive feature extraction model, so that the user cognitive feature extraction model outputs the cognitive features corresponding to each of the network users, and extracts the demographic features corresponding to each of the network users from the user information corresponding to each of the network users.

[0093] The user profile generation module 40 is used to construct multi-dimensional user profile framework data for each of the network users based on their respective demographic characteristics, posting behavior characteristics, and cognitive characteristics, so as to generate network user profile data for each of the network users based on the multi-dimensional user profile framework data.

[0094] The embodiments of the multimodal and multidimensional network user profiling device provided in this application can be used to execute the processing flow of the embodiments of the multimodal and multidimensional network user profiling method described above. Its functions will not be repeated here, but can be referred to the detailed description of the embodiments of the multimodal and multidimensional network user profiling method described above.

[0095] The multimodal and multidimensional network user profiling device can perform the multimodal and multidimensional network user profiling process either on a server or on a client device. The choice depends on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any limitations in this regard. If all operations are performed on the client device, the client device may further include a processor for the specific processing of the multimodal and multidimensional network user profiling.

[0096] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0097] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0098] As described above, the multimodal and multi-dimensional network user profiling device provided in this application embodiment can effectively improve the effectiveness, accuracy, and comprehensiveness of obtaining multi-dimensional characteristics of network users based on cross-platform and multimodal data. This can effectively improve the accuracy and comprehensiveness of the generated network user profile, thereby helping enterprises and design teams to better understand the needs of network users and the design direction of products or services, providing more precise services to network users, and better providing personalized services, product recommendations, and precision marketing. It can also make business decisions based on network user profiles, such as ranking statistics, regional analysis, industry trends, and competitor analysis, thereby improving the user experience of network users and service enterprises.

[0099] To further illustrate the embodiments of the above-mentioned multimodal and multidimensional network user profiling method and apparatus, this application also provides a specific application example of the multimodal and multidimensional network user profiling method. The basic principle of this application example is to collect post information published by different user accounts on various platforms, including text information, image information, and video information. Cross-platform user account alignment technology is used to align accounts belonging to the same user on different platforms, obtaining a multi-platform post dataset of different users. Then, multimodal content analysis technology is used to analyze the corresponding text, image, video, and audio data, conducting user profiling analysis from multiple dimensions to construct a multi-dimensional network user profile. The main innovation of this application example includes applying cross-platform user account alignment and multimodal content analysis technology to profiling analysis technology, which has not been applied in previous profiling analysis technology systems. Simultaneously, in terms of profiling dimensions, a cognitive dimension-related profile is added, which is also one of the technical innovations of this application.

[0100] See Figure 4 The multimodal and multidimensional network user profiling method provided in this application application example specifically includes the following:

[0101] (1) Multimodal data acquisition and cleaning

[0102] User personal information, account information, and corresponding post data are collected from multiple social media platforms. Post data includes multimodal data such as text, images, videos, and audio. The raw post data needs to be cleaned to improve the quality of the collected dataset. After cleaning, all collected data undergoes standardization processing, including removing duplicate data, standardizing data formats, and handling missing values, to ensure data structure consistency.

[0103] (2) Cross-platform user account alignment

[0104] Cross-platform user account alignment matches and associates user accounts across different platforms, thereby identifying accounts used by the same user on various social media platforms. Based on deep learning and natural language processing technologies, cross-platform user alignment extracts features from user data (such as usernames, email addresses, and phone numbers) and uses rule-based matching algorithms and text similarity algorithms to associate accounts. Cross-platform user alignment also mines user posting behavior characteristics, such as activity time, language habits, and content bias, to perform user account association authentication, ensuring the accuracy and reliability of cross-platform user account alignment. After completing user account alignment, a cross-platform post information dataset is built for each user, aggregating all user information and posting information for further analysis.

[0105] (3) Multimodal user data analysis

[0106] For cross-platform post datasets, this application example uses multimodal data analysis technology to analyze user-related data. Multimodal data analysis refers to the analysis and mining of different types of data (such as text, images, audio, and video) to obtain richer and more comprehensive information and insights. Multimodal data analysis uses modality transformation technology, leveraging various data mining and data augmentation techniques to convert data types, unify data formats, and combine image and audio data to improve the accuracy of video content understanding. Multimodal user data analysis also uses sentiment analysis and emotion recognition technology, combining text content, audio tone, and image facial expressions to infer users' emotional states and emotions, and analyze their personality traits. Through multimodal user data analysis technology, all user-related data can be deeply mined and analyzed to form user profile features.

[0107] (4) Construction of a multi-dimensional user profile framework

[0108] Online user profiling is based on a multi-dimensional user profiling framework, conducting user profile analysis from multiple dimensions and aspects. This multi-dimensional framework includes three categorized profiling dimensions: demographic information, behavioral characteristics, and cognitive characteristics. The demographic dimension includes categories such as occupation, age, and location; users with similar demographic characteristics tend to interact more and naturally form groups. The behavioral characteristic dimension includes categories such as interests, online platform preferences, language habits, and preferred online time; users with similar interests will engage in discussions and socialize based on these shared interests. The cognitive characteristic dimension includes categories such as personality traits, attitudinal tendencies, and emotional characteristics; users with similar cognitive characteristics are more likely to receive similar content.

[0109] (5) Multi-dimensional user profile analysis

[0110] Different technical solutions are used to analyze user profile feature information for different dimensions of various multidimensional user profiling frameworks. For the demographic dimension, relevant information can usually be directly obtained from the personal information section of the account without further processing or analysis. For the behavioral feature dimension, data statistics, word frequency analysis, and other techniques are used to analyze user behavior profiles based on the user account's profile feature data. For the cognitive feature dimension, natural language processing and large-scale model analysis techniques are used to analyze various cognitive characteristics of users to complete the user cognitive feature profile analysis. After completing the profile analysis of the demographic, behavioral, and cognitive feature dimensions, the results of the profile analysis of each dimension are combined to generate a multidimensional user profile result.

[0111] In summary, the key points of the multimodal and multidimensional web user profiling method provided in this application's application examples are cross-platform user account alignment, multimodal user data analysis, and multidimensional user profile construction. Specifically:

[0112] 1. Cross-platform user account alignment

[0113] Cross-platform user account alignment technology ensures the comprehensiveness of user-related information collection in the application examples of this application, and is a key foundation for multi-dimensional and multi-modal network user profiling analysis technology.

[0114] 2. Multimodal user data analysis

[0115] Multimodal user data analysis can comprehensively analyze different types of user-related data, which is beneficial for conducting in-depth mining of user characteristics.

[0116] 3. Building Multi-Dimensional User Profiles

[0117] Multi-dimensional user profiling, based on a multi-dimensional and refined classification model of network users, is one of the core technologies of the application example in this application. It can perform multi-dimensional, three-dimensional and multi-faceted user profile analysis and construct network user profiles.

[0118] Based on this, the multimodal and multidimensional network user profiling method provided in the application examples of this application has the following beneficial effects:

[0119] 1. Taking into account the situation of multiple platforms

[0120] The multi-dimensional, multi-modal network user profiling analysis technology proposed in this application integrates information from multiple platforms. It uses cross-platform account alignment technology to collect unified user account information and posting information on various social media platforms, solving the problem of scattered and inconsistent network user data on different platforms, and ensuring the comprehensiveness and completeness of the final user profiling analysis results.

[0121] 2. Comprehensively consider multiple dimensions to create a profile

[0122] The multi-dimensional, multi-modal network user profile analysis technology proposed in this application example analyzes user profiles from multiple dimensions, making it easier to capture various user characteristics and complete multi-dimensional, three-dimensional, and multi-angle profile analysis.

[0123] 3. Employing multimodal analysis techniques

[0124] The multi-dimensional, multi-modal network user profiling analysis technology proposed in this application uses Transformer-based multi-modal data analysis technology to comprehensively analyze various types of post content on social media platforms, including text, images, videos, and audio. This can reduce the omission of information from network user posts and improve the accuracy of user profiling analysis.

[0125] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the multimodal and multi-dimensional network user profiling method mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and the memory via wired or wireless means.

[0126] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0127] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the multimodal and multi-dimensional network user profiling method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the multimodal and multi-dimensional network user profiling method in the above method embodiments.

[0128] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] The one or more modules are stored in the memory, and when executed by the processor, they perform the multimodal and multidimensional network user profiling method in the implementation embodiment.

[0130] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0131] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0132] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0133] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned multimodal and multi-dimensional network user profiling method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0134] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned multimodal and multidimensional network user profiling method.

[0135] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0136] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0137] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0138] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multi-modal and multi-dimensional network user profiling method, characterized in that, The method comprises the following steps: obtaining multi-modal post data from each network social platform, and collecting user information and posting behavior characteristics of each post data in the corresponding network social platform; clustering the post data belonging to the same network user according to the user information and posting behavior characteristics corresponding to each post data, to obtain a cross-platform post information dataset containing a plurality of post data corresponding to each network user; inputting the cross-platform post information dataset into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each network user, and extracts the demographic characteristics corresponding to each network user from the user information corresponding to each network user; constructing a multi-dimensional user portrait framework data corresponding to each network user according to the demographic characteristics, posting behavior characteristics and cognitive characteristics corresponding to each network user, and generating network user portrait data of each network user based on the multi-dimensional user portrait framework data. 2.The multi-modal and multi-dimensional network user profiling method of claim 1, wherein, The method comprises the following steps: collecting multi-modal post data and user information corresponding to each post data from each network social platform, wherein the modalities of the post data include text, pictures, videos and audio, and the user information includes user personal information and account information in the corresponding network social platform; performing data cleaning and standardization processing on each post data; extracting posting behavior characteristics corresponding to each post data from each network social platform. 3.The multi-modal and multi-dimensional network user profiling method of claim 2, wherein, The method comprises the following steps: According to the account information in each user information, the account information with at least one same information in the account information is confirmed as the account information belonging to the same network user, and the post data corresponding to the account information belonging to the same network user is clustered to obtain a cross-platform post information data group corresponding to each network user. According to the post behavior characteristics corresponding to each of the post data, the post behavior characteristics corresponding to each of the post data in each of the cross-platform post information data groups are matched, and based on the corresponding matching results, each of the current unclustered post data is added to each of the cross-platform post information data groups to form a cross-platform post information data set containing the cross-platform post information data group corresponding to each of the network users. 4.The multi-modal and multi-dimensional network user profiling method of claim 3, wherein, The cross-platform post information data set is input into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each of the network users, and the demographic characteristics corresponding to each of the network users are extracted from the user information corresponding to each of the network users. The cross-platform post information data set is input into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each of the network users, and the demographic characteristics corresponding to each of the network users are extracted from the user information corresponding to each of the network users. The cross-platform post information data set is input into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each of the network users, and the demographic characteristics corresponding to each of the network users are extracted from the user information corresponding to each of the network users. 5.The multi-modal and multi-dimensional network user profiling method of claim 1, wherein, According to the demographic characteristics, the post behavior characteristics and the cognitive characteristics corresponding to each of the network users, the multi-dimensional user portrait framework data corresponding to each of the network users is constructed respectively, and the network user portrait data of each of the network users is generated based on the multi-dimensional user portrait framework data. According to the demographic characteristics, the post behavior characteristics and the cognitive characteristics corresponding to each of the network users, the multi-dimensional user portrait framework data corresponding to each of the network users is constructed respectively, and the network user portrait data of each of the network users is generated based on the multi-dimensional user portrait framework data. The post behavior characteristics and the cognitive characteristics in each of the multi-dimensional user portrait framework data are analyzed respectively to obtain the post behavior summary characteristics corresponding to the post behavior characteristics and the cognitive summary characteristics corresponding to the cognitive characteristics in each of the multi-dimensional user portrait framework data, so that the demographic characteristics, the post behavior summary characteristics and the cognitive summary characteristics in the multi-dimensional user portrait framework data of each of the network users are respectively taken as the network user portrait data of each of the network users. 6.The multi-modal and multi-dimensional network user profiling method according to any one of claims 1 to 5, characterized in that, The demographic characteristics include occupation category, user age and region.

7. The multi-modal and multi-dimensional network user profiling method according to any one of claims 1 to 5, characterized in that, The post behavior characteristics include interest topics, network platform preferences, language habits and online time preferences.

8. The multi-modal and multi-dimensional network user profiling method according to any one of claims 1 to 5, characterized in that, The cognitive characteristics include personality characteristics, attitude tendencies and emotional characteristics.

9. A multi-modal and multi-dimensional network user profiling apparatus, characterized in that, The demographic characteristics include occupation category, user age and region. The post behavior characteristics include interest topics, network platform preferences, language habits and online time preferences. The cognitive characteristics include personality characteristics, attitude tendencies and emotional characteristics. The cross-platform acquisition module of multi-modal data is used for acquiring multi-modal post data respectively from each network social platform, and collecting user information and post behavior characteristics of each post data in the corresponding network social platform; The user clustering module is used for clustering the post data belonging to the same network user according to the user information and post behavior characteristics corresponding to each post data, so as to obtain a cross-platform post information data set containing a plurality of post data corresponding to each network user; The multi-dimensional feature extraction module is used for inputting the cross-platform post information data set into a preset user cognitive characteristic extraction model, so that the user cognitive characteristic extraction model outputs the cognitive characteristics corresponding to each network user respectively, and extracts the demographic characteristics corresponding to each network user from the user information corresponding to each network user respectively; The user portrait generation module is used for constructing a multi-dimensional user portrait framework data corresponding to each network user according to the demographic characteristics, post behavior characteristics and cognitive characteristics corresponding to each network user respectively, and generating network user portrait data of each network user based on each multi-dimensional user portrait framework data.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the multi-modal and multi-dimensional network user portrait method of any one of claims 1 to 8.

Images