Target portrait digitization method and device in combination with virtual and real space knowledge

Abstract

The invention discloses a target portrait digitization method and device in combination with virtual and real space knowledge, and belongs to the field of artificial intelligence, and the method comprises the steps: constructing a hierarchical target portrait attribute tag from the aspects of basic features, psychological features and association relationships of a target person; target user information is collected and obtained from a social network platform and a real physical domain and then is processed, so that cross-domain virtual and real identity association verification between virtual space social accounts and between physical space target users is realized; respectively constructing attribute tag instantiation methods according to the obtained target information in combination with the constructed hierarchical target portrait attribute tags; and according to the target attribute features, different visualization modes are adopted for display. The target portrait dimension is enriched, the target attribute instantiation method is expanded, and the accuracy of the target portrait is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically, to a method and device for digitizing a target image by combining virtual and real space knowledge. Background Art

[0002] With the development of Internet technology, news websites, virtual social software, social network platforms, etc. have been deeply integrated into all aspects of people's lives. As an important part of social network platforms, people, communities, institutions and other targets are constantly producing social media data, which has led to an exponential growth of social media data on the Internet. Data filled with violence, illegality, pornography, vulgarity, malicious smears, false information, etc. are also flooding. In order to improve the quality and effectiveness of early detection and early prevention, it is necessary to start from the source, combine the user's historical speech mining, forwarding, likes, comments and other network behavior analysis, abstract target attribute labels, and construct a hierarchical attribute label set that can accurately describe the target user, that is, target portrait digitization. Target portrait is an extension of character / user portrait, and the target type mainly refers to character targets. Accurate, comprehensive and fresh target portrait data plays an important role in auxiliary analysis and decision support in network monitoring, network security management, e-commerce and other fields. Therefore, the research on target portrait digitization technology has high academic research value and practical application value, and is one of the frontier hot areas of current research.

[0003] At present, user portrait technology for specific directions or simple tasks has been relatively mature. However, in application scenarios such as network monitoring and e-commerce, target portrait digitalization technology still faces a series of challenges. First, traditional target portraits are mostly carried out based on data from a certain platform, community, news website, etc. for a specific target. The dimension of the target portrait is closely related to the data source. The dimension of the target portrait from different data sources varies greatly. In particular, some leaders in some fields, company executives, traffic celebrities, and netizen leaders need to be portrayed from multiple angles and in an all-round way. This requires the acquisition of physical space and virtual space, and virtual space and virtual space information to be associated and integrated. This is also one of the key and difficult issues in building a full-dimensional digital target portrait. Secondly, the traditional method for determining the dimension of target portrait attribute labels only considers the dimensions clearly displayed by the data source, and does not conduct in-depth analysis and understanding of the target's historical speeches, likes, forwarding, comments, etc. The dimension of the target portrait is simple and single, and needs to be further enriched to improve the practicality of the target portrait. In addition, the instantiation method of the target portrait attribute label is still mainly based on attribute extraction, statistical calculation, classification and clustering. It is necessary to further combine intelligent methods such as artificial intelligence and large models to empower and increase the efficiency of target portrait digitalization. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art, combine the association of virtual and real space targets and the extraction of target attributes based on large models, and provide a target portrait digitization method and device combining virtual and real space knowledge, which enriches the target portrait dimension, expands the target attribute instantiation method, and improves the accuracy of the target portrait.

[0005] The object of the present invention is achieved through the following solutions:

[0006] A method for digitizing a target image by combining virtual and real space knowledge comprises the following steps:

[0007] S1, build a target portrait attribute label system based on computers: build hierarchical target portrait attribute labels from the basic characteristics, psychological characteristics and association relationships of the target person;

[0008] S2, target information acquisition and cross-domain association: collect and obtain target user information from social network platforms and real physical domains and process them to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users;

[0009] S3, target image attribute label instantiation: for the acquired target information, combined with the hierarchical target image attribute labels constructed in S1, attribute label instantiation methods are constructed respectively;

[0010] S4, target portrait visualization: different visualization methods are used to display the target according to its attribute characteristics.

[0011] Furthermore, in step S1, the basic characteristics include basic attributes, growth experience, life behavior and network behavior; the psychological characteristics include interests and hobbies, opinions, ideals and beliefs, sense of belonging, Big Five personality types and MBTI types; the association relationships include family, friends and colleagues.

[0012] Further, in step S2, the target user information is collected and obtained from the social network platform and the real physical domain and then processed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users, which specifically includes sub-steps:

[0013] First, the information data associated with each social account is classified, processed and organized into three categories: target attribute knowledge, social structure knowledge and content knowledge;

[0014] Then, based on the idea of ​​ensemble learning, the similarity scores of the three types of knowledge for each account on different platforms are comprehensively summarized;

[0015] Finally, the target cross-domain association results are obtained based on the scoring results and sorting.

[0016] Further, in step S2, the target user information is collected and obtained from the social network platform and the real physical domain and then processed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users, which specifically includes sub-steps:

[0017] Cross-platform account identity recognition process based on target attribute knowledge: By calculating the similarity between users' account attribute information, we can determine whether the accounts on different social platforms are the same target in the physical space; calculate and summarize different modes of target information separately; calculate the similarity between account social structures, and use the common neighbor indicator to measure the similarity between account social structures.

[0018] Further, in step S2, the target user information is collected and obtained from the social network platform and the real physical domain and then processed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users, which specifically includes sub-steps:

[0019] The cross-platform account identity recognition process based on content data: obtain the theme features of the target by analyzing the social account, calculate and compare the similarities between social accounts on different platforms, and complete the matching judgment of the same target in the virtual space; accurately match the key information of the target information data obtained in the physical space with the target data set after associating and summarizing the virtual social platform. If it is accurately matched, it means that the association of the virtual and real space targets has been successfully completed; if it cannot be accurately matched, the content-based topic model construction and the heterogeneous network interest embedding model calculation are completed in turn, and the target association of the virtual and real space is completed based on the calculation results and the set threshold.

[0020] Furthermore, the content-based topic model construction specifically includes the following sub-steps:

[0021] For a given social media account structure network G, the latent Dirichlet allocation LDA model is used to model the user's word set V = {V u The topics in |u∈U} are as follows:

[0022]

[0023] Among them, α and β are hyperparameters, Dir(·) represents Dirichlet distribution, and Multi(·) represents multinomial distribution. For each account u, from the parameter α(θ i ~Dir(α)) extracts its topic distribution θ from the Dirichlet distribution i ; For each word, from the user-topic distribution z ij ~Multi(θ i ) to extract the topic z ij Then, according to the subject-word dictionary Select the word w ij , subject-word dictionary φ k The parameter is β(φ k ~Dir(β)).

[0024] Furthermore, the calculation of the heterogeneous network interest embedding model specifically includes the following sub-steps: through the network embedding representation of accounts and accounts, accounts and topics, based on the idea of ​​graph similarity calculation, Sim-GNN is constructed by combining twin neural networks and graph convolutional networks to capture account similarities based on association relationships and topics to complete target cross-domain association.

[0025] Further, in step S3, the target information obtained is combined with the hierarchical target portrait attribute labels constructed in S1 to construct attribute label instantiation methods, including constructing an attribute extraction model based on contextual prompts to complete the instantiation processing of basic characteristics, interests and hobbies, opinions, ideals and beliefs, and sense of belonging of the person. The specific steps are as follows:

[0026] First, based on the large language model, the attribute prompt template is constructed in combination with the target attribute labels that can be instantiated through attribute extraction.

[0027] Then, corresponding thinking chain prompt templates are constructed for some attribute labels;

[0028] Then, based on the attribute extraction prompt template and the thought chain prompt template, the large model training data is constructed, and the large model is fine-tuned;

[0029] Finally, the target-related data is accessed and the target attribute labels are instantiated in the form of interactive questions and answers.

[0030] Further, in step S3, the target information obtained is combined with the hierarchical target portrait attribute labels constructed in S1 to construct attribute label instantiation methods, including constructing a personality trait discrimination model based on classification ideas to complete the instantiation processing of the target's Big Five personality and MBTI feature labels. The specific steps are as follows:

[0031] For multimodal data in social networks, feature extraction networks are constructed to achieve CNN-based single-modal feature extraction;

[0032] Among them, for text data, after dividing long chapters into sentences, they are encoded based on the Bert pre-training model to obtain an L×768-dimensional matrix, and then connected to three CNN network layers and maximum pooling layers respectively to obtain three 1×64-dimensional deep feature outputs. After the outputs are connected and passed through a fully connected layer with 64 neurons, a 64-dimensional text feature vector is finally obtained;

[0033] The audio data is sampled through sound waves and then input into a CNN with three convolutional layers, a global average pooling layer, and a fully connected layer with 64 neurons to extract a 64-dimensional audio waveform and intensity feature representation.

[0034] Among them, for video data, firstly, multiple key frames of the video are obtained by combining the video key frame extraction algorithm, and they are respectively input into the VGG-16 network model to obtain an N×64-dimensional feature vector; secondly, the obtained modal features are connected and input into a Contextual LSTM structure to obtain contextual association features, and a Dense network layer and a Softmax layer are added after the network to obtain 5-dimensional / 16-dimensional prediction results corresponding to three different modalities of text, audio and video; then, combined with the idea of ​​ensemble learning algorithm, based on the weights of each dimensional feature corresponding to each modality, the final result is calculated as follows: Where w represents the weight and p represents the corresponding score. Finally, the personality trait type, general description of the type and target information data of the target are combined and input into the general large language model to obtain the personality trait type and its text description of the target.

[0035] Furthermore, in step S4, different visualization methods are used to display the target attribute characteristics, which specifically includes the following sub-steps:

[0036] First, the numerical dimension, dimension type, and numerical span features are extracted for the attribute label data. At the same time, features are extracted for each visualization style in the visualization style set and a style feature library is constructed.

[0037] Then, combining the idea of ​​feature similarity, the corresponding features of the attribute label data are matched and calculated with the visualization style feature library; if the feature similarity is greater than or equal to the given threshold δ, it is considered that the attribute label data can completely match the visualization style features, and only the attribute label data and the corresponding dimensions of the visualization style need to be associated and filled; if the feature similarity is lower than the given threshold δ, the most matching visualization style is recommended to the user, and the user is supported to manually configure the style he wants to display based on the attribute label data;

[0038] Finally, the digital presentation of the target portrait is completed based on the selected visualization style.

[0039] A computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, any of the above methods is executed.

[0040] The beneficial effects of the present invention include:

[0041] (1) The method of the present invention proposes a method strategy for associating virtual and real space targets. Starting from three types of knowledge, namely target attributes, content and social structure, combined with the results of group clustering analysis, colleague relationship probability analysis, peer probability analysis, etc., an identity recognition method based on multi-dimensional criterion confidence fusion is implemented to realize cross-domain virtual and real identity association verification between virtual social accounts and real targets, improve the accuracy of multi-source target information association, and enrich the multi-dimensional portrait basic data of the target.

[0042] (2) The method of the present invention proposes a target portrait attribute label modeling method. In addition to hierarchical sorting and division of the target basic characteristics, it also adds attribute labels for psychological characteristics, making the target portrait more three-dimensional, more intuitive, and more vivid.

[0043] (3) The method of the present invention proposes a target portrait digitization method that combines virtual and real space knowledge. It can associate and fuse target information data obtained from multiple platforms and channels to enhance the richness of target information, cooperate with target portrait attribute label modeling, expand the dimension of target portrait, and design corresponding attribute label instantiation methods respectively, so as to improve the accuracy, convenience and practicality of target portrait construction, and perform visual presentation in a way that is more in line with users' information acquisition habits.

[0044] (4) The method of the present invention solves the problems existing in existing target profiling methods, such as a single source channel for target-related information, incomplete dimensions of target profiling, and a lack of methods for instantiating target profiling label attributes. At the same time, it realizes comprehensive, accurate, and digital target profiling knowledge that can cover both the real social physical domain and the cognitive domain of social platforms, so as to support the implementation of downstream tasks such as information recommendation and target lobbying. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A schematic diagram of a target portrait attribute label system according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of target information acquisition and cross-domain association principles in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the instantiation principle of the target portrait attribute label according to an embodiment of the present invention;

[0049] Figure 4This is a schematic diagram of the target portrait visualization principle of an embodiment of the present invention. DETAILED DESCRIPTION

[0050] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0051] In one embodiment, a target portrait digitization method combining virtual and real space knowledge is provided, including four parts: target portrait attribute label system, target information acquisition and cross-domain association, target portrait attribute label instantiation, and target portrait visualization.

[0052] Target portrait attribute label system: construct hierarchical target portrait attribute labels from three aspects: basic characteristics, psychological characteristics and association relationships of the target person. Among them, the basic characteristics of the person can be divided into four categories: basic attributes, growth experience, life behavior and network behavior; the psychological characteristics of the person include interests and hobbies, opinions, ideals and beliefs, sense of belonging, Big Five personality type and MBTI type; the association relationships of the person include family, friends, colleagues, etc.

[0053] Target information acquisition and cross-domain association: Target user information is collected from social networking platforms and real physical domains, and cleaned to obtain target attribute knowledge, content knowledge, and social network structure knowledge. Based on the target data, group clustering, organizational relationship probability, companion probability and other association analyses are performed, and multi-dimensional confidence fusion is performed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users.

[0054] Target portrait attribute label instantiation: Based on the acquired target information and the target portrait attribute label, attribute label instantiation methods are constructed respectively. Among them, the basic characteristics of the character, hobbies, opinions, ideals and beliefs, and sense of belonging are completed using the attribute extraction model based on contextual prompts, and the Big Five personality and MBTI types are classified by building the Big Five personality model and MBTI model based on social network data respectively. The character association relationship is completed by building a character social relationship inference model combining social network analysis and graph mining.

[0055] Target portrait visualization: Different visualization methods are used to display the target's attribute characteristics. For example, the target person's learning experience and work experience can be displayed in a timeline format, interests and hobbies can be displayed in a word cloud diagram, the Big Five personality types can be displayed in a radar chart, and character relationships can be displayed in a network diagram.

[0056] In another more detailed embodiment, the present invention provides a method for digitizing a target image by combining virtual and real space knowledge, and the specific implementation process is as follows:

[0057] 1. Target image attribute label system

[0058] like Figure 1 As shown, in view of the problem that the current dimensions of target person portraits in the industry mainly remain at the basic features, the portrait dimensions vary greatly, and the comprehensiveness needs to be further improved, in order to more scientifically and effectively carry out comprehensive and in-depth characterization, convenient management and agile iteration of key targets, and support typical tasks in different application scenarios, the present invention proposes to construct a hierarchical target portrait attribute label system from the dimensions of basic features, psychological feature information, target association relationships, etc.

[0059] In this embodiment, the basic features are divided into basic attributes of characters, growth experience characteristics, life behavior characteristics and network behavior characteristics: among them, the basic attributes of characters include identity information tags such as name, gender, nationality, hometown, date of birth, etc., contact information tags such as telephone, email, home address, etc., and personal asset information tags such as fixed assets, cash assets, and asset levels. Growth experience characteristics mainly include education experience tags such as primary school, middle school, university, and on-the-job training, work experience tags such as work unit, position, duration, and main tasks, party experience tags such as party affiliation, party position, party rewards and punishments, and political views, and professional expertise such as professional skills and professional awards. Life behavior characteristics mainly include life hobbies, travel information, consumption habits, medical information, and tainted bad record information. At the same time, each dimension also supports further subdivision. For example, life hobbies can be divided into sports hobbies, food hobbies, religious beliefs, and clothing hobbies. Similarly, network behavior characteristics include social behavior tags such as posting, forwarding, liking, and commenting, web browsing information tags such as web browsing preferences and browsing history, and network activity information tags such as platform usage, time, and related news statistics. Psychological characteristics can be divided into labels such as interests and hobbies, personality traits, opinions and speeches, ideals and beliefs, sense of belonging, emotional state, Big Five personality types, MBTI personality types, emotional state, and psychological state; association relationships can be divided into association relationship labels obtained in real life (family, couples, brothers, etc.) or in cyberspace (running friends, friends in the film and television industry, etc.), that is, physical space association relationship labels and virtual space association relationship labels.

[0060] It is worth noting that the target portrait attribute label system constructed by the present invention, which includes three dimensions: basic characteristics, psychological characteristics and correlation relationships, provides division ideas and basic segmentation plans respectively, and supports users to make adjustments according to the tasks to be processed, which greatly increases the flexibility and ease of use of the label system.

[0061] 2. Target information acquisition and cross-domain association

[0062] like Figure 2As shown, the target information acquisition and cross-domain association part, its input is the information data associated with the set of social accounts in the virtual space and the target information data collected in the physical space, and its output is the result of the association between the virtual and real space targets.

[0063] First, the information data associated with each social account is classified, processed and organized to form three categories: target attribute knowledge, social structure knowledge and content knowledge. Then, based on the idea of ​​ensemble learning, the similarity scores of the three types of knowledge for each account on different platforms are summarized. Finally, the target cross-domain association results are obtained based on the score results and sorting. Among them, the cross-platform account identity recognition method based on target attribute knowledge mainly calculates the similarity between account attribute information such as user name, nickname, date of birth, avatar, signature, contact information, etc., and then determines whether the accounts on different social platforms are the same target in the physical space. Different modes of target information are calculated and summarized separately. For example, the image similarity calculation method is used for the avatar of the social platform account, and the user name, contact information, etc. are accurately matched in the form of strings. The nickname, signature, etc. calculate the text semantic similarity, and the geographic location information uses the coordinate information for distance calculation. For the same target, there is a high probability that there will be the same friends on different social platforms. The social network accounts with more common neighbors are likely to be the same target in the real space. Therefore, the cross-platform account identification method based on social account structure information is based on this idea by calculating the similarity between the social structures of accounts, and using the common neighbor index to measure the similarity between the social structures of accounts. The cross-platform account identification method based on content data mainly obtains the target's writing characteristics, daily hobbies, focus and other thematic features by analyzing the posting content, comment information, interactive content, etc. of the social account, and calculates and compares the similarity between social accounts on different platforms to complete the matching judgment of the same target in the virtual space. The target information data obtained in the physical space is accurately matched with the target data set after the association and aggregation of the virtual social platform, such as phone number, email address, passport number and other key information. If it is accurately matched, it means that the association of the virtual and real space targets is successfully completed. If it cannot be accurately matched, the content-based topic model construction and the heterogeneous network interest embedding model calculation are completed in sequence, and the target association in the virtual and real space is completed based on the calculation results and the set threshold. Among them, when constructing a content-based topic model, it is necessary to avoid being disturbed by cross-domain content differences and to remove noise data to extract important information. For a given social media account structure network G, the Latent Dirichlet Allocation (LDA) model is used to model the user's word set V = {V u The topics in |u∈U} are as follows:

[0064]

[0065] Among them, α and β are hyperparameters, Dir(·) represents Dirichlet distribution, and Multi(·) represents multinomial distribution. For each account u, from the parameter α(θ i ~Dir(α)) extracts its topic distribution θ from the Dirichlet distribution i For each word, from the user-topic distribution z ij ~Multi(θ i ) to extract the topic z ij Then, according to the subject-word dictionary Select the word w ii , subject-word dictionary φ k The parameter is β(φ k The heterogeneous network interest embedding model uses the network embedding representation of accounts and accounts, accounts and topics, and builds Sim-GNN based on the idea of ​​graph similarity calculation, combining twin neural networks and graph convolutional networks to capture account similarity based on association relationships and topics to complete target cross-domain association.

[0066] 3. Attribute tag instantiation

[0067] like Figure 3 As shown, the attribute label instantiation method proposed in the present invention has the following specific steps: for social network target information data and physical domain target information data, combined with target portrait attribute labels, they are instantiated through attribute extraction models based on contextual cues or personality trait discrimination models based on classification ideas.

[0068] The construction process of the attribute extraction model based on contextual prompts is as follows: First, based on open source large language models such as ChatGLM, LLAMA or Baichuan, combined with the sorted target attribute labels that can be instantiated through attribute extraction (including basic features, interests and hobbies, opinions and speeches, etc.), complete the prompt template construction, for example: "{According to the target-related information content, extract the target's name, nationality,..., hometown and phone number, and output them in json format, the format is as follows: {"Name": X, "Nationality": string,..., "Hometown": string, "Phone": int}. The target-related information is as follows: "Target X related information data", the user input is as follows: "Please give the full-dimensional attribute label of target X"}". Then, for some attribute labels (for example, sense of belonging, ideals and beliefs, real needs, personality traits, etc.), they need to be obtained through reasoning, so it is necessary to construct high-quality thinking chain prompts corresponding to the attribute labels. For example, a sense of belonging is a deep psychological attitude of the target. If you want to instantiate it, you need to propose multimodal features from the multimodal data, combine statistical summary, emotion polarity change measurement, etc., to give an evaluation conclusion; among them, for text data, the main analysis is the use of vocabulary and emotion changes, for voice data, the main analysis is the changes in volume, pitch, speaking speed and other features, for image data, the movements and expression changes of the target's eyebrows, eyes, mouth and other facial parts can be analyzed, and for video data, the changes in the target's posture, movements, gestures, body shape, etc., as well as the changes in the target's eye gaze point, blinking frequency, scanning range, etc. over a period of time. The constructed prompt template is as follows: "Based on the target-related information content, give the target X's sense of belonging to the company, mainly evaluated from the following dimensions: When talking about company-related content, 1. The frequency of use of positive emotional words; 2. The frequency of use of positive modal particles; 3. The volume and pitch change greatly when describing the company's positive image; 4. The speech speed changes frequently and slightly when describing the company's positive image; 5. The type of expression and the intensity of change are positive; 6. The eye gaze point and blinking frequency change slightly. The target dimensions meet the situation N. When N≥5, the sense of belonging is "extremely strong"; 5>N≥3, the sense of belonging is "strong"; 3>N≥1, the sense of belonging is "general"; N<1, the sense of belonging is "weak". The target-related information is as follows: "Target X's related information content". ". Third, based on the attribute extraction prompt template and the thinking chain prompt template, a large model training data with a small amount of data is constructed, and the large model is fine-tuned to improve the attribute extraction performance. Finally, the target-related data is connected to complete the instantiation of the target attribute label in the form of interactive question and answer.

[0069] The personality trait discrimination model based on classification idea is mainly used to complete the instantiation of the target's Big Five personality and MBTI feature labels. The main steps are as follows: First, construct feature extraction networks for multimodal data in social networks to realize CNN-based single-modal feature extraction. For text data, after dividing long chapters into sentences, they are encoded based on the Bert pre-training model to obtain an L×768-dimensional matrix, and then connected to three CNN network layers and maximum pooling layers respectively to obtain three 1×64-dimensional deep feature outputs. After connecting the outputs and passing through a fully connected layer with 64 neurons, a 64-dimensional text feature vector is finally obtained; similarly, after the audio data is sampled through the sound wave, it is input into a layer containing three CNN convolutional layers, a global average pooling layer, and a fully connected layer with 64 neurons to extract a 64-dimensional audio waveform and intensity feature representation; for video data, firstly, multiple key frames of the video are obtained by combining the video key frame extraction algorithm, and then they are input into the VGG-16 network model to obtain an N×64-dimensional feature vector. Secondly, the obtained features of each modality are connected and input into a Contextual LSTM structure to obtain contextual features, and a Dense network layer and a Softmax layer are added after the network to obtain 5-dimensional / 16-dimensional prediction results corresponding to three different modalities of text, audio and video. Then, combined with the idea of ​​ensemble learning algorithm, based on the weights of each dimensional feature corresponding to each modality, the final result is calculated as: Among them, w represents the weight and p represents the corresponding score. Finally, the personality trait type of the target, the general description of the type and the target information data are combined and input into the general large language model to obtain the personality trait type and its text description of the target.

[0070] 4. Target portrait visualization

[0071] like Figure 4 As shown in the figure, a machine-assisted target portrait visualization processing flow is designed. First, multiple features such as numerical dimension, dimension type, numerical span, etc. are extracted for the attribute label data. At the same time, features are extracted for the visualization styles in the visualization style set and a style feature library is constructed. Then, combined with the idea of ​​feature similarity, the corresponding features of the attribute label data are matched and calculated with the visualization style feature library; if the feature similarity is greater than a given threshold δ (for example, δ = 0.97), it is considered that the attribute label data can be completely matched with the visualization style feature, and it is only necessary to associate and fill the attribute label data with the corresponding dimension of the visualization style; if the feature similarity is lower than the given threshold δ, the most matching visualization style can be recommended to the user, and the user can be supported to manually configure the style to be displayed according to the attribute label data; finally, the digital presentation of the target portrait is completed based on the selected visualization style.

[0072] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0073] According to one aspect of an embodiment of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations.

[0074] As another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

Claims

1. A method for digitizing target images by combining virtual and real space knowledge, characterized in that: The following steps are involved: S1, build a target portrait attribute label system based on computers: build hierarchical target portrait attribute labels from the basic characteristics, psychological characteristics and association relationships of the target person; S2, target information acquisition and cross-domain association: collect and obtain target user information from social network platforms and real physical domains and process them to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users; S3, target image attribute label instantiation: for the acquired target information, combined with the hierarchical target image attribute labels constructed in S1, attribute label instantiation methods are constructed respectively; S4, target portrait visualization: different visualization methods are used to display the target according to its attribute characteristics.

2. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S1, the basic characteristics include basic attributes, growth experience, life behavior and network behavior; the psychological characteristics include interests and hobbies, opinions, ideals and beliefs, sense of belonging, Big Five personality types and MBTI types; the association relationships include family, friends and colleagues.

3. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S2, the target user information is collected and obtained from the social network platform and the real physical domain and then processed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users, which specifically includes sub-steps: First, the information data associated with each social account is classified, processed and organized into three categories: target attribute knowledge, social structure knowledge and content knowledge; Then, based on the idea of ​​ensemble learning, the similarity scores of the three types of knowledge for each account on different platforms are comprehensively summarized; Finally, the target cross-domain association results are obtained based on the scoring results and sorting.

4. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S2, the target user information is collected and obtained from the social network platform and the real physical domain and then processed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users, which specifically includes sub-steps: Cross-platform account identity recognition process based on target attribute knowledge: By calculating the similarity between users' account attribute information, we can determine whether the accounts on different social platforms are the same target in the physical space; calculate and summarize different modes of target information separately; calculate the similarity between account social structures, and use the common neighbor indicator to measure the similarity between account social structures.

5. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S2, the target user information is collected and obtained from the social network platform and the real physical domain and then processed to achieve cross-domain virtual and real identity association verification between virtual space social accounts and physical space target users, which specifically includes sub-steps: Cross-platform account identification process based on content data: by analyzing social accounts to obtain the subject features of the target, calculate and compare the similarities between social accounts on different platforms to complete the matching judgment of the same target in the virtual space; The target information data obtained from the physical space is accurately matched with the target data set associated and summarized by the virtual social platform for key information. If an accurate match is achieved, it means that the association of the virtual and real space targets is successfully completed. If an accurate match cannot be achieved, the content-based topic model construction and the heterogeneous network interest embedding model calculation are completed in sequence, and the target association of the virtual and real space is completed based on the calculation results and the set threshold.

6. The method for digitizing target images by combining virtual and real space knowledge according to claim 5, characterized in that: The content-based topic model construction specifically includes the following sub-steps: For a given social media account structure network G, the latent Dirichlet allocation LDA model is used to model the user's word set V = {V u The topics in |u∈U} are as follows: Among them, α and β are hyperparameters, Dir(·) represents Dirichlet distribution, and Multi(·) represents multinomial distribution. For each account u, from the parameter α(θ i ~Dir(α)) extracts its topic distribution θ from the Dirichlet distribution i ; For each word, from the user-topic distribution z ij ~Multi(θ i ) to extract the topic z ij Then, according to the topic-word dictionary φ zij Select the word w ij , subject-word dictionary φ k The parameter is β(φ k ~Dir(β)).

7. The method for digitizing target images by combining virtual and real space knowledge according to claim 5, characterized in that: The heterogeneous network interest embedding model calculation specifically includes the following sub-steps: through the network embedding representation of accounts and accounts, accounts and topics, based on the idea of ​​graph similarity calculation, Sim-GNN is constructed by combining twin neural networks and graph convolutional networks to capture account similarities based on association relationships and topics to complete target cross-domain association.

8. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S3, the target information obtained is combined with the hierarchical target portrait attribute labels constructed in S1 to construct attribute label instantiation methods, including constructing an attribute extraction model based on contextual prompts to complete the instantiation processing of basic characteristics, interests and hobbies, opinions, ideals and beliefs, and sense of belonging of the character. The specific steps are as follows: First, based on the large language model, the attribute prompt template is constructed by combining the sorted target attribute labels that can be instantiated through attribute extraction; Then, corresponding thinking chain prompt templates are constructed for some attribute labels; Then, based on the attribute extraction prompt template and the thought chain prompt template, the large model training data is constructed, and the large model is fine-tuned; Finally, the target-related data is accessed and the target attribute labels are instantiated in the form of interactive questions and answers.

9. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S3, the target information obtained is combined with the hierarchical target portrait attribute labels constructed in S1 to construct attribute label instantiation methods, including constructing a personality trait discrimination model based on classification ideas to complete the instantiation processing of the target's Big Five personality and MBTI feature labels. The specific steps are as follows: For multimodal data in social networks, feature extraction networks are constructed to achieve CNN-based single-modal feature extraction; Among them, for text data, after dividing long chapters into sentences, they are encoded based on the Bert pre-training model to obtain an L×768-dimensional matrix, and then connected to three CNN network layers and maximum pooling layers respectively to obtain three 1×64-dimensional deep feature outputs. After the outputs are connected and passed through a fully connected layer with 64 neurons, a 64-dimensional text feature vector is finally obtained; The audio data is sampled through sound waves and then input into a CNN with three convolutional layers, a global average pooling layer, and a fully connected layer with 64 neurons to extract a 64-dimensional audio waveform and intensity feature representation. Among them, for video data, firstly, multiple key frames of the video are obtained by combining the video key frame extraction algorithm, and they are respectively input into the VGG-16 network model to obtain an N×64-dimensional feature vector; secondly, the obtained modal features are connected and input into a Contextual LSTM structure to obtain contextual association features, and a Dense network layer and a Softmax layer are added after the network to obtain 5-dimensional / 16-dimensional prediction results corresponding to three different modalities of text, audio and video; then, combined with the idea of ​​ensemble learning algorithm, based on the weights of each dimensional feature corresponding to each modality, the final result is calculated as follows: Where w represents the weight and p represents the corresponding score. Finally, the personality trait type, general description of the type and target information data of the target are combined and input into the general large language model to obtain the personality trait type and its text description of the target.

10. The method for digitizing target images by combining virtual and real space knowledge according to claim 1, characterized in that: In step S4, different visualization methods are used to display the target attribute characteristics, which specifically includes the following sub-steps: First, extract the numerical dimension, dimension type, and numerical span features for the attribute label data. At the same time, extract features for the visualization styles in the visualization style set and build a style feature library. Then, combining the idea of ​​feature similarity, the corresponding features of the attribute label data are matched and calculated with the visualization style feature library; if the feature similarity is greater than or equal to the given threshold δ, it is considered that the attribute label data can completely match the visualization style features, and only the attribute label data and the corresponding dimensions of the visualization style need to be associated and filled; if the feature similarity is lower than the given threshold δ, the most matching visualization style is recommended to the user, and the user is supported to manually configure the style he wants to display based on the attribute label data; Finally, the digital presentation of the target portrait is completed based on the selected visualization style.

11. A computer device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to any one of claims 1 to 10 is executed.

Images