A multi-modal user portrait construction method and system based on dynamic evolution map

Abstract

The application discloses a kind of multi-modal user portrait construction method and system based on dynamic evolution atlas.The method includes: obtaining the multi-modal feature data of user by explicit interaction loop and implicit behavior analysis loop;Using large language model as conflict resolver, based on the evolution logic, the relationship between new features and inventory features in atlas is determined as replacement, supplement, merge or irrelevant by semantic comparison;According to the determination result, the topology structure of portrait atlas is dynamically updated, and the obsolete features are automatically archived and the time sequence evolution link is established.The application solves the problem that the traditional user portrait can only be simply incrementally stored and cannot effectively handle the time sequence change and logic conflict of user attributes, and realizes the adaptive evolution and high consistency maintenance of user portrait.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, and in particular to a method and system for constructing multimodal user profiles based on dynamic evolutionary graphs. Background Technology

[0002] With the rapid development of artificial intelligence technology, user profiles, as the core foundation of personalized services, directly impact service accuracy and user experience through their construction quality. Currently, most mainstream user profile construction technologies employ an incremental storage model, simply accumulating various user feature data. This lacks effective management of the temporal and logical nature of these features, resulting in the following key problems: Lack of handling for time-series changes: User attributes (such as occupation, interests, and needs) change dynamically over time. Traditional methods cannot identify such changes and update the profile, resulting in a disconnect between the profile and the user's actual state, leading to problems such as service recommendation bias. Feature conflicts are difficult to resolve: User features obtained from different channels may have logical conflicts (such as the coexistence of old and new occupational information). Existing technologies lack effective conflict resolution mechanisms and can only passively retain multi-source features, which reduces the consistency and reliability of the profile. Insufficient multimodal feature fusion: Existing profile construction focuses on a single type of feature (such as text commands or behavior logs), failing to fully integrate the multimodal features of explicit commands and implicit behaviors, resulting in a single profile dimension that cannot comprehensively depict user needs; Lack of evolutionary capability: Traditional profiles are statically stored and do not have the ability to adapt and evolve according to changes in user characteristics. They require manual intervention to update, which increases the operation and maintenance costs and cannot adapt to the dynamic needs of users in real time.

[0003] Therefore, how to build a user profile system that can proactively handle temporal changes, resolve feature conflicts, integrate multimodal data, and achieve adaptive evolution has become a pressing technical challenge in the field of human-computer interaction. Summary of the Invention

[0004] This invention aims to address the problems in traditional user profile construction techniques, such as lack of handling of temporal changes, inability to resolve feature conflicts, insufficient multimodal fusion, and lack of adaptive evolution capabilities. It provides a multimodal user profile construction method and system based on dynamic evolutionary graphs, which enables dynamic updating of user profiles, maintenance of logical consistency, and adaptive evolution, thereby improving the accuracy and timeliness of the profiles.

[0005] The objective of this invention is achieved through the following technical solution: a method for constructing multimodal user profiles based on dynamic evolutionary graphs, comprising: User features are collected using an explicit command channel and an implicit behavior analysis channel. The explicit command channel is responsible for receiving natural language commands from users, while the implicit behavior analysis channel deploys a bypass behavior analysis agent to monitor the interaction data between users and the intelligent agent system in real time. When the interaction trajectory is received, a large language model is used to infer the user's potential attribute labels and form a candidate feature set containing content and timestamps. The acquired candidate features are transformed into vector representations, and semantically related existing features are retrieved from the portrait knowledge base based on semantic similarity. A mapping between candidate features and existing features is then established. Based on the decision logic of the large language model, the system loads the preset profile evolution rules, receives candidate feature and existing feature data, and outputs the evolutionary relationship between the two. The topology of the image map is dynamically updated based on the judgment results.

[0006] Furthermore, the explicit instruction channel receives natural language instructions input by the user as parameters; encapsulates the user instructions into a standardized explicit feature dictionary, which includes information such as feature type, feature content, confidence level, and current timestamp; finally, the encapsulated explicit features are added to the explicit feature storage list, and the feature dictionary is returned.

[0007] Furthermore, the implicit behavior analysis channel employs application-layer aspect-oriented programming or message middleware technology to implement implicit listening. In the backend main control program of the interactive system, a listening hook is pre-set to realize the implantation of asynchronous analysis tasks. When the main control program distributes user instructions or receives results from the tool, the behavior analysis hook is automatically triggered. Without blocking the main dialogue flow, the current context data is asynchronously captured, encapsulated into a standardized interaction log object, and the behavior analysis is performed.

[0008] Furthermore, the retrieval of semantically related existing features in the portrait knowledge base based on semantic similarity includes: It accepts the local persistent storage path as an initialization parameter, initializes the vector database client based on the path, and automatically creates a new collection if the index file does not exist under the specified path. Load a pre-trained feature vector transformation model and receive the feature content text as input parameters; encode the feature content through the initialized vector model to transform the unstructured text features into computer-processable standardized vectors; The algorithm receives candidate features as input parameters. First, it transforms the content of the candidate features into a query vector. Then, based on this vector, it performs semantic retrieval in the vector database, calculates the cosine similarity between the query vector and the existing feature vectors in the database, and retrieves the top-K most semantically similar existing features. It extracts the ID, original content, metadata including timestamps and sources, and similarity scores of these features from the database, encapsulates them into a list and returns them, and establishes an association mapping between candidate features and related existing features.

[0009] Furthermore, the decision logic of the large language model includes: built-in preset portrait evolution rules, which clearly define the criteria for determining four types of evolutionary relationships: The relationships are as follows: replacement relationship, where candidate features represent the natural changes in user state over time; supplementary relationship, where candidate features are a refinement, dimensional expansion, or supplementary explanation of existing features; merging relationship, where candidate features and existing features can be merged into a single meta-feature; and irrelevant relationship, where candidate features and existing features have no semantic connection. The system receives candidate features and existing features as input parameters; it constructs interactive instructions for the large language model, explicitly informing the model of its role as a feature conflict decision maker, and simultaneously inputs preset evolutionary rules, candidate features, and existing features; it selects the large language model to perform inference, requiring the model to output only evolutionary relationship types; it standardizes the model's returned results, converting the four types of relationships—replacement, supplementation, merging, and irrelevant—through a mapping table and passing them to the database; if the returned results are abnormal, they are defaulted to irrelevant relationships and returned.

[0010] Furthermore, the dynamic updating of the topology of the image map based on the judgment result includes: It accepts candidate features, existing features, and evolutionary relationships as input parameters; First, a unique ID for the candidate feature is generated using the UUID generation mechanism. Then, the candidate feature is transformed into a feature vector. Subsequently, candidate feature nodes are created in the vector database, and subsequent operations are performed based on the evolutionary relationship. If it is a replacement relationship and there are existing features, the state of the existing feature node is updated to archived, and a replacement time-series evolution link is established from the existing feature node to the candidate feature node, with the link carrying the current timestamp; when the archiving operation is performed, a cascading update mechanism is triggered, and the large language model is used to determine all downstream supplementary relationship nodes of the archived node, so as to realize the synchronous archiving of related features or the redirection of parent nodes. If it is a complementary relationship and there are existing features, establish a complementary association link from the existing feature node to the candidate feature node, with the link carrying the current timestamp; If there is a merging relationship and existing features exist, then a meta-strategy node is created as the parent node, and the candidate features and existing features are mounted as its child nodes. If the relationship is unrelated, only the candidate feature node is created, and no association link is established with the existing features.

[0011] According to another aspect of the specification, a multimodal user profile construction system based on dynamic evolutionary graphs for implementing the method is also provided, including: a multimodal perception module: used to collect user features through explicit command channels and implicit behavior analysis channels; The memory evolution management module is used to coordinate the retrieval, comparison, and storage processes of features, and to perform node archiving, cascading updates, or merging and reorganization based on evolutionary relationships. Conflict decision module: Built-in decision logic based on a large language model, used to output the evolution relationship between old and new features; Dynamic graph storage module: Used to store feature nodes and their edge relationships, and supports filtering and querying of archived nodes.

[0012] Furthermore, the multimodal perception module includes an explicit instruction receiving unit and an implicit behavior analysis unit; the explicit instruction receiving unit is used to receive user natural language instructions and extract explicit features; the implicit behavior analysis unit deploys a bypass behavior analysis agent to realize interactive data monitoring and potential feature inference.

[0013] Furthermore, the dynamic graph storage module adopts a vector database structure. The feature nodes stored in the vector database include id, content, type, confidence level, timestamp, vector, and status attributes. The status attributes include active status and archived status.

[0014] According to another aspect of the specification, a multimodal user profile construction device based on a dynamic evolutionary graph is also provided, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements the multimodal user profile construction method based on a dynamic evolutionary graph.

[0015] The beneficial effects of this invention are: 1. Solve the problem of temporal change: Through temporal evolution linking and archiving mechanisms, accurately capture the dynamic changes of user attributes, realize the adaptive evolution of profiles, and avoid the problem of traditional static profiles being out of touch with the actual state of users; 2. Resolve feature conflicts: Utilize a large language model to build a conflict decision-maker, and intelligently determine feature relationships based on preset evolutionary rules to ensure the logical consistency and reliability of the profile and improve the accuracy of personalized services; 3. Multimodal feature fusion: By collecting user data through explicit and implicit dual loops, and integrating multimodal features such as natural language commands, tool call trajectories, and file operation logs, user needs can be comprehensively depicted and the user profile dimensions enriched. 4. Reduced operation and maintenance costs: The system automates feature collection, conflict resolution, map updates, and archiving of outdated features without human intervention, significantly reducing the operation and maintenance costs of profile maintenance and improving the real-time performance of profile updates. 5. Flexible and scalable: The dynamic graph structure supports the addition of new feature nodes and evolutionary relationship types, which can adapt to the user profile needs in different scenarios and has strong versatility and scalability.

[0016] 6. Enhance knowledge consistency: Through cascading updates and redirection mechanisms, the problems of "isolated nodes" and "logical gaps" that are common in the dynamic evolution of knowledge graphs are solved, ensuring the structural rigor of user profiles under complex evolution paths. Attached Figure Description

[0017] Figure 1 This is a system architecture diagram for constructing multimodal user profiles based on dynamic evolutionary graphs, as presented in this invention. Figure 2 This is a flowchart of the multimodal user profile construction method based on dynamic evolutionary graph of the present invention; Figure 3 This is a schematic diagram of the internal logic of the conflict resolution module of the present invention; Figure 4 This is a schematic diagram of the topological changes in the dynamic map update of the present invention; Figure 5 This is a schematic diagram of the support device for the method of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] The core technical solution of this invention is to construct a closed-loop system of "multimodal feature acquisition - context retrieval - conflict resolution - dynamic graph update". It acquires multimodal features through explicit and implicit dual-loop acquisition, uses a large language model to resolve feature conflicts, and maintains the temporal relationship and logical consistency of features based on a dynamic evolutionary graph, as detailed below: 1. A method for constructing multimodal user profiles based on dynamic evolutionary graphs The method includes the following steps: S1. Data Acquisition: Receive natural language commands from users through explicit loops to directly acquire the attributes and needs explicitly expressed by users; collect user tool call trajectories and file operation logs through implicit loops; deploy a bypass behavior analysis agent to monitor the interaction data between users and the intelligent agent system in real time; when a specific tool call (such as file management or code retrieval) is detected, inference analysis is triggered, and the user's potential attribute labels are inferred in reverse using a large language model to form a candidate feature set containing confidence evidence. S2. Contextual Retrieval: The candidate features obtained in step S1 are converted into vector representations. Based on semantic similarity, semantically related existing features are retrieved from the portrait knowledge base, and a mapping between candidate features and existing features is established. S3. Conflict Decision: A conflict decision maker is constructed using a large language model. Candidate features and retrieved existing features are input into the decision maker, and based on preset profile evolution rules, the evolutionary relationship between the two is output. The evolutionary relationship includes at least replacement, supplementation, merging, and irrelevance. Specifically, when a candidate feature represents a natural change in a user's state over time (such as career advancement or interest shift), it is determined to be a "replacement" relationship; when a candidate feature is a detailed improvement or description of a different dimension of an existing feature, it is determined to be a "supplementation" relationship; the condition for determining a "merging" relationship is that the candidate feature and the existing feature are highly semantically related and complementary, requiring integration into a higher-level meta-feature; when the candidate feature and the existing feature have no semantic connection, it is determined to be an "irrelevance" relationship. S4. Graph Update: Dynamically update the topology of the portrait graph based on the judgment result of step S3. If the relationship is replacement, mark the existing features as archived and establish a temporal evolution link from the existing features to the candidate features. When performing the archiving operation, if the cascading update mechanism is triggered, the large language model is used to judge all downstream "supplementary" relationship nodes of the archived node to achieve synchronous archiving of related features or redirection of parent nodes. If the relationship is supplementary, establish an association link between the existing features and the candidate features. If the relationship is merged, create a meta-strategy node as the parent node and mount the candidate features and existing features as its child nodes. If the relationship is unrelated, add the candidate features as new nodes to the portrait graph and establish independent temporal nodes. At the same time, automatically archive outdated features to maintain the timeliness and simplicity of the graph.

[0020] Based on the method provided above, this embodiment also provides a system for implementing the specific operation of the method. The implementation of this system depends on the following hardware and software environment: Hardware requirements: Intel Core i7 or higher processor, ≥16GB RAM, ≥500GB hard drive, and terminal devices that support network communication; Software environment: Operating system is Windows 10 or above or Linux CentOS 7 or above, programming language is Python 3.8 or above, and the core libraries required include: large language model calling library (such as OpenAI SDK), vector processing library (such as NumPy, PyTorch), vector database driver (such as ChromaDB Client), and multi-agent orchestration framework (such as LangChain).

[0021] Environment setup steps: Install the above core dependency libraries using a Python package manager, ensuring that the versions of each library are compatible and meet the basic dependency requirements for system operation.

[0022] The specific dependency installation can be performed through the corresponding package management command, which will automatically complete the download and configuration of all core libraries without the need to manually configure the library paths.

[0023] See Figure 1 The system, built upon a multimodal user profile based on a dynamic evolutionary graph, includes the following implementation modules: (1) Implementation of multimodal sensing module The core functions of the multimodal perception module are to receive explicit instructions, listen to implicit behaviors, and infer latent features. This is implemented using class encapsulation, and the specific implementation steps are as follows: 1. Module Initialization: Receive the large language model API key as an initialization parameter and complete the initialization configuration of the large language model client using this key; at the same time, create two feature storage lists to store explicit features and implicit inference features respectively, to ensure that the two types of feature data are classified and managed and do not interfere with each other.

[0024] 2. Explicit Instruction Reception Function: Design an explicit instruction reception method to receive natural language instructions input by the user as parameters. Inside the method, the user instruction is encapsulated into a standardized explicit feature dictionary. This dictionary contains information such as feature type (marked as "explicit"), feature content (i.e., the original text of the user instruction), confidence level (the confidence level of explicit instructions is set to 1.0 by default, representing the user's explicitly expressed needs), and current timestamp. Finally, the encapsulated explicit features are added to the explicit feature storage list, and the feature dictionary is returned.

[0025] 3. Implicit Behavior Monitoring Function: Implicit monitoring is implemented using application-layer aspect-oriented programming or message middleware technology. In this embodiment, a monitoring hook is pre-set in the backend main control program of the interactive system to embed asynchronous analysis tasks. When the main control program distributes user commands or receives results from tools, the behavior analysis hook is automatically triggered. Without blocking the main dialogue flow, the current context data is asynchronously captured, encapsulated into a standardized interaction log object, and directly pushed to the behavior analysis module, avoiding the decryption cost and latency caused by underlying network packet capture.

[0026] 4. Data Packet Analysis and Latent Feature Inference: After receiving the interaction trajectory, the behavior analysis module loads a preset behavior analysis prompt template. This template defines the rules for extracting behavioral features (such as "inferring occupation from file operations" and "inferring technology stack from code queries"). The interaction trajectory is filled into the template, and a large language model is called for inference. The JSON format data returned by the model is parsed, and latent attribute labels, inference basis, and confidence levels are extracted.

[0027] 5. Candidate Feature Set Acquisition Function: Design a method for acquiring candidate feature sets. By merging the data in the explicit feature storage list and the implicit feature storage list, a complete candidate feature set is generated and returned, providing data input for the subsequent context retrieval process.

[0028] 6. Timestamp utility function: Design an internal timestamp acquisition utility function that uses the current system time to generate a standardized timestamp in the format of "year-month-day hour:minute:second", providing a unified basis for time marking of various features.

[0029] (2) Implementation of the memory evolution management module (corresponding code module: memory_manager.py) The core functions of the memory evolution management module are feature vector transformation, context retrieval, and evolutionary process coordination. It is implemented using a class encapsulation approach, and the specific implementation logic is as follows: 1. Module Initialization: Receives the local persistent storage path as an initialization parameter, initializes the vector database client based on the path, and automatically creates a new set if the index file does not exist under the specified path; at the same time, it loads a pre-trained feature vector transformation model to convert unstructured text features into computer-processable standardized vectors.

[0030] 2. Feature Vector Conversion Function: Design a feature vector conversion method that takes the feature content text as input parameter; encodes the feature content through an initialized vector model, and outputs a 384-dimensional feature vector, realizing the conversion of text features into a vector form that can be used for semantic retrieval, providing a data foundation for subsequent contextual retrieval.

[0031] 3. Contextual Retrieval Function: Design a contextual retrieval method based on vector similarity, which receives candidate features as input parameters; firstly, call the feature vector transformation method to convert the content of the candidate features into a query vector; then, perform semantic retrieval in the vector database based on this vector, calculate the cosine similarity between the query vector and the existing feature vectors in the database, and retrieve the top-K (e.g., K=10) existing features that are semantically most similar; extract the ID, original content, metadata (including timestamp and source) and similarity score of these features from the database, encapsulate them into a list and return them, establish the association mapping between candidate features and related existing features, and provide contextual basis for the subsequent "conflict decision" step.

[0032] 4. Database connection management function: Design an automatic persistence mechanism to ensure that after each write or update operation, the storage engine synchronously writes the vector index and metadata in memory to the local disk file to prevent data loss; at the same time, it provides a resource release method to properly clean up the model's occupied video memory and file handle resources when the system is shut down.

[0033] (3) Implementation of the conflict resolution module (corresponding code module: adjudicator.py) See Figure 2 The core function of the conflict resolution module is to intelligently determine the evolutionary relationship between candidate features and existing features based on a large language model. It is implemented using a class encapsulation approach, and the specific implementation logic is as follows: 1. Module Initialization: Receives the large language model API key as an initialization parameter, and completes the configuration and initialization of the large language model client using this key; it also has built-in preset profile evolution rules, which clearly define the criteria for four types of evolutionary relationships: First, replacement relationship, where candidate features represent the natural changes in user status over time (such as career advancement, interest shift, and need update); second, supplementary relationship, where candidate features are a refinement, dimensional expansion, or supplementary explanation of existing features; third, merging relationship, where candidate features and existing features can be merged into a single meta-feature; and fourth, irrelevant relationship, where candidate features and existing features have no semantic connection.

[0034] 2. Evolutionary Relationship Judgment Function: Design an evolutionary relationship judgment method that accepts candidate features and existing features as input parameters; construct interactive instructions for the large language model, explicitly informing the model of its role as a feature conflict judge, and synchronously inputting preset evolutionary rules, candidate features (including content and timestamps), and existing features (including content and timestamps); select the large language model to perform inference, requiring the model to only output the evolutionary relationship type; standardize the model's returned results, using a mapping table to convert "replace," "supplement," "merge," and "irrelevant" to their corresponding values ​​and pass them to the memory evolution management module; if the returned result is abnormal, it is judged as an irrelevant relationship by default and returned.

[0035] (4) Implementation of the dynamic graph storage module (corresponding code module: chroma_store.py) The core function of the dynamic map storage module is to realize the dynamic updating and archiving management of the image map topology. It is implemented in a class encapsulation manner, and the specific implementation logic is as follows: 1. Module Initialization: Receives the local persistent storage path as an initialization parameter, reuses the vector database client instance already initialized by the memory evolution management module, or re-establishes a read connection to the local vector index file based on this path; simultaneously, it loads the node metadata parsing configuration, used to map the unstructured metadata returned by the vector database to the node attributes and edge relationships of the graph, ensuring that the graph operation layer can correctly identify the underlying storage structure. This provides the foundation for subsequent graph operations.

[0036] 2. Atlas update function: See Figure 3The design of the graph update method accepts candidate features, existing features (None when there is no relationship), and evolutionary relationships as input parameters. First, a unique ID of the candidate feature is generated (using the UUID generation mechanism), and the feature vector transformation method of the memory evolution management module is called to obtain the 384-dimensional vector of the candidate feature. Then, a candidate feature node is created in the vector database. This node contains attributes such as ID, content, type, confidence, timestamp, vector, and state (default is set to "active"). Subsequent operations are performed according to the evolutionary relationship: if it is a replacement relationship and there is an existing feature, the state of the existing feature node is updated to "archived" and a "replacement" time-series evolution link from the existing feature node to the candidate feature node is established, with the link carrying the current timestamp. When performing an archiving operation, a cascading update mechanism is triggered. The large language model is used to determine all downstream "supplementary" relationship nodes of the archived node, so as to realize the synchronous archiving of related features or the redirection of parent nodes. If it is a supplementary relationship and there are existing features, a "supplementary" relationship link is established from the existing feature node to the candidate feature node, and the link carries the current timestamp. If it is a merge relationship and there are existing features, a meta-strategy node is created as the parent node, and the candidate feature and the existing feature are mounted as its child nodes. If it is an unrelated relationship, only the candidate feature node is created, and no relationship link with the existing feature is established.

[0037] 3. Active Feature Query Function: Design an active feature query method to filter archived features and obtain currently valid user profile features; through vector database query statements, filter feature nodes with the status of "active", extract the node ID, content, type, timestamp and confidence information, sort in descending order of timestamp and return, to ensure that the query results can reflect the latest status of users.

[0038] 4. Database connection management function: Design a database connection closing method to actively close the vector database driver connection after the module finishes running, thereby releasing system resources.

[0039] 3. Overall System Operation Flow See Figure 4 The overall operation flow of the user profile building system is as follows: It realizes the fully automated execution of the entire process from feature collection to user profile map update. The specific operation logic is as follows: 1. Module Initialization Phase: Receives the large language model API key and the local persistent storage path as input, and sequentially completes the initialization configuration of the multimodal perception module, memory evolution management module, conflict resolution module, and dynamic graph storage module. The system automatically checks whether historical profile data exists under the specified path; if it exists, the index is loaded; otherwise, a new storage set is initialized, establishing the foundation for collaborative communication between modules.

[0040] 2. Implicit Behavior Listening Startup Phase: The implicit behavior listening function of the multimodal perception module is started using a background thread mechanism. The thread is set as a daemon thread to ensure that the interaction data between the user and the intelligent agent system can be listened to in real time during the main process of the system without blocking the execution of the main process.

[0041] 3. Explicit Feature Acquisition Stage: Receive explicit natural language commands input by the user (example command is "I am a programmer and need to find information on Python code optimization"), extract explicit features through the explicit command receiving function of the multimodal perception module, and complete the storage of explicit features.

[0042] 4. Candidate Feature Set Acquisition Stage: The candidate feature set acquisition function of the multimodal perception module is invoked to merge the data in the explicit feature and implicit feature storage lists to generate a complete candidate feature set, providing unified data input for subsequent retrieval and decision-making.

[0043] 5. Retrieval-Decision-Update Closed-Loop Stage: Traverse each candidate feature in the candidate feature set and perform the following operations in sequence: First, contextual retrieval: Obtain a list of semantically related existing features through the memory evolution management module. If related features exist, select the existing feature with the highest similarity. Second, conflict decision: If related existing features exist, determine the evolutionary relationship between the two through the conflict decision module. If not, determine that they are unrelated. Third, graph update: Input the candidate features, existing features (if any), and evolutionary relationships into the dynamic graph storage module to complete the update of the graph topology.

[0044] 6. Active Profile Query Stage: Call the active feature query function of the dynamic graph storage module to obtain the user features of all users in the current active state, and output and display the user's current active profile features and corresponding confidence levels to provide a basis for personalized services.

[0045] 7. Resource Release Phase: An exception handling final execution mechanism is adopted. Regardless of whether the main process executes normally, the vector database connection between the memory evolution management module and the dynamic graph storage module is closed after the process ends to ensure normal resource release.

[0046] When the system is running, the large language model API key and vector database connection parameters need to be replaced with actual valid information to start the fully automated operation without human intervention.

[0047] Corresponding to the aforementioned embodiment of a multimodal user profile construction method based on dynamic evolutionary graphs, the present invention also provides an embodiment of a multimodal user profile construction device based on dynamic evolutionary graphs.

[0048] See Figure 5The present invention provides a multimodal user profile construction device based on dynamic evolutionary graph, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a multimodal user profile construction method based on dynamic evolutionary graph in the above embodiment.

[0049] The embodiment of the multimodal user profile construction device based on dynamic evolutionary graph provided by this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 5 The diagram shown is a hardware structure diagram of any data processing-capable device, including the multimodal user profile construction device based on dynamic evolutionary graphs provided by this invention. (Except for...) Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0050] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0051] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0052] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a method for constructing a multimodal user profile based on a dynamic evolutionary graph as described in the above embodiments.

[0053] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0054] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for constructing a multimodal user profile based on a dynamic evolutionary graph.

[0055] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0056] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for constructing multimodal user profiles based on dynamic evolutionary graphs, characterized in that, include: User characteristics are collected using explicit command channels and implicit behavior analysis channels; The explicit instruction channel is responsible for receiving natural language instructions from users, while the implicit behavior analysis channel deploys a bypass behavior analysis agent to monitor the interaction data between users and the intelligent agent system in real time. When the interaction trajectory is received, a large language model is used to infer the user's potential attribute labels and form a candidate feature set containing content and timestamps. The acquired candidate features are transformed into vector representations, and semantically related existing features are retrieved from the portrait knowledge base based on semantic similarity. A mapping between candidate features and existing features is then established. Based on the decision logic of the large language model, the system loads the preset profile evolution rules, receives candidate feature and existing feature data, and outputs the evolutionary relationship between the two. The topology of the image map is dynamically updated based on the judgment results.

2. The method for constructing a multimodal user profile based on a dynamic evolutionary graph according to claim 1, characterized in that, The explicit instruction channel receives natural language instructions input by the user as parameters; encapsulates the user instructions into a standardized explicit feature dictionary, which includes feature type, feature content, confidence level, and current timestamp information; finally, the encapsulated explicit features are added to the explicit feature storage list, and the feature dictionary is returned.

3. The method for constructing a multimodal user profile based on a dynamic evolutionary graph according to claim 1, characterized in that, The implicit behavior analysis channel employs application-layer aspect-oriented programming or message middleware technology to implement implicit listening. In the backend main control program of the interactive system, a listening hook is pre-set to realize the implantation of asynchronous analysis tasks. When the main control program distributes user instructions or receives results from the tool, the behavior analysis hook is automatically triggered. Without blocking the main dialogue flow, the current context data is asynchronously captured, encapsulated into a standardized interaction log object, and the behavior analysis is performed.

4. The method for constructing a multimodal user profile based on a dynamic evolutionary graph according to claim 1, characterized in that, The process of retrieving semantically related existing features from the portrait knowledge base based on semantic similarity includes: It accepts the local persistent storage path as an initialization parameter, initializes the vector database client based on the path, and automatically creates a new collection if the index file does not exist under the specified path. Load a pre-trained feature vector transformation model and receive the feature content text as input parameters; encode the feature content through the initialized vector model to transform the unstructured text features into computer-processable standardized vectors; The algorithm receives candidate features as input parameters. First, it transforms the content of the candidate features into a query vector. Then, based on this vector, it performs semantic retrieval in the vector database, calculates the cosine similarity between the query vector and the existing feature vectors in the database, and retrieves the top-K most semantically similar existing features. It extracts the ID, original content, metadata including timestamps and sources, and similarity scores of these features from the database, encapsulates them into a list and returns them, and establishes an association mapping between candidate features and related existing features.

5. The method for constructing a multimodal user profile based on a dynamic evolutionary graph according to claim 1, characterized in that, The decision logic of the large language model includes: built-in preset portrait evolution rules, which clearly define the criteria for determining four types of evolutionary relationships: The relationships are as follows: Substitution relationship: Candidate features represent the natural changes in user state over time; Complementary relationship: Candidate features are the refinement, dimensional expansion, or supplementary explanation of existing features; Merging relationship: Candidate features and existing features can be merged into a single meta-feature; Irrelevant relationship: Candidate features and existing features have no semantic connection. The system receives candidate features and existing features as input parameters; it constructs interactive instructions for the large language model, explicitly informing the model of its role as a feature conflict decision maker, and simultaneously inputs preset evolutionary rules, candidate features, and existing features; it selects the large language model to perform inference, requiring the model to output only evolutionary relationship types; it standardizes the model's returned results, converting the four types of relationships—replacement, supplementation, merging, and irrelevant—through a mapping table and passing them to the database; if the returned results are abnormal, they are defaulted to irrelevant relationships and returned.

6. The method for constructing a multimodal user profile based on a dynamic evolutionary graph according to claim 1, characterized in that, The topology structure for dynamically updating the image map based on the judgment result includes: It accepts candidate features, existing features, and evolutionary relationships as input parameters; First, a unique ID for the candidate feature is generated using the UUID generation mechanism. Then, the candidate feature is transformed into a feature vector. Subsequently, candidate feature nodes are created in the vector database, and subsequent operations are performed based on the evolutionary relationship. If it is a replacement relationship and there are existing features, the state of the existing feature node is updated to archived, and a replacement time-series evolution link is established from the existing feature node to the candidate feature node, with the link carrying the current timestamp; when the archiving operation is performed, a cascading update mechanism is triggered, and the large language model is used to determine all downstream supplementary relationship nodes of the archived node, so as to realize the synchronous archiving of related features or the redirection of parent nodes. If it is a complementary relationship and there are existing features, establish a complementary association link from the existing feature node to the candidate feature node, with the link carrying the current timestamp; If there is a merging relationship and existing features exist, then a meta-strategy node is created as the parent node, and the candidate features and existing features are mounted as its child nodes. If the relationship is unrelated, only the candidate feature node is created, and no association link is established with the existing features.

7. A multimodal user profile construction system based on dynamic evolutionary graphs for implementing the method of any one of claims 1-6, characterized in that, include: Multimodal perception module: used to collect user characteristics through explicit command channels and implicit behavior analysis channels; The memory evolution management module is used to coordinate the retrieval, comparison, and storage processes of features, and to perform node archiving, cascading updates, or merging and reorganization based on evolutionary relationships. Conflict decision module: Built-in decision logic based on a large language model, used to output the evolution relationship between old and new features; Dynamic graph storage module: Used to store feature nodes and their edge relationships, and supports filtering and querying of archived nodes.

8. The system according to claim 7, characterized in that, The multimodal perception module includes an explicit instruction receiving unit and an implicit behavior analysis unit; the explicit instruction receiving unit is used to receive user natural language instructions and extract explicit features; the implicit behavior analysis unit deploys a bypass behavior analysis agent to realize interactive data monitoring and potential feature inference.

9. The system according to claim 7, characterized in that, The dynamic graph storage module adopts a vector database structure. The feature nodes stored in the vector database include id, content, type, confidence, timestamp, vector, and status attributes. The status attributes include active status and archived status.

10. A multimodal user profile construction device based on dynamic evolutionary graphs, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that... When the processor executes the executable code, it implements a method for constructing a multimodal user profile based on a dynamic evolutionary graph as described in any one of claims 1-6.

Images