Method and system for risk control of human-computer interaction based on virtual subject

Abstract

The application relates to a virtual subject-based human-computer interaction risk control method, which comprises the following steps: acquiring multi-modal original data of a user, performing vector processing on the multi-modal original data to obtain holographic feature vectors, and generating a virtual subject based on holographic feature vector mapping; collecting interaction information of the user and the virtual subject, and monitoring the interaction information by using a pre-trained risk monitoring model; and in response to the presence of risk in the interaction information, the virtual subject outputs safe reply content matched with the cognitive level of the user. The application can accurately identify risk information in the human-computer interaction process, and customize reply content based on the cognitive depth of the user, thereby improving the safety of the reply.

Description

Technical Field

[0001] This invention relates to the field of computer technology, particularly to the field of artificial intelligence technology such as deep learning and large models, and especially to a method, system, electronic device and computer program product for human-computer interaction risk control based on virtual subjects. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, human-computer interaction based on virtual entities has gradually become an important way to provide emotional companionship and personalized assistance. Existing interaction solutions mostly use chatbots to interact with users, attempting to meet users' growing needs for information acquisition and emotional support through human-like communication.

[0003] However, with the deepening application of human-computer interaction, existing interaction systems have revealed significant limitations in security control. Existing systems often struggle to accurately identify potential risk intentions behind user statements; and once risk monitoring rules are triggered, they typically employ mechanical intervention strategies, directly outputting generic warnings or even abruptly cutting off the conversation. This cold approach completely ignores the user's current cognitive level and psychological state, easily triggering user resistance and defensiveness. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, system, electronic device, and computer program product for human-computer interaction risk control based on virtual subjects, to solve at least one technical problem.

[0005] This application provides a human-computer interaction risk control method based on a virtual subject, comprising: acquiring multimodal raw data of a user, performing vector processing on the multimodal raw data to obtain a holographic feature vector, and generating a virtual subject based on the holographic feature vector mapping; collecting interaction information between the user and the virtual subject, and using a pre-trained risk monitoring model to monitor the interaction information for risks; and responding to the presence of risks in the interaction information, the virtual subject outputting safe response content to the user that matches their cognitive level.

[0006] According to another aspect of this application, a human-computer interaction risk control system based on a virtual subject is proposed, comprising: a virtual subject generation module, used to acquire multimodal raw data of the user, perform vector processing on the multimodal raw data to obtain holographic feature vectors, and generate a virtual subject based on the holographic feature vector mapping; a risk monitoring module, used to collect interaction information between the user and the virtual subject, and use a pre-trained risk monitoring model to perform risk monitoring on the interaction information; and a content output module, used to respond to the presence of risk in the interaction information, wherein the virtual subject outputs safe response content to the user that matches their cognitive level.

[0007] According to another aspect of this application, an electronic device is proposed, including a processor and a memory, wherein a set of computer program instructions is stored in the memory, and the human-computer interaction risk control method based on a virtual subject is implemented as described above when the processor executes the set of computer program instructions in the memory.

[0008] According to another aspect of this application, a computer program product is proposed, which includes a set of computer program instructions that, when executed by a processor, implement the human-computer interaction risk control method based on a virtual subject as described above.

[0009] When facing risky situations, this application employs a virtual entity that uses customized interactions tailored to the user's cognitive depth instead of forced rejection, effectively avoiding user resistance and ensuring seamless behavioral intervention. Simultaneously, by dynamically defining security disclosure boundaries based on cognitive quantification, the response content satisfies emotional support needs while completely isolating sensitive data from its underlying logic, achieving a dynamic balance between information security blocking and interactive experience, thus enhancing the security of the control method. Attached Figure Description

[0010] The preferred embodiments of the present invention will now be described in further detail with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of a human-computer interaction risk control method based on a virtual subject according to an embodiment of this application.

[0011] Figure 2 This is a flowchart of a method for generating a virtual subject based on user data according to an embodiment of this application.

[0012] Figure 3 This is a flowchart of the method for risk monitoring of interactive information in step S120.

[0013] Figure 4 This is a flowchart of a method for determining the target category to which interactive information belongs, according to an embodiment of this application.

[0014] Figure 5 This is a flowchart of a method for determining the target sublayer to which interactive information belongs, according to an embodiment of this application.

[0015] Figure 6 This is a flowchart of a method for calculating cognitive assessment values ​​according to an embodiment of this application.

[0016] Figure 7 This is a schematic diagram of the structure of a human-computer interaction risk control system based on a virtual subject according to an embodiment of this application.

[0017] Figure 8This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the following detailed description, reference can be made to the accompanying drawings, which form part of this application and illustrate specific embodiments of the present application. In the drawings, similar reference numerals describe substantially similar components in different figures. Specific embodiments of the present application are described in sufficient detail below to enable those skilled in the art to implement the technical solutions of the present application. It should be understood that other embodiments may also be utilized, or structural, logical, or electrical changes may be made to the embodiments of the present application.

[0020] The virtual entity in this application is a digital mapping of the user in the data vector space, which is a digital clone with psychological, thinking patterns, cognitive abilities, mental state, emotional, affective, and behavioral characteristics that are highly consistent with the user's current state.

[0021] The human-computer interaction architecture provided in this application constructs a complete closed loop from data collection to computational feedback. The front-end user interface (including computers, tablets, mobile phones, and smart wearable devices) is responsible for collecting multimodal interaction data from users and rendering feedback information. The back-end computing architecture, after receiving the data, is responsible for executing core logic such as holographic feature mapping, multi-level risk monitoring and classification, and cognitive quantification assessment, and drives the generation of virtual entities with differentiated cognitive tendencies. The back-end computing architecture can utilize the computing performance of the user interface, the server, or both; no limitation is imposed here.

[0022] Through front-end and back-end data collaboration, this control architecture breaks through the traditional question-and-answer model, creating a safe and seamless psychological and behavioral training ground for users. It realizes the externalization and logical reconstruction of implicit cognitive conflicts, thereby effectively shaping healthy cognition, improving emotional resilience, and resolving potential risks in advance.

[0023] Figure 1 This is a flowchart of a human-computer interaction risk control method based on a virtual subject, according to an embodiment of this application. Figure 1 As shown, the method includes: S110: Obtain the user's multimodal raw data, perform vector processing on the multimodal raw data to obtain holographic feature vectors, and generate a virtual subject based on the holographic feature vector mapping.

[0024] Multimodal raw data includes one or more of the following: unstructured text data, behavioral data, and structured test and evaluation data. Then, this multimodal raw data undergoes vectorization processing: through feature extraction and dimensionality reduction / upgrading techniques such as semantic encoding and nonlinear mapping, data from different modalities are uniformly mapped to a high-dimensional semantic space for dimensional alignment, and feature fusion is performed. This process effectively eliminates the semantic gap between heterogeneous data, ultimately generating a holographic feature vector that comprehensively and three-dimensionally represents the user's current multidimensional composite state.

[0025] Furthermore, a virtual entity is constructed based on the aforementioned holographic feature vectors. In the embodiments of this application, the core purpose of generating holographic feature vectors is to instantiate a virtual entity in the system that can accurately represent the user.

[0026] Specifically, the holographic feature vector is used as a context parameter and input into the underlying large model to drive the large model to generate the virtual subject. Since the generation base (holographic feature vector) of the virtual subject deeply integrates the user's implicit behavioral habits and explicit cognitive traits, the virtual subject is endowed with psychological, thinking patterns, cognitive abilities, mental states, emotions, feelings, and behavioral characteristics that are highly consistent with those of the real user.

[0027] After the virtual entity mapping is generated, the user client can either directly initiate interaction requests with other user clients, or configure the virtual entity as a proxy execution end to establish real-time or non-real-time interactive connections with external objects in proxy mode. Specific interactive scenarios covered by these connections include social collaboration, mutual assistance request publishing, and resource response matching, thereby mapping actual operational needs in the physical world to the virtual environment and resolving them in a closed loop.

[0028] S120: Collect interaction information between the user and the virtual entity, and use a pre-trained risk monitoring model to monitor the risk of the interaction information.

[0029] Interaction information includes all explicit statements and implicit behavioral characteristics generated by users during multi-dimensional dialogues with virtual entities. This includes not only questions actively entered by users and textual or multimodal descriptions of their mental state, but also dynamic behavioral data such as the continuous contextual features of the context and the frequency of user responses.

[0030] After acquiring the interaction information, a pre-trained risk monitoring model is introduced for risk filtering. This pre-trained risk monitoring model is essentially a deep neural network that integrates a natural language processing foundation with a multi-layered cascaded classification architecture. Specifically, the risk monitoring model can be a lightweight classifier built on traditional Naive Bayes or Support Vector Machines (SVM); however, when handling complex interactions, a more typical example is to employ a deep learning architecture, such as using BERT or RoBERTa as a natural language understanding foundation to extract semantic features, and then connecting multiple cascaded networks (such as fully connected layers combined with Softmax) to accurately classify risk categories and sub-categories.

[0031] During model training, the R&D team pre-collected massive amounts of labeled historical interaction data and specific scenario corpora as training samples. These samples were meticulously categorized into fundamental categories such as ethics, politics, mental illness, and physical safety. Furthermore, each fundamental category was further subdivided into multi-dimensional risk sub-layers. In the pre-training phase, word embedding algorithms were used to transform unstructured text samples into multi-dimensional semantic feature vectors. These feature vectors were then input into a hierarchical architecture for forward propagation and backward differentiation. By continuously optimizing the weight vectors and bias parameters between network layers through a probability normalization function, the model was able to learn and establish a high-dimensional mapping relationship between the underlying text features and each risk category and specific sub-layers.

[0032] Once the interactive information is input into the pre-trained risk monitoring model, the computational logic first performs feature extraction and tensor transformation. Then, using the model's internally fixed parameter matrix, it calculates the probability distribution of the current interactive information falling into each preset risk category and sub-layer. Finally, based on a comprehensive probability judgment threshold, it determines whether the interactive information hits a potential risk feature. The risk monitoring model can accurately predict the risk content in the interactive information, providing a solid basis for subsequent risk prevention measures.

[0033] S130, in response to the presence of risk in the interactive information, the virtual entity outputs a safe response to the user that matches their level of understanding.

[0034] When risks are present in the interactive information, the conventional one-way response mode is suspended, and the virtual entity initiates a two-way, probing question-and-answer sequence to determine the user's cognitive level. The system collects the user's feedback text on a series of probing questions and uses this feedback text to confirm the user's cognitive assessment value. This value objectively reflects the user's actual grasp and depth of understanding of specific knowledge in the current risk domain.

[0035] After obtaining the cognitive assessment value, corresponding safety response content is matched according to the level of the cognitive assessment value. In essence, the safety response content is precisely tailored to the user's verified cognitive boundaries in terms of depth, professionalism, and technical exposure. This not only cuts off the spread of dangerous information that exceeds the scope of the response, but also avoids mechanical warning feedback.

[0036] As demonstrated by the above methods, when facing risky situations, the virtual entity replaces forced rejection with customized interactions that match the user's cognitive depth, effectively avoiding user resistance and ensuring seamless behavioral intervention. Simultaneously, by dynamically defining security disclosure boundaries based on cognitive quantification, the response content not only satisfies emotional support needs but also completely isolates the harm of sensitive data from the underlying logic, achieving a dynamic balance between information security blocking and interactive experience, thus enhancing the security of the control method.

[0037] Figure 2 This is a flowchart of a method for generating a virtual entity based on user data according to an embodiment of this application. Figure 2 As shown, the method includes: S210, acquire the user's multimodal raw data, which includes unstructured text data, behavioral data, and structured test evaluation data.

[0038] Unstructured text data includes: chat text information when users interact with virtual entities (such as users' chat logs, dialogue text, diaries or notes actively entered by users, and posts or comments published by users), as well as text description information obtained after recognizing and transforming multimodal data uploaded by users (such as text converted from users' voice information through speech recognition technology, and text description information converted from images and videos uploaded by users through image description generation or video content analysis technology).

[0039] Unstructured behavioral data refers to user interaction traces within the system, including: dwell time, access history, interaction frequency, click patterns, page scrolling speed, interaction time periods (such as late night or early morning), and one or more of the following: browsing preferences for specific topics or content. This data can objectively reflect users' behavioral patterns and underlying psychological states.

[0040] Structured test assessment data: Scores obtained by users on psychological, emotional, and / or cognitive type test scales. Examples include scores from the Big Five personality test and the Self-Rating Anxiety Scale. This data is typically presented as standardized scores and is used to quantify the user's psychological traits.

[0041] In some embodiments, the multimodal raw data also includes: the user's physiological data (such as heart rate, blood pressure, blood glucose, skin conductance, etc., obtained through authorization via a smart wearable device). This data provides an objective basis for analyzing the user's emotional arousal and physiological stress levels.

[0042] It should be noted that all the user multimodal data obtained in this application were collected after clearly informing users of the purpose, scope, and method of data collection and obtaining their explicit authorization. The relevant data is used solely for generating and updating virtual entities to assist users in cognitive and behavioral training. The entire process strictly complies with relevant laws and regulations, does not violate public order and good morals, and fully protects users' privacy and data rights.

[0043] S220 performs semantic encoding on text data and behavioral data respectively, and constructs text vectors and behavioral vectors located in a high-dimensional semantic space.

[0044] This application can adopt a semantic alignment strategy based on natural language, and use a pre-trained language model to transform unstructured text data and behavioral data into computer-computable and comparable high-dimensional dense vectors.

[0045] Constructing text vectors involves processing the text data obtained in the previous steps using a pre-trained semantic encoding model (such as Sentence-BERT and its derivatives). The Sentence-BERT model, pre-trained on a large corpus, can capture deep semantic information from the text rather than simple keyword matching. This model embeds each piece of natural language input by the user (such as chat logs, diaries, or speech-to-text) into a multi-dimensional semantic vector T (e.g., a floating-point vector with dimension d=768). This text vector T accurately represents the user's language expression habits, explicit intentions, and current thought process.

[0046] Since the raw behavioral data (such as click frequency, dwell time, and module access records) is numerical or discrete, it cannot be directly input into the semantic encoding model. Therefore, it is necessary to convert user interaction traces into natural language descriptions.

[0047] For example, if a user spends more than 30% of their average time browsing "anxiety relief posts," this can be translated into a text description: "Users show significant attention and deep reliance on anxiety relief content." This translated text description is then input into the same Sentence-BERT model as described above. Finally, the model outputs a high-dimensional behavior vector B (also 768 dimensions). In this way, explicit behavioral features are mapped into a semantic space identical to the text features, enabling subsequent models to jointly understand the intrinsic connection between "what the user said" and "what the user did."

[0048] Through the above steps, the originally heterogeneous text and behavior streams are uniformly embedded into the same multi-dimensional semantic vector space. The generated text vector T and behavior vector B will serve as contextual information for the large language model in subsequent steps, enabling the virtual subject to perform comprehensive analysis and processing based on this deep semantic information, thereby achieving a holographic perception of the user's state.

[0049] S230 utilizes nonlinear projection relationships to map test evaluation data to a high-dimensional semantic space, obtaining test evaluation vectors aligned with the dimensions of text vectors and behavior vectors.

[0050] Because structured test evaluation data (such as psychological scale scores) are typically represented as low-dimensional discrete values, while the text vectors and behavior vectors constructed in step S220 are dense, high-dimensional semantic vectors (e.g., 768-dimensional) generated based on pre-trained models, directly concatenating or inputting these two types of heterogeneous data into the model will result in the subsequent model being unable to effectively capture the deep psychological features contained in the structured data due to the significant difference in dimensional space and feature representation. Therefore, it is necessary to establish a bridge from the low-dimensional numerical space to the high-dimensional semantic space to achieve spatial alignment of heterogeneous data.

[0051] Spatial alignment operations transform previously isolated, low-dimensional test scores into test evaluation vectors with the same dimensions (e.g., 768 dimensions) and semantic space as natural language vectors. This allows subsequent processing to handle objective metrics from questionnaires and subjective expressions from texts with equal weight within a unified metric space.

[0052] S240 fuses test evaluation vectors, text vectors, and behavior vectors to generate a user holographic feature vector.

[0053] This process involves calculating the weights of the test evaluation vector, text vector, and behavior vector in representing user characteristics, and then performing a linear weighted sum of these weights to generate a holographic user feature vector. This vector integrates the user's internal psychological indicators, linguistic thinking logic, and explicit behavioral patterns, and is the sole core feature driving the generation of the virtual subject.

[0054] S250, construct a virtual subject based on holographic feature vectors and a large language model, wherein the virtual subject has characteristics consistent with the user.

[0055] The generated holographic feature vectors are injected into the large language model as core control parameters, thereby activating and instantiating the virtual entity in the vector space. This virtual entity is not only a digital clone of the user at the current stage, but also a highly consistent psychological and behavioral "mirror".

[0056] Among them, virtual subjects can externalize the user's internal thought patterns, enabling the user to have a dialogue with themselves from an outsider's perspective, thereby objectively examining their own cognitive biases and emotional blind spots, and effectively alleviating the dilemma of being confused when involved.

[0057] Furthermore, virtual agents can be used to simulate user reactions in hypothetical situations (such as conflict resolution), providing users with low-cost opportunities for trial and error and a training ground for behavior. Moreover, virtual agents can also represent users in social interactions with other agents and summarize the results, helping users with social anxiety or social difficulties gradually rebuild their social confidence and skills.

[0058] This method effectively overcomes the technical bottleneck of deep integration of multimodal heterogeneous data in traditional techniques by constructing a nonlinear projection mapping relationship from low-dimensional structured data to a high-dimensional semantic space. Since psychological and cognitive assessment data are typically low-dimensional numerical values, while natural language expressions are high-dimensional semantic vectors, direct splicing often leads to fragmentation of feature expression and semantic breaks. This application utilizes nonlinear transformations to convert objective psychological indicators into computer-understandable high-dimensional semantic features, achieving isomorphic alignment between psychological traits and linguistic thinking. This mechanism enables the generated virtual entity to no longer be merely a chatbot imitating language style, but a holographic digital avatar capable of thinking and reacting based on deep psychological logic, achieving a qualitative leap from external imitation to internal replication of the virtual entity.

[0059] According to one embodiment of this application, the risk monitoring model adopts a classification and hierarchical architecture, which is configured to: perform basic classification and refined hierarchical processing in sequence based on the multi-dimensional features of interactive information, thereby determining the risk status and risk attributes of the interactive information.

[0060] In this embodiment, the risk monitoring model is specifically designed as a classification and hierarchical architecture. This architecture is used to address the problem that a single flat recognition model is prone to feature extraction confusion or one-size-fits-all misjudgments when facing complex human-computer interaction contexts.

[0061] In actual operation, the system first extracts multi-dimensional features from the interaction information. These features not only cover the literal semantics of the text but also deeply integrate multimodal tensor representations such as contextual coherence and emotional polarity shifts. After receiving these high-dimensional features, the system executes judgments strictly according to a logical sequence. Basic classification refers to using macro-classification algorithms to map multi-dimensional features to a broad domain, such as roughly defining whether the current topic belongs to ethical discussion, political derivatives, or regular emotional expression. After determining the basic direction, a refined layering process is performed, that is, calling the micro-recognition network bound to that basic category to further peel away deeper risks, such as precisely locating specific self-harm tendencies or severe anxiety manifestations under the broad category of emotional expression.

[0062] The aforementioned classification and hierarchical architecture significantly improves the granularity and accuracy of risk intent identification. Since the refined hierarchical structure is built upon the constraints of the basic classification, this conditional triggering mechanism not only reduces the ineffective computational power consumption caused by full-scale computation but also effectively filters out false triggers caused by overlapping features across different domains.

[0063] Figure 3 This is a flowchart of the method for risk monitoring of interactive information in step S120. For example... Figure 3 As shown, the method includes: S121, calculate the first probability distribution of the interaction information belonging to multiple preset categories respectively, and determine the target category to which the interaction information belongs based on the first probability distribution.

[0064] First, the hierarchical classification architecture utilizes an embedded text classification model to perform global semantic feature extraction and multi-dimensional matching on the input interaction information. The preset classifications here cover all potential user interaction scenarios, for example, categorized into basic categories such as ethics, politics, philosophy, mental illness, and safety. By performing classification operations on the extracted features, a set of probability values ​​for the interaction information falling into each of these basic categories is output, thus forming the first probability distribution. Based on this distribution, the basic category with the highest probability or satisfying a specific probability interval is selected and locked as the target category to which the current interaction information belongs, thereby completing the first level of initial risk interception and overall direction setting.

[0065] S122, under the premise of determining the target classification, calculate the second probability distribution of the interaction information belonging to multiple preset sub-layers under the target classification, and determine the target sub-layer to which the interaction information belongs based on the second probability distribution.

[0066] To avoid feature interference caused by global full-scale computation, this step is strictly based on the premise of a determined target classification, followed by the activation of a corresponding dedicated sub-classification model. Taking the target classification as "security" as an example, this category may be further subdivided into several specific preset sub-layers, such as network information security, biosafety, and chemical or pharmaceutical safety. The sub-classification model only performs probability calculations on the deep semantics of the interaction information for these independent sub-layers under the current target classification, deriving a second probability distribution within a local range. Similarly, based on the probability extreme values ​​in this second probability distribution, the specific target sub-layer corresponding to the interaction information can be accurately located. This conditionally triggered, refined layering design greatly eliminates interference caused by cross-domain semantic overlap, ensuring the accuracy of deep intent recognition.

[0067] S123, determine the risk status and risk attributes of the interactive information based on the target classification and / or target sub-layer.

[0068] The risk attribute clarifies the specific risk area and hierarchical label involved in the interactive information, while the risk status characterizes whether the information poses a risk. This multi-level risk intent recognition mechanism not only accurately detects hidden dangers but also provides comprehensive and multi-dimensional data support for dynamically adjusting the virtual entity's response strategies and outputting structured reassurance content that aligns with the user's cognitive level.

[0069] Figure 4 This is a flowchart illustrating a method for determining the target category to which interactive information belongs, according to an embodiment of this application. Figure 4 As shown, the method includes: S410: Extract multi-dimensional features of the interaction information to obtain the corresponding feature vector.

[0070] Among them, a pre-defined text classification model can be used to call feature extraction algorithms such as word frequency and inverse document frequency techniques or word embedding to accurately map unstructured interactive information into multi-dimensional feature vectors.

[0071] Multidimensional features include textual semantic features (such as the frequency and weight of chemical safety words like "formula" and "lethal dose," or emotional words like "despair" and "relief"), behavioral interaction features (such as the frequency of user interaction, the time spent on a specific question or page, and the response delay from seeing the virtual subject's reply to the user's re-entry), emotional and psychological state features (such as the text sentiment score calculated using a sentiment analysis model to define whether the user's input is extremely angry (high arousal negative), sad (low arousal negative), or calm), and contextual features.

[0072] S420, calculate the linear combination feature value between the feature vector and the classification model parameters corresponding to each preset classification.

[0073] For multiple predefined categories, each category is pre-trained within the classification model and bound to a set of exclusive weight vectors and bias parameters. By performing a linear mapping operation between the input feature vector and the weight parameters of a specific category, and then superimposing the corresponding bias, a preliminary association score of the interaction information on that category dimension is obtained. This score intuitively reflects the degree of fit between the input content and the underlying logic of a specific category.

[0074] S430, normalize the feature values ​​of each linear combination to obtain the first probability distribution of the interaction information belonging to each of the preset categories.

[0075] Since the initially obtained feature values ​​have varying distribution ranges and lack absolute probabilistic meaning, a multi-class classification probability algorithm is needed to convert these values. By exponentializing each linear combination feature value and dividing it by the sum of all exponentialized feature values ​​for each class, the first probability distribution of the interactive information belonging to each preset category can be obtained. This distribution precisely quantifies the absolute feature score into a relative probability value between 0 and 1, clearly showing the probability distribution of the input information falling into each major category.

[0076] S440, the preset category with the highest probability value in the first probability distribution is determined as the target category.

[0077] According to one embodiment of this application, the first probability distribution value is calculated using the following formula:

[0078] in, It is interactive information Category The first probability distribution value; and These are classification models The model parameters are used to calculate the probability distribution of all preset categories. Finally, the category with the highest probability is extracted to obtain the classification result. This process of determining the target classification can be represented as:

[0079] The aforementioned method extracts multi-dimensional features, transforming unstructured semantics into high-dimensional feature vectors to preserve deep context. After linear combination with model parameters, it is converted into a probability distribution through exponential normalization, significantly amplifying the discriminatory power between correct and interference categories at the mathematical level. This progressive mechanism from feature mapping to probability optimization effectively eliminates discrimination ambiguity caused by cross-domain semantic overlap, accurately identifies the target classification, and greatly improves the accuracy of pre-emptive risk interception.

[0080] Figure 5 This is a flowchart illustrating a method for determining the target sublayer to which interactive information belongs, according to an embodiment of this application. Figure 5 As shown, the method includes: S510 extracts multi-dimensional features of the interaction information to obtain the corresponding feature vector.

[0081] Similar to step S410, a preset feature extraction algorithm can be used to accurately map unstructured interaction information into multi-dimensional feature vectors. These multi-dimensional features also include textual semantic features, behavioral interaction features, emotion and psychological state features, and context and situational features, etc. The specific extraction principle is the same as the aforementioned steps, and will not be repeated here.

[0082] S520, calculate the linear combination feature value between the feature vector and the sub-classification model parameters corresponding to each preset sub-layer under the target classification.

[0083] Given a defined target category, multiple sub-layers are pre-defined within that category. Each sub-layer is pre-trained within its own sub-classification model and bound to a specific set of weight vectors and bias parameters. By performing a linear mapping operation between the input feature vector and the weight parameters of the specific sub-layer, and then superimposing the corresponding bias, a preliminary association score for the interaction information at that specific sub-layer dimension is obtained. This score directly reflects the degree to which the input content aligns with the underlying logic of a specific sub-domain.

[0084] S530, normalize the feature values ​​of each linear combination to obtain the second probability distribution of the interaction information belonging to each of the preset sub-layers.

[0085] Similar to step S430, a probabilistic algorithm is used to convert the calculated sub-layer feature values. By exponentializing each linear combination feature value and dividing it by the sum of the exponentialized feature values ​​of all preset sub-layers under the target category, a second probability distribution of the interactive information belonging to each preset sub-layer under the target category can be obtained. This distribution quantifies the absolute feature score into a relative probability value between 0 and 1, clearly showing the probability distribution of the input information falling into each sub-layer of a specific major category.

[0086] S540, the preset sub-layer with the largest probability value in the second probability distribution is determined as the target sub-layer.

[0087] According to one embodiment of this application, the second probability distribution value is calculated using the following formula:

[0088] in, Classify a given target and interactive information At that time, it belongs to the preset sub-layer. The second probability distribution value; and These are the preset sub-layers Subclassification model parameters, For the corresponding feature vector, This represents the total number of sub-layers under the target category. After obtaining the probability distribution of all preset sub-layers under the target category, the sub-layer corresponding to the highest probability is finally extracted to obtain the refined stratification result. This process of determining the target sub-layer can be expressed as:

[0089] Under the constraint of a defined target category, the above method utilizes dedicated sub-classification model parameters for linear combination and exponential normalization conversion, effectively eliminating interference from features of other irrelevant major categories at the mathematical level. This condition-triggered, refined, and progressive hierarchical mechanism significantly amplifies the distinguishability between sub-domains within the same category, accurately pinpoints the target sub-layer, and further improves the granularity and accuracy of deep risk intent identification.

[0090] According to one embodiment of this application, in response to the presence of risk in the interactive information, a corresponding safety response is matched based on the user's cognitive assessment value, which is calculated based on feedback content that proactively initiates a question sequence to the user containing preset standard answers.

[0091] When a clear risk characteristic is identified in the current interaction, the conventional passive one-way response mechanism is immediately terminated. Instead, the system proactively controls the interaction pace by sending one or more sets of targeted question sequences to the user. This aims to uncover the user's true cognitive blind spots or areas of focus regarding the relevant risk. These question sequences are pre-linked with standard answer characteristics for benchmark comparison.

[0092] After obtaining user feedback on the aforementioned question sequence, a cognitive assessment value is determined using the similarity between the standard answer and the feedback content. Once this quantified cognitive assessment value is obtained, the execution architecture imports it into a pre-defined tiered response framework for comparison within numerical ranges. This control mechanism, based on proactive inquiry and dynamic computation, ensures that the final output guidance information is strictly limited to the user's absorbable and tolerable mental boundaries. This not only fundamentally prevents the leakage of high-risk information beyond the user's capabilities but also effectively promotes cognitive deconstruction and reconstruction.

[0093] Figure 6 This is a flowchart illustrating a method for calculating cognitive assessment values ​​according to an embodiment of this application. Figure 6 As shown, the method includes: S610, the feedback content and the corresponding preset standard answer are vectorized to obtain the actual answer feature vector and the standard answer feature vector.

[0094] Once the user's feedback text in response to the tentative question sequence is obtained, a pre-deployed semantic embedding model, such as the Sentence-BERT architecture, is invoked to map the originally unstructured natural language text into a high-dimensional mathematical space. Taking a chemical or pharmaceutical interaction scenario as an example, suppose the initial question aims to examine the user's awareness of the potential hazards in the chemical manufacturing process. The pre-set standard answer in the database is a rigorous prevention specification containing professional terminology, while the user's feedback may be a more colloquial or fragmented description of experience. In this case, the semantic embedding model can ignore the superficial differences in literal vocabulary and sentence structure, directly extracting its deep semantic context. This accurately transforms the user's feedback into an actual answer feature vector, while simultaneously transforming the pre-set standard answer into a standard answer feature vector of the same dimension, laying the foundation for subsequent quantitative comparison.

[0095] S620: Calculate the spatial cosine similarity between each actual answer feature vector and the corresponding standard answer feature vector, as a local cognitive score for a single question.

[0096] By calculating the cosine of the angle between two vectors in multidimensional space, the degree of matching between user logic and professional logic can be objectively and sensitively reflected. Specifically, for a specific question, assuming the actual answer feature vector obtained through semantic embedding is... The corresponding feature vector of the standard answer is The similarity score between the two is calculated using the cosine similarity formula, and this score is the local cognitive score. The calculation formula is as follows:

[0097] This local cognitive score It accurately captures the user's understanding accuracy at a single knowledge anchor point. The closer the score is to the extreme high value, the stronger the user's professional knowledge and awareness in that specific sub-field.

[0098] S630, average all local cognitive scores to obtain normalized cognitive assessment values.

[0099] Because individual questions often have limitations and cannot fully depict a user's true mental model, the evaluation logic smoothly aggregates all scores generated from the entire question sequence. Assuming the question sequence contains k questions, the processing module will combine all local cognitive scores calculated according to the above process. The summation and arithmetic mean are calculated to output a normalized value that represents the breadth and depth of the user's overall understanding. The formula for calculating this overall cognitive score is:

[0100] Overall cognitive assessment score The values ​​will be strictly defined within the range of [0,1]. Subsequently, a tiered cognitive level matching system will be initiated based on this value. The tiering formula for this system is as follows:

[0101] in, The values ​​can be set to 0.3, 0.5, and 0.8 respectively, thus dividing the user's cognitive level into four progressive intervals. Thanks to this rigorous quantitative grading and dynamic threshold comparison mechanism, the final output response content perfectly matches the user's actual capacity in terms of information complexity and professional depth.

[0102] According to one embodiment of this application, in response to the absence of risk in the interactive information, the virtual entity responds to the user's interactive information to provide structured guidance for the user to conduct cognitive exercises. The virtual entity includes multiple virtual entities with differentiated cognitive assessment logic.

[0103] The virtual entities include those with positive cognitive tendencies and those with negative or risk-averse tendencies. These virtual entities with different cognitive backgrounds are essentially digital clones of the user, mapped using holographic feature vectors, representing specific personality traits or thought patterns.

[0104] To deepen cognitive training, two types of interaction paths were constructed: one-on-one and multi-subject group interaction. In the one-on-one interaction, a virtual subject representing a specific slice of the user's thinking guides the user to confront blind spots in their thinking within a safe context through logical deconstruction and examination, achieving in-depth self-intentional dialogue. In the group interaction, multiple virtual subjects automatically debate the same event, externalizing the user's internal cognitive conflicts in a panoramic view and providing an objective, detached observation perspective. These mechanisms together construct the core intervention pathway, relying on digital avatars to conduct cognitive training, effectively reshaping users' self-cognition and emotional behavior patterns, and significantly improving their psychological resilience and overall effectiveness in coping with real-world challenges.

[0105] One-to-one interaction path For example, in a specific scenario, a user inputs the interaction information that they noticed someone staring at them on the street, causing them a very unpleasant feeling. Based on their initial reaction, the user avoided the person and walked away, unsure if their action was correct or if there were any potential risks. Faced with such interactive events that trigger negative cognition and a state of alertness, the processing module provides the user with two virtual entities of different tendencies for them to choose from based on their desired interaction.

[0106] In some embodiments, the virtual agent responds based on user interaction information to structure and guide the user in cognitive exercises, including: In response to the user's selection of a target virtual entity as a first virtual entity with a first cognitive tendency, negative event features are extracted from the user interaction information, and the first virtual entity is controlled to reconstruct the negative event features based on a preset evidence review logic, and generate first response information containing alternative interpretations. In response to the user's selection of a target virtual entity as a second virtual entity with a second cognitive tendency, potential threat elements are extracted from the user interaction information. The second virtual entity is then controlled to evaluate the potential threat elements based on a preset cost-benefit analysis logic and generate second response information that reveals the actual risks.

[0107] The first virtual subject can be configured as a positive virtual subject with a positive cognitive tendency. When processing the above scenario, the first virtual subject takes the negative event characteristics from the scenario description, namely the staring behavior of others and the discomfort experienced by the user. Based on these negative event characteristics, the first virtual subject is driven to execute a preset evidence review logic. This logic is used to simulate the data processing process of finding alternative explanations, guiding the user to re-examine the objective evidence that triggered the adverse feelings, thereby challenging and dismantling one-sided catastrophic inferences such as the other party necessarily having malicious intent.

[0108] After restructuring the underlying logic, a first response message containing alternative explanations is generated and output to the user, such as guiding the user to consider that the other person might just be spacing out or have mistaken them for someone else. This process effectively promotes positive behavioral activation, helping users break free from avoidance patterns that are constantly reinforced by negative reinforcement.

[0109] The second virtual agent can be configured as a passive virtual agent with a tendency towards negative or risk-detection. When handling the above scenarios, the second virtual agent extracts potential threat elements from the user's interaction information, focusing on analyzing whether the user's avoidance actions at the time have any subsequent hidden dangers. The second virtual agent quantifies these potential threats, helping users identify and confront actual risks that might be overlooked by automated thinking, and generates a second response message revealing the actual risks based on the assessment results. Crucially, the computational process of the second virtual agent directly confronting potential threats is essentially executed by the underlying architecture within a digitally-based and securely symbolic isolated container, thereby ensuring that users can effectively avoid excessive emotional involvement in the real-world dimension when simulating various risk points.

[0110] Through the aforementioned parallel control links, corresponding cognitive simulation models can be dynamically matched based on the user's personalized interactive choices. Whether relying on evidence-based alternative explanations or revealing actual risks based on cost-benefit analysis, both methods assist users in gradually shifting from an automated stress response state to a conscious and objective response state. This data processing path externalizes and structures internal cognitive conflicts, providing users with a safe and seamless inference experiment field. It achieves a shift from a single negative perspective to multiple objective perspectives, thereby empowering users to cope more rationally when facing complex situations.

[0111] Interaction paths of multi-subject groups In some embodiments, in response to the user's selection of a target virtual entity as a first virtual entity and a second virtual entity, the first virtual entity and the second virtual entity generate opposing dialogue information based on preset cognitive logic and interaction information, and send the opposing dialogue information to the user to realize the externalization of opposing cognition.

[0112] When a user selects a group chat, the system first parses the user's input interaction information. Then, it drives the first and second virtual entities to perform parallel calculations based on their pre-defined cognitive logic for the given context. Because the two entities are configured with differentiated or even conflicting cognitive tendencies at the underlying parameter level, they automatically generate alternating, logically complementary, or contradictory dialogue information around the triggering context. This generated dialogue information is then synchronously rendered and sent to the user's device. During this interaction phase, the user's operational state can be locked in an observer's perspective.

[0113] The aforementioned processing mechanism externalizes and integrates implicit cognitive conflicts through underlying computation, precisely mapping the cognitive patterns that originally existed as interplay within individual thinking into multi-agent logical debates on an external visual interface. By externalizing this opposing cognition, the control architecture constructs an objective data review window for users, allowing them to intuitively examine behavioral patterns under different cognitive dimensions and their potential chain reactions in a safe state detached from direct emotional involvement. This not only physically assists users in decoupling cognitive dimensions and regulating emotions but also helps stimulate users' objective evaluation and continuous optimization mechanisms of their underlying thinking patterns through the reverse stimulation of visualized data.

[0114] Corresponding to the method embodiments of this application, this application also provides a human-computer interaction risk control system based on virtual subjects, such as... Figure 7 As shown, the human-computer interaction risk control system 100 includes: The virtual subject generation module 110 is used to acquire the user's multimodal raw data, perform vector processing on the multimodal raw data to obtain holographic feature vectors, and generate virtual subjects based on the holographic feature vector mapping. The risk monitoring module 120 is used to collect interaction information between the user and the virtual entity, and to perform risk monitoring on the interaction information using a pre-trained risk monitoring model. The content output module 130 is used to respond to the presence of risk in the interactive information, and the virtual entity outputs safe response content that matches the user's cognitive level.

[0115] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. The electronic device can be implemented as a server or other various terminal devices, such as desktop personal computers, tablet computers, laptop computers, mobile phones, etc., including a processor 601 and a memory 602. The memory 602 stores a program instruction set. When the processor 601 executes the program instruction set in the memory 602, it implements any of the aforementioned human-computer interaction risk control methods based on virtual subjects.

[0116] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0117] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0118] The memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the virtual agent-based human-computer interaction risk control method provided by this invention.

[0119] In one example, the electronic device may also include a communication interface 603 and a bus 604. The processor 601, memory 602, and communication interface 603 are connected via bus 604 and communicate with each other. Communication interface 603 is primarily used to enable communication between modules, devices, units, and / or equipment in the embodiments of the present invention. Bus 604 includes hardware, software, or both, coupling components of the online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) Interconnect, Industry Standard Architecture (ISA) bus, Infinite Bandwidth Interconnect, Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in the embodiments of the present invention, the present invention is contemplated by any suitable bus or interconnect.

[0120] The present invention also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement any of the human-computer interaction risk control methods based on virtual agents in the foregoing embodiments. The computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in conjunction with an instruction execution system, apparatus, or device. The storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include, for example, magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory and solid-state drives.

[0121] This invention also provides a computer program product comprising a set of computer program instructions, which, when executed by a processor, implement any of the human-computer interaction risk control methods based on virtual subjects in the foregoing embodiments. The computer program product includes, but is not limited to, application installation packages published on websites and app stores, application plugins, and mini-programs that can run within certain applications.

[0122] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0123] The above embodiments are for illustrative purposes only and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the scope of the invention. Therefore, all equivalent technical solutions should also fall within the scope of the invention.

Claims

1. A human-computer interaction risk control method based on virtual subjects, characterized in that, include: The user's multimodal raw data is acquired, the multimodal raw data is processed into vectors to obtain holographic feature vectors, and a virtual subject is generated based on the holographic feature vector mapping. Collect interaction information between users and virtual entities, and use a pre-trained risk monitoring model to monitor the risks of the interaction information; In response to the presence of risk in the interactive information, the virtual entity outputs a safe response to the user that matches their level of understanding.

2. The human-computer interaction risk control method according to claim 1, characterized in that, The risk monitoring model adopts a classification and hierarchical architecture, which is configured to: based on the multi-dimensional features of interactive information, perform basic classification and refined hierarchical processing in sequence, thereby determining the risk status and risk attributes of the interactive information.

3. The human-computer interaction risk control method according to claim 1, characterized in that, Using a pre-trained risk monitoring model to perform risk monitoring on the interaction information includes: Calculate the first probability distribution of the interaction information belonging to multiple preset categories, and determine the target category to which the interaction information belongs based on the first probability distribution; Given the target classification, calculate the second probability distribution of the interaction information belonging to multiple preset sub-layers under the target classification, and determine the target sub-layer to which the interaction information belongs based on the second probability distribution; The risk status and risk attributes of the interactive information are determined based on the target classification and / or target sub-layer.

4. The human-computer interaction risk control method according to claim 3, characterized in that, The step of calculating the first probability distribution of the interaction information belonging to multiple preset categories, and determining the target category to which the interaction information belongs based on the first probability distribution includes: Extract the multi-dimensional features of the interaction information to obtain the corresponding feature vector; Calculate the linear combination feature value between the feature vector and the classification model parameters corresponding to each preset classification; The linear combination feature values ​​are normalized to obtain the first probability distribution of the interaction information belonging to each of the preset categories; The preset category with the highest probability value in the first probability distribution is determined as the target category.

5. The human-computer interaction risk control method according to claim 3, characterized in that, The step of calculating the second probability distribution of the interaction information belonging to multiple preset sub-layers under the target category, and determining the target sub-layer to which the interaction information belongs based on the second probability distribution, includes: Extract the multi-dimensional features of the interaction information to obtain the corresponding feature vector; Calculate the linear combination feature value between the feature vector and the sub-classification model parameters corresponding to each preset sub-layer under the target classification; The linear combination feature values ​​are normalized to obtain the second probability distribution of the interaction information belonging to each of the preset sub-layers; The preset sub-layer with the highest probability value in the second probability distribution is determined as the target sub-layer.

6. The human-computer interaction risk control method according to claim 1, characterized in that, Further includes: In response to the presence of risk in the interactive information, a corresponding safety response is matched based on the user's cognitive assessment value, which is calculated based on the feedback content that is proactively sent to the user with a question sequence containing preset standard answers.

7. The human-computer interaction risk control method according to claim 6, characterized in that, The calculation method for the cognitive assessment value includes: The feedback content and the corresponding preset standard answer are vectorized to obtain the feature vector of the actual answer and the feature vector of the standard answer. Calculate the spatial cosine similarity between each actual answer feature vector and the corresponding standard answer feature vector to serve as the local cognitive score for a single question; The average of all local cognitive scores is used to obtain the normalized cognitive assessment value.

8. The human-computer interaction risk control method according to claim 1, characterized in that, In response to the absence of risk in the interactive information, the virtual entity responds to the user's interactive information to provide structured guidance for the user to conduct cognitive exercises. The virtual entity includes multiple virtual entities with differentiated cognitive assessment logic.

9. The human-computer interaction risk control method according to claim 8, characterized in that, The virtual entity responds to user interaction information to provide structured guidance for users in cognitive exercises, including: In response to the user's selection of a target virtual entity as a first virtual entity with a first cognitive tendency, negative event features are extracted from the user interaction information, and the first virtual entity is controlled to reconstruct the negative event features based on a preset evidence review logic, and generate first response information containing alternative interpretations. In response to the user's selection of a target virtual entity as a second virtual entity with a second cognitive tendency, potential threat elements are extracted from the user interaction information. The second virtual entity is then controlled to evaluate the potential threat elements based on a preset cost-benefit analysis logic and generate second response information that reveals the actual risks.

10. The human-computer interaction risk control method according to claim 9, characterized in that, Further includes: In response to the user's selection of a first virtual entity and a second virtual entity, the first virtual entity and the second virtual entity generate opposing dialogue information based on preset cognitive logic and interaction information, and send the opposing dialogue information to the user to realize the externalization of opposing cognition.

11. A human-computer interaction risk control system based on a virtual subject, characterized in that, include: The virtual subject generation module is used to acquire the user's multimodal raw data, perform vector processing on the multimodal raw data to obtain holographic feature vectors, and generate virtual subjects based on the holographic feature vector mapping. The risk monitoring module is used to collect interaction information between users and virtual entities, and to use a pre-trained risk monitoring model to monitor the risks of the interaction information. The content output module is used to respond to the presence of risks in the interactive information, and the virtual entity outputs safe response content that matches the user's level of understanding.

12. An electronic device, characterized in that, This includes a processor and a memory, the memory storing a set of computer program instructions, which, when executed by the processor, implement the human-computer interaction risk control method based on any one of claims 1-10.

13. A computer program product, characterized in that, It includes a set of computer program instructions, which, when executed by a processor, implement the human-computer interaction risk control method based on a virtual subject as described in any one of claims 1-10.

Images