Method for identifying implicit cognitive discrepancies

Abstract

The application relates to the field of artificial intelligence and natural language processing technology, and provides a method for identifying implicit cognitive divergence, which comprises the following steps: obtaining the collaborative interaction content of a plurality of first participants in a target collaborative interaction scene, wherein the collaborative interaction content comprises internal cognitive activities and external interaction behaviors; inputting the collaborative interaction content of the plurality of first participants into a detection stimulus generation model, and outputting a plurality of detection stimuli through the detection stimulus generation model; the detection stimulus generation model is generated based on pre-labeled effective detection stimulus samples and ineffective detection stimulus samples; for each detection stimulus, obtaining the detection response of each first participant to the detection stimulus; and identifying whether there is an implicit cognitive divergence between the plurality of first participants based on the plurality of detection responses of the first participants. The technical scheme provided by the application can realize automatic identification of the implicit cognitive divergence, and improve the communication quality and the collaboration efficiency.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and natural language processing technology, and in particular to a method for identifying implicit cognitive disagreements. Background Technology

[0002] In collaborative scenarios involving multiple parties, a common phenomenon exists: while participants may superficially reach a consensus on a particular issue, solution, or concept, significant unspoken disagreements remain at the underlying cognitive level, such as in goal setting, key concept definition, hidden assumptions, execution paths, and value priorities. These disagreements are highly concealed and often only surface long after the collaborative interaction has ended, severely impacting communication quality and collaboration efficiency.

[0003] Currently, in collaborative interaction scenarios, the focus is mainly on analyzing dialogue sentiment and emotion, extracting dialogue summaries and key information, and detecting explicit dialogue consistency and contradictions. These methods can only process expressed language signals or explicit conflicts, and cannot reach the implicit differences at the underlying cognitive level of the participants.

[0004] If significant, unexpressed disagreements are recorded as implicit cognitive disagreements, then how to achieve automated identification of implicit cognitive disagreements, thereby improving communication quality and collaboration efficiency, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a method for identifying implicit cognitive disagreements, enabling automated identification of implicit cognitive disagreements, thereby improving communication quality and collaboration efficiency.

[0006] This application provides a method for identifying implicit cognitive disagreements, including: Acquire the collaborative interaction content of each of the multiple first participants in the target collaborative interaction scenario, wherein the collaborative interaction content includes internal cognitive activities and external interactive behaviors; The collaborative interaction content of each of the multiple first participants is input into the probe stimulus generation model, and the probe stimulus generation model outputs multiple probe stimuli; wherein, the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; For each of the aforementioned detection stimuli, the detection response generated by each of the first participants in response to the detection stimuli is obtained; Based on multiple probe responses corresponding to each of the first participants, identify whether there are implicit cognitive disagreements among the multiple first participants.

[0007] According to the implicit cognitive divergence identification method provided in this application, for each first participant, obtaining the probe response generated by the first participant in response to the probe stimulus includes at least one of the following: Initiate a probe inquiry to the first participant and obtain the probe response generated by the first participant in response to the probe stimulus; Based on at least one of the first participant’s historical interaction records, decision records, publicly available information, language signals, behavioral signals, document information, etc., obtain the first participant’s detection response to the detection stimulus; The detection response of the first participant to the detection stimulus is obtained by a single sampling or by multiple sampling. The detection response of the first participant to the detection stimulus is obtained by synchronous or asynchronous acquisition.

[0008] According to the implicit cognitive divergence identification method provided in this application, the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples, including: For each of the multiple collaborative interaction scenarios, obtain the collaborative interaction content of each second participant in the collaborative interaction scenario; The collaborative interaction content of each of the multiple second participants is input into the initial probe stimulus generation model, and multiple probe stimulus samples are output through the initial probe stimulus generation model; For each of the aforementioned probe stimulus samples, obtain the probe response samples generated by each of the second participants in response to the probe stimulus samples; Based on multiple probe response samples from each of the second participants, identify whether there are implicit cognitive disagreements among the multiple second participants; Based on the identification results, the valid probe stimulus sample and the invalid probe stimulus sample are determined from the plurality of probe stimulus samples; Based on the valid probe stimulus samples and the invalid probe stimulus samples, the initial probe stimulus generation model is trained to obtain the probe stimulus generation model.

[0009] According to the implicit cognitive divergence identification method provided in this application, the step of determining the valid probe stimulus sample and the invalid probe stimulus sample from the plurality of probe stimulus samples based on the identification result includes: When the identification result indicates the existence of implicit cognitive divergence, it is determined whether the probe stimulus sample with implicit cognitive divergence meets the preset conditions. The preset conditions include at least one of the following: the probe stimulus sample is clearly expressed, the divergence revealed is implicit and not explicit, and it does not constitute redundancy with existing probe stimulus samples. Based on the judgment results, the multiple probe stimulus samples are classified, and the classification results include: probe stimulus samples that reveal a preset divergence, probe stimulus samples that reveal other real divergences besides the preset divergence, and probe stimulus samples that do not have divergences. The effective probe stimulus samples include probe stimulus samples that reveal a preset divergence and / or probe stimulus samples that reveal other real divergences besides the preset divergence, and the invalid probe stimulus samples include probe stimulus samples that do not have divergences.

[0010] According to the method for identifying implicit cognitive disagreement provided in this application, the method further includes: Based on the classification results, a comprehensive evaluation index corresponding to the initial detection stimulus generation model is constructed. The comprehensive evaluation index is used to reflect the recognition ability of the initial probe stimulus generation model. The recognition ability includes at least one of the following: the ability to cover the preset divergence, the ability to discover other real divergences, and the refinement of the generated multiple probe stimulus samples.

[0011] According to the implicit cognitive divergence identification method provided in this application, the step of training the initial probe stimulus generation model based on the effective probe stimulus samples and the invalid probe stimulus samples includes: Determine the effective detection ratio of the effective probe stimulus samples and the invalid detection ratio of the invalid probe stimulus samples; The effective detection ratio is constructed as a positive reward signal, the ineffective detection ratio is constructed as a negative reward signal, and the initial detection stimulus generation model is trained using a policy gradient algorithm in conjunction with the comprehensive evaluation index.

[0012] According to the implicit cognitive divergence identification method provided in this application, the step of training the initial probe stimulus generation model based on the effective probe stimulus samples and the invalid probe stimulus samples includes: The effective probe stimulus samples are identified as preferred samples, and the invalid probe stimulus samples are identified as non-preferred samples, thus constructing preference pair data; Based on the preference pair data, a preference learning algorithm is used to train the initial probe stimulus generation model.

[0013] According to the implicit cognitive divergence identification method provided in this application, the step of training the initial probe stimulus generation model based on the effective probe stimulus samples and the invalid probe stimulus samples includes: Based on the effective probe stimulus samples and the invalid probe stimulus samples, a supervised learning dataset is constructed; Based on the supervised learning dataset, the initial probe stimulus generation model is trained using a supervised fine-tuning algorithm.

[0014] According to the implicit cognitive disagreement identification method provided in this application, obtaining the collaborative interaction content of each second participant in the collaborative interaction scenario includes: Seed content is collected from multiple domain knowledge sources; the seed content includes at least one of domain knowledge text, case description text, and question-and-answer interaction text. Based on the seed content, a collaborative interaction scenario description is generated for the collaborative interaction scenario. Based on the description of the collaborative interaction scenario, the collaborative interaction content of each of the second participants is generated.

[0015] According to the implicit cognitive disagreement identification method provided in this application, the step of generating a collaborative interaction scenario description based on the seed content includes: The seed content is input into the large language model, and the large language model outputs the initial collaborative interaction scenario description of the collaborative interaction scenario. The initial collaborative interaction scenario description is subjected to a multi-dimensional quality assessment to obtain multiple first assessment scores; wherein, the multi-dimensional assessment includes at least two of the following: the concealment of the disagreement, the specificity, the credibility, the domain authenticity, and the diversity. If all of the first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, then the initial collaborative interaction scenario description is determined as the collaborative interaction scenario description. If at least one of the multiple first evaluation scores is less than the score threshold of the corresponding dimension, the initial collaborative interaction scenario description is optimized, and the optimized collaborative interaction scenario description is re-evaluated in multiple dimensions until all the obtained multiple first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, or until a preset first maximum number of optimizations is reached, and the optimized collaborative interaction scenario description is determined as the collaborative interaction scenario description.

[0016] According to the implicit cognitive disagreement identification method provided in this application, the step of generating the respective collaborative interaction content of each second participant based on the collaborative interaction scenario description includes: Based on the description of the collaborative interaction scenario, generate the initial collaborative interaction content for each of the second participants; The initial collaborative interaction content is subjected to a multi-dimensional quality assessment to obtain multiple second assessment scores; wherein, the multi-dimensional assessment includes at least two of the following: naturalness, concealment, coverage, role consistency, and completeness of auxiliary information; If all of the multiple second evaluation scores are greater than or equal to the score threshold of the corresponding dimension, then the initial collaborative interaction content is determined as the collaborative interaction content. If at least one of the multiple second evaluation scores is less than the score threshold of the corresponding dimension, the initial collaborative interaction content is optimized, and the optimized collaborative interaction content is re-evaluated in multiple dimensions until all the obtained multiple second evaluation scores are greater than or equal to the score threshold of the corresponding dimension, or until a preset second maximum number of optimizations is reached, and the optimized collaborative interaction content is determined as the collaborative interaction content.

[0017] This application also provides a device for identifying implicit cognitive disagreements, comprising: The acquisition unit is used to acquire the collaborative interaction content of each of the multiple first participants in the target collaborative interaction scenario, wherein the collaborative interaction content includes internal cognitive activities and external interactive behaviors. The processing unit is used to input the collaborative interaction content of the multiple first participants into the probe stimulus generation model, and output multiple probe stimuli through the probe stimulus generation model; wherein, the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; The acquisition unit is further configured to acquire, for each of the detection stimuli, the detection response generated by each of the first participants in response to the detection stimuli; It is a unit used to identify whether there is a implicit cognitive disagreement among the multiple first participants based on multiple probe responses corresponding to each first participant.

[0018] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the implicit cognitive divergence identification method as described above.

[0019] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for identifying implicit cognitive divergences as described above.

[0020] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method for identifying implicit cognitive disagreement as described above.

[0021] The implicit cognitive disagreement identification method provided in this application obtains the collaborative interaction content of multiple first participants in a target collaborative interaction scenario. This content includes internal cognitive activities and explicit interactive behaviors. The collaborative interaction content of each first participant is input into a probe stimulus generation model, which outputs multiple probe stimuli. The probe stimulus generation model is trained based on pre-labeled valid and invalid probe stimulus samples. For each probe stimulus, the probe response of each first participant is obtained. Based on the multiple probe responses of each first participant, the existence of implicit cognitive disagreements among the multiple first participants is identified. By applying probe stimuli to the first participants and obtaining their probe responses, implicit cognitive disagreements, which were originally not directly observable, are transformed into objectively comparable differences in probe responses. This achieves automated identification of implicit cognitive disagreements, overcoming the problem that traditional methods can only handle expressed linguistic signals or explicit conflicts, failing to address the implicit differences at the underlying cognitive level of participants. This provides an objective and reusable technical means for disagreement warning, communication quality improvement, and decision optimization in multi-party collaborative interaction scenarios, thereby effectively improving communication quality and collaboration efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a method for identifying implicit cognitive disagreements provided in an embodiment of this application.

[0024] Figure 2 This is a schematic diagram illustrating a training method for a stimulus generation model provided in an embodiment of this application.

[0025] Figure 3 This is a schematic diagram of the structure of a device for identifying implicit cognitive disagreement, provided in an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0029] The technical solution provided in this application can be widely applied in various collaborative interaction scenarios where there is a risk of implicit cognitive disagreement. Depending on when the collaborative interaction occurs, it can be deployed in three forms: a preventive detection system, a real-time intervention system, and an offline analysis system.

[0030] A preventative detection system proactively sends detection stimuli to all participants before the formal commencement of collaborative interactions or at key decision-making junctures. This helps identify potential cognitive differences in advance and prevents misunderstandings during subsequent collaboration. It is suitable for scenarios such as project kickoff meetings, pre-contract confirmation, cross-team coordination, and the initiation of interdisciplinary collaborations.

[0031] The real-time intervention system identifies potential implicit cognitive disagreements during collaborative interactions and alerts or intervenes to participants at appropriate times and in appropriate ways. It is suitable for scenarios such as real-time meeting support, real-time medical consultation support, real-time customer service support, and real-time educational tutoring.

[0032] The offline analysis system performs debriefing analysis after collaborative interactions, identifies implicit cognitive disagreements, and generates reports for participants to review. It is suitable for scenarios such as meeting debriefing, medical case discussion debriefing, legal consultation record debriefing, and classroom teaching analysis.

[0033] Furthermore, depending on the types of participants in the collaborative interaction, it can also be deployed in the following system forms: The human-machine collaboration alignment enhancement module allows the assistant to proactively detect potential discrepancies between itself and the user in terms of goals, concepts, and assumptions when collaborating with an AI assistant, thus avoiding outputs that do not meet the user's expectations due to goal alignment failures.

[0034] A multi-agent collaboration assistance module, embedded in a multi-agent collaboration system, serves as a disagreement identification and coordination module, improving the alignment of multi-agent collaboration and the quality of task completion. It is suitable for scenarios such as multi-agent programming collaboration, multi-agent research collaboration, and multi-agent decision support.

[0035] Cross-cultural and cross-organizational communication support systems identify implicit cognitive differences caused by background differences and provide targeted communication suggestions when collaborating with participants from different cultural backgrounds, organizations, and professional fields.

[0036] Education and training support systems identify implicit cognitive differences between learners or between learners and learning objectives in collaborative learning scenarios involving learners and teachers, or learners among themselves, thereby assisting in the implementation of personalized teaching.

[0037] In summary, the technical solution provided in this application can be applied to various multi-party collaborative interaction scenarios, such as team collaboration and project communication, interdisciplinary research discussions, medical consultation and remote consultation, legal consultation and contract negotiation, business negotiations, educational interaction, cross-cultural / cross-organizational negotiation and docking, multi-agent system task coordination, and user and AI assistant goal alignment in human-machine collaboration scenarios. The specific settings can be configured according to actual needs.

[0038] Currently, in collaborative interaction scenarios, the focus is mainly on analyzing dialogue sentiment and emotion, extracting dialogue summaries and key information, and detecting explicit dialogue consistency and contradictions. These methods can only process expressed language signals or explicit conflicts, and cannot reach the implicit differences at the underlying cognitive level of the participants.

[0039] If unexpressed significant disagreements are categorized as implicit cognitive disagreements, this application provides a method for identifying implicit cognitive disagreements to automate their identification and improve communication quality and collaboration efficiency. The method will be described in detail below through several specific embodiments. It is understood that these specific embodiments can be combined with each other, and similar concepts or processes may not be repeated in some embodiments.

[0040] Figure 1 A flowchart illustrating a method for identifying implicit cognitive disagreement provided in this application embodiment is shown below. For example, please refer to... Figure 1 As shown, the method for identifying implicit cognitive divergence may include: S101. Obtain the collaborative interaction content of each of the multiple first participants in the target collaborative interaction scenario. The collaborative interaction content includes internal cognitive activities and external interactive behaviors.

[0041] Among them, the target collaborative interaction scenario refers to any collaborative situation in which multiple parties are involved and there may be implicit cognitive differences.

[0042] For example, in some embodiments, the types of collaborative interaction scenarios include: natural language dialogue scenarios, covering written dialogues such as email, instant messaging, document comments, etc., and oral dialogues such as face-to-face meetings, telephone conferences, video conferences, etc.; multimodal interaction scenarios, integrating multiple information channels such as voice, text, screen sharing, whiteboards, and gestures; document collaboration and asynchronous communication scenarios, such as multiple people jointly editing documents, task comments and status updates in project management systems; message passing and task coordination scenarios between multiple agents; and human-machine hybrid collaboration scenarios, i.e., task execution and decision-making processes jointly participated in by human users and artificial intelligence systems, etc., which can be specifically set according to actual needs.

[0043] The first participant refers to the various parties involved in a collaborative interaction scenario. For example, the types of first participants can include human participants, intelligent agents, or a hybrid combination of humans and intelligent agents, which can be configured according to actual needs. Intelligent agents include, for example, Artificial Intelligence (AI) assistants and automated systems. Multiple first participants can come from the same or different organizational, cultural, or disciplinary backgrounds, such as participants in cross-organizational collaboration, cross-cultural negotiation, or interdisciplinary research discussions, which can be configured according to actual needs.

[0044] In scenarios involving human participants, each participant possesses their own cognitive state, background knowledge, and value judgments; in scenarios involving intelligent agents, each participant has a pre-defined objective function, reasoning logic, and decision-making strategy. By using the first participant as the subject of disagreement identification, the implicit cognitive differences between different participants can be effectively captured.

[0045] Collaborative interaction content refers to all information records generated by multiple primary participants in a target collaborative interaction scenario, which may include: dialogue text, speech-to-text content, screen sharing, whiteboard annotations, document modification records, system operation logs, task assignment records, decision voting records, etc.

[0046] Internal cognitive activity refers to the inner thoughts, reasoning processes, decision-making basis, potential assumptions, and value judgments of the first participant during collaborative interaction that are not expressed externally. It is a concrete manifestation of the participant's private cognitive state.

[0047] For example, internal cognitive activities may include: a participant’s true views on the current issue, a deep interpretation of others’ words and actions, unspoken concerns, strategic intentions of expression, and potential assumptions formed based on their own professional background and experience.

[0048] Explicit interaction behavior refers to the verbal expressions and actions of the first participant during collaborative interaction that can be directly observed externally. This includes the participant's spoken or written content, actions such as screen sharing, whiteboard annotation, document editing, and task assignment, as well as nonverbal cues such as tone of voice, facial expressions, and gestures. Explicit interaction behavior constitutes the publicly accessible portion of collaborative interaction content and is the primary focus of traditional interaction analysis methods. In some embodiments, explicit interaction behavior and internal cognitive activity together constitute each participant's complete interaction record. By comparing the differences between the internal cognitive activity and explicit interaction behavior of the same participant, as well as the differences in explicit interaction behavior among different participants, implicit cognitive discrepancies that could not be directly identified from surface communication can be revealed.

[0049] For example, the method of collecting collaborative interaction content does not have to be determined based on the type of the target collaborative interaction scenario. For example, audio and video can be collected through meeting recording devices, chat records can be obtained through instant messaging tool interfaces, version history can be obtained through document collaboration platforms, or communication logs can be obtained through the message bus of multi-agent systems. The specific method can be set according to actual needs.

[0050] S102. Input the collaborative interaction content of multiple first participants into the probe stimulus generation model, and output multiple probe stimuli through the probe stimulus generation model; wherein, the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples.

[0051] The probe stimulus generation model refers to a machine learning model used to transform unstructured collaborative interaction content into structured, targeted probe stimuli. It can automatically identify potential cognitive disagreements within the input collaborative interaction content and generate corresponding probe stimuli for subsequent verification, thereby achieving proactive detection of implicit cognitive disagreements. By training with pre-labeled valid and invalid probe stimulus samples, the probe stimulus generation model can distinguish which probe stimuli effectively reveal disagreements and which do not, thus continuously optimizing the quality and effectiveness of its generated probe stimuli.

[0052] For example, the stimulus generation model can be a large language model based on the Transformer architecture, such as LLaMA, GPT, Qwen, ChatGLM, etc., or a task-specific model obtained by fine-tuning it; or it can be a special model obtained through multi-stage combined training, which includes at least one training paradigm among supervised fine-tuning, preference learning, and reinforcement learning.

[0053] In this context, the probe stimulus refers to a standardized or semi-standardized question, task, or situation designed to detect implicit cognitive discrepancies among participants. The design aims to enable participants to generate comparable probe responses based on their individual cognitive states when faced with the same probe stimulus independently, thereby transforming indirect internal cognitive differences into observable and determinate external response differences.

[0054] For example, in some embodiments, the forms of probe stimuli may include: multiple-choice questions requiring participants to select from multiple preset options; yes / no questions requiring participants to make binary judgments on a specific proposition; open-ended questions allowing participants to freely express their opinions and views; ranking or prioritization questions requiring participants to rank multiple options according to specific criteria; situational hypothesis questions, such as "How would you handle it if X happened?", examining participants' decision-making tendencies in a specific situation; decision-making simulation questions, requiring participants to make specific action choices in a given situation; concept definition extraction questions, such as "What is your understanding of X?", examining participants' private understanding of key concepts; key parameter estimation questions, involving the estimation of parameters such as cost, time, and scale; value or preference probing questions, examining participants' value priorities; behavior prediction questions, such as "What will you do next?"; resource allocation questions or risk assessment questions, examining participants' preferences in resource allocation or risk assessment. The above forms of probe stimuli can be used individually, in combination, or in variations thereof, and can be specifically set according to actual needs.

[0055] It should be noted that the above-mentioned forms of probe stimuli are merely illustrative examples. They are highly flexible and diverse. Any form of probe stimuli can be used in this application as long as it enables participants to independently generate comparable responses based on their private cognitive states and to make meaningful judgments on the differences in responses.

[0056] Valid and invalid probe stimulus samples are the labeled data used to train the probe stimulus generation model. Valid probe stimulus samples are those that, after being verified by a response difference validation mechanism, elicit significant differences in response among participants, meaning the probe stimulus successfully reveals implicit cognitive discrepancies among participants. Invalid probe stimulus samples, on the other hand, are those that, after validation, fail to elicit significant differences in response, meaning the probe stimulus fails to reveal meaningful cognitive discrepancies. Through the positive guidance of valid samples and the negative constraint of invalid samples, the probe stimulus generation model can progressively distinguish between "good" and "bad" probe stimuli, thereby improving the relevance and effectiveness of its generated probe stimuli to a certain extent. It is understood that the labeled data is obtained based on a pre-executed response difference validation process, eliminating the need for manual annotation and thus achieving a fully automated closed-loop training process for the probe stimulus generation model.

[0057] S103. For each probe stimulus, obtain the probe response generated by each first participant in response to the probe stimulus.

[0058] In this context, the probe response refers to the independent answer or reaction generated by each participant based on their own private cognitive state after receiving the probe stimulus. The probe response is the core data source for the response difference verification mechanism, and its quality directly affects the accuracy of subsequent disagreement determination. The acquisition of probe responses must ensure that each participant is independent of the others; that is, when any participant generates a probe response, it should not be influenced by the responses of other participants, in order to ensure that the response differences truly reflect the cognitive differences among the participants.

[0059] Private cognitive states refer to the implicit cognitive information that participants do not disclose during collaborative interactions. This can include: hidden goals, unexpressed assumptions, private understandings of key concepts, value priorities, risk preferences, and potential judgments based on their professional background and experience. The specific settings can be customized according to actual needs. Private cognitive states are the intrinsic basis for participants' probing responses and the fundamental reason for the different responses of different participants to the same probing stimulus. For example, private cognitive states can be modeled and inferred through various means such as participant profiling, historical interaction records, decision-making records, publicly available data, verbal signals, behavioral signals, and document information.

[0060] For example, obtaining the probe response of each first participant to the probe stimulus may include at least one of the following methods: In the first possible approach, a probing question can be posed to the first participant to obtain their probing response to the probing stimulus. This method is suitable for real-time collaborative scenarios, as it obtains the participant's immediate response by directly asking questions, reflecting the participant's true cognitive state at the current moment.

[0061] In the second possible acquisition method, the first participant's probe response to the probe stimulus can be obtained based on at least one of the following: historical interaction records, decision-making records, publicly available information, verbal signals, behavioral signals, and document information. This method is suitable for scenarios where it is impossible to directly question the participant, such as when the participant has left, is unwilling to cooperate, or requires post-event analysis. By learning from and inferring from the participant's past behavioral records and publicly available information, the participant's potential probe response can be simulated or predicted.

[0062] In the third possible acquisition method, the probe response of the first participant to the probe stimulus can be obtained through single sampling or multiple sampling. Single sampling is suitable for scenarios where response stability requirements are not high and has the advantage of high efficiency; multiple sampling is suitable for scenarios where response consistency needs to be evaluated. By aggregating the results of multiple samplings, such as majority voting, confidence weighting, and distribution fitting, the reliability and robustness of the probe response can be improved.

[0063] In the fourth possible acquisition method, the probe response of the first participant to the probe stimulus can be acquired synchronously or asynchronously. Synchronous acquisition is suitable for scenarios where all participants are online simultaneously and real-time interaction is required, avoiding information leakage due to time differences. Asynchronous acquisition is suitable for scenarios where participants cannot participate simultaneously, allowing each participant to generate a response independently at different times, offering greater flexibility. The choice between synchronous and asynchronous acquisition can be made flexibly based on the specific needs of the actual collaborative scenario.

[0064] It is understood that the above four possible acquisition methods are only examples in the embodiments of this application. For example, the acquisition methods of probe response can also include: role-playing acquisition based on participant profiles using a large language model; comprehensive modeling inference based on multi-source signals, including language, behavior, decision, documents, etc.; and mixed use of probe response and simulated response, etc. As long as each participant can independently generate comparable probe response based on their private cognitive state, any response acquisition method can be adopted in this application.

[0065] It should be noted that the above-mentioned acquisition methods can be combined to achieve flexible adaptation to different application scenarios.

[0066] In this application, the core requirement for obtaining probe responses is to ensure that each participant's probe response remains independent during the generation process; that is, no participant should know the content of other participants' responses when generating a probe response. This is because the basic principle of the response difference verification mechanism is that if there is some unexpressed implicit cognitive divergence between two or more participants, then when these participants are independently subjected to the same appropriately designed probe stimulus, their responses based on their private cognitive states will exhibit observable differences; conversely, if all participants respond highly consistently to the same probe stimulus, then there is no meaningful implicit divergence in the cognitive dimensions covered by the stimulus. This principle transforms the originally unobservable problem of "internal cognitive differences" into an observable, repeatable, and automatically judgeable problem of "response differences," and the independence of the participants is a prerequisite for ensuring the effectiveness of this transformation.

[0067] It should be noted that the response difference verification mechanism involved in the embodiments of this application is not limited to any specific form of detection stimulus or method of detection response acquisition. Any specific implementation that enables participants to independently generate comparable responses based on their private cognitive state and to make meaningful judgments on response differences falls within the protection scope of this application.

[0068] S104. Based on the multiple probe responses corresponding to each first participant, identify whether there are implicit cognitive disagreements among the multiple first participants.

[0069] Implicit cognitive disagreement refers to a situation where multiple participants, during collaborative interaction, superficially reach a consensus on a certain issue, solution, or concept, but significant unexpressed differences exist at the underlying cognitive level, such as in goal setting, key concept definition, hidden assumptions, implementation paths, and value priorities. This type of implicit cognitive disagreement is highly concealed and cannot be identified by directly observing participants' overt interactive behavior; rather, it requires analysis of the differences in each participant's response to the same probing stimulus to reveal it.

[0070] For example, methods for identifying response differences among multiple first participants can include: exact matching of discrete options, suitable for scenarios where the probe stimulus is in the form of multiple-choice or true / false questions, directly comparing whether the participants' choices are consistent; semantic similarity determination based on embedding vectors or large language models, suitable for unstructured response scenarios such as open-ended questions, calculating semantic similarity by mapping the response text to vector representations; distribution-level difference measures, such as KL divergence, Jensen-Shannon divergence, and Wasserstein distance, suitable for scenarios where responses are presented in the form of probability distributions; weighted fusion of multi-dimensional differences, suitable for scenarios where the probe stimulus involves multiple cognitive dimensions, weighting and synthesizing the differences in each dimension; and determination methods based on rules, learning, or a combination of both, which can be set according to actual needs. It should be noted that the above response difference determination methods can be used individually or in combination, and the specific choice can be made flexibly according to the form of the probe stimulus and the actual application requirements.

[0071] When identifying whether there is implicit cognitive disagreement among multiple first participants based on multiple probe responses corresponding to each first participant, it means comparing the degree of consistency among multiple probe responses corresponding to each first participant according to a preset response difference judgment method. If there are significant differences among multiple probe responses corresponding to each first participant, it indicates that there is unexpressed implicit cognitive disagreement among the participants in the cognitive dimension covered by the probe stimulus. If there is a high degree of consistency among multiple probe responses corresponding to each first participant, it indicates that there is no meaningful implicit cognitive disagreement in the cognitive dimension covered by the probe stimulus.

[0072] The response difference verification mechanism is the core mechanism of this application. Its basic principle is as follows: if there is some unexpressed implicit cognitive divergence between two or more participants, then when these participants are independently subjected to the same appropriately designed probing stimulus, their responses based on their private cognitive states will exhibit observable differences. Conversely, if all participants respond highly consistently to the same probing stimulus, then there is no meaningful implicit divergence in the cognitive dimension covered by the stimulus. This mechanism transforms the originally subjective and unobservable judgment of "whether implicit cognitive divergence exists" into the objectively comparable, repeatable, and automatically verifiable factual question of "whether participants' probing responses to the same probing stimulus are consistent," thus laying the foundation for objectivity and automation for the entire technical solution.

[0073] As can be seen, in this embodiment, the collaborative interaction content of multiple first participants in the target collaborative interaction scenario is obtained. The collaborative interaction content includes internal cognitive activities and external interactive behaviors. The collaborative interaction content of multiple first participants is input into the probe stimulus generation model, and the probe stimulus generation model outputs multiple probe stimuli. The probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples. For each probe stimulus, the probe response generated by each first participant is obtained. Based on the multiple probe responses corresponding to each first participant, it is identified whether there is implicit cognitive disagreement among the multiple first participants. In this way, by applying probe stimuli to the first participants and obtaining the probe responses generated by each first participant in response to the probe stimuli, the implicit cognitive disagreement that was originally not directly observable is transformed into objectively comparable differences in probe responses. This achieves the automatic identification of implicit cognitive disagreement, overcomes the problem that traditional methods can only handle expressed language signals or explicit conflicts and cannot reach the implicit differences at the underlying cognitive level of the participants. It provides an objective and reusable technical means for disagreement warning, communication quality improvement and decision optimization in multi-party collaborative interaction scenarios, thereby effectively improving communication quality and collaboration efficiency.

[0074] The following section will describe in detail how to train and generate a probe stimulus generation model based on pre-labeled valid and invalid probe stimulus samples. Please refer to the following... Figure 2 The example shown.

[0075] Figure 2 This is a schematic diagram of a training method for a stimulus generation model provided in an embodiment of this application. For example, see [link to relevant documentation]. Figure 2 As shown, the method may include: S201. For each collaborative interaction scenario in multiple collaborative interaction scenarios, obtain the collaborative interaction content of each second participant in the collaborative interaction scenario.

[0076] Collaborative interaction scenarios refer to simulated collaborative situations containing pre-defined implicit cognitive divergences, constructed for training probe stimulus generation models. Unlike the target collaborative interaction scenario in S101, which is geared towards practical applications, the collaborative interaction scenario in S201 is a human-controlled training data generation environment with known cognitive state truth values. Its core feature is that, while generating collaborative interaction content, it simultaneously generates all the auxiliary information needed to support response difference verification, including participant profiles, private cognitive states, internal reasoning annotations, and pre-defined divergences. This ensures that the generated data can directly serve subsequent response difference verification mechanisms and model training. By constructing multiple collaborative interaction scenarios covering multiple domains and task types, a large-scale, high-quality, and diverse data foundation can be provided for training probe stimulus generation models.

[0077] For example, obtaining the collaborative interaction content of each second participant in a collaborative interaction scenario may include: Seed content is collected from multiple domain knowledge sources; the seed content includes at least one of domain knowledge text, case description text, and question-and-answer interaction text; based on the seed content, a collaborative interaction scenario description is generated; based on the collaborative interaction scenario description, collaborative interaction content for each second participant is generated.

[0078] Multi-domain knowledge sources include, but are not limited to, academic paper repositories, patent databases, industry report databases, case study databases, and public Q&A communities. By collecting seed content from these multi-domain knowledge sources, it is possible to ensure that the generated collaborative interaction scenarios have broad domain coverage, encompassing multiple target application areas such as machine learning, software engineering, medicine, law, business, education, and interdisciplinary research; and possess professional depth in content. The seed content itself originates from high-quality professional knowledge sources, guaranteeing the domain authenticity of the collaborative interaction scenario content; and exhibits diversity in collaborative interaction scenario modes, with the differences in content characteristics from different knowledge sources providing a material foundation for subsequently generating different types of collaborative scenarios, avoiding data homogenization.

[0079] For example, when generating a collaborative interaction scenario description based on seed content, it may include: The seed content is input into the large language model, and the large language model outputs an initial collaborative interaction scenario description. The initial collaborative interaction scenario description is subjected to multi-dimensional quality evaluation to obtain multiple first evaluation scores. Among them, the multi-dimensional evaluation includes at least two of the following: the concealment of disagreement, specificity, credibility, domain authenticity and diversity.

[0080] If multiple first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, then the initial collaborative interaction scenario description is determined as the collaborative interaction scenario description.

[0081] If at least one of the multiple first evaluation scores is lower than the score threshold for the corresponding dimension, the initial collaborative interaction scenario description is optimized. The optimized collaborative interaction scenario description is then re-evaluated across multiple dimensions until all the obtained first evaluation scores are greater than or equal to the score threshold for the corresponding dimension, or until a preset first maximum number of optimizations is reached. The optimized collaborative interaction scenario description is then determined as the final collaborative interaction scenario description. This iterative closed loop of "generation-evaluation-optimization" effectively ensures that the generated collaborative interaction scenario description possesses sufficiently high quality before entering the subsequent interaction content generation stage, avoiding problems such as low-quality or unusable data in subsequent collaborative interaction content generation due to insufficient quality of the collaborative interaction scenario description. Furthermore, by introducing a first maximum number of optimizations as the iteration termination condition, generation efficiency is considered while ensuring data quality, avoiding infinite loops and making the data generation process highly feasible and controllable.

[0082] The value of the first maximum number of optimizations can be set according to actual needs, and this application embodiment does not impose specific restrictions.

[0083] For example, in some embodiments, in addition to large language models, other generative models can be used to transform seed content into a structured initial collaborative interaction scenario description, the degree of which directly determines the controllability and quality of the generated collaborative interaction content.

[0084] For example, the type of the initial collaborative interaction scenario description may include at least one of the following: Collaborative tasks can be organized according to McGrath's group task taxonomy or other task classification systems. Public profiles of each second participant, such as their role, professional background, and personality traits; The private cognitive states of each second participant, such as hidden goals, unexpressed assumptions, private understanding of key concepts, value priorities, etc. Pre-defined implicit cognitive disagreements, each of which includes at least: apparent consensus, actual understanding of each party, and reasons for not being discovered; An overall outline or script for collaborative interaction.

[0085] For example, when conducting a multi-dimensional quality assessment of the initial collaborative interaction scenario description, a large language model or a dedicated assessment model can be used as an evaluator to score the initial collaborative interaction scenario description item by item from multiple preset dimensions, and obtain the first assessment score corresponding to each dimension. Among these criteria, the implicitness of the disagreement is used to assess whether the pre-set implicit cognitive disagreements are sufficiently hidden beneath the participants' surface consensus, rather than being directly stated; the higher the implicitness, the more it meets the "implicit" requirement. Specificity is used to assess whether the collaborative interaction scenario description contains sufficient detailed information, such as the participants' specific words and actions, specific contextual elements, and specific manifestations of disagreement, to support the subsequent generation of collaborative interaction content; the higher the specificity, the richer the generated content. Credibility is used to assess whether the initial collaborative interaction scenario description is logically consistent, whether the participants' behavioral motivations are reasonable, and whether the emergence of disagreement has a causal inevitability; the higher the credibility, the more realistic and credible the scenario. Domain authenticity is used to assess whether the initial collaborative interaction scenario description conforms to the real collaborative situations in the target domain, whether the use of professional terminology is accurate, and whether the scenario setting conforms to the actual working mode of the domain; the higher the domain authenticity, the higher the professional value of the scenario. Finally, diversity is used to assess whether the types of disagreements, task types, and participant interaction patterns presented in the initial collaborative interaction scenario description are diverse; the higher the diversity, the wider the coverage of the dataset.

[0086] Each of the above dimensions is configured with a corresponding scoring threshold. The scoring thresholds for each dimension can be the same or different, and can be flexibly set according to the actual application scenario and quality requirements. After the evaluator scores each dimension independently, it compares the evaluation score of each dimension with the corresponding scoring threshold to determine whether the initial collaborative interaction scenario description meets the quality standards.

[0087] It is understood that the above evaluation dimensions are merely illustrative examples. In actual implementation, evaluation dimensions can be added, reduced, or adjusted according to the characteristics of the target application domain and data quality requirements, as long as they can effectively evaluate the quality of the collaborative interaction scenario description. This application embodiment does not impose specific limitations on this.

[0088] If multiple first-evaluation scores are greater than or equal to the corresponding dimension's score threshold, it indicates that the current collaborative interaction scenario description meets the preset standards in every quality dimension, and the initial collaborative interaction scenario description is directly determined as the final collaborative interaction scenario description. If at least one dimension's score is lower than the corresponding score threshold among multiple first-evaluation scores, it indicates that the current collaborative interaction scenario description fails to meet the preset standards in at least one quality dimension and needs optimization. In this case, the initial collaborative interaction scenario description is automatically rewritten and optimized using a large language model, focusing on improving the content defects corresponding to the unmet dimensions. After optimization, the optimized collaborative interaction scenario description is re-evaluated across all dimensions to check whether each dimension has met the standards. The above iterative process of "evaluation-optimization-re-evaluation" continues until one of the following two conditions is met: first, all evaluation scores in each dimension reach or exceed the corresponding score threshold; second, the number of iterations has reached the preset first maximum number of optimizations, and even if there are still dimensions that do not meet the standards, the iteration stops, and the collaborative interaction scenario description after the last optimization is determined as the final collaborative interaction scenario description. By introducing the first maximum number of optimizations as a termination condition, we can avoid infinite loops caused by the inability to meet the standards for low-quality scenario descriptions, thereby ensuring the controllability and engineering feasibility of the data generation process.

[0089] For example, in some embodiments, generating the collaborative interaction content for each second participant based on the collaborative interaction scenario description may include: Based on the collaborative interaction scenario description, initial collaborative interaction content is generated for each second participant. This initial collaborative interaction content undergoes a multi-dimensional quality assessment, resulting in multiple second-dimensional assessment scores. These multi-dimensional scores include at least two of the following: naturalness, concealment, coverage, role consistency, and completeness of auxiliary information. If all multiple second-dimensional assessment scores are greater than or equal to the corresponding dimension's score threshold, the initial collaborative interaction content is identified as collaborative interaction content. If at least one of the multiple second-dimensional assessment scores is less than the corresponding dimension's score threshold, the initial collaborative interaction content is optimized. The optimized collaborative interaction content undergoes a new multi-dimensional quality assessment until all multiple second-dimensional assessment scores are greater than or equal to the corresponding dimension's score threshold, or until a preset second maximum number of optimizations is reached. The optimized collaborative interaction content is then identified as collaborative interaction content. This iterative closed loop of "generation-evaluation-optimization" allows for multi-dimensional quality assessment and automatic optimization of the initial collaborative interaction content. This effectively ensures that the generated content possesses sufficiently high quality before entering subsequent data organization and storage stages, avoiding problems such as unusable labeled data or poor model training results due to insufficient content quality (e.g., unnatural dialogue, overly obvious disagreements, inconsistent character behavior, or incomplete auxiliary information). Furthermore, by introducing a second maximum number of optimization iterations as the iteration termination condition, generation efficiency is considered while maintaining content quality, avoiding infinite loops and ensuring the collaborative interaction content generation process has good engineering feasibility and controllability. In addition, by generating content round by round, simultaneously producing internal cognitive activities and external interactive behaviors in each round, implicit cognitive disagreements are naturally embedded in the collaborative interaction content as "internal and external differences," rather than being manually added through post-annotation. This ensures the concealment and authenticity of the disagreements. Moreover, the entire process can be fully automated without human intervention, significantly reducing the human and time costs of data construction and providing reliable labeled data support for training high-quality probe-stimulus generation models.

[0090] The value of the second maximum number of optimizations can be set according to actual needs, and this application embodiment does not impose specific restrictions.

[0091] For example, when generating the initial collaborative interaction content for each second participant based on the collaborative interaction scenario description, a generation strategy of "parallel within rounds, sequential between rounds" can be adopted. That is, the internal cognitive activities and explicit interactive behaviors of each second participant are generated in parallel within the same round to improve the efficiency of collaborative interaction content generation. Specifically, within the same round, the internal cognitive activities and explicit interactive behaviors of each second participant can be generated independently without considering the output content of other second participants in this round. After all the outputs of all second participants have been generated, the round context is updated uniformly, and the generation of the next round begins. In other embodiments, a generation strategy of "sequential within rounds" can also be adopted. That is, within the same round, the explicit interactive behaviors of each second participant are generated sequentially according to a preset speaking order. Participants who speak later can adjust their own expression strategies based on the explicit interactive behaviors of those who speak earlier. The specific settings can be configured according to actual needs.

[0092] For example, when conducting multi-dimensional quality assessment of initial collaborative interaction content, a large language model or a dedicated assessment model can be used as the evaluator. The initial collaborative interaction content to be evaluated and the corresponding collaborative interaction scenario description are input into the evaluator, which scores each item according to the preset assessment dimensions and outputs the second assessment score corresponding to each dimension.

[0093] For example, when conducting multi-dimensional quality assessments of initial collaborative interaction content, a large language model or a dedicated assessment model can be used as the evaluator. The initial collaborative interaction content to be evaluated, along with the corresponding description of the collaborative interaction scenario, is input into the evaluator. The evaluator scores each item according to preset assessment dimensions and outputs a second assessment score for each dimension. Among these dimensions: Naturalness assesses whether the initial collaborative interaction content is fluent and natural, and whether it conforms to human dialogue habits and expression in real collaborative scenarios. Implicitness assesses whether preset implicit cognitive disagreements are adequately hidden beneath the participants' surface consensus, rather than being directly or overtly expressed. Coverage assesses whether the initial collaborative interaction content fully covers all preset disagreements, key issues, and scenario elements in the collaborative interaction scenario description, and whether each participant's public profile and private cognitive state are fully reflected during the interaction. Role Consistency assesses whether each participant's overt interactive behaviors, such as speaking style, vocabulary habits, decision-making tendencies, and emotional reactions, are consistent with their publicly defined role identity, professional background, and personality traits. The completeness of auxiliary information is used to assess whether auxiliary information, such as internal cognitive activities, is complete and usable in the initial collaborative interaction content. This includes whether the internal cognitive activities cover key elements such as reasoning processes, decision-making basis, and strategic considerations, and whether the correspondence between internal cognitive activities and explicit interactive behaviors in each round is clear and traceable. The higher the score of auxiliary information completeness, the stronger the explanatory power of the internal cognitive activities for explicit interactive behaviors, and the more sufficient the data support for role simulation in the subsequent response difference verification mechanism.

[0094] Each of the above dimensions is configured with a corresponding scoring threshold. The scoring thresholds for each dimension can be the same or different, and can be flexibly set according to the actual application scenario and quality requirements. After the evaluator scores each dimension independently, it compares the evaluation score of each dimension with the corresponding scoring threshold to determine whether the description of the initial collaborative interaction content meets the quality standards.

[0095] It is understood that the above evaluation dimensions are merely illustrative examples. In actual implementation, evaluation dimensions can be added, reduced, or adjusted according to the characteristics of the target application domain and data quality requirements, as long as the quality of the initial collaborative interaction content can be effectively evaluated. This application embodiment does not impose specific limitations on this.

[0096] S202. Input the collaborative interaction content of multiple second participants into the initial probe stimulus generation model, and output multiple probe stimulus samples through the initial probe stimulus generation model.

[0097] Collaborative interaction content includes both internal cognitive activities and external interactive behaviors.

[0098] For example, the initial probe stimulus generation model can be any of the general large language models based on the Transformer architecture, such as LLaMA, GPT, Qwen, ChatGLM, etc., and can be set according to actual needs.

[0099] It should be noted that the specific implementation of outputting multiple probe stimulus samples through the initial probe stimulus generation model in S202 is similar to the specific implementation of outputting multiple probe stimuli through the probe stimulus generation model in S102 above. Please refer to the relevant description in S102 above. Here, the embodiments of this application will not be described again.

[0100] S203. For each probe stimulus sample, obtain the probe response samples generated by each second participant in response to the probe stimulus sample.

[0101] It should be noted that the specific implementation of obtaining the probe response samples generated by each second participant in response to the probe stimulus in S203 is similar to the specific implementation of obtaining the probe response generated by each first participant in response to the probe stimulus in S103 above. Please refer to the relevant description in S103 above. Here, the embodiments of this application will not be repeated.

[0102] S204. Based on multiple probe response samples from each second participant, identify whether there are implicit cognitive disagreements among the multiple second participants.

[0103] It should be noted that the specific implementation of identifying whether there is implicit cognitive disagreement among multiple second participants based on multiple probe response samples of each second participant in S204 is similar to the specific implementation of identifying whether there is implicit cognitive disagreement among multiple first participants based on multiple probe responses corresponding to each first participant in S104 above. Please refer to the relevant description in S104 above. Here, the embodiments of this application will not be repeated.

[0104] S205. Based on the identification results, determine the valid and invalid probe stimulus samples from multiple probe stimulus samples.

[0105] For example, in some embodiments, determining valid and invalid probe stimulus samples from a plurality of probe stimulus samples based on the identification results may include: When the identification results indicate the existence of implicit cognitive disagreement, it is determined whether the probe stimulus sample with implicit cognitive disagreement meets the preset conditions. The preset conditions include: the probe stimulus sample is clearly expressed, that is, the question itself is unambiguous; the second participant can accurately understand and respond; the revealed disagreement is implicit and not explicit, that is, the disagreement has not been publicly expressed by the second participant in the collaborative interaction content; it does not constitute redundancy with existing probe stimulus samples, that is, the cognitive dimension covered by the probe stimulus sample does not overlap with existing probe stimulus templates. Based on the judgment results, multiple probe stimulus samples are classified. The classification results include: probe stimulus samples that reveal preset disagreement, that is, successfully detect the pre-set disagreement point in the collaborative interaction scenario; probe stimulus samples that reveal other real disagreements outside the preset disagreement, that is, discover other cognitive differences that exist between participants but are not at the preset disagreement point; and probe stimulus samples that do not have disagreement, that is, probe stimulus samples that fail to elicit any real response differences. Among them, valid probe stimulus samples include probe stimulus samples that reveal preset disagreement and / or probe stimulus samples that reveal other real disagreements outside the preset disagreement, and invalid probe stimulus samples include probe stimulus samples that do not have disagreement.

[0106] This two-stage judgment mechanism, which combines "response difference verification" with "preset condition screening," enables refined quality grading of probe stimulus samples. On the one hand, it ensures that effective probe stimulus samples possess high quality and high information density, providing reliable positive supervision for the training of the probe stimulus generation model. On the other hand, by separately classifying "revealing other real disagreements beyond the preset," it endows this application with the open-ended identification capability to discover other real disagreements beyond the preset. Simultaneously, by explicitly marking samples without disagreements as invalid probe stimulus samples, it provides clear negative sample constraints for the training of the probe stimulus generation model, further strengthening the model's ability to identify and generate effective probe stimuli.

[0107] For example, in some embodiments, after obtaining the classification results, a comprehensive evaluation index corresponding to the initial probe stimulus generation model can be constructed based on the classification results. The comprehensive evaluation index reflects the recognition capability of the initial probe stimulus generation model, and the recognition capability includes at least one of the following: the ability to cover preset divergences, the ability to detect other real divergences, and the refinement of the generated multiple probe stimulus samples.

[0108] Among them, coverage capability reflects how many preset divergences the probe stimulus samples output by the initial probe stimulus generation model can hit; detection capability reflects how many real divergences outside the preset can be detected by the initial probe stimulus generation model; and refinement reflects the proportion of effective probe stimulus samples in the probe stimulus samples output by the initial probe stimulus generation model.

[0109] For example, comprehensive evaluation metrics may include, but are not limited to, precision, recall, F1 score and its variants, weighted score, area integral metrics, and composite metrics based on multi-objective fusion, which can be set according to actual needs.

[0110] S206. Based on valid and invalid probe stimulus samples, train the initial probe stimulus generation model to obtain the probe stimulus generation model.

[0111] For example, training an initial probe stimulus generation model based on valid and invalid probe stimulus samples can include at least three possible implementations: One possible implementation is to use a reinforcement learning-based training paradigm to train a probe stimulus generation model.

[0112] The effective detection ratio of valid probe stimuli and the invalid detection ratio of invalid probe stimuli are determined. The effective detection ratio is constructed as a positive reward signal, and the invalid detection ratio as a negative reward signal. Combined with a comprehensive evaluation index, a policy gradient algorithm is used to train the initial probe stimulus generation model. Specifically, the effective detection ratio reflects the initial probe stimulus generation model's ability to generate valid probe stimuli, the invalid detection ratio reflects the model's tendency to generate invalid probe stimuli, and the comprehensive evaluation index comprehensively reflects the model's overall performance in terms of coverage, discovery ability, and refinement. By constructing the reward signal from these three aspects, the initial probe stimulus generation model can be guided to increase the generation of valid probe stimuli and reduce the generation of invalid probe stimuli during training, thereby continuously optimizing the quality of its probe stimulus output.

[0113] Among them, policy gradient algorithms may include, but are not limited to, PPO, GRPO, variants of the RLHF paradigm, and Actor-Critic algorithms. It should be noted that the specific construction form of the reward signal, such as the weight ratio of each component, whether to perform normalization processing, and whether to introduce auxiliary signals for diversity or simplicity, can be flexibly configured according to actual application requirements, and the embodiments of this application do not impose any restrictions on this.

[0114] In another possible implementation, a preference-based learning training paradigm can be used to train the probe stimulus generation model.

[0115] Valid probe stimuli are designated as preferred samples, and invalid probe stimuli are designated as non-preferred samples, constructing preference pair data. Based on this preference pair data, a preference learning algorithm is used to train the initial probe stimulus generation model. Specifically, each preference pair consists of a valid probe stimulus and an invalid probe stimulus in the same collaborative interaction scenario, with the valid probe stimulus serving as the preferred sample and the invalid probe stimulus as the non-preferred sample. The preference pair data conveys the message to the model: "For a given collaborative interaction, a valid probe stimulus is preferred over an invalid probe stimulus." Through training with a large amount of preference pair data, the probe stimulus generation model gradually learns the generation pattern of valid probe stimuli, thus becoming more inclined to generate probe stimuli that reveal implicit cognitive disagreements in subsequent inferences.

[0116] Among them, preference learning algorithms may include, but are not limited to, DPO, Iterative DPO, KTO, IPO and their variants.

[0117] In another possible implementation, a training paradigm based on supervised fine-tuning (SFT) can be used to train the probe stimulus generation model.

[0118] A supervised learning dataset is constructed based on valid and invalid probe stimulus samples. A supervised fine-tuning algorithm is then used to train the initial probe stimulus generation model based on this dataset. Specifically, the supervised learning dataset consists of input-output pairs, where the input is the collaborative interaction content, and the output is either a valid probe stimulus sample (positive sample) or an invalid probe stimulus sample (negative sample). Through supervised fine-tuning, the probe stimulus generation model learns "which probe stimuli are valid and which are invalid for a given collaborative interaction content," thereby improving its ability to generate valid probe stimuli.

[0119] The supervised fine-tuning algorithm can employ the standard SFT training process. It should be noted that this supervised fine-tuning training can serve as a warm-up stage for the aforementioned reinforcement learning or preference learning training, helping the stimulus generation model obtain a better initial policy before entering reinforcement learning or preference learning; it can also be used independently to directly complete model training in scenarios with limited computational resources or where complex training strategies are not required.

[0120] It is understood that in some embodiments, a multi-stage combined training strategy can also be adopted, that is, the above three training paradigms can be combined with the strategy in a certain order. For example, supervised fine-tuning can be performed first using effective and ineffective probe stimulus samples to warm up the probe stimulus generation model and enable it to obtain good initial performance; then preference learning can be used for fine-tuning to further optimize the output quality of the probe stimulus generation model; finally, reinforcement learning can be used for deep optimization, and the comprehensive evaluation index can be used as a reward signal to drive the probe stimulus generation model to iterate towards higher performance; the training paradigm can also be dynamically switched according to the performance changes during the actual training process, etc., which can be set according to actual needs.

[0121] The common feature of all the above training paradigms is that they all use the validity judgment results automatically generated by the response difference verification mechanism, namely valid probe stimulus samples and invalid probe stimulus samples, as the source of training supervision signals. They do not need to rely on large-scale manually labeled data, thus constructing an end-to-end automated closed loop from data generation, capability assessment to probe stimulus generation model training. This significantly reduces the human and time costs of probe stimulus generation model training, making the training of probe stimulus generation models highly automated and with good engineering feasibility.

[0122] The implicit cognitive disagreement identification device provided in this application will be described below. The implicit cognitive disagreement identification device described below and the implicit cognitive disagreement identification method described above can be referred to in correspondence.

[0123] Figure 3 A schematic diagram of a device for identifying implicit cognitive disagreement provided in an embodiment of this application is shown below. For example, please refer to... Figure 3 As shown, the implicit cognitive divergence identification device 30 may include: The acquisition unit 301 is used to acquire the collaborative interaction content of each of the multiple first participants in the target collaborative interaction scenario, wherein the collaborative interaction content includes internal cognitive activities and external interactive behaviors. The processing unit 302 is used to input the collaborative interaction content of each of the multiple first participants into the probe stimulus generation model, and output multiple probe stimuli through the probe stimulus generation model; wherein, the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; The acquisition unit 301 is further configured to acquire the detection response generated by each of the first participants in response to each detection stimulus; The identification unit 303 is used to identify whether there is a latent cognitive disagreement among the multiple first participants based on the multiple probe responses corresponding to each first participant.

[0124] For example, in some embodiments, for each of the first participants, the acquisition unit 301 is configured to acquire the probe response generated by the first participant in response to the probe stimulus, including at least one of the following: Initiate a probe inquiry to the first participant and obtain the probe response generated by the first participant in response to the probe stimulus; Based on at least one of the first participant’s historical interaction records, decision records, publicly available information, language signals, behavioral signals, document information, etc., obtain the first participant’s detection response to the detection stimulus; The detection response of the first participant to the detection stimulus is obtained by a single sampling or by multiple sampling. The detection response of the first participant to the detection stimulus is obtained by synchronous or asynchronous acquisition.

[0125] For example, in some embodiments, the probe stimulus generation model is trained based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples, including: For each of the multiple collaborative interaction scenarios, obtain the collaborative interaction content of each second participant in the collaborative interaction scenario; The collaborative interaction content of each of the multiple second participants is input into the initial probe stimulus generation model, and multiple probe stimulus samples are output through the initial probe stimulus generation model; For each of the aforementioned probe stimulus samples, obtain the probe response samples generated by each of the second participants in response to the probe stimulus samples; Based on multiple probe response samples from each of the second participants, identify whether there are implicit cognitive disagreements among the multiple second participants; Based on the identification results, the valid probe stimulus sample and the invalid probe stimulus sample are determined from the plurality of probe stimulus samples; Based on the valid probe stimulus samples and the invalid probe stimulus samples, the initial probe stimulus generation model is trained to obtain the probe stimulus generation model.

[0126] For example, in some embodiments, determining the valid probe stimulus sample and the invalid probe stimulus sample from the plurality of probe stimulus samples based on the identification result includes: When the identification result indicates the existence of implicit cognitive divergence, it is determined whether the probe stimulus sample with implicit cognitive divergence meets the preset conditions. The preset conditions include at least one of the following: the probe stimulus sample is clearly expressed, the divergence revealed is implicit and not explicit, and it does not constitute redundancy with existing probe stimulus samples. Based on the judgment results, the multiple probe stimulus samples are classified, and the classification results include: probe stimulus samples that reveal a preset divergence, probe stimulus samples that reveal other real divergences besides the preset divergence, and probe stimulus samples that do not have divergences. The effective probe stimulus samples include probe stimulus samples that reveal a preset divergence and / or probe stimulus samples that reveal other real divergences besides the preset divergence, and the invalid probe stimulus samples include probe stimulus samples that do not have divergences.

[0127] For example, in some embodiments, it also includes: Based on the classification results, a comprehensive evaluation index corresponding to the initial detection stimulus generation model is constructed. The comprehensive evaluation index is used to reflect the recognition ability of the initial probe stimulus generation model. The recognition ability includes at least one of the following: the ability to cover the preset divergence, the ability to discover other real divergences, and the refinement of the generated multiple probe stimulus samples.

[0128] For example, in some embodiments, training the initial probe stimulus generation model based on the valid probe stimulus samples and the invalid probe stimulus samples includes: Determine the effective detection ratio of the effective probe stimulus samples and the invalid detection ratio of the invalid probe stimulus samples; The effective detection ratio is constructed as a positive reward signal, the ineffective detection ratio is constructed as a negative reward signal, and the initial detection stimulus generation model is trained using a policy gradient algorithm in conjunction with the comprehensive evaluation index.

[0129] For example, in some embodiments, training the initial probe stimulus generation model based on the valid probe stimulus samples and the invalid probe stimulus samples includes: The effective probe stimulus samples are identified as preferred samples, and the invalid probe stimulus samples are identified as non-preferred samples, thus constructing preference pair data; Based on the preference pair data, a preference learning algorithm is used to train the initial probe stimulus generation model.

[0130] For example, in some embodiments, training the initial probe stimulus generation model based on the valid probe stimulus samples and the invalid probe stimulus samples includes: Based on the effective probe stimulus samples and the invalid probe stimulus samples, a supervised learning dataset is constructed; Based on the supervised learning dataset, the initial probe stimulus generation model is trained using a supervised fine-tuning algorithm.

[0131] For example, in some embodiments, obtaining the collaborative interaction content of each second participant in the collaborative interaction scenario includes: Seed content is collected from multiple domain knowledge sources; the seed content includes at least one of domain knowledge text, case description text, and question-and-answer interaction text. Based on the seed content, a collaborative interaction scenario description is generated for the collaborative interaction scenario. Based on the description of the collaborative interaction scenario, the collaborative interaction content of each of the second participants is generated.

[0132] For example, in some embodiments, generating a collaborative interaction scenario description based on the seed content includes: The seed content is input into the large language model, and the large language model outputs the initial collaborative interaction scenario description of the collaborative interaction scenario. The initial collaborative interaction scenario description is subjected to a multi-dimensional quality assessment to obtain multiple first assessment scores; wherein, the multi-dimensional assessment includes at least two of the following: the concealment of the disagreement, the specificity, the credibility, the domain authenticity, and the diversity. If all of the first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, then the initial collaborative interaction scenario description is determined as the collaborative interaction scenario description. If at least one of the multiple first evaluation scores is less than the score threshold of the corresponding dimension, the initial collaborative interaction scenario description is optimized, and the optimized collaborative interaction scenario description is re-evaluated in multiple dimensions until all the obtained multiple first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, or until a preset first maximum number of optimizations is reached, and the optimized collaborative interaction scenario description is determined as the collaborative interaction scenario description.

[0133] For example, in some embodiments, generating the respective collaborative interaction content of each second participant based on the collaborative interaction scenario description includes: Based on the description of the collaborative interaction scenario, generate the initial collaborative interaction content for each of the second participants; The initial collaborative interaction content is subjected to a multi-dimensional quality assessment to obtain multiple second assessment scores; wherein, the multi-dimensional assessment includes at least two of the following: naturalness, concealment, coverage, role consistency, and completeness of auxiliary information; If all of the multiple second evaluation scores are greater than or equal to the score threshold of the corresponding dimension, then the initial collaborative interaction content is determined as the collaborative interaction content. If at least one of the multiple second evaluation scores is less than the score threshold of the corresponding dimension, the initial collaborative interaction content is optimized, and the optimized collaborative interaction content is re-evaluated in multiple dimensions until all the obtained multiple second evaluation scores are greater than or equal to the score threshold of the corresponding dimension, or until a preset second maximum number of optimizations is reached, and the optimized collaborative interaction content is determined as the collaborative interaction content.

[0134] The implicit cognitive disagreement identification device 90 provided in this application embodiment can execute the technical solution of the implicit cognitive disagreement identification method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the implicit cognitive disagreement identification method described above. Please refer to the implementation principle and beneficial effects of the implicit cognitive disagreement identification method described above. It will not be repeated here.

[0135] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for identifying implicit cognitive divergence. This method includes: acquiring the collaborative interaction content of multiple first participants in a target collaborative interaction scenario, the collaborative interaction content including internal cognitive activities and external interactive behaviors; inputting the collaborative interaction content of the multiple first participants into a probe stimulus generation model, and outputting multiple probe stimuli through the probe stimulus generation model; wherein the probe stimulus generation model is trained based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; acquiring the probe response generated by each first participant in response to each probe stimulus; and identifying whether there is implicit cognitive divergence among the multiple first participants based on the multiple probe responses corresponding to each first participant.

[0136] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the implicit cognitive divergence identification method provided by the above methods. The method includes: acquiring the collaborative interaction content of multiple first participants in a target collaborative interaction scenario, the collaborative interaction content including internal cognitive activities and external interactive behaviors; inputting the collaborative interaction content of the multiple first participants into a probe stimulus generation model, and outputting multiple probe stimuli through the probe stimulus generation model; wherein, the probe stimulus generation model is trained based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; acquiring the probe response generated by each first participant in response to each probe stimulus; and identifying whether there is an implicit cognitive divergence among the multiple first participants based on the multiple probe responses corresponding to each first participant.

[0138] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for identifying implicit cognitive divergences provided by the above methods. The method includes: acquiring the collaborative interaction content of multiple first participants in a target collaborative interaction scenario, the collaborative interaction content including internal cognitive activities and external interactive behaviors; inputting the collaborative interaction content of the multiple first participants into a probe stimulus generation model, and outputting multiple probe stimuli through the probe stimulus generation model; wherein the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; acquiring the probe response generated by each first participant in response to each probe stimulus; and identifying whether there is an implicit cognitive divergence among the multiple first participants based on the multiple probe responses corresponding to each first participant.

[0139] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for identifying implicit cognitive disagreement, characterized in that, include: Acquire the collaborative interaction content of each of the multiple first participants in the target collaborative interaction scenario, wherein the collaborative interaction content includes internal cognitive activities and external interactive behaviors; The collaborative interaction content of each of the multiple first participants is input into the probe stimulus generation model, and the probe stimulus generation model outputs multiple probe stimuli; wherein, the probe stimulus generation model is generated based on pre-labeled valid probe stimulus samples and invalid probe stimulus samples; For each of the aforementioned detection stimuli, the detection response generated by each of the first participants in response to the detection stimuli is obtained; Based on multiple probe responses corresponding to each of the first participants, identify whether there are implicit cognitive disagreements among the multiple first participants.

2. The method for identifying implicit cognitive disagreement according to claim 1, characterized in that, For each of the first participants, obtaining the probe response of the first participant to the probe stimulus includes at least one of the following: Initiate a probe inquiry to the first participant and obtain the probe response generated by the first participant in response to the probe stimulus; Based on at least one of the first participant’s historical interaction records, decision records, publicly available information, language signals, behavioral signals, document information, etc., obtain the first participant’s detection response to the detection stimulus; The detection response of the first participant to the detection stimulus is obtained by a single sampling or by multiple sampling. The detection response of the first participant to the detection stimulus is obtained by synchronous or asynchronous acquisition.

3. The method for identifying implicit cognitive disagreement according to claim 1, characterized in that, The probe stimulus generation model is trained based on pre-labeled valid and invalid probe stimulus samples, including: For each of the multiple collaborative interaction scenarios, obtain the collaborative interaction content of each second participant in the collaborative interaction scenario; The collaborative interaction content of each of the multiple second participants is input into the initial probe stimulus generation model, and multiple probe stimulus samples are output through the initial probe stimulus generation model; For each of the aforementioned probe stimulus samples, obtain the probe response samples generated by each of the second participants in response to the probe stimulus samples; Based on multiple probe response samples from each of the second participants, identify whether there are implicit cognitive disagreements among the multiple second participants; Based on the identification results, the valid probe stimulus sample and the invalid probe stimulus sample are determined from the plurality of probe stimulus samples; Based on the valid probe stimulus samples and the invalid probe stimulus samples, the initial probe stimulus generation model is trained to obtain the probe stimulus generation model.

4. The method for identifying implicit cognitive disagreement according to claim 3, characterized in that, The step of determining the valid and invalid probe stimulus samples from the plurality of probe stimulus samples based on the identification result includes: When the identification result indicates the existence of implicit cognitive divergence, it is determined whether the probe stimulus sample with implicit cognitive divergence meets the preset conditions. The preset conditions include at least one of the following: the probe stimulus sample is clearly expressed, the divergence revealed is implicit and not explicit, and it does not constitute redundancy with existing probe stimulus samples. Based on the judgment results, the multiple probe stimulus samples are classified, and the classification results include: probe stimulus samples that reveal a preset divergence, probe stimulus samples that reveal other real divergences besides the preset divergence, and probe stimulus samples that do not have divergences. The effective probe stimulus samples include probe stimulus samples that reveal a preset divergence and / or probe stimulus samples that reveal other real divergences besides the preset divergence, and the invalid probe stimulus samples include probe stimulus samples that do not have divergences.

5. The method for identifying implicit cognitive disagreement according to claim 4, characterized in that, The method further includes: Based on the classification results, a comprehensive evaluation index corresponding to the initial detection stimulus generation model is constructed. The comprehensive evaluation index is used to reflect the recognition ability of the initial probe stimulus generation model. The recognition ability includes at least one of the following: the ability to cover the preset divergence, the ability to discover other real divergences, and the refinement of the generated multiple probe stimulus samples.

6. The method for identifying implicit cognitive disagreement according to claim 5, characterized in that, The step of training the initial probe stimulus generation model based on the effective probe stimulus samples and the invalid probe stimulus samples includes: Determine the effective detection ratio of the effective probe stimulus samples and the invalid detection ratio of the invalid probe stimulus samples; The effective detection ratio is constructed as a positive reward signal, the ineffective detection ratio is constructed as a negative reward signal, and the initial detection stimulus generation model is trained using a policy gradient algorithm in conjunction with the comprehensive evaluation index.

7. The method for identifying implicit cognitive disagreement according to claim 3, characterized in that, The step of training the initial probe stimulus generation model based on the effective probe stimulus samples and the invalid probe stimulus samples includes: The effective probe stimulus samples are identified as preferred samples, and the invalid probe stimulus samples are identified as non-preferred samples, thus constructing preference pair data; Based on the preference pair data, a preference learning algorithm is used to train the initial probe stimulus generation model.

8. The method for identifying implicit cognitive disagreement according to claim 3, characterized in that, The step of training the initial probe stimulus generation model based on the effective probe stimulus samples and the invalid probe stimulus samples includes: Based on the effective probe stimulus samples and the invalid probe stimulus samples, a supervised learning dataset is constructed; Based on the supervised learning dataset, the initial probe stimulus generation model is trained using a supervised fine-tuning algorithm.

9. The method for identifying implicit cognitive disagreement according to any one of claims 3-8, characterized in that, The step of obtaining the collaborative interaction content of each second participant in the collaborative interaction scenario includes: Seed content is collected from multiple domain knowledge sources; the seed content includes at least one of domain knowledge text, case description text, and question-and-answer interaction text. Based on the seed content, a collaborative interaction scenario description is generated for the collaborative interaction scenario. Based on the description of the collaborative interaction scenario, the collaborative interaction content of each of the second participants is generated.

10. The method for identifying implicit cognitive disagreement according to claim 9, characterized in that, The step of generating a collaborative interaction scenario description based on the seed content includes: The seed content is input into the large language model, and the large language model outputs the initial collaborative interaction scenario description of the collaborative interaction scenario. The initial collaborative interaction scenario description is subjected to a multi-dimensional quality assessment to obtain multiple first assessment scores; wherein, the multi-dimensional assessment includes at least two of the following: the concealment of the disagreement, the specificity, the credibility, the domain authenticity, and the diversity. If all of the first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, then the initial collaborative interaction scenario description is determined as the collaborative interaction scenario description. If at least one of the multiple first evaluation scores is less than the score threshold of the corresponding dimension, the initial collaborative interaction scenario description is optimized, and the optimized collaborative interaction scenario description is re-evaluated in multiple dimensions until all the obtained multiple first evaluation scores are greater than or equal to the score threshold of the corresponding dimension, or until a preset first maximum number of optimizations is reached, and the optimized collaborative interaction scenario description is determined as the collaborative interaction scenario description.

Images