Digital human contradictory dispute mediation method based on intelligent perception and emotion regulation

Abstract

The invention discloses a digital human contradictory dispute mediation method based on intelligent perception and emotion regulation. The method comprises the following steps: firstly, acquiring voice information and visual information of two contradictory parties; calculating a voice emotion score by using the voice information, calculating a visual emotion score by using the visual information, and carrying out weighted calculation to obtain a comprehensive emotion score; then generating an initial speech speed, an initial expression and an initial gesture of the digital human; matching a proper response strategy from the knowledge base and transmitting the response strategy to the digital person; and finally, mapping voice information in the voice emotion score to a voice synthesis parameter, generating emotional voice signals consistent with both parties of a contradictory dispute in language by combining a response strategy, and outputting the emotional voice signals by a digital person for contradictory dispute mediation. In the mediation process, the digital person monitors the comprehensive emotion score and the contradictory scene in real time for dynamic adjustment. The real emotional state of the user is accurately captured based on intelligent perception, emotion mediation is carried out based on emotion perception, and the reality sense of digital human interaction and the user trust degree are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of conflict and dispute mediation and artificial intelligence, and specifically relates to a digital human conflict and dispute mediation method based on intelligent perception and emotion regulation. Background Art

[0002] Dispute mediation is a process where a third party intervenes to help conflicting parties reach a consensus. Its core lies in balancing emotional engagement with rational negotiation. Traditional mediation relies on human mediators, who intervene through face-to-face communication and body language observation. However, this approach suffers from low efficiency and uneven resource allocation. With the development of artificial intelligence (AI), mediation systems based on digital humans are gaining popularity. However, existing digital human systems primarily rely on voice or text interaction, lacking multimodal feedback such as facial expressions and gestures. This results in a stilted and mechanical approach to conflict mediation. Furthermore, existing systems exhibit significant delays in capturing subtle changes in user emotions, often failing to recognize and respond promptly to micro-expressions and body language. This delays emotional feedback and misses the golden window for mediation response. Furthermore, existing technologies often rely on static knowledge bases for responses, failing to dynamically adjust wording based on real-time user emotions, which can easily lead to greater resistance when users become emotionally agitated. The lack of synergy between the movements and speech output by digital humans, such as sudden pauses while speaking or mismatches between gestures and semantics, further reduces user trust and mediation effectiveness. Summary of the Invention

[0003] The main purpose of this invention is to overcome the shortcomings and deficiencies of the existing technology and provide a method for mediating conflicts and disputes involving digital humans based on intelligent perception and emotional regulation. The method adjusts the digital human's speech speed, expression, and gestures in real time according to emotions, making the interaction natural and smooth, solving the problem of disconnection between traditional digital human movements and voice, and greatly improving the human-computer interaction experience.

[0004] In order to achieve the above objectives, the present invention provides, on one hand, a method for mediating conflicts and disputes among digital humans based on intelligent perception and emotion regulation, comprising: Multimodal data acquisition: Voice and visual information from both parties of the conflict is acquired through voice acquisition equipment and cameras. The visual information includes facial expression key point data and body movements. The body movements are quantified into movement openness indicators using skeletal key point detection technology. Emotion analysis: Use voice information to calculate the voice emotion score, use visual information to calculate the visual emotion score, and then perform a weighted calculation to obtain a comprehensive emotion score; Digital Human Initialization: Generates the initial speech speed, initial expression, and initial gesture of the digital human based on the comprehensive emotion score; Response strategy matching: Based on the voice information of both parties in the dispute, voice recognition and semantic analysis are performed, and the appropriate response strategy is matched from the knowledge base based on the comprehensive emotion score and transmitted to the digital human; Speech output generation: The speech information in the speech emotion score is mapped to speech synthesis parameters, and combined with the response strategy, an emotional speech signal consistent with the language of the two parties in the conflict is generated, which is then output by the digital human to mediate the conflict; Dynamic response of the digital human: When the digital human mediates conflicts and disputes, it dynamically adjusts the digital human's appearance, facial expressions, gestures, speaking speed and response strategy based on the real-time comprehensive emotional scores of both parties and the conflict scenario.

[0005] As a preferred technical solution, the voice emotion score is calculated as follows: , in, E voice is the speech emotion score, F 0 is the fundamental frequency of the voice information, S is the speech speed of the voice information, E is the energy of voice information, F 0min is the minimum value set for the fundamental frequency of speech, F 0max The maximum value set for the fundamental frequency of speech; S max 、 E max is the normalized threshold of speech rate and energy, a 1. a 2. a 3 are the weights of fundamental frequency, speech rate and energy respectively; The visual emotion score is calculated as follows: E face = b 1. I brow + b 2. θ mouth , in, E face is the visual emotion score, I brow is the frown intensity, θ mouth is the angle of drooping corner of mouth, b 1. b 2 are weights of frown intensity and mouth corner droop angle respectively; the frown intensity and mouth corner droop angle are quantified using facial key point detection technology; The comprehensive emotion score calculation formula is: E total = a · E voice + b · E face , in, E total is the comprehensive emotion score, a 、 b are the weights of the voice emotion score and visual emotion score respectively.

[0006] As a preferred technical solution, the basic speaking speed, initial expression and initial gesture of the digital human are generated based on the comprehensive emotion score, specifically: Set the basic speaking speed and initialize the initial speaking speed of the digital human based on the comprehensive emotion score and the basic speaking speed: S output = S base ·(1 – s 1. E total ), in, S output is the initial speaking speed of the digital person, S base To set the basic speaking speed, s 1 is the speech speed adjustment factor, E total is the comprehensive emotion score; Define the smile intensity to generate the initial expression of the digital human. The smile intensity is defined as: I smile = 1 – E total , in, I smile for smile intensity; Calculate the frequency of the open gesture as the initial gesture of the digital human. The frequency calculation formula of the open gesture is: F gesture = 2·(1 – E total ), in, F gesture is the frequency of the open gesture; the open gesture is a palm-up gesture; the frequency of the open gesture is quantified based on limb movements through skeleton key point detection technology.

[0007] As a preferred technical solution, the matching of a suitable response strategy from the knowledge base is specifically as follows: Perform speech recognition and semantic analysis on the speech information, extract keywords and intent categories, and form structured semantic input Q; Match the appropriate response strategy from the set knowledge base, expressed as: K = g match ( Q , E total , DB ), in, K For the response strategy, g match () is a matching function based on rule engine or machine learning, Q For the semantic input of both parties in the conflict, E total is the comprehensive emotion score, DB For the knowledge base; Monitor the overall emotion score. E total When the emotion exceeds the set threshold, the appeasement strategy subset is preferentially retrieved from the knowledge base, and empathy words are automatically inserted to construct the response text: Response = EmpathyPhrase + Solution soft , in, Response For the response text, EmpathyPhrase For empathy, Solution soft For appeasement strategies; Otherwise, the answer text is generated by matching the regular strategy from the knowledge base.

[0008] As a preferred technical solution, the generation of an emotional voice signal consistent with the language of the two parties in the conflict is specifically as follows: The digital human voice emotion parameter vector is set based on the voice emotion score, which is expressed as: , in, is the pitch of the digital person, calculated as: , E voice is the speech emotion score, E norm is the set neutral sentiment score; is the speaking speed of the digital person, and the calculation formula is: ; is the digital human voice softening degree, and the calculation formula is: ; α, β and γ are all preset weight coefficients; Use pre-trained language recognition technology to automatically detect the voice information of the conflicting parties and obtain the target language L ; Call target language L The corresponding TTS speech engine generates emotional speech signals: , in, For emotional speech signals, s is the response text of the response strategy, h TTS This is the text-to-speech function.

[0009] As a preferred technical solution, the dynamic adjustment of the digital human's appearance, facial expressions, gestures, speech speed and response strategy is specifically as follows: Monitor the comprehensive emotional scores of both parties in a conflict in real time and determine the label of the current conflict scenario; Based on the comprehensive emotion score and conflict situation label, the system searches for the appropriate output control parameter combination in the preset behavior parameter mapping table to determine the appearance parameters, facial expression parameters, gesture parameters, speech speed adjustment factor, and response strategy. A synchronous scheduling mechanism coordinates the output rhythm of gestures and semantic keywords to ensure consistency between speech and action. Call the animation rendering engine to generate the corresponding facial expressions, body movements and appearance of the digital human according to the current control parameters.

[0010] The second aspect of the present invention provides a digital human conflict and dispute mediation system based on intelligent perception and emotion regulation, which is applied to the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation, and includes a multimodal data acquisition module, an emotion analysis module, a digital human initialization module, a strategy matching module, a speech generation module, and a dynamic response module; The multimodal data acquisition module is used to acquire voice information and visual information of the conflicting parties through a voice acquisition device and a camera; the visual information includes facial expression key point data and body movements; the body movements are quantified into movement openness indicators through skeletal key point detection technology; The emotion analysis module is used to calculate the speech emotion score using speech information, calculate the visual emotion score using visual information, and perform weighted calculation to obtain a comprehensive emotion score; The digital human initialization module is used to generate the initial speech speed, initial expression and initial gesture of the digital human according to the comprehensive emotion score; The strategy matching module is used to perform speech recognition and semantic analysis based on the voice information of the two parties in the conflict and dispute, and match the appropriate response strategy from the knowledge base based on the comprehensive emotion score and transmit it to the digital human; The speech generation module is used to map the speech information in the speech emotion score to speech synthesis parameters, and combine the response strategy to generate an emotional speech signal consistent with the language of the two parties in the conflict, which is output by the digital human to mediate the conflict; The dynamic response module is used to dynamically adjust the digital human's appearance, facial expressions, gestures, speech speed and response strategy according to the real-time comprehensive emotional scores of both parties in the conflict and the conflict scenario when the digital human mediates conflicts and disputes.

[0011] A third aspect of the present invention is to provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation.

[0012] The fourth aspect of the present invention is to provide a computer-readable storage medium storing a program, which, when executed by a processor, implements the digital human conflict and dispute mediation method based on intelligent perception and emotional regulation.

[0013] The fifth aspect of the present invention is to provide a computer program product, including a computer program or instructions, which, when executed by a processor, implements the digital human conflict and dispute mediation method based on intelligent perception and emotional regulation.

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. Multimodal emotion recognition mechanism integrating body movement features This invention incorporates the openness of body movements (such as palm direction and arm angle) into the emotion analysis system, and combines facial expression key points (frown intensity, mouth corner angle) with voice signals (fundamental frequency, speaking rate, energy) to construct a multimodal emotion fusion model; by introducing skeletal key point detection technology, body features are quantified into computable indicators, which can more accurately capture the user's true emotional state than traditional single recognition methods that rely only on voice or facial expressions.

[0015] 2. Dynamic response strategy generation mechanism based on emotion drive The present invention no longer uses a fixed template or static knowledge base response method, but combines semantic input containing keywords and intent categories with real-time comprehensive emotion scores to build a "two-layer response strategy matching mechanism": that is, the basic response content is matched from the conventional strategy library first, and when it is detected that the user is in a highly excited emotional state ( E total When the emotion is greater than the set emotional threshold, the system automatically switches to the soothing strategy sub-library and inserts empathy language to significantly reduce confrontational emotions and enhance communication fluency.

[0016] 3. Digital human behavior control driven by voice-action collaboration During the output process, the present invention constructs a functional mapping relationship between voice parameters (such as speaking speed and pitch), facial expressions (smile intensity), gesture frequency and emotion scores, ensuring that when the digital human outputs voice, its facial expressions and movements are synchronously matched, avoiding unnatural phenomena of the digital human, and greatly improving the realism of the interaction and user trust.

[0017] 4. Multilingual adaptive speech generation capabilities The present invention integrates language recognition technology, which can automatically identify the language type according to user input and call the matching TTS voice engine to generate emotional voice output that conforms to the user's language, overcoming the limitations of language barriers on mediation effects and expanding the applicable population and scenarios of the system.

[0018] 5. Real-time dynamic response digital human adaptation mechanism This invention constructs a response system of "emotion-scene perception-appearance behavior linkage", which dynamically adjusts the digital human's appearance, facial expressions, gesture frequency, speech speed, response strategy and other parameters according to the user's emotional changes and conflict scene labels (such as "fierce quarrel" and "business negotiation"), significantly improving the user's emotional acceptance and mediation success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is an overall flow chart of the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation in an embodiment of the present invention.

[0021] Figure 2 This is an overall framework diagram of the digital human conflict and dispute mediation method in an embodiment of the present invention.

[0022] Figure 3The flowchart of matching a suitable response strategy in an embodiment of the present invention is shown.

[0023] Figure 4 This is a structural diagram of a digital human conflict and dispute mediation system based on intelligent perception and emotion regulation in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0026] like Figure 1 As shown, this embodiment provides a method for mediating conflicts and disputes between digital humans based on intelligent perception and emotion regulation, comprising the following steps: S1. Multimodal data acquisition: Voice and visual information of both parties in the conflict is acquired through voice acquisition equipment and cameras. Visual information includes facial expression key point data (such as frown intensity and mouth corner angle) and body movements (such as gesture openness and body posture). Body movements are quantified into movement openness indicators (such as palm direction and arm extension range) through skeletal key point detection technology.

[0027] S2. Emotion analysis: Use voice information to calculate the voice emotion score, use visual information to calculate the visual emotion score, and perform weighted calculation to obtain a comprehensive emotion score.

[0028] Furthermore, to obtain the emotional information in the multimodal data, first obtain the voice emotional information from the voice information and calculate the voice emotional score. The calculation method is: , in, E voice is the speech emotion score, F 0 is the fundamental frequency of the voice information, S is the speech speed of the voice information, E The energy of voice information; F0min is the minimum value set for the fundamental frequency of speech, F 0max The maximum value, minimum value and maximum value of the fundamental frequency of the speech are determined by training with historical mediation data; S max 、 E max is the normalized threshold of speech rate and energy, a 1. a 2. a 3 are the weights of fundamental frequency, speech rate and energy respectively.

[0029] Next, we use facial key point detection technology to quantify the visual emotion information in the visual information and calculate the visual emotion score. The calculation formula is: E face = b 1. I brow + b 2. θ mouth , in, E face is the visual emotion score, I brow is the frown intensity, θ mouth is the angle of drooping corner of mouth, b 1. b 2 are the weights of frown intensity and mouth corner drooping angle respectively; wherein, frown intensity and mouth corner drooping angle are quantified using facial key point detection technology.

[0030] Finally, the weighted fusion method is used to fuse the voice emotion information and visual emotion information to obtain a comprehensive emotion score, which is calculated as follows: E total = a · E voice + b · E face , in, E total is the comprehensive emotion score, a 、 b are the weights of the voice emotion score and visual emotion score respectively.

[0031] In this embodiment, a 1. a 2. a 3 takes values ​​of 0.5, 0.3, and 0.2 respectively. b 1.b 2 takes values ​​of 0.6 and 0.4 respectively, a 、 b The values ​​are 0.7 and 0.3 respectively.

[0032] S3. Digital Human Initialization: Generate the initial speaking speed, initial expression and initial gesture of the digital human based on the comprehensive emotion score.

[0033] Furthermore, the digital human is initialized, including: S3.1. Set the basic speaking speed and initialize the initial speaking speed of the digital human based on the comprehensive emotion score and the basic speaking speed: S output = S base ·(1 – s 1. E total ), in, S output is the initial speaking speed of the digital person, S base To set the basic speaking speed, s 1 is the speech speed adjustment factor, E total is the comprehensive emotion score; the unit of speech speed is words per minute. In this embodiment, the basic speech speed is set to 120 words per minute.

[0034] S3.2. Define the smile intensity to generate the initial expression of the digital human. The smile intensity is defined as: I smile = 1 – E total , in, I smile For smile intensity.

[0035] S3.3. Calculate the frequency of the open gesture as the initial gesture of the digital human. The frequency calculation formula of the open gesture is: F gesture = 2·(1 – E total ), in, F gesture is the frequency of open gestures; where the palm facing up is an open gesture; the frequency of open gestures is quantified based on limb movements using skeleton key point detection technology.

[0036] S4. Response strategy matching: Perform voice recognition and semantic analysis based on the voice information of both parties in the conflict, and match the appropriate response strategy from the knowledge base based on the comprehensive emotion score and transmit it to the digital human.

[0037] Furthermore, the specific method of matching the appropriate response strategy is as follows: S4.1. First, perform speech recognition and semantic analysis on the voice information of both parties in the dispute, extract keywords and intent categories (such as complaint, negotiation, accusation, etc.), and form a structured semantic input Q.

[0038] In this embodiment, the specific method of obtaining the semantic input Q is as follows: (1) Speech recognition: The speech information of the two parties in the dispute is recognized and processed, and an end-to-end speech recognition model (such as Whisper-large-v3) is used to convert the speech signal into a text sequence; (2) Semantic parsing: Applying a BERT-based natural language understanding model to the identified text sequence for semantic parsing and extracting structured semantic information, including: Keyword extraction: extracting key elements related to the mediation content (such as "compensation amount", "responsibility division", "contract terms", etc.); Intent classification: identifying the intention labels of user speech (such as [negotiation request], [responsibility complaint], [emotional catharsis], etc.); Emotional tendency recognition: combining comprehensive emotion scores E total , annotate the emotional tendency of semantic sentences (such as "calm", "neutral", "excited") to provide support for subsequent response strategy selection.

[0039] S4.2. Match the appropriate response strategy from the set knowledge base, expressed as: K = g match ( Q , E total , DB ), in, K For the response strategy, g match () is a matching function based on rule engines or machine learning (for example, using the TF-IDF algorithm to calculate semantic similarity and combining sentiment score weights for sorting and matching). Q For the semantic input of both parties in the conflict, E total is the comprehensive emotion score, DB For the knowledge base.

[0040] S4.3, monitor the comprehensive emotion score. E totalWhen the emotion exceeds the set threshold, the appeasement strategy subset is first retrieved from the knowledge base and empathy phrases (such as "I understand how you feel now...") are added to construct the response text: Response = EmpathyPhrase + Solution soft , in, Response For the response text, EmpathyPhrase For empathy, Solution soft For appeasement strategies; S4.3. Otherwise, generate the response text by matching the conventional strategy from the knowledge base K .

[0041] In this embodiment, when the comprehensive emotion score is greater than the set emotion threshold of 0.8, a soothing strategy is selected and empathy language is added to construct a response strategy, thereby making the mediation more friendly.

[0042] S5. Voice output generation: The voice information in the voice emotion score is mapped to voice synthesis parameters, and combined with the response strategy, an emotional voice signal consistent with the language of the two parties in the conflict is generated and output by the digital human to mediate the conflict.

[0043] Furthermore, based on the acquired response strategy and the voice information in the voice emotion, a voice signal is generated and output by the digital human. Specifically: S5.1. Set the digital human voice emotion parameter vector based on the voice emotion score, expressed as: , in, is the pitch of the digital person, calculated as: , E voice is the speech emotion score, E norm is the set neutral sentiment score; is the speaking speed of the digital person, and the calculation formula is: ; is the digital human voice softening degree, and the calculation formula is: ; α, β and γ are all preset weight coefficients.

[0044] S5.2. Use language recognition technology (such as FastText) to automatically detect the voice information of the conflicting parties and obtain the target language L .

[0045] S5.3. Calling the target language LThe corresponding TTS speech engine (such as the multilingual version of Google Text-to-Speech) generates emotional speech signals: , in, For emotional speech signals, s is the response text of the response strategy, h TTS This is the text-to-speech function.

[0046] S6. Dynamic adjustment of the digital human: When the digital human mediates conflicts and disputes, it dynamically adjusts the digital human's appearance, facial expressions, gestures, speaking speed and response strategy based on the real-time comprehensive emotional scores of both parties and the conflict scenario.

[0047] Further, according to E total and scenario requirements Sc , dynamically adjust the digital human's appearance, facial expressions, gestures, speaking speed and response strategies, specifically: S6.1. Emotion-Scene Perception: Real-time Monitoring of the Comprehensive Emotional Scores of Both Parties in a Conflict E total , and determine the current conflict scenario label Sc .

[0048] S6.2. Parameter Mapping Initialization: Based on the Comprehensive Emotion Score E total Tags with contradictory scenarios Sc , search for the appropriate output control parameter combination in the preset behavior parameter mapping table, determine the appearance parameters (such as clothing style, color tone and style), facial expression parameters (such as smile intensity, frown degree), gesture parameters (such as gesture frequency, palm direction, arm amplitude), speech rate adjustment factor and response strategy, etc.

[0049] S6.3. Synchronous control of gestures and speech: During speech output, a synchronous scheduling mechanism (such as the dynamic time warping algorithm) is used to coordinate the output rhythm of gestures and semantic keywords to ensure consistency between speech and gesture performance.

[0050] S6.4, Digital Human Performance Rendering: Call the animation rendering engine to generate the corresponding facial expressions, body movements and appearance of the digital human according to the current control parameters; such as through the use of technologies such as skeletal animation interpolation, expression BlendShape and dynamic switching of clothing materials to ensure smooth and natural transitions.

[0051] S6.5. Continuous Feedback and Iterative Updates: Continuously Monitoring the Composite Emotional Score During Mediation E totalIf there are significant changes in the user's emotions or changes in the scene status, the system automatically repeats the above process and regenerates the adapted appearance and behavioral response to maintain the real-time and emotional consistency of the digital human output.

[0052] In this embodiment, during the emotional scene monitoring, the comprehensive emotional score is updated every second. E total , and identify conflict scene labels (such as "fierce quarrel" and "business negotiation") through a scene classification model (such as LSTM). The parameter mapping rules are: Appearance: When the conflict scene is labeled as intense, the digital human switches to warm orange (RGB: 255, 165, 0) silk clothing and the hairstyle is adjusted to a low ponytail; when the conflict scene is labeled as stable, the digital human adopts business blue (RGB: 0, 128, 255) suit fabric clothing and the hairstyle is a standard bun / short hair.

[0053] Facial expression: When E total >0.8, smile intensity I smile Set to 0.3+0.7(1- E total ), frown intensity I brow Cleared, the eye gaze frequency decreased to 3 times / minute; when E total <0.3, restore the standard smile ( I smile =0.5) and neutral eyebrow shape ( I brow =0.1).

[0054] Gestures: When emotions are high, the frequency of open gestures increases. F gesture The frequency is set to 1 time / minute, and only soothing gestures with the palm facing up and the arm angle ≤45° are allowed; when the mood is stable, the gesture frequency is increased to 4 times / minute, and enumeration and guidance gestures that match the semantics are supported.

[0055] Voice-Action Synergy: Based on E total Dynamically adjust speech speed ( S output = S base (1 – 0.2 E total )), and ensure that the synchronization rate between gestures and semantic keywords is ≥95% through action-speech synchronization algorithms (such as DTW dynamic time warping).

[0056] Finally, through Unreal Engine's MetaHuman animation system, the digital human's posture is updated based on the skeletal animation interpolation algorithm (time step of 200ms). Facial expressions are driven in real time through BlendShape, and clothing materials are dynamically switched through Shader to achieve the dynamic response of the digital human.

[0057] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0058] Based on the same concept as the digital human conflict and dispute mediation method based on intelligent perception and emotional regulation in the above-mentioned embodiment, the present invention also provides a digital human conflict and dispute mediation system based on intelligent perception and emotional regulation, which can be used to execute the above-mentioned digital human conflict and dispute mediation method based on intelligent perception and emotional regulation. For ease of explanation, the structural diagram of the embodiment of the digital human conflict and dispute mediation system based on intelligent perception and emotional regulation only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0059] like Figure 3 As shown, another embodiment of the present invention provides a digital human conflict and dispute mediation system based on intelligent perception and emotion regulation, including a multimodal data acquisition module, an emotion analysis module, a digital human initialization module, a strategy matching module, a speech generation module and a dynamic response module; The multimodal data acquisition module is used to acquire voice and visual information of both parties in the conflict through voice acquisition equipment and cameras; the visual information includes facial expression key point data and body movements; the body movements are quantified into movement openness indicators through skeletal key point detection technology; The emotion analysis module is used to calculate the speech emotion score using speech information, calculate the visual emotion score using visual information, and perform weighted calculation to obtain a comprehensive emotion score; The digital human initialization module is used to generate the digital human's initial speech speed, initial expression, and initial gesture based on the comprehensive emotion score; The strategy matching module is used to perform speech recognition and semantic analysis based on the voice information of both parties in the conflict. It also matches the appropriate response strategy from the knowledge base based on the comprehensive emotion score and transmits it to the digital human. The speech generation module is used to map the speech information in the speech emotion score to speech synthesis parameters, and combines it with the response strategy to generate emotional speech signals consistent with the language of the two parties in the conflict, which are then output by the digital human to mediate the conflict. The dynamic response module is used to dynamically adjust the digital human's appearance, facial expressions, gestures, speaking speed and response strategy based on the real-time comprehensive emotional scores of both parties in the conflict and the conflict scenario when the digital human is mediating conflicts and disputes.

[0060] It should be noted that the digital human conflict and dispute mediation system based on intelligent perception and emotional regulation of the present invention corresponds one-to-one to the digital human conflict and dispute mediation method based on intelligent perception and emotional regulation of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the digital human conflict and dispute mediation method based on intelligent perception and emotional regulation are applicable to the embodiment of the digital human conflict and dispute mediation system based on intelligent perception and emotional regulation. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0061] In addition, in the implementation of the digital human conflict and dispute mediation system based on intelligent perception and emotional regulation in the above-mentioned embodiment, the logical division of each program module is only an example. In actual application, the above-mentioned functions can be distributed to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the digital human conflict and dispute mediation system based on intelligent perception and emotional regulation is divided into different program modules to complete all or part of the functions described above.

[0062] like Figure 4 As shown, in one embodiment, a computer-readable storage medium is provided, which stores a program in a memory. When the program is executed by a processor, the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation is implemented, specifically: The audio and visual information of both parties in the conflict is obtained through voice acquisition equipment and cameras. The visual information includes facial expression key point data and body movements. Body movements are quantified into movement openness indicators using skeletal key point detection technology. The emotion analysis module is used to calculate the speech emotion score using speech information, calculate the visual emotion score using visual information, and perform weighted calculation to obtain a comprehensive emotion score; Generate the digital human's initial speech speed, initial expression, and initial gesture based on the comprehensive emotion score; Perform speech recognition and semantic analysis based on the voice information of both parties in the dispute, and match the appropriate response strategy from the knowledge base based on the comprehensive emotion score and transmit it to the digital human; The speech information in the speech emotion score is mapped to speech synthesis parameters, and combined with the response strategy, an emotional speech signal consistent with the language of the two parties in the conflict is generated, which is output by the digital human to mediate the conflict; When the digital human mediates conflicts and disputes, it dynamically adjusts the digital human's appearance, facial expressions, gestures, speaking speed and response strategy based on the real-time comprehensive emotional scores of both parties and the conflict scenario.

[0063] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0064] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A digital human conflict and dispute mediation method based on intelligent perception and emotion regulation, characterized by: include: Multimodal data acquisition: Voice and visual information from both parties of the conflict is acquired through voice acquisition equipment and cameras. The visual information includes facial expression key point data and body movements. The body movements are quantified into movement openness indicators using skeletal key point detection technology. Emotion analysis: Use voice information to calculate the voice emotion score, use visual information to calculate the visual emotion score, and then perform a weighted calculation to obtain a comprehensive emotion score; Digital Human Initialization: Generates the initial speech speed, initial expression, and initial gesture of the digital human based on the comprehensive emotion score; Response strategy matching: Based on the voice information of both parties in the dispute, voice recognition and semantic analysis are performed, and the appropriate response strategy is matched from the knowledge base based on the comprehensive emotion score and transmitted to the digital human; Speech output generation: The speech information in the speech emotion score is mapped to speech synthesis parameters, and combined with the response strategy, an emotional speech signal consistent with the language of the two parties in the conflict is generated, which is then output by the digital human to mediate the conflict; Dynamic response of the digital human: When the digital human mediates conflicts and disputes, it dynamically adjusts the digital human's appearance, facial expressions, gestures, speaking speed and response strategy based on the real-time comprehensive emotional scores of both parties and the conflict scenario.

2. The digital human conflict and dispute mediation method based on intelligent perception and emotion regulation according to claim 1 is characterized in that: The voice emotion score is calculated as follows: ， in, E voice is the speech emotion score, F 0 is the fundamental frequency of the voice information, S is the speech speed of the voice information, E is the energy of voice information, F 0min is the minimum value set for the fundamental frequency of speech, F 0max The maximum value set for the fundamental frequency of speech; S max 、 E max is the normalized threshold of speech rate and energy, a 1. a 2. a 3 are the weights of fundamental frequency, speech rate and energy respectively; The visual emotion score is calculated as follows: E face = b 1· I brow + b 2· θ mouth ， in, E face is the visual emotion score, I brow is the frown intensity, θ mouth is the angle of drooping corner of mouth, b 1. b 2 are weights of frown intensity and mouth corner droop angle respectively; the frown intensity and mouth corner droop angle are quantified using facial key point detection technology; The comprehensive emotion score calculation formula is: E total = a · E voice + b · E face ， in, E total is the comprehensive emotion score, a 、 b are the weights of the voice emotion score and visual emotion score respectively.

3. The digital human conflict and dispute mediation method based on intelligent perception and emotion regulation according to claim 1 is characterized in that: The basic speaking speed, initial expression and initial gesture of the digital human are generated based on the comprehensive emotion score, specifically: Set the basic speaking speed and initialize the initial speaking speed of the digital human based on the comprehensive emotion score and the basic speaking speed: S output = S base ·(1 – s 1· E total )， in, S output is the initial speaking speed of the digital person, S base To set the basic speaking speed, s 1 is the speech speed adjustment factor, E total is the comprehensive emotion score; Define the smile intensity to generate the initial expression of the digital human. The smile intensity is defined as: I smile = 1 – E total ， in, I smile for smile intensity; Calculate the frequency of the open gesture as the initial gesture of the digital human. The frequency calculation formula of the open gesture is: F gesture = 2·(1 – E total )， in, F gesture is the frequency of the open gesture; the open gesture is a palm-up gesture; the frequency of the open gesture is quantified based on limb movements through skeleton key point detection technology.

4. The digital human conflict and dispute mediation method based on intelligent perception and emotion regulation according to claim 1 is characterized in that: The matching of appropriate response strategies from the knowledge base is specifically as follows: Perform speech recognition and semantic analysis on the speech information, extract keywords and intent categories, and form structured semantic input Q; Match the appropriate response strategy from the set knowledge base, expressed as: K = g match ( Q , E total , DB )， in, K For the response strategy, g match () is a matching function based on rule engine or machine learning, Q For the semantic input of both parties in the conflict, E total is the comprehensive emotion score, DB For the knowledge base; Monitor the overall emotion score. E total When the emotion exceeds the set threshold, the appeasement strategy subset is preferentially retrieved from the knowledge base, and empathy words are automatically inserted to construct the response text: Response = EmpathyPhrase + Solution soft ， in, Response For the response text, EmpathyPhrase For empathy, Solution soft For soothing strategies; Otherwise, the answer text is generated by matching the regular strategy from the knowledge base.

5. The digital human conflict and dispute mediation method based on intelligent perception and emotion regulation according to claim 1 is characterized in that: The generation of an emotional speech signal consistent with the language of the two parties in the conflict is specifically as follows: The digital human voice emotion parameter vector is set based on the voice emotion score, which is expressed as: ， in, is the pitch of the digital person, calculated as: , E voice is the speech emotion score, E norm is the set neutral sentiment score; is the speaking speed of the digital person, and the calculation formula is: ; is the digital human voice softening degree, and the calculation formula is: ; α, β and γ are all preset weight coefficients; Use pre-trained language recognition technology to automatically detect the voice information of the conflicting parties and obtain the target language L ; Call target language L The corresponding TTS speech engine generates emotional speech signals: ， in, For emotional speech signals, s is the response text of the response strategy, h TTS This is the text-to-speech function.

6. The digital human conflict and dispute mediation method based on intelligent perception and emotion regulation according to claim 1 is characterized in that: The dynamic adjustment of the digital human's appearance, facial expressions, gestures, speech speed and response strategy is specifically as follows: Monitor the comprehensive emotional scores of both parties in a conflict in real time and determine the label of the current conflict scenario; Based on the comprehensive emotion score and conflict situation label, the system searches for the appropriate output control parameter combination in the preset behavior parameter mapping table to determine the appearance parameters, facial expression parameters, gesture parameters, speech speed adjustment factor, and response strategy. A synchronous scheduling mechanism coordinates the output rhythm of gestures and semantic keywords to ensure consistency between speech and action. Call the animation rendering engine to generate the corresponding facial expressions, body movements and appearance of the digital human according to the current control parameters.

7. The digital human conflict and dispute mediation system based on intelligent perception and emotion regulation is characterized by: The digital human conflict and dispute mediation method based on intelligent perception and emotion regulation, as described in any one of claims 1 to 6, comprises a multimodal data acquisition module, an emotion analysis module, a digital human initialization module, a strategy matching module, a speech generation module, and a dynamic response module; The multimodal data acquisition module is used to acquire voice information and visual information of the conflicting parties through a voice acquisition device and a camera; the visual information includes facial expression key point data and body movements; the body movements are quantified into movement openness indicators through skeletal key point detection technology; The emotion analysis module is used to calculate the speech emotion score using speech information, calculate the visual emotion score using visual information, and perform weighted calculation to obtain a comprehensive emotion score; The digital human initialization module is used to generate the initial speech speed, initial expression and initial gesture of the digital human according to the comprehensive emotion score; The strategy matching module is used to perform speech recognition and semantic analysis based on the voice information of the two parties in the conflict and dispute, and match the appropriate response strategy from the knowledge base based on the comprehensive emotion score and transmit it to the digital human; The speech generation module is used to map the speech information in the speech emotion score to speech synthesis parameters, and combine the response strategy to generate an emotional speech signal consistent with the language of the two parties in the conflict, which is output by the digital human to mediate the conflict; The dynamic response module is used to dynamically adjust the digital human's appearance, facial expressions, gestures, speech speed and response strategy according to the real-time comprehensive emotional scores of both parties in the conflict and the conflict scenario when the digital human mediates conflicts and disputes.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation as described in any one of claims 1-6.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by the processor, the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by the processor, the digital human conflict and dispute mediation method based on intelligent perception and emotion regulation as described in any one of claims 1 to 6 is implemented.

Images