Intelligent intervention generation method and system for vehicle-mounted embarrassing scene

Abstract

The invention belongs to the technical field of vehicle-mounted multi-person voice interaction, and relates to a vehicle-mounted embarrassing scene-oriented intelligent intervention generation method and system, and the method comprises the steps: continuously monitoring in-vehicle multi-mode state signals, recognizing and classifying embarrassing scenes, and generating and executing an intervention strategy. According to the invention, the fundamental defect problem that the social relationship between users cannot be perceived and understood in the prior art is solved, so that the vehicle-mounted assistant is no longer a machine which can only passively respond to an instruction, but is an intelligent agent which can preliminarily understand the interpersonal atmosphere and the social state in the vehicle. According to the method, the virtual assistant does not only meet functional requirements of the user any more, can more actively provide positive emotional value, and plays a role of a social lubricant when the user needs most, so that the humanization and comfort of riding experience are greatly improved. In addition, the problem that in the prior art, conflicts among the users are impossible is solved, potential social conflicts in the vehicle can be effectively avoided or solved, and the journey of multi-person riding is more harmonious and pleasant.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of vehicle-mounted multi-person voice interaction, and particularly relates to an intelligent intervention generation method and system for a vehicle-mounted awkward scene. BACKGROUND

[0002] In the process of evolution of a vehicle-mounted intelligent cockpit towards multi-user and multi-scene interaction, how to effectively process voice instructions simultaneously or alternately issued by multiple passengers in the vehicle is a main technical challenge to improve the interactive experience. Therefore, the prior art realizes the differentiation and recognition of different sound sources in the vehicle through the combination of software and hardware to support basic multi-person simultaneous interaction.

[0003] According to the technical implementation path, the interaction types can be roughly summarized as follows: the first type is a sound source positioning and separation technology based on a microphone array. Chinese patent CN110070868B discloses a voice interaction method of a vehicle-mounted system. As shown in Figure 1 , first, the sound source signals of each sound area in the vehicle are collected through a vehicle-mounted microphone array; then, the system simultaneously recognizes these sound source signals to distinguish which signals are valid voice from different users. After recognizing the voice signals of multiple users, the system generates corresponding voice instructions for each user's voice signal and performs multi-thread processing to execute operations corresponding to each instruction. This method realizes the differentiation of sound sources in different physical positions through a microphone array and a signal processing algorithm, so that multiple users' instructions can be "simultaneously heard" and "separately executed". The second type introduces more advanced voiceprint recognition and clustering algorithms. Chinese patent CN117854517A discloses a vehicle-mounted multi-person real-time intelligent voice interaction system. As shown in Figure 2 , the system clusters the collected audio through an algorithm, which means that the system can not only distinguish the direction of the sound source, but also determine whether it is "A's voice" or "B's voice" through voiceprint feature analysis. After completing the speaker clustering, the system uses a natural language processing module to perform real-time voice-to-text conversion on each speaker's audio. Finally, the system performs human-computer interaction processing according to the target speaker. This system strives to improve the recognition accuracy of multi-person interaction by more accurately binding the voice content and the speaker's identity. The third type integrates visual information to assist voice recognition. Chinese patent CN117133292A also discloses an in-vehicle voice interaction method based on audio-visual fusion. As shown in Figure 3As shown, this method first acquires visual information of passengers' lips using an in-vehicle camera and performs lip movement detection to determine who the current speaker is. Based on this, it controls the microphones at the corresponding locations for directional sound pickup. Finally, the system fuses the acquired lip movement sequence images with the directionally acquired speech signals for joint speech recognition. This method introduces a visual dimension to achieve more accurate speaker identification and wake-free interaction, and improves recognition robustness in noisy environments.

[0004] However, the current development focus of in-vehicle multi-person interaction technology is entirely concentrated on solving technical problems at the "physical recognition" level, while there is a significant technological gap in the higher-dimensional "social atmosphere perception and intervention." Although the aforementioned solutions have made efforts in "hearing clearly" and "distinguishing" multiple speakers, their design goals are fundamentally limited, leading to the following shortcomings:

[0005] 1. Lacks understanding of the "social state" inside the vehicle and can only process parallel, independent commands.

[0006] Whether based on sound source localization (e.g., CN110070868B), speaker clustering (e.g., CN117854517A), or audio-visual fusion (e.g., CN117133292A), the ultimate goal of these technologies is to separate voice signals from different users and process them as independent, unrelated commands. However, currently, there is no mechanism to analyze the interaction relationships between different users. For example, the system cannot determine whether the situation inside the car is a "long period of silence" or a "power struggle among multiple people due to different demands." Existing technologies can only identify the physical fact of "no one speaking," or the two independent commands "user A wants to open the window" and "user B wants to close the window," but they cannot infer and understand these independent events as a specific, awkward social scenario requiring intervention.

[0007] 2. The interaction mode is passive, lacking the "emotional intelligence" to proactively break social deadlock.

[0008] Because current technology cannot understand the social atmosphere inside a car, its interaction mode is inevitably passively triggered. Virtual assistants only react after receiving explicit voice commands. None of the aforementioned technical solutions provide a means to proactively identify social awkwardness, such as prolonged silence, and initiate interaction to ease the tension. For example, when passengers in a car fall into a prolonged silence, the virtual assistant in existing technology simply remains silent along with them, without receiving any instructions. It lacks a mechanism similar to human "emotional intelligence"—that is, the ability to proactively break the silence by recommending music, introducing scenery, or initiating an interesting topic after sensing an awkward silence, thus proactively providing emotional value to the user.

[0009] 3. Unable to handle the "social conflict" between users, only mechanical response or no response.

[0010] When there are more complex social situations in the car, such as verbal conflicts between passengers due to disagreement, the limitations of the prior art are more fully exposed. The technical solution described above aims to "identify" the voice content, rather than "understand" the emotions and social dynamics behind it. Therefore, when the system identifies a conversation containing "negative emotions" and "conflict keywords", such as quarrels and disputes, it has no ability to judge that this is a scene that needs "mediation". The prior art cannot generate and output a "harmonious" neutral language that aims to ease the conflict, for example, "both of you don't be angry, safety first is the most important". Similarly, when "mutually exclusive instructions" are identified for the same device, such as the air conditioner, the prior art cannot actively make a "compromise suggestion". It either mechanically executes the last instruction or cannot execute due to conflict, both of which can exacerbate the conflict between passengers rather than solve the problem.

[0011] In summary, the technical goal of the prior art in the field of vehicle multi-person interaction is technical rather than social. The technical solution only solves the problem of "who is speaking" and "what is being said", but completely misses the two deeper problems of "how are their relationships and atmosphere" and "as a virtual assistant, how should I respond to this social situation". Based on this, there is an urgent need to develop an intelligent intervention generation method and system for vehicle awkward scenarios to effectively solve the above problems. SUMMARY

[0012] The purpose of the present application is to provide an intelligent intervention generation method for vehicle awkward scenarios, and to provide an intelligent intervention generation system for vehicle awkward scenarios, so as to solve the problem that the vehicle virtual assistant can only perform voice recognition at the physical layer in a multi-person interaction scenario, completely lacks understanding of the social state in the vehicle, and lacks the ability to actively intervene in awkward social scenarios.

[0013] The purpose of the present application is achieved by the following technical solutions:

[0014] An intelligent intervention generation method for vehicle awkward scenarios, comprising the following steps:

[0015] Step S401, continuously monitor the multi-modal state signals in the vehicle, including audio signals, text content, user instruction flow and vehicle sensor signals;

[0016] Step S402, embarrassment scene recognition and classification, input the multi-modal signals continuously monitored in step S401 into an embarrassment scene judgment module, which can analyze the embarrassment scene and determine whether it is interpersonal relationship embarrassment, interpersonal conflict embarrassment or control right contention embarrassment according to the judgment conditions; if any of the above conditions is not met, it is determined that the current is a non-embarrassment scene, and a regular passive interaction process is executed;

[0017] Step S403, according to the specific embarrassment scene type identified, for interpersonal relationship embarrassment, i.e. cold scene, the system will execute an ice-breaking intervention strategy;

[0018] Step S404, according to the specific embarrassment scene type identified, for interpersonal conflict embarrassment, i.e. quarrel, the system will execute a mediation intervention strategy;

[0019] Step S405, according to the specific embarrassment scene type identified, for control right contention embarrassment, the system will generate and execute a mediation intervention strategy.

[0020] Further, in step S401, the audio signal is used to analyze whether there is human voice in the vehicle, the volume of human voice, and whether there is long silence; the text content is used to analyze the conversation content and identify whether it contains negative emotions or preset conflict keywords by converting the collected human voice into text in real time; the user instruction stream is used to record all control instruction sequences issued by the user to the vehicle-mounted system; the vehicle sensor signal includes a seat belt sensor signal, which is used to determine the number of passengers in the vehicle through the seat belt sensor signal.

[0021] Further, in step S402, the embarrassment scene judgment module is built-in a set of rule and model-based judgment logic for real-time analysis and identification of whether there is a predefined embarrassment scene.

[0022] Further, in step S402, the judgment of interpersonal relationship embarrassment, i.e. cold scene: when the system detects that the seat belt signal shows that the number of people is greater than or equal to 2 and no valid human voice conversation is detected within a preset time length, and the current audio focus is not on media playing, the system determines the current state as interpersonal relationship embarrassment.

[0023] Further, in step S402, the judgment of interpersonal conflict embarrassment, i.e. quarrel: when the system identifies the preset conflict keywords from the voice-to-text content, or detects the acoustic characteristics of the conversation parties through voiceprint analysis, including sudden increase in volume, accelerated speech, which conforms to the negative emotion model, the system determines the current state as interpersonal conflict embarrassment.

[0024] Further, step S402, the determination of the control right contention embarrassment: when the system monitors that in the preset round, there are mutual control instructions for the same vehicle-mounted device in the instruction sequence distinguished by voiceprint or sound source positioning from different users, the system determines the current state as control right contention embarrassment.

[0025] Further, step S403, the ice-breaking intervention strategy is to recommend music and introduce the destination.

[0026] Further, step S404, the mediation intervention strategy is conflict mediation and safety reminder.

[0027] Further, step S405, the mediation intervention strategy is to make a compromise suggestion.

[0028] An intelligent intervention system for a vehicle-mounted embarrassing scene includes a state monitoring module, an embarrassing scene determination module, and a strategy generation and execution module. The state monitoring module is used to continuously collect multi-modal signals in the vehicle. The embarrassing scene determination module is used to determine the embarrassing scene of the social state according to the signals. The strategy generation and execution module is used to automatically generate and execute the intervention strategy according to the determination result.

[0029] Compared with the prior art, the beneficial effects of the present application are:

[0030] 1. The present application realizes the leap from "physical identification" to "social understanding", and gives the vehicle-mounted assistant the ability to understand the social atmosphere in the vehicle. The technical limit of the prior art in handling multi-person interaction is to "distinguish" and "identify" independent voice instructions of different users, and it can only handle physical acoustic signals in essence. The present application realizes a fundamental technological breakthrough by continuously monitoring the multi-modal state signals in the vehicle and establishing an embarrassing scene determination module. The module can infer and comprehensively judge a series of isolated physical signals such as "no sound duration", "seat belt signal (number of people)", "conflict keywords", and "mutually exclusive instruction sequence" into specific scenes with clear social implications, such as "cold scene", "quarrel", and "contention". The present application solves the fundamental defect of the prior art that it cannot perceive and understand the social relationship between users, so that the vehicle-mounted assistant is no longer a "machine" that can only respond to instructions passively, but an "intelligent agent" that can preliminarily understand the interpersonal atmosphere and social state in the vehicle.

[0031] 2. This invention represents a transformation from a "passive response" to a "proactive intervention" interaction model, providing users with proactive emotional value. Existing technologies operate entirely passively, only activating when explicitly instructed by the user. In the face of social awkwardness, existing virtual assistants often fall silent. This invention, however, fundamentally changes this model through its designed proactive intervention strategy generation and execution mechanism. When the system recognizes awkward situations such as "awkward silences," it can proactively initiate interaction without user input, breaking the ice by recommending music, introducing attractions, etc. This shift from passive waiting to proactive care addresses the pain point of functional deficiencies in existing technologies within social scenarios. Virtual assistants no longer merely fulfill functional needs but proactively provide positive emotional value, acting as a "social lubricant" when users need it most, significantly enhancing the humanization and comfort of the travel experience.

[0032] 3. This invention upgrades decision-making capabilities from "mechanical execution" to "intelligent mediation," effectively resolving in-vehicle social conflicts. When faced with conflicts or contradictions between users, such as arguments or conflicting instructions, existing technologies either mechanically execute one party's instructions or crash due to inability to handle the situation, both of which can escalate the conflict. This invention, through its unique intervention strategy, endows the virtual assistant with basic "emotional intelligence" for handling complex social conflicts. For example, in the case of "interpersonal conflict," the system generates not just functional responses, but mediating statements that guide both parties to focus on safety; in the case of "control disputes," the system can proactively propose a compromise, conciliatory solution. This decision-making logic, aimed at "solving problems and easing conflicts," completely transcends the mechanical "instruction execution" model of existing technologies. This invention overcomes the shortcomings of existing technologies in dealing with user conflicts, effectively avoiding or resolving potential social conflicts in vehicles, making journeys with multiple passengers more harmonious and enjoyable. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the multi-person voice interaction technology process based on sound source localization.

[0035] Figure 2 This is a schematic diagram of the multi-person voice interaction technology process based on speaker clustering;

[0036] Figure 3A flowchart of a multi-person voice interaction technology based on audio-visual fusion;

[0037] Figure 4 A flowchart of the intelligent intervention for the vehicle-mounted awkward scene of the present application. DETAILED DESCRIPTION

[0038] The present application will be further described below in conjunction with embodiments:

[0039] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for the purpose of explanation and are not limiting of the present application. In addition, it should be noted that only the parts related to the present application are shown in the accompanying drawings for the purpose of description, rather than all the structures.

[0040] It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", etc. are merely used for differentiation and cannot be understood as indicating or implying relative importance.

[0041] In order to solve the technical problem in the prior art that the vehicle-mounted virtual assistant can only perform voice recognition at the physical level in a multi-person interaction scene, and completely lacks the ability to understand the social state in the vehicle and actively intervene in awkward social scenes, the present application proposes a method and system for generating intelligent intervention for vehicle-mounted awkward scenes.

[0042] The fundamental difference between the present application and the prior art is that instead of passively and isolatedly processing each user's voice command as the final goal, the present application innovatively introduces a vehicle-mounted social atmosphere perception and decision intervention layer. This layer can actively monitor and identify a plurality of predefined awkward scenes, and generate and execute a personified interaction with "emotional quotient" aiming to alleviate the atmosphere or reconcile contradictions according to the scene type.

[0043] As shown in Figure 4 The method for generating intelligent intervention for vehicle-mounted awkward scenes of the present application includes the following steps:

[0044] Step S401: Continuously monitor the multi-modal state signals in the vehicle, including audio signals, text content, user instruction flow and vehicle sensor signals. The system will continuously and actively monitor a plurality of state signals reflecting the social atmosphere in the vehicle during the operation of the vehicle, rather than passively responding only when the user issues a command as in the prior art. These monitored signals at least include:

[0045] Audio signal: used to analyze whether there is human voice in the vehicle, the volume of human voice, and whether there is long-term silence.

[0046] Text content: Real-time speech-to-text conversion of collected human voice for analyzing the content of the conversation, especially identifying whether it contains negative emotions or pre-defined conflict keywords.

[0047] User instruction stream: Record all the sequences of control instructions issued by users to the vehicle-mounted system.

[0048] Vehicle sensor signals: For example, determine the number of passengers in the vehicle through seat belt sensor signals.

[0049] Step S402, embarrassment scenario recognition and classification. This step is one of the core innovations of the present application and is completely unavailable in the prior art. The system inputs the multi-modal signals continuously monitored in step S401 into an embarrassment scenario determination module. The module has a set of rule and model-based determination logic for real-time analysis and identification of pre-defined embarrassment scenarios, and further determination of interpersonal relationship embarrassment, interpersonal conflict embarrassment, or control power struggle embarrassment. Specifically,

[0050] 1. Interpersonal relationship embarrassment determination: When the system detects that the seat belt signal shows that the number of people is greater than or equal to 2 and no valid human voice conversation is detected within a pre-set time period, for example, 5 minutes, and the current audio focus is not on media playback, the system determines the current state as interpersonal relationship embarrassment.

[0051] 2. Interpersonal conflict embarrassment determination: When the system identifies pre-defined conflict keywords, such as "get out of here", "annoying", "shut up", etc., from the content of the speech-to-text conversion, or detects the acoustic characteristics of the conversation parties, such as sudden volume increase, speed up, etc., which meet the "angry", "irritated" and other negative emotion models, the system determines the current state as interpersonal conflict embarrassment.

[0052] 3. Control power struggle embarrassment determination: When the system monitors that within a pre-set number of turns, for example, 5 turns of conversation, there are mutually exclusive control instructions for the same vehicle-mounted device, such as air conditioner, audio, etc., in the instruction sequences from different users distinguished by voiceprint or sound source positioning, for example, user A instructs "air conditioner temperature to 18 degrees" and user B instructs "air conditioner temperature to 26 degrees", the system determines the current state as control power struggle embarrassment.

[0053] If any of the above conditions are not met, the system determines that the current state is not an embarrassment scenario, and executes the regular passive interaction process (F).

[0054] Steps S403 to S405, generation and execution of intervention strategies. Once the specific embarrassment scenario type is identified in step S402, the system will automatically generate and execute the corresponding intelligent intervention strategy, rather than mechanically executing the instructions or not responding as in the prior art.

[0055] In step S403, to address awkward interpersonal relationships, i.e., silence, the system will implement ice-breaking intervention strategies. This could include recommending music or introducing the destination. The system will proactively generate and broadcast personalized dialogue, such as: "It's been a while since we've spoken. Would you like to listen to some music? I recommend a currently popular playlist." Or, combining navigation information, it might say: "We're about to cross the XX Bridge; the night view here is very famous."

[0056] In step S404, for awkward interpersonal conflicts such as arguments, the system will execute mediation intervention strategies, such as conflict resolution and safety reminders. It will generate and broadcast statements with mediation and safety reminder functions, such as: "Hey, both of you, don't be angry. Peace brings wealth. Besides, arguing while driving might affect driving safety; safety is the most important thing."

[0057] In step S405, addressing the awkward situation of the struggle for control, the system will generate and execute a conciliatory intervention strategy, such as proposing a compromise. It will proactively suggest a compromise or neutral solution, for example: "Haha, it seems everyone's perception of temperature is a bit different. How about we set a neutral temperature of 22 degrees Celsius and try it out?"

[0058] Through the above steps, this invention upgrades from passive response to active intervention, enabling the in-vehicle virtual assistant to possess basic social intelligence and emotional intelligence, effectively resolving various typical awkward situations in the car, thereby greatly enhancing the riding experience and emotional value when multiple passengers travel together.

[0059] This invention also provides an intelligent intervention system for awkward scenarios in vehicles, including a status monitoring module, an awkward scenario determination module, and a strategy generation and execution module. The status monitoring module continuously collects multimodal signals from inside the vehicle. The awkward scenario determination module determines awkward scenarios in social situations based on the signals. The strategy generation and execution module actively generates and executes intervention strategies based on the determination results.

[0060] This invention proposes an intelligent intervention generation method and system for awkward in-vehicle scenarios, based on existing in-vehicle multi-person voice interaction technologies such as microphone array signal processing, sound source localization, and speaker recognition, as well as research on human-computer interaction sociability (HCI). Existing research in fields such as multimodal signal processing, dialogue systems, and computational sociology provides a macro-level theoretical background and technical foundation for the technical solution of this invention.

[0061] To further clarify the specific implementation and technical effects of the present invention, one or more embodiments will be described in detail below. These embodiments are intended to more clearly reveal the core ideas of the present invention, but do not constitute any limitation on the scope of protection of the present invention. In these embodiments, unless otherwise specified, the scenarios, parameters, and steps involved are exemplary.

[0062] Example 1

[0063] Interventions for awkward or silent interpersonal situations:

[0064] 1. Step S401: Continuously monitor the multimodal status signals inside the vehicle:

[0065] In a car equipped with the system of this invention, there is one passenger in the driver's seat and one in the front passenger seat. The system identifies two people in the vehicle via seatbelt sensor signals. Over the next five minutes, the system continuously monitors via a microphone array and detects no valid human conversation. Furthermore, through audio focus analysis, it confirms that the vehicle is not playing music or broadcasting.

[0066] 2. Step S402, Awkward Scene Recognition and Classification:

[0067] Upon receiving the aforementioned signal, the awkward scene determination module triggered the judgment rules for awkward interpersonal relationships, i.e., silence: "Number of people ≥ 2" AND "Silence duration ≥ 5 minutes" AND "Audio focus is not media". Therefore, the system determined the current atmosphere inside the car to be "awkward interpersonal relationships".

[0068] 3. Step S403: Generate and execute an icebreaking intervention strategy:

[0069] The system then proactively generated and executed an intervention strategy. Based on the navigation information, the system detected that the vehicle was about to approach a well-known cross-river bridge, so it used the TTS engine to announce in a natural and friendly tone: "We are about to enter the XX Bridge. The night view from the bridge is very beautiful. Speaking of which, this bridge is a landmark building in our city."

[0070] By proactively initiating a neutral topic relevant to the current situation, the system effectively broke the silence in the car, providing passengers with an opportunity to continue the conversation and thus resolving the awkward silence.

[0071] Example 2

[0072] Interventions targeting awkward interpersonal conflict scenarios, i.e., arguments:

[0073] 1. Step S401: Continuously monitor the multimodal status signals inside the vehicle;

[0074] During a family trip, the driver and front passenger got into an argument. The system captured the audio of the conversation using a microphone array and transcribed it into text in real time.

[0075] 2. Step S402, Awkward Scene Recognition and Classification:

[0076] Within the text content, the system identified preset conflict keywords, such as "Why did you take the wrong road again!" and "Aren't you annoying me!". Simultaneously, the voiceprint sentiment analysis module detected increased volume and faster speaking speed from both speakers, consistent with the emotion models of "anger" and "irritability." Therefore, the awkward scenario determination module classified the current state as "interpersonal conflict awkwardness."

[0077] 3. Step S404: Generate and execute a "mediation" type intervention strategy:

[0078] The system immediately generated and executed a mediation strategy, broadcasting in a calm and slightly concerned tone: "Oh dear, please don't be angry! Everyone gets lost sometimes, and arguing won't get us anywhere. Besides, let's not argue while driving; safety comes first!"

[0079] By using anthropomorphic persuasive language and cleverly steer the conversation toward the "safety" issue that both parties are concerned about, the system not only eased the tense atmosphere but also served as a driving safety reminder, demonstrating a high level of social intelligence.

[0080] Example 3

[0081] Intervention for scenarios involving "awkward struggles for control":

[0082] 1. Step S401: Continuously monitor the multimodal status signals inside the vehicle:

[0083] Inside the car, the driver (user A) said via voice command, "Turn the air conditioning temperature to 18 degrees." The system executed the command. Immediately afterwards, the front passenger (user B) said, "No, it's too cold, turn it to 26 degrees."

[0084] 2. Step S402, Awkward Scene Recognition and Classification:

[0085] Within a five-round dialogue monitoring cycle, the system identified mutually exclusive control commands from different users, distinguishing them as targeting the same device, the air conditioner, through sound source localization. The awkward scenario determination module therefore classified the current state as an "awkward struggle for control."

[0086] 3. Step S405: Generate and implement a "harmonious" intervention strategy:

[0087] At this point, the system did not mechanically execute the latter instruction, but instead proactively generated and executed a reconciliation strategy. It announced in a lighthearted and humorous tone: "Haha, it seems everyone's perceived temperature varies quite a bit! How about we set a neutral temperature of 22 degrees Celsius and try it out? If that still doesn't feel right, we can turn on the air conditioning zone control, which would be perfect for both of us!"

[0088] By proposing a fair compromise and informing the public of a better solution (partition control) that meets the needs of both parties, the system successfully transformed a potential point of conflict into a relaxed negotiation process, resolving the awkwardness or unpleasantness that might arise from the struggle for control.

[0089] As can be seen from the three embodiments above, the present invention can accurately identify a variety of typical awkward social scenarios in vehicles, and proactively generate and execute highly human-like, emotionally intelligent intervention strategies based on the characteristics of different scenarios. This solves the fundamental deficiency of existing technologies in dealing with in-vehicle social dynamics, and greatly enhances the intelligent, humanized, and emotional experience when multiple people are traveling together.

[0090] This invention provides a method for automatically identifying and classifying various specific awkward social scenarios in vehicles by continuously monitoring multimodal state signals and based on a set of preset judgment logic or models. These scenarios include at least awkward / cold interpersonal relationships, awkward / arguing interpersonal conflicts, and awkward situations involving power struggles. This method overcomes the limitation of existing technologies that can only identify physical signals, elevating physical signals to the level of social scenario judgment. After identifying a specific awkward social scenario, this method, without user instruction, proactively generates and executes corresponding anthropomorphic intervention strategies aimed at easing tensions or reconciling conflicts. This represents a fundamental shift in interaction mode from "passive response" to "proactive intervention," demonstrating proactive behavioral capabilities. Specifically addressing awkward situations involving power struggles, this invention can identify mutually exclusive instructions from different users and proactively generate a conciliatory or compromise solution, such as suggesting a middle value, as an intervention strategy. Unlike existing technologies that merely mechanically execute or report errors, this method upgrades from simple "instruction execution" to "intelligent decision-making."

[0091] In addition, step S401 primarily determines the scenario by monitoring audio, text, command streams, and seatbelt signals. The input signal sources used for judgment can be further expanded. For example, visual signals acquired by in-vehicle cameras can be introduced. By analyzing passenger expressions, such as frowning or anger, and body language, such as a defensive posture with crossed arms, lack of eye contact between passengers, and direction of gaze, the degree of "awkward silence" or "conflict" can be more accurately determined. Furthermore, physiological signal sensors, such as heart rate and skin conductance sensors integrated into the steering wheel or seat, can be introduced. By monitoring the driver's or passenger's physiological indicators, such as a sudden increase in heart rate or a strong skin conductance response, the presence of emotions such as tension or anger can be assessed, thereby improving the accuracy and robustness of recognizing "awkward interpersonal conflict" scenarios. In step S402, a set of judgment logic based on preset rules is mainly used. This implementation can also be replaced by an end-to-end machine learning model. Specifically, a large amount of in-vehicle multimodal interaction data can be collected and labeled to train a multi-classification scene recognition model, for example, based on a Transformer or CNN-LSTM architecture. This model can take the multimodal signals obtained in step S401 as input and directly output the probability that the current scene belongs to one of the categories of "normal," "awkward silence," "conflict," or "competition." The system's judgment of awkward scenes will be upgraded from hard-coded rules to a data-driven, more generalizable intelligent model, with the same goal of automatically identifying and classifying awkward scenes. In steps S403 to S405, the system executes a preset intervention strategy that is strongly bound to the scene type. The system can also use a large language model (LLM) to dynamically generate intervention scripts. The LLM's prompt not only includes the currently identified awkward scene type but can also combine the user's historical interaction preferences, current personality model (such as MBTI parameters), and even environmental information outside the vehicle, such as weather and location contextual information. In this way, the intervention scripts generated by the system will no longer be uniform templates but highly contextualized and personalized. For example, in the case of "awkward silence," the system may prioritize recommending music for users who prefer music, while prioritizing introducing information about the attraction for users who are about to arrive at a scenic spot. In this invention, the intervention strategy is primarily executed through voice interaction. The execution of the intervention can also be extended to multimodal output. For example, when an "awkward silence" is detected, in addition to broadcasting voice commands, the system can proactively switch the central control screen to the ambient lighting adjustment interface and play soothing background music, creating a relaxed atmosphere through the combined effects of vision and hearing. When an awkward struggle for control is detected, the system can display a neutral suggestion graphically on the screen, such as an animation of a thermometer pointing to the midpoint temperature of 22 degrees Celsius, allowing the user to confirm visually rather than solely relying on voice.

[0092] Note that the design of the digital prototype model, the parameter settings for each material, and the selection of the measuring plane described above are merely preferred embodiments and technical principles of the present invention. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for generating intelligent intervention for an in-vehicle awkward scene, characterized in that, Comprising the following steps: Step S401, continuously monitor the multi-modal state signals in the vehicle, including audio signals, text content, user instruction flow and vehicle sensor signals; Step S402, embarrassment scene recognition and classification, input the multi-modal signals continuously monitored in step S401 into an embarrassment scene judgment module, which can analyze the embarrassment scene and determine whether it is interpersonal relationship embarrassment, interpersonal conflict embarrassment or control right contention embarrassment according to the judgment conditions; If any of the above conditions is not met, it is judged that the current is not an embarrassing scene, and a regular passive interaction process is executed; Step S403, according to the specific type of embarrassing scene identified, for interpersonal relationship embarrassment, i.e. cold scene, the system will execute the ice-breaking intervention strategy; Step S404, according to the specific type of embarrassing scene identified, for interpersonal conflict embarrassment, i.e. quarrel, the system will execute the mediation intervention strategy; Step S405, according to the specific type of embarrassing scene identified, for control right contention embarrassment, the system will generate and execute the mediation intervention strategy.

2. The intelligent intervention generation method for an in-vehicle embarrassing scene according to claim 1, characterized in that: Step S401, the audio signal is used to analyze whether there is human voice in the vehicle, the volume of human voice, and whether there is long silence; the text content is used to analyze the conversation content and identify whether it contains negative emotions or preset conflict keywords by converting the collected human voice into text in real time; the user instruction flow is used to record all control instruction sequences issued by the user to the vehicle system; the vehicle sensor signal includes a seat belt sensor signal, which is used to determine the number of passengers in the vehicle through the seat belt sensor signal.

3. The method of claim 1, wherein: Step S402, in the embarrassment scene judgment module, a set of judgment logic based on rules and models is built in for real-time analysis and identification of whether there is a predefined embarrassing scene.

4. The intelligent intervention generation method for an in-vehicle embarrassing scene according to claim 1, characterized in that, Step S402, judgment of interpersonal relationship embarrassment, i.e. cold scene: when the system detects that the seat belt signal shows that the number of people is greater than or equal to 2 and no effective human voice conversation is detected within a preset time period, and the current audio focus is not on media playing, the system will determine the current state as interpersonal relationship embarrassment.

5. The method of claim 1, wherein, Step S402, judgment of interpersonal conflict embarrassment, i.e. quarrel: when the system identifies the preset conflict keywords from the text converted from the voice, or detects the acoustic characteristics of the conversation parties through voiceprint analysis, including sudden increase in volume and acceleration of speech speed, which conforms to the negative emotion model, the system will determine the current state as interpersonal conflict embarrassment.

6. The method of claim 1, wherein, Step S402, judgment of control right contention embarrassment: when the system monitors that within a preset round, there are mutually exclusive control instructions for the same vehicle-mounted device in the instruction sequence from different users distinguished by voiceprint or sound source positioning, the system will determine the current state as control right contention embarrassment.

7. The method of claim 1, wherein: Step S403, the ice-breaking intervention strategy is to recommend music and introduce the destination.

8. The method of claim 1, wherein: Step S404, the mediation intervention strategy is conflict mediation and safety reminder.

9. The method of claim 1, wherein: Step S405, the mediation intervention strategy is to propose a compromise.

10. An intelligent intervention system for in-vehicle awkward scenarios, characterized in that: The system comprises a state monitoring module, an embarrassing situation judging module and a strategy generating and executing module. The state monitoring module is used for continuously collecting multi-modal signals in the vehicle. The embarrassing situation judging module is used for judging an embarrassing situation of a social state according to the signals. The strategy generating and executing module is used for automatically generating and executing an intervention strategy according to the judgment result.

Images