Human-computer interaction method, system, device and medium based on speech differential processing

Abstract

The present invention belongs to the field of artificial intelligence technology and specifically discloses a human-computer interaction method, system, device and medium based on voice differential processing, including extracting voice data sequences and scene features, building a background noise database, collecting user voice information, training a voiceprint recognition model, obtaining a noise reducer, performing noise reduction analysis, extracting the time interval of the noise-reduced voice data sequence, performing dialogue segmentation, determining multiple semantic data subsequences, performing differential semantic analysis on the multiple semantic data subsequences, determining multiple interaction instructions and interaction times, performing action execution plan analysis based on the interaction instructions and time, determining a robot action execution plan, and interactively controlling the robot according to the action execution plan; the method can better improve the robot's responsiveness to user voice instructions, adapt to the user's personalized needs, and thus perform efficient interactive applications in a home environment.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a human-computer interaction method, system, device and medium based on speech differential processing. Background Art

[0002] Artificial intelligence technology is developing rapidly. Technically, breakthroughs in advanced algorithms such as deep learning have greatly improved the learning capabilities of machines. In terms of data, massive amounts of network and IoT data provide rich material for AI training. Currently, in a home environment, users want to control the robot to turn left through voice commands. However, due to interference from the sounds of electronic devices and other sound sources, traditional systems often make recognition errors, often misidentifying "forward" or missing recognition problems, which in turn leads to response failures. When there are multiple guests visiting a home environment, and the user issues voice commands to the robot at the same time, multiple guests are talking loudly, causing the user's voice to overlap with the guests' voices. The robot cannot accurately distinguish the overlapping voices and determine the voice command to be executed, and thus cannot execute the action required by the user's voice command.

[0003] In addition, in a home environment, when users are accustomed to issuing multiple commands in succession (such as "forward," "stop," and "turn right") to control the robot to complete corresponding continuous actions, traditional systems are easily confused by too many commands and cannot accurately judge the order. They are prone to misrecognition and response delays, making it impossible for robots to achieve intelligent and efficient voice interaction experience in different scenarios.

[0004] In view of this, the present invention proposes a human-computer interaction method, system, device and medium based on speech differential processing. Summary of the Invention

[0005] The purpose of the present invention is to provide a human-computer interaction method, system, device and medium based on voice differential processing to solve the technical problems in the prior art that robots have a high incidence of incorrect recognition, missed recognition and response failures in voice control command recognition.

[0006] To achieve the above object, the present invention provides the following technical solution: a human-computer interaction method based on speech differential processing, comprising the following steps:

[0007] S1: extracting a speech data sequence received by a target robot within a preset time window, performing scene feature extraction on an application scene of the target robot, obtaining a scene feature set, and constructing a background noise database based on the scene feature set;

[0008] S2: Collecting user voice information, combining it with the electronic device sound information and guest voice information in the background noise database, performing voiceprint recognition model training, obtaining a background voiceprint recognition denoiser, and performing noise reduction analysis on the scene feature set and the voice data sequence using the background voiceprint recognition denoiser to obtain a noise-reduced voice data sequence;

[0009] S3: extracting the time intervals between adjacent noise-reduced speech data in the noise-reduced speech data sequence, performing dialogue segmentation on the noise-reduced speech data sequence according to the extraction result, and determining a plurality of noise-reduced semantic data subsequences;

[0010] S4: performing differential semantic analysis on the multiple noise reduction semantic data subsequences to determine multiple interaction instructions, where the multiple interaction instructions include multiple interaction times;

[0011] S5: Analyzing the target robot's action execution plan based on the multiple interaction instructions and the multiple interaction times, and determining the target robot's action execution plan;

[0012] S6: interactively controlling the target robot according to the target robot action execution plan.

[0013] As a preferred technical solution of the first aspect of the present invention, a preset time window is configured according to the actual application requirements of the robot, the preset time window is used as the collection period of voice data, and voice samples are obtained based on the preset time window.

[0014] As a preferred technical solution of the first aspect of the present invention, the logic for obtaining the scene feature set is:

[0015] During the preset time window, the microphone is enabled to collect the user's voice data in real time and transmit the collected audio data in the form of digital signals;

[0016] At the same time, the robot's environmental sensors are activated to obtain environmental feature data of the current application scenario based on the environmental sensors, and the microphone is activated to collect background noise data in the home scenario. The background noise data includes the sound of electronic devices and the sound information of guests, and the scene data is obtained;

[0017] The collected scene data is preprocessed, including outlier removal and normalization, and speech recognition scene features are screened out based on a specific threshold, and the scene features are combined into a scene feature set.

[0018] As a preferred technical solution of the first aspect of the present invention, the construction process of the background voiceprint recognition noise reducer is as follows:

[0019] A1: Noise identification is performed on electronic device sound information and guest sound information in the background noise database to obtain a noise sample set, user voice information is collected to perform command sound identification, and a recognition sample set is obtained. The noise sample set and the recognition sample set are mixed in a preset ratio to obtain a mixed sample set;

[0020] A2: Utilize audio processing technology to traverse the mixed sample set to perform voiceprint feature analysis, and obtain a voiceprint feature sample set to perform initial training on the discriminator. The discriminator is used to identify user voice information from noisy voice data. During training, the training process is supervised by the recognition voiceprint feature set corresponding to the recognition sample set with the command sound identification, helping the discriminator learn how to distinguish the user's command voice from background noise. The training process is analyzed using a loss function, and the hyperparameters of the discriminator are continuously adjusted, including the learning rate, batch size, and number of network layers. The hyperparameters are adjusted until the loss converges, and a trained voiceprint denoising recognizer is obtained.

[0021] A3: The loss function is the cross entropy loss function, and the function formula is as follows:

[0022] ;

[0023] in, is the loss function value, N is the number of samples in the mixed sample set, and is an integer greater than or equal to 1. is the true label of the i-th voiceprint feature in the mixed sample set (1 indicates that the voiceprint feature comes from the noise sample set, and 0 indicates that the voiceprint feature comes from the recognition sample set), is the predicted probability of the discriminator for the i-th recognition voiceprint feature from the recognition sample set.

[0024] As a preferred technical solution of the first aspect of the present invention, the acquisition logic of the noise reduction semantic data subsequence;

[0025] Extract and record the start and end time of each segment of the acquired noise-reduced speech data sequence to ensure that the subsequent time interval calculation has an accurate start and end point. Calculate the time interval between every two adjacent segments of speech data in the noise-reduced speech data sequence, and store the time intervals of all adjacent segments in an interval list.

[0026] Set time interval thresholds based on scenarios and user habits, and use them to distinguish between continuous commands and pauses in conversation.

[0027] Traverse the time interval list and compare each interval with the set time interval threshold. If the time interval exceeds the threshold, the point is regarded as a dialogue segmentation point and an independent semantic data subsequence is generated; if it does not exceed the threshold, the speech data is classified into the same subsequence;

[0028] According to the segmentation results, the divided speech data segments are organized into multiple noise-reduced semantic data subsequences. Each subsequence contains a complete speech command and records the corresponding timestamp.

[0029] As a preferred technical solution of the first aspect of the present invention, the logic for obtaining the interaction instruction is:

[0030] Perform semantic analysis on each denoised semantic data subsequence, extract the main speech content and convert it into a processable text form;

[0031] Generate a timestamp for each subsequence, record the time when the voice command was issued, and store it in the corresponding semantic subsequence;

[0032] The interaction time uses the start time or end time of the subsequence, and the appropriate time point is selected according to the needs;

[0033] Compare the semantic content of adjacent subsequences. If the subsequence content is repeated, remove the repeated instructions. Use the differential algorithm to analyze the content differences of adjacent subsequences and extract the new or unique semantic information in each subsequence.

[0034] Extracting semantic features from the plurality of denoised semantic data subsequences by using a semantic feature extraction network layer to obtain a plurality of denoised semantic feature subsequences, and performing differential semantic analysis on the plurality of denoised semantic data subsequences:

[0035] When the semantic difference is greater than the preset semantic difference, two interaction instructions are generated respectively according to the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences;

[0036] When the semantic difference is greater than the preset semantic difference, the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences are fused to generate an interaction instruction, thereby achieving the goal of removing duplicate instructions.

[0037] Determine the differences between instructions, identify each new instruction of the user, organize the independent instructions obtained after differential analysis into an interaction instruction set, and attach the corresponding interaction time. Each instruction corresponds to its timestamp to form a set containing interaction instructions and interaction time.

[0038] In the interaction instruction set, the execution order or priority of the instructions is optimized according to the instruction content and interaction time, and the interaction instruction set after differential semantic analysis and optimization is output to the action execution module to provide clear instruction input for the robot's action plan.

[0039] As a preferred technical solution of the first aspect of the present invention, the determination logic of the target robot action execution plan is:

[0040] Arrange the interaction instructions in the order of interaction time to form a time-series instruction chain. Each instruction corresponds to its interaction time, indicating that the execution follows the order of user intention;

[0041] Perform dependency analysis on adjacent instructions in the instruction sequence. If there is a conflict, retain the instruction with higher priority or determine the execution instruction based on the time sequence. Generate a specific action execution plan based on the classification, priority and dependency analysis results.

[0042] The execution time of each instruction is corrected according to the interaction time, and the execution plan is simulated within the system to detect whether there are unreasonable or unexecutable instructions in the plan, and the plan is adjusted or replanned; the final action execution plan is output to the robot's control module and prepared for execution.

[0043] In a second aspect, the present invention provides a human-computer interaction system based on voice differential processing, which is based on the implementation of the first aspect and includes a background noise extraction module, a voice information extraction module, a voice data processing module, an interaction instruction extraction module, an execution plan analysis module, and a control module, wherein the modules are connected to each other via wired or wireless connections:

[0044] A background noise extraction module extracts the speech data sequence received by the target robot within a preset time window, performs scene feature extraction on the application scene of the target robot, obtains a scene feature set, and constructs a background noise database based on the scene feature set;

[0045] A voice information extraction module collects the user's voice information, combines it with the electronic device sound information and the guest sound information in the background noise database, trains a voiceprint recognition model, obtains a background voiceprint recognition denoiser, and uses the background voiceprint recognition denoiser to perform noise reduction analysis on the scene feature set and the voice data sequence to obtain a noise-reduced voice data sequence;

[0046] a speech data processing module, which extracts the time intervals between adjacent noise-reduced speech data in the noise-reduced speech data sequence, performs dialogue segmentation on the noise-reduced speech data sequence according to the extraction result, and determines a plurality of noise-reduced semantic data subsequences;

[0047] An interaction instruction extraction module performs differential semantic analysis on the multiple noise reduction semantic data subsequences to determine multiple interaction instructions, wherein the multiple interaction instructions include multiple interaction times;

[0048] an execution plan analysis module, which analyzes the action execution plan of the target robot based on the multiple interaction instructions and the multiple interaction times, and determines the action execution plan of the target robot;

[0049] A control module interactively controls the target robot according to the target robot action execution plan.

[0050] In a third aspect, the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0051] The processor executes the first aspect by calling the computer program stored in the memory.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the first aspect.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention collects voice data and scene features in a specific time period, uses a background voiceprint recognition denoiser to improve the accuracy of voice command recognition in noisy environments, enhances the system's processing capability and response efficiency to continuous commands through dialogue segmentation and differential semantic analysis, and simultaneously considers the timing and content of the commands to form a reasonable action execution plan. The system's intelligence and interactivity are enhanced through real-time interactive control, thereby achieving efficient and precise control of robots in complex environments, significantly improving the adaptability and control accuracy of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Flowchart of a human-computer interaction method based on voice differential processing in an embodiment of the present invention;

[0056] Figure 2 This is a framework diagram of a human-computer interaction system based on voice differential processing in an embodiment of the present invention;

[0057] Figure 3 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] In the description of the present invention, it should be noted that the terms "vertical", "up", "down", "horizontal", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limiting the present invention.

[0060] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0061] Example 1

[0062] like Figure 1 As shown, the present invention provides a technical solution: a human-computer interaction method based on speech differential processing, comprising the following steps:

[0063] S1: extracting a speech data sequence received by a target robot within a preset time window, performing scene feature extraction on an application scene of the target robot, obtaining a scene feature set, and constructing a background noise database based on the scene feature set;

[0064] Specifically, based on the robot's actual application needs (e.g., home use), a reasonable time window (e.g., 3 or 5 seconds) is set as the voice data collection period. Within each time window, the voice collection module is activated, and the frequency of data collection is determined to obtain sufficient voice samples within that window. By limiting the duration and frequency of data collection, complete voice data is collected within a fixed timeframe, adapting to diverse application scenarios.

[0065] During a preset time window, the microphone is activated to collect the user's voice data in real time, and the collected audio data is transmitted to the system in the form of a digital signal. The collected voice data will be used as raw input for subsequent processing.

[0066] The robot's environmental sensors (such as noise, light, and distance sensors) are simultaneously activated to collect environmental characteristic data for the current application scenario. The microphone is also activated to collect various background noise data in home settings (such as the sounds of electronic devices and the voices of guests and users), generating a scene feature set. This data includes, but is not limited to, ambient noise levels, background sound characteristics (such as pitch and frequency), light intensity, the presence of obstacles, and guest voiceprint information. By capturing the environmental characteristics surrounding the robot, additional context is provided for subsequent noise reduction, distinguishing and recognizing user voices, and analyzing commands, enabling the system to adapt to environmental changes in different scenarios.

[0067] The collected scene data is then preprocessed, such as removing outliers and normalizing it, to ensure data stability. Furthermore, features with a significant impact on speech recognition are screened based on specific thresholds (e.g., noise exceeding a certain decibel level). The processed data forms a scene feature set.

[0068] Through the above steps, the target robot’s speech data and scene feature set are obtained in each time window, providing comprehensive data support for subsequent noise reduction processing, command segmentation, and differential analysis, enabling the robot to better adapt to user needs and environmental changes in different scenarios.

[0069] S2: Collecting user voice information, combining it with the electronic device sound information and guest voice information in the background noise database, performing voiceprint recognition model training, obtaining a background voiceprint recognition denoiser, and performing noise reduction analysis on the scene feature set and the voice data sequence using the background voiceprint recognition denoiser to obtain a noise-reduced voice data sequence;

[0070] It should be noted that S2 uses background voiceprint recognition and noise reduction to effectively filter out environmental noise and noise caused by conversations between guests and users, ensuring accurate voice data processing of the robot in different scenarios, thereby improving its response to user commands.

[0071] Specifically, the construction process of the background voiceprint recognition noise reducer is as follows:

[0072] A1: Noise identification is performed on the electronic device sound information and the guest sound information in the background noise database to obtain a noise sample set. The user's voice information is collected to issue a command for sound identification to obtain a recognition sample set. The noise sample set and the recognition sample set are mixed in a preset ratio (e.g., a 1:1 mix, a 1:2 mix, etc.) to obtain a mixed sample set.

[0073] More specifically, the mixing ratio is 1:1 mixing or 1:2 mixing;

[0074] A2: Use audio processing technology to traverse the mixed sample set to perform voiceprint feature analysis, and obtain a voiceprint feature sample set to perform initial training on the discriminator, wherein the discriminator is used to identify user voice information from voice data containing noise. During training, the training process is supervised by the recognition voiceprint feature set corresponding to the recognition sample set with the command sound identification, to help the discriminator learn how to distinguish the user's command sound from background noise, and use the loss function to analyze the training process and continuously adjust the hyperparameters of the discriminator, including the learning rate, batch size and number of network layers; until the loss converges, a trained voiceprint denoising recognizer is obtained.

[0075] A3: The loss function is the cross entropy loss function, and the function formula is as follows:

[0076] ;

[0077] in, is the loss function value, N is the number of samples in the mixed sample set, and is an integer greater than or equal to 1. is the true label of the i-th voiceprint feature in the mixed sample set (1 indicates that the voiceprint feature comes from the noise sample set, and 0 indicates that the voiceprint feature comes from the recognition sample set), is the predicted probability of the discriminator for the i-th recognition voiceprint feature from the recognition sample set.

[0078] Through the above steps, the neural network-based background noise reducer can learn and eliminate background noise from scene features and voice data, providing high-quality audio input for the robot's voice command system, improving the accuracy of the robot's execution of actions, and enhancing the quality of human-computer interaction.

[0079] S3: extracting the time intervals between adjacent noise-reduced speech data in the noise-reduced speech data sequence, performing dialogue segmentation on the noise-reduced speech data sequence according to the extraction result, and determining a plurality of noise-reduced semantic data subsequences;

[0080] Specifically, each segment of the acquired denoised speech data sequence is timestamped. First, the start and end times of each segment are extracted and recorded to ensure accurate start and end points for subsequent time interval calculations. This provides the foundational data for subsequent time interval calculations, enabling the system to accurately determine the intervals between speech segments. The time interval is calculated for each two adjacent segments in the denoised speech data sequence using the following formula: Time interval = Current segment start time - Previous segment end time. The time intervals for all adjacent segments are stored in an interval list for subsequent processing.

[0081] Set a time interval threshold (for example, 0.5 seconds or 1 second) based on the scenario and user habits. This threshold is used to distinguish continuous commands from pauses in the conversation. The threshold can be adjusted dynamically based on the application scenario. For example, a shorter threshold might be set in a noisy environment to reduce false segmentation. Thus, the segmentation criteria for voice commands are determined, and the threshold setting ensures segmentation accuracy and avoids false segmentation of consecutive commands.

[0082] The system traverses the list of time intervals, comparing each interval to a set threshold. If the interval exceeds the threshold, it is considered a conversation segmentation point, generating an independent semantic data subsequence. If it does not exceed the threshold, the speech data is grouped into the same subsequence. This separates the speech command segments, ensuring that each subsequence corresponds to a complete command or statement, laying the foundation for subsequent semantic analysis. Based on the segmentation results, the divided speech data segments are organized into multiple noise-reduced semantic data subsequences. Each subsequence contains a complete speech command and records the corresponding timestamp for subsequent chronological processing.

[0083] Verify the initial segmentation of subsequences to ensure that the segmentation points match the natural pauses of the voice command. If the verification process finds excessive or insufficient segmentation, you can optimize it by adjusting the time interval threshold to further improve the rationality of the dialogue segmentation.

[0084] Through the above steps, the user's voice pause points can be accurately identified in the noise-reduced voice data sequence, and the voice data can be reasonably divided into independent semantic subsequences, ensuring the robot's accurate response to each command, improving the system's multi-command processing capabilities and user experience.

[0085] S4: performing differential semantic analysis on the multiple noise reduction semantic data subsequences to determine multiple interaction instructions, wherein the multiple interaction instructions include multiple interaction times;

[0086] Semantic parsing is performed on each de-noised semantic data subsequence to extract the main speech content and convert it into a processable text form. A natural language processing (NLP) model is used to convert the speech data in the subsequence into text instructions for subsequent semantic differential analysis.

[0087] A timestamp is generated for each subsequence, recording the time the voice command was issued and storing it in the corresponding semantic subsequence. The interaction time can be the start or end time of the subsequence, choosing the appropriate time point based on needs. The semantic content of adjacent subsequences is compared to check for identical or highly similar content. If the subsequence content is repeated, the duplicate commands are removed (for example, due to a user slip of the tongue, the robot may issue the same command twice), ensuring that the system only processes meaningful, independent commands. This reduces the error of issuing the same command multiple times, improves system efficiency, and avoids erroneous operations caused by repeated commands.

[0088] Adjacent subsequences are analyzed for content differences using a differential algorithm to extract new or unique semantic information from each subsequence. Semantic features (word vectors or sentence vectors) are extracted from the multiple denoised semantic data subsequences using a semantic feature extraction network layer to obtain multiple denoised semantic feature subsequences.

[0089] Furthermore, differential semantic analysis of the multiple de-noised semantic data subsequences is performed to identify and compare the semantic differences between individual elements (e.g., words, phrases, or sentences) in the text or speech between two adjacent de-noised semantic data subsequences. This processing method typically involves comparing different inputs to identify differences in meaning, sentiment, or other characteristics. This facilitates understanding the user's intent based on sequential semantic information, improving the accuracy of the robot's action execution.

[0090] Preferably, a differential processing formula is used to perform differential semantic analysis on the multiple noise reduction semantic feature subsequences.

[0091] The differential processing formula is: ΔI= − ΔI is the semantic difference between two adjacent denoised semantic data subsequences, is the semantic feature of the nth denoised semantic data subsequence, is the semantic feature of the n-1th denoised semantic data subsequence.

[0092] When the semantic difference is greater than the preset semantic difference, two interaction instructions are generated respectively according to the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences;

[0093] When the semantic difference is greater than the preset semantic difference, the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences are fused to generate an interaction instruction, thereby achieving the goal of removing duplicate instructions.

[0094] The system identifies the differences between commands and recognizes each new command from the user, ensuring that the system accurately captures changes in user needs. The independent commands obtained after differential analysis are organized into an interactive command set, along with the corresponding interaction time. Each command is associated with its timestamp, forming a set containing the interaction command and interaction time. This ensures the independence and temporal consistency of each command, providing a clear and complete command chain for subsequent robot action execution.

[0095] Within the interactive instruction set, the execution order or priority of instructions is optimized based on the instruction content and interaction time. For example, if the user issues a "stop" command, the system will prioritize it to ensure a timely response to urgent instructions. The interactive instruction set, which has undergone differential semantic analysis and optimization, is output to the action execution module, providing clear instruction input for the robot's motion planning.

[0096] Through the above steps, the system can accurately extract the user's independent instructions from multiple noise-reduced semantic data subsequences, remove repeated or invalid instructions, and form a time-ordered interactive instruction set, so that the robot can quickly respond to diverse voice needs and enhance the intelligent interactive experience.

[0097] S5: Analyzing the target robot's action execution plan based on the multiple interaction instructions and the multiple interaction times, and determining the target robot's action execution plan;

[0098] Interaction commands are arranged in chronological order, forming a time-series chain of commands. Each command corresponds to its interaction time, ensuring sequential consistency so that the system follows the order of user intent during execution. Each command in the command sequence is categorized (e.g., move, stop, turn, etc.) and assigned different priorities based on its content. For example, a "stop" command typically has the highest priority, followed by "forward" or "turn" commands. Prioritizing commands based on their importance facilitates handling of conflicting or urgent commands in complex scenarios.

[0099] Dependency analysis is performed on adjacent instructions within a command sequence. For example, "Forward" followed by "Stop" are coherent commands, but "Forward" and "Backward" require conflict detection. If a conflict exists, the higher-priority instruction is retained, or the execution is determined based on chronological order. Based on the classification, priority, and dependency analysis results, a specific action execution plan is generated. For example, if the command sequence is "Forward -> Turn -> Stop," it is converted into a robot motion path plan, executed in the order of "Forward -> Turn -> Stop."

[0100] The execution time of each instruction is corrected based on the interaction time. For example, if the time interval between two instructions is short, the robot needs to speed up the execution; if the interval is long, the system can delay the execution appropriately. This process ensures that the rhythm of instruction execution is synchronized with the frequency of user instructions. The execution plan is simulated within the system to detect whether there are unreasonable or unexecutable instructions in the plan, such as route intersections, time overlaps, etc. If problems arise, adjustments or replanning are made. The final action execution plan is output to the robot's control module and prepared for execution. The output plan should include the content, execution order and time interval information of each instruction. Provide clear operational guidance for the robot control system to ensure that the robot completes the corresponding actions according to user requirements.

[0101] Through the above steps, the system can generate a logically coherent and prioritized action execution plan based on user interaction instructions and time arrangements, ensuring that the robot can efficiently respond to user needs in complex environments and provide a smooth intelligent interactive experience.

[0102] S6: interactively controlling the target robot according to the target robot action execution plan.

[0103] Before executing an action plan, the robot's control module, including the drive motor, direction control module, and sensor monitoring module, is activated. This ensures the proper functioning of all hardware components and establishes a real-time communication connection with the central control system. From the generated action plan, the first action instruction and its corresponding timing parameters are sequentially read and loaded into the control module's execution queue, preparing for execution. The control module then executes the robot's specific actions, such as forward movement, turning, and stopping, based on the loaded action instructions. During execution, the robot's sensors provide real-time feedback to the central control system, including position, direction, and speed.

[0104] During the execution of an action, the robot's operating status is continuously monitored to check for anomalies (such as obstacles, tilt, or directional deviation). If an anomaly is detected, an interrupt mechanism or adjustment plan is immediately triggered. Based on the time parameters in the execution plan, the next instruction is loaded into the control module at the instruction interval, keeping the execution rhythm consistent with the frequency of user instructions. If the previous instruction is not completed on time, the loading of the next instruction is delayed or the execution speed is adjusted. After each instruction is completed, the control module sends a completion signal to the central control system, marking the completion of the instruction. After receiving the signal, the system loads the next instruction and continues to control the robot's execution until all instructions in the action execution plan are completed.

[0105] When all commands are executed, the system generates summary feedback, including the robot's trajectory and command response status, and notifies the user through voice or screen notifications that the command has been successfully executed. Providing user feedback on execution results enhances the interactive experience and allows users to understand the robot's execution status and completion level. After all action commands are completed, the control module automatically returns to standby mode, awaiting the user's next command input. If the next task command is issued, the system automatically enters the next control flow. This ensures that the system is always on standby, improving user convenience. Through these steps, the system can accurately interactively control the robot based on the action execution plan, enabling the robot to complete continuous actions under user command, providing an efficient and stable intelligent control experience.

[0106] Example 2

[0107] See also Figure 2 The human-computer interaction system based on speech differential processing includes a background noise extraction module, a speech information extraction module, a speech data processing module, an interaction instruction extraction module, an execution plan analysis module, and a control module. Each module is connected via wired or wireless connections:

[0108] A background noise extraction module extracts the speech data sequence received by the target robot within a preset time window, performs scene feature extraction on the application scene of the target robot, obtains a scene feature set, and constructs a background noise database based on the scene feature set;

[0109] A voice information extraction module collects the user's voice information, combines it with the electronic device sound information and the guest sound information in the background noise database, trains a voiceprint recognition model, obtains a background voiceprint recognition denoiser, and uses the background voiceprint recognition denoiser to perform noise reduction analysis on the scene feature set and the voice data sequence to obtain a noise-reduced voice data sequence;

[0110] a speech data processing module, which extracts the time intervals between adjacent noise-reduced speech data in the noise-reduced speech data sequence, performs dialogue segmentation on the noise-reduced speech data sequence according to the extraction result, and determines a plurality of noise-reduced semantic data subsequences;

[0111] An interaction instruction extraction module performs differential semantic analysis on the multiple noise reduction semantic data subsequences to determine multiple interaction instructions, wherein the multiple interaction instructions include multiple interaction times;

[0112] an execution plan analysis module, which analyzes the action execution plan of the target robot based on the multiple interaction instructions and the multiple interaction times, and determines the action execution plan of the target robot;

[0113] A control module interactively controls the target robot according to the target robot action execution plan.

[0114] Preferably, a preset time window is configured according to the actual application requirements of the robot, the preset time window is used as the collection period of voice data, and voice samples are obtained based on the preset time window.

[0115] Preferably, the logic for acquiring the scene feature set is:

[0116] During the preset time window, the microphone is enabled to collect the user's voice data in real time and transmit the collected audio data in the form of digital signals;

[0117] At the same time, the robot's environmental sensors are activated to obtain environmental feature data of the current application scenario based on the environmental sensors, and the microphone is activated to collect background noise data in the home scenario. The background noise data includes the sound of electronic devices and the sound information of guests, and the scene data is obtained;

[0118] The collected scene data is preprocessed, including outlier removal and normalization, and speech recognition scene features are screened out based on a specific threshold, and the scene features are combined into a scene feature set.

[0119] Preferably, the construction process of the background voiceprint recognition noise reducer is as follows:

[0120] A1: Noise identification is performed on electronic device sound information and guest sound information in the background noise database to obtain a noise sample set, user voice information is collected to perform command sound identification, and a recognition sample set is obtained. The noise sample set and the recognition sample set are mixed in a preset ratio to obtain a mixed sample set;

[0121] A2: Utilize audio processing technology to traverse the mixed sample set to perform voiceprint feature analysis, and obtain a voiceprint feature sample set to perform initial training on the discriminator. The discriminator is used to identify user voice information from noisy voice data. During training, the training process is supervised by the recognition voiceprint feature set corresponding to the recognition sample set with the command sound identification, helping the discriminator learn how to distinguish the user's command voice from background noise. The training process is analyzed using a loss function, and the hyperparameters of the discriminator are continuously adjusted, including the learning rate, batch size, and number of network layers. The hyperparameters are adjusted until the loss converges, and a trained voiceprint denoising recognizer is obtained.

[0122] A3: The loss function is the cross entropy loss function, and the function formula is as follows:

[0123] ;

[0124] in, is the loss function value, N is the number of samples in the mixed sample set, and is an integer greater than or equal to 1. is the true label of the i-th voiceprint feature in the mixed sample set (1 indicates that the voiceprint feature comes from the noise sample set, and 0 indicates that the voiceprint feature comes from the recognition sample set), is the predicted probability of the discriminator for the i-th recognition voiceprint feature from the recognition sample set.

[0125] Preferably, the acquisition logic of the noise-reduced semantic data subsequence;

[0126] Extract and record the start and end time of each segment of the acquired noise-reduced speech data sequence to ensure that the subsequent time interval calculation has an accurate start and end point. Calculate the time interval between every two adjacent segments of speech data in the noise-reduced speech data sequence, and store the time intervals of all adjacent segments in an interval list.

[0127] Set time interval thresholds based on scenarios and user habits, and use them to distinguish between continuous commands and pauses in conversation.

[0128] Traverse the time interval list and compare each interval with the set time interval threshold. If the time interval exceeds the threshold, the point is regarded as a dialogue segmentation point and an independent semantic data subsequence is generated; if it does not exceed the threshold, the speech data is classified into the same subsequence;

[0129] According to the segmentation results, the divided speech data segments are organized into multiple noise-reduced semantic data subsequences. Each subsequence contains a complete speech command and records the corresponding timestamp.

[0130] Preferably, the acquisition logic of the interaction instruction:

[0131] Perform semantic analysis on each denoised semantic data subsequence, extract the main speech content and convert it into a processable text form;

[0132] Generate a timestamp for each subsequence, record the time when the voice command was issued, and store it in the corresponding semantic subsequence;

[0133] The interaction time uses the start time or end time of the subsequence, and the appropriate time point is selected according to the needs;

[0134] Compare the semantic content of adjacent subsequences. If the subsequence content is repeated, remove the repeated instructions. Use the differential algorithm to analyze the content differences of adjacent subsequences and extract the new or unique semantic information in each subsequence.

[0135] Extracting semantic features from the plurality of denoised semantic data subsequences by using a semantic feature extraction network layer to obtain a plurality of denoised semantic feature subsequences, and performing differential semantic analysis on the plurality of denoised semantic data subsequences:

[0136] When the semantic difference is greater than the preset semantic difference, two interaction instructions are generated respectively according to the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences;

[0137] When the semantic difference is greater than the preset semantic difference, the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences are fused to generate an interaction instruction, thereby achieving the goal of removing duplicate instructions.

[0138] Determine the differences between instructions, identify each new instruction of the user, organize the independent instructions obtained after differential analysis into an interaction instruction set, and attach the corresponding interaction time. Each instruction corresponds to its timestamp to form a set containing interaction instructions and interaction time.

[0139] In the interaction instruction set, the execution order or priority of the instructions is optimized according to the instruction content and interaction time, and the interaction instruction set after differential semantic analysis and optimization is output to the action execution module to provide clear instruction input for the robot's action plan.

[0140] Preferably, the determination logic of the target robot action execution plan is:

[0141] Arrange the interaction instructions in the order of interaction time to form a time-series instruction chain. Each instruction corresponds to its interaction time, indicating that the execution follows the order of user intention;

[0142] Perform dependency analysis on adjacent instructions in the instruction sequence. If there is a conflict, retain the instruction with higher priority or determine the execution instruction based on the time sequence. Generate a specific action execution plan based on the classification, priority and dependency analysis results.

[0143] The execution time of each instruction is corrected according to the interaction time, and the execution plan is simulated within the system to detect whether there are unreasonable or unexecutable instructions in the plan, and the plan is adjusted or replanned; the final action execution plan is output to the robot's control module and prepared for execution.

[0144] Example 3

[0145] According to an exemplary embodiment, an electronic device includes: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0146] The processor executes the above-mentioned human-computer interaction method based on speech difference processing by calling the computer program stored in the memory.

[0147] Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may have relatively large differences due to different configurations or performances. It can include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the human-computer interaction method based on voice differential processing provided in each of the above method embodiments. The electronic device can also include other components for implementing the functions of the device. For example, the electronic device can also have components such as a wired or wireless network interface and an input / output interface for input and output. The embodiments of the present application are not described in detail here.

[0148] Example 4

[0149] According to an exemplary embodiment, a computer-readable storage medium is shown, on which a rewritable computer program is stored;

[0150] When the computer program is run on a computer device, the computer device is enabled to execute the above-mentioned human-computer interaction method based on speech difference processing.

[0151] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including at least one computer program. The at least one computer program is executable by a processor to implement the human-computer interaction method based on speech differential processing in the above-described embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, or an optical data storage device.

[0152] In an exemplary embodiment, a computer program product or computer program is also provided, comprising one or more program codes stored in a computer-readable storage medium. One or more processors of an electronic device can read the one or more program codes from the computer-readable storage medium and execute the one or more program codes, thereby enabling the electronic device to perform the above-described human-computer interaction method based on speech differential processing.

[0153] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0154] It should be understood that determining B based on A does not mean determining B based solely on A. B can also be determined based on A and / or other information.

[0155] Those skilled in the art will understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0156] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0157] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only one type. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0159] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0160] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A human-computer interaction method based on speech differential processing, characterized by: The following steps are involved: S1: extracting a speech data sequence received by a target robot within a preset time window, performing scene feature extraction on an application scene of the target robot, obtaining a scene feature set, and constructing a background noise database based on the scene feature set; S2: Collecting user voice information, combining it with the electronic device sound information and guest voice information in the background noise database, performing voiceprint recognition model training, obtaining a background voiceprint recognition denoiser, and performing noise reduction analysis on the scene feature set and the voice data sequence using the background voiceprint recognition denoiser to obtain a noise-reduced voice data sequence; S3: extracting the time intervals between adjacent noise-reduced speech data in the noise-reduced speech data sequence, performing dialogue segmentation on the noise-reduced speech data sequence according to the extraction result, and determining a plurality of noise-reduced semantic data subsequences; S4: performing differential semantic analysis on the multiple noise reduction semantic data subsequences to determine multiple interaction instructions, where the multiple interaction instructions include multiple interaction times; S5: Analyzing the target robot's action execution plan based on the multiple interaction instructions and the multiple interaction times, and determining the target robot's action execution plan; S6: interactively controlling the target robot according to the target robot action execution plan.

2. The human-computer interaction method based on speech differential processing according to claim 1, characterized in that: Configure a preset time window according to the actual application requirements of the robot, use the preset time window as the collection period of voice data, and obtain voice samples based on the preset time window.

3. The human-computer interaction method based on speech differential processing according to claim 1, characterized in that: The logic for obtaining the scene feature set: During the preset time window, the microphone is enabled to collect the user's voice data in real time and transmit the collected audio data in the form of digital signals; At the same time, the robot's environmental sensors are activated to obtain environmental feature data of the current application scenario based on the environmental sensors, and the microphone is activated to collect background noise data in the home scenario. The background noise data includes the sound of electronic devices and the sound information of guests, and the scene data is obtained; The collected scene data is preprocessed, including outlier removal and normalization, and speech recognition scene features are screened out based on a specific threshold, and the scene features are combined into a scene feature set.

4. The human-computer interaction method based on speech differential processing according to claim 1, characterized in that: The construction process of the background voiceprint recognition noise reducer is as follows: A1: Noise identification is performed on electronic device sound information and guest sound information in the background noise database to obtain a noise sample set, user voice information is collected to perform command sound identification, and a recognition sample set is obtained. The noise sample set and the recognition sample set are mixed in a preset ratio to obtain a mixed sample set; A2: Utilize audio processing technology to traverse the mixed sample set to perform voiceprint feature analysis, and obtain a voiceprint feature sample set to perform initial training on the discriminator. The discriminator is used to identify user voice information from noisy voice data. During training, the training process is supervised by the recognition voiceprint feature set corresponding to the recognition sample set with the command sound identification, helping the discriminator learn how to distinguish the user's command voice from background noise. The training process is analyzed using a loss function, and the hyperparameters of the discriminator are continuously adjusted, including the learning rate, batch size, and number of network layers. The hyperparameters are adjusted until the loss converges, and a trained voiceprint denoising recognizer is obtained. A3: The loss function is the cross entropy loss function, and the function formula is as follows: ； in, is the loss function value, N is the number of samples in the mixed sample set, and is an integer greater than or equal to 1. is the true label of the i-th voiceprint feature in the mixed sample set (1 indicates that the voiceprint feature comes from the noise sample set, and 0 indicates that the voiceprint feature comes from the recognition sample set), is the predicted probability of the discriminator for the i-th recognition voiceprint feature from the recognition sample set.

5. The human-computer interaction method based on speech differential processing according to claim 1, characterized in that: Logic for acquiring the noise-reduced semantic data subsequence; Extract and record the start and end time of each segment of the acquired noise-reduced speech data sequence to ensure that the subsequent time interval calculation has an accurate start and end point. Calculate the time interval between every two adjacent segments of speech data in the noise-reduced speech data sequence, and store the time intervals of all adjacent segments in an interval list. Set time interval thresholds based on scenarios and user habits, and use them to distinguish between continuous commands and pauses in conversation. Traverse the time interval list and compare each interval with the set time interval threshold; If the time interval exceeds the threshold, the conversation pause is regarded as the conversation segmentation point, and an independent semantic data subsequence is generated; if it does not exceed the threshold, the speech data is classified into the same subsequence; According to the segmentation results, the divided speech data segments are organized into multiple noise-reduced semantic data subsequences. Each subsequence contains a complete speech command and records the corresponding timestamp.

6. The human-computer interaction method based on speech differential processing according to claim 2, characterized in that: The acquisition logic of the interaction instruction: Perform semantic analysis on each denoised semantic data subsequence, extract the main speech content and convert it into a processable text form; Generate a timestamp for each subsequence, record the time when the voice command was issued, and store it in the corresponding semantic subsequence; The interaction time uses the start time or end time of the subsequence, and the appropriate time point is selected according to the needs; Compare the semantic content of adjacent subsequences. If the subsequence content is repeated, remove the repeated instructions. Use the differential algorithm to analyze the content differences of adjacent subsequences and extract the new or unique semantic information in each subsequence. Extracting semantic features from the plurality of denoised semantic data subsequences by using a semantic feature extraction network layer to obtain a plurality of denoised semantic feature subsequences, and performing differential semantic analysis on the plurality of denoised semantic data subsequences: When the semantic difference is greater than the preset semantic difference, two interaction instructions are generated respectively according to the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences; When the semantic difference is greater than the preset semantic difference, the two user intentions reflected by the two denoised semantic feature subsequences corresponding to the two adjacent denoised semantic data subsequences are fused to generate an interactive instruction, thereby achieving the goal of removing duplicate instructions; Determine the differences between commands, identify each new command from the user, organize the independent commands obtained after differential analysis into an interactive command set, and attach the corresponding interaction time. Each command corresponds to its timestamp, forming a set containing interactive commands and interaction times; In the interaction instruction set, the execution order or priority of the instructions is optimized according to the instruction content and interaction time, and the interaction instruction set after differential semantic analysis and optimization is output to the action execution module to provide clear instruction input for the robot's action plan.

7. The human-computer interaction method based on speech differential processing according to claim 4, characterized in that: The determination logic of the target robot action execution plan: Arrange the interaction instructions in the order of interaction time to form a time-series instruction chain. Each instruction corresponds to its interaction time, indicating that the execution follows the order of user intention; Perform dependency analysis on adjacent instructions in the instruction sequence. If there is a conflict, retain the instruction with higher priority or determine the execution instruction based on the time sequence. Generate a specific action execution plan based on the classification, priority and dependency analysis results. The execution time of each instruction is corrected according to the interaction time, and the execution plan is simulated within the system to detect whether there are unreasonable or unexecutable instructions in the plan, and the plan is adjusted or replanned; the final action execution plan is output to the robot's control module and prepared for execution.

8. A human-computer interaction system based on speech differential processing, based on the implementation of the human-computer interaction method based on speech differential processing according to any one of claims 1 to 7, characterized in that: It includes background noise extraction module, voice information extraction module, voice data processing module, interactive instruction extraction module, execution plan analysis module and control module. Each module is connected by wired or wireless communication: A background noise extraction module extracts the speech data sequence received by the target robot within a preset time window, performs scene feature extraction on the application scene of the target robot, obtains a scene feature set, and constructs a background noise database based on the scene feature set; A voice information extraction module collects the user's voice information, combines it with the electronic device sound information and the guest sound information in the background noise database, trains a voiceprint recognition model, obtains a background voiceprint recognition denoiser, and uses the background voiceprint recognition denoiser to perform noise reduction analysis on the scene feature set and the voice data sequence to obtain a noise-reduced voice data sequence; a speech data processing module, which extracts the time intervals between adjacent noise-reduced speech data in the noise-reduced speech data sequence, performs dialogue segmentation on the noise-reduced speech data sequence according to the extraction result, and determines a plurality of noise-reduced semantic data subsequences; An interaction instruction extraction module performs differential semantic analysis on the multiple noise reduction semantic data subsequences to determine multiple interaction instructions, wherein the multiple interaction instructions include multiple interaction times; an execution plan analysis module, which analyzes the action execution plan of the target robot based on the multiple interaction instructions and the multiple interaction times, and determines the action execution plan of the target robot; A control module interactively controls the target robot according to the target robot action execution plan.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; The processor executes the human-computer interaction method based on speech difference processing according to any one of claims 1 to 7 by calling the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the human-computer interaction method based on speech difference processing as described in any one of claims 1 to 7.

Images