Intelligent robot voice control method and system based on environment perception

Abstract

This invention discloses a voice control method and system for intelligent robots based on environmental perception, relating to the field of robot voice control. The method includes steps S1: synchronously acquiring voice and multimodal environmental data through a multi-source sensing array; Step S2: performing semantic parsing on the voice and extracting an environmental feature matrix; Step S3: constructing a fusion feature space to establish a mapping relationship between intent and environment; Step S4: calculating a command score using a safety confidence assessment algorithm; and Step S5: dynamically adjusting motion control parameters based on the score and triggering obstacle avoidance and mitigation logic when confidence is insufficient. This application achieves deep coupling between semantic features and environmental constraints, dynamically correcting the execution sequence through a quantified risk assessment mechanism, effectively solving the problems of motion deviation and safety risks in voice control under complex environments, and significantly improving the robot's intelligence level.

Description

Technical Field

[0001] This invention relates to the field of robot voice control, and in particular to a method and system for intelligent robot voice control based on environmental perception. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, intelligent robots are increasingly being used in fields such as home services, industrial logistics, and medical care. Voice interaction, as one of the most natural interaction methods for robots, directly affects the user experience and the level of system intelligence. To ensure that robots can accurately execute user commands and ensure operational safety in complex and dynamic environments, building a voice control system with environmental awareness capabilities has become a hot topic in the industry.

[0003] Among them, the voice control method for intelligent robots based on environmental perception focuses on integrating voice semantic features with multimodal environmental perception data. By monitoring information such as obstacle distance, light intensity, and road slope in real time, it uses a multimodal fusion algorithm to dynamically evaluate the safety confidence of the executed commands. This technical approach aims to improve the robot's adaptability to complex environments and achieve deep decoupling and safe collaboration between voice control commands and physical environmental constraints through precise adjustment of drive parameters and obstacle avoidance mechanisms.

[0004] Existing technologies for robot voice control still suffer from insufficient integration between command execution and environmental feedback. For example, some systems do not adequately consider the impact of real-time environmental characteristics on movement safety when receiving voice commands, leading to execution risks or action deviations in complex terrain or obstacle-filled scenarios. Therefore, this paper proposes an intelligent robot voice control method and system based on environmental perception to address these issues. Summary of the Invention

[0005] The main objective of this invention is to provide a voice control method and system for intelligent robots based on environmental perception, so as to solve the problems mentioned in the background above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: a voice control method and system for intelligent robots based on environmental perception, comprising the following steps: Step S1: Synchronously collect multimodal raw data of the external environment and user voice interaction signals through a multi-source sensing array integrated on the surface of the robot shell; Step S2: Perform semantic parsing on the collected voice interaction signals to extract semantic feature vectors containing action intentions, target objects, and motion constraints; perform spatiotemporal synchronization processing on the multimodal raw data to extract environmental feature matrices containing spatial distribution of obstacles, ambient light intensity, and physical slope of the road surface. Step S3: Construct a fusion feature space of semantic and environmental features, input the semantic feature vector and environmental feature matrix into the preset cross-association model, and establish the mapping relationship between voice commands at the intent level and the physical environment level. Step S4: Based on the fusion feature space, the executability of the current voice command is dynamically evaluated using a preset security confidence assessment algorithm, and a security confidence score for the current command is calculated and generated. Step S5: Based on the safety confidence score, adjust the model dynamically to calculate the robot's motion control parameters through preset driving parameters, and trigger the obstacle avoidance remediation mechanism for commands with confidence scores below the preset threshold to generate a corrected execution sequence; Step S6: Transmit the motion control parameters and execution sequence to the robot's underlying drive mechanism to execute the corresponding physical actions, and output the execution status information of the current task through the voice feedback module.

[0007] Furthermore, the multi-source sensing array in S1 includes a laser scanning radar, a wide-angle CMOS imaging sensor, a ring microphone array, and an attitude heading reference system. The laser radar is fixed to the center of the top of the robot by a bracket, and its scanning plane is parallel to the horizontal reference plane of the robot. The wide-angle CMOS imaging sensor is fixed to the front and side of the robot shell by bolts, and its optical axis covers the robot's travel envelope area. The ring microphone array is suspended in the internal cavity of the robot by flexible shock-absorbing pads, and its acquisition holes are distributed at predetermined positions on the shell. The attitude perception mode is acquired by an inertial measurement unit and tilt sensor integrated on the chassis control motherboard.

[0008] Furthermore, S1 specifically includes the following steps: S101: Laser scanning radar constructs a 3D point cloud around the robot; S102: Wide-angle CMOS imaging sensor captures ambient light intensity and obstacle texture information; S103: Circular microphone array for acquiring voice interaction signals; S104: Real-time data reading from the attitude and heading reference system.

[0009] Furthermore, the processing of the voice interaction signal in step S2 includes: S201: Removes environmental background noise from the acoustic acquisition mode using a preset noise reduction algorithm, and converts the speech signal into a text sequence using an acoustic model and a language model; S202: Use a pre-built natural language processing model to perform dependency parsing on the text sequence, extract action predicates as action intentions, extract noun phrases as target objects, extract degree adverbs and locative words as motion constraints, and encode them into fixed-length semantic feature vectors. The processing of the multimodal raw data in step S2 includes: S203: Use statistical filtering algorithms to remove outlier noise points in the laser point cloud and project the point cloud data to a unified robot coordinate system through coordinate transformation; S204: Use visual recognition algorithms to extract the illumination histogram features in the image and calculate the ambient light intensity; use the output value of the tilt sensor to calculate the robot's current pitch and roll angles, and obtain the physical slope of the road surface through gravity vector decomposition; S2 further includes: S205: Combine the above data into a multidimensional environmental feature matrix.

[0010] Furthermore, the process of constructing the fused feature space in step S3 is as follows: S301: Using a preset extrinsic calibration matrix, the pixel coordinates of the visual imaging mode, the spatial coordinates of the laser scanning mode, and the sound source localization coordinates of the acoustic acquisition mode are spatially aligned. S302: Employs a timestamp alignment mechanism to ensure that the data of each modality remains synchronized within a preset time window; S303: The semantic feature vector is used as the query vector, and the environmental feature matrix is ​​used as the key vector and value vector. The input is fed into the self-attention mechanism layer to calculate the spatial attention distribution weight between semantic intent and environmental constraints, thereby generating a fusion feature vector that represents the degree of coupling of voice commands in the current physical environment.

[0011] Furthermore, the working logic of the security confidence assessment algorithm in step S4 is as follows: S401: Pre-establish a security assessment neural network model, which includes a feature input layer, a nonlinear mapping layer, and a confidence output layer; S402: Calculate the ratio of obstacle distance to robot braking distance; S403: The influence factor of coupled road surface slope on motor output torque and the interference coefficient of ambient light intensity on visual positioning accuracy; S404: The confidence output layer outputs a safety confidence score within a predetermined numerical range based on the weighted calculation result; S405: Determine whether the score is within the safe threshold range.

[0012] Furthermore, the working logic of the driving parameter adjustment model in step S5 is as follows: S501: When the safety confidence score is higher than the first preset threshold, calculate the target speed and steering angle of the drive motor according to the original intent corresponding to the semantic feature vector; S502: When the safety confidence score is between the first preset threshold and the second preset threshold, the output power of the drive motor is reduced by a preset attenuation function, and the sampling frequency of the obstacle avoidance sensor is increased. S503: When the safety confidence score is lower than the second preset threshold, the current voice command execution chain is automatically cut off, and an obstacle avoidance and recovery mechanism is triggered. The mechanism includes: calculating a recovery path that deviates from the obstacle and has a gentle slope change through a path planning algorithm, and updating the driving parameters to control commands that drive along the recovery path.

[0013] Furthermore, in step S6, the underlying drive mechanism includes a wheel drive motor, a steering servo, and a brake; the motion control parameters are transmitted to the drivers of each drive motor via a controller local area network bus, and the torque and speed of the motor are controlled by pulse width modulation signals; the voice feedback module calls the corresponding status description text from a preset voice library according to the level of safety confidence score, drives the speaker to emit sound through digital-to-analog conversion and power amplification circuit, and informs the user of the safety of the command execution and the obstacle avoidance action currently taken.

[0014] Furthermore, the present invention also provides an intelligent robot voice control system based on environmental perception, the system comprising: The data acquisition module is used to acquire voice interaction signals and raw multimodal environmental data through a multi-source sensing array; The feature extraction module is used to perform semantic parsing on speech signals to generate semantic feature vectors and to preprocess environmental data to generate environmental feature matrices. The spatial fusion module is used to construct a fusion feature space of semantic features and environmental features, and to calculate the coupling weights between intent and environment. The security assessment module is used to calculate the security confidence score of the current instruction using a security confidence assessment algorithm. The control and adjustment module is used to dynamically calculate motion control parameters based on the safety confidence score, and to trigger obstacle avoidance recovery logic when the confidence score is insufficient. The execution feedback module is used to drive the actions of the underlying mechanisms and provide voice feedback on the task status.

[0015] Furthermore, the multi-source sensing array includes a laser scanning radar, a wide-angle CMOS imaging sensor, a ring microphone array, and an attitude and heading reference system; the underlying drive mechanism includes a wheel drive motor, a steering servo, and a brake; the voice feedback module includes a voice synthesis unit, a digital-to-analog converter circuit, and a speaker; the feature extraction module includes a semantic parsing engine and an environmental feature extraction unit; the spatial fusion module includes a spatiotemporal synchronization unit and a cross-correlation model; the safety assessment module includes a pre-trained safety assessment neural network model; and the control adjustment module includes a drive parameter adjustment model and an obstacle avoidance and recovery logic unit.

[0016] The present invention has the following beneficial effects: 1. This invention achieves deep coupling of speech semantic features and multimodal environmental data through a multi-source sensing array. It uses a cross-association model to directly map the user's abstract speech commands to specific physical environmental constraints, changing the isolated state of command execution and environmental perception in traditional voice control. This enables the robot to dynamically calibrate semantic understanding deviations based on real-time environmental features, improving the accuracy of command parsing in complex dynamic environments.

[0017] 2. The safety confidence assessment algorithm introduced in this invention establishes a quantitative risk assessment mechanism for the execution of voice commands by comprehensively considering multiple physical dimensions such as obstacle distance, road slope, and light intensity. By using the safety confidence score as feedback input for adjusting driving parameters, the robot can automatically execute safety protection strategies such as deceleration, avoidance, or refusal to execute when it receives voice commands with potential risks, effectively reducing the execution risk of the robot in complex terrain or obstacle-dense scenarios.

[0018] 3. The obstacle avoidance and recovery mechanism and dynamic adjustment model of driving parameters designed in this invention realize deep decoupling and safe coordination between voice control commands and physical environmental constraints; by automatically generating a corrected execution sequence when the confidence level is low, the robot has the ability to autonomously complete safe obstacle avoidance without completely relying on the precise commands of the user, which significantly enhances the robot's adaptability to non-ideal interactive environments.

[0019] 4. This invention employs spatiotemporal synchronization processing technology and a self-attention mechanism to improve the fusion accuracy of multimodal data at the feature level. By unifying the coordinate transformation and timestamp alignment of visual, laser, acoustic, and posture data, it eliminates observation differences between different sensors, provides high-fidelity data support for safety confidence assessment, reduces motion deviations caused by sensor noise or environmental interference, and makes the robot's motion performance more in line with the expected level of intelligence. Attached Figure Description

[0020] Figure 1This is an overall flowchart of the method in the intelligent robot voice control method and system based on environmental perception according to the present invention. Figure 2 This is a flowchart of S1 in an intelligent robot voice control method and system based on environmental perception, according to the present invention. Figure 3 This is a flowchart of S2 in an intelligent robot voice control method and system based on environmental perception, according to the present invention. Figure 4 This is a flowchart of S3 in an intelligent robot voice control method and system based on environmental perception, according to the present invention. Figure 5 This is a flowchart of S4 in an intelligent robot voice control method and system based on environmental perception, according to the present invention. Figure 6 This is a flowchart of S5 in an intelligent robot voice control method and system based on environmental perception, according to the present invention. Detailed Implementation

[0021] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0022] Please refer to Figures 1 to 6 As shown: A voice control method and system for intelligent robots based on environmental perception. The specific implementation of this invention uses a multi-source perception and intelligent voice collaborative control system deployed on an indoor mobile chassis as a carrier, detailing the complete operation flow of the voice control method and system for intelligent robots based on environmental perception. The system consists of three main parts: a front-end multi-source perception array, a mid-end information processing module, and a back-end execution feedback module. Each part is logically coupled with a synchronous trigger controller via an internal high-speed bus. The front-end sensors include a laser scanning radar located on the top of the robot's shell, wide-angle CMOS imaging sensors on the front and sides, an internally suspended ring microphone array, and an attitude and heading reference system integrated into the chassis. The mid-end processing module includes a semantic parsing engine, a spatiotemporal synchronization unit, a cross-association model, and a safety confidence evaluator. The back-end consists of a low-level drive controller, an obstacle avoidance and recovery logic unit, and a voice synthesis feedback unit. The entire system captures environmental modalities and voice commands in real time during robot operation, dynamically adjusts the motion state, and outputs a corrected execution sequence.

[0023] The laser scanning radar adopts a linear pulse structure, installed at a height of 0.8 meters, with a scanning frequency set at 15Hz and a horizontal field of view of 360°, used to construct a 3D spatial point cloud around the robot. Three wide-angle CMOS imaging sensors are configured, covering the front, left front, and right front directions respectively, with a resolution of 1080P and a frame rate of 60fps, used to capture ambient light intensity and obstacle texture information. The ring microphone array consists of six high-sensitivity digital silicon microphones evenly spaced, with a sampling rate set at 48kHz, fixed to the inner cavity of the housing by flexible shock-absorbing pads, and has sound source localization and echo cancellation functions. The attitude and heading reference system consists of a nine-axis inertial sensor and a high-precision inclinometer; its output, after fusion filtering, provides real-time attitude data of the robot relative to the gravity vector, with its time synchronization error controlled within a specified range. Within this range, ensure that the data of each modality has strict temporal consistency at the source of collection.

[0024] After configuration, proceed to step S1, which is: synchronously collect multimodal raw data of the external environment and user voice interaction signals through a multi-source sensing array integrated on the surface of the robot shell; Specifically, the following steps are included: S101: Laser scanning radar constructs a 3D point cloud around the robot; S102: Wide-angle CMOS imaging sensor captures ambient light intensity and obstacle texture information; S103: Circular microphone array for acquiring voice interaction signals; S104: Real-time data reading from the attitude and heading reference system.

[0025] After data acquisition is completed in step S1, the process proceeds to step S2, which involves speech semantic analysis and environmental feature extraction. Step S2 involves: performing semantic analysis on the acquired speech interaction signals to extract semantic feature vectors containing action intentions, target objects, and motion constraints; and performing spatiotemporal synchronization processing on the multimodal raw data to extract environmental feature matrices containing spatial distribution of obstacles, ambient light intensity, and physical slope of the road surface. Specifically, the following steps are included: S201: First, preprocess the voice interaction signal collected by the ring microphone array: apply the spectral subtraction noise reduction algorithm to remove the background noise generated by mechanical transmission, and use the Hidden Markov Model (HMM) to convert the acoustic features into a text sequence. S202: Use a natural language processing model to perform dependency parsing on the text and extract verb predicates as action intentions. Extract noun phrases as target objects Extracting directional adverbs as motion constraints And map it into a fixed-length semantic feature vector; S203: Process the multimodal raw environmental data: Use a statistical filter to remove noise from the laser point cloud, set the neighborhood radius to 0.1 meters and the standard deviation multiple to 2.0; Project the point cloud to the robot's central coordinate system through homogeneous coordinate transformation; S205: Extract the global brightness histogram using the visual imaging modality and calculate the ambient light intensity. ; Calculate the chassis pitch angle using the inclinometer output value. With roll angle The physical slope of the current road surface is obtained through gravity vector decomposition. ; S205: Combine the above data into a multidimensional environmental feature matrix.

[0026] Proceed to step S3: Construct a fusion feature space of semantic and environmental features, input the semantic feature vector and environmental feature matrix into the preset cross-association model, and establish the mapping relationship between voice commands at the intent level and the physical environment level. Specifically, the following steps are included: S301: Using a preset external parameter calibration matrix The pixel coordinates of the CMOS imaging sensor are affine aligned with the spatial coordinates of the laser scanning radar. S302: Employs a timestamp alignment mechanism to extract observation records from each sensor within the same trigger cycle; S303: Using the semantic feature vector as the query vector and the environmental feature matrix as the key and value vectors, input them into the self-attention mechanism network layer to calculate the attention weight distribution of semantic intent under the current physical space constraints. This generates a fusion feature vector that represents the degree of coupling between voice commands and the physical environment. This vector can quantitatively reflect the difficulty of executing abstract commands in the current environment.

[0027] Proceed to step S4: Based on the fusion feature space, use the preset security confidence evaluation algorithm to dynamically evaluate the executability of the current voice command and calculate and generate a security confidence score for the current command; Specifically, the following steps are included: S401: The system calls a pre-trained security evaluation neural network model, which contains a three-layer perceptron structure; the fused feature vector is input to the feature input layer; S402: In the nonlinear mapping layer, the system executes weighted operation logic: by calculating the distance to obstacles. safe braking distance from the robot proportionality coefficient ; S403: Factors influencing motor load based on road slope and the interference coefficient caused by light intensity on visual positioning accuracy ; S404: Performs a non-linear mapping on the fused feature vectors and outputs a security confidence score. ; S405: Determine whether the score is within the safe threshold range, for example: when or At that time, the confidence output layer will forcibly shift the score towards the lower score range, where To establish a safe distance, This is the preset climbing threshold.

[0028] In step S5, based on the safety confidence score, the model dynamically calculates the robot's motion control parameters by adjusting the preset drive parameters, and triggers an obstacle avoidance remediation mechanism for commands with confidence scores below a preset threshold, generating a corrected execution sequence. Specifically, the following steps are included: S501: When the security confidence score is... When the speed is higher than the first preset threshold, the target speed Ntar and steering angle δ of the drive motor are calculated according to the original intent corresponding to the semantic feature vector. S502: When the security confidence score is... When the output power of the drive motor is between the first preset threshold and the second preset threshold, a preset attenuation function is used to reduce the output power of the drive motor. Reduce and increase the sampling frequency of the obstacle avoidance sensor; S503: When the security confidence score is... When the value falls below the second preset threshold, the current voice command execution chain is cut off, triggering the obstacle avoidance and recovery mechanism. The specific working logic is shown in the table below: Table 1 is the working logic table of the driving parameter adjustment model in this implementation. ; When the obstacle avoidance and recovery mechanism is triggered, the system uses a dynamic window method to search for a path in the local environment map that avoids high-risk obstacles and has a slope change rate. The remedy path is determined, and the torque command of the drive motor is updated to a corrected sequence that travels along the remedy path.

[0029] Proceed to step S6: Transmit the motion control parameters and execution sequence to the robot's underlying drive mechanism, execute the corresponding physical actions, and output the execution status information of the current task through the voice feedback module; Specifically, the drive controller sends control frames to the wheel drive motor via the controller area network bus, and adjusts the duty cycle using pulse width modulation signals to achieve precise control of speed and torque; simultaneously, the voice feedback module... The classification results retrieve status descriptions from a local preset text library, such as: "The slope ahead is too steep, and the system has automatically switched to low-speed climbing mode." The speaker output is driven through digital-to-analog conversion to achieve an interactive closed loop.

[0030] To enable those skilled in the art to fully understand and implement this invention, the following section further supplements the specific implementation principle of this invention by taking a specific application scenario of an indoor logistics robot in a complex corridor environment in a hospital as an example.

[0031] In steps S1 and S2: When the hospital pharmacy staff gives the robot the voice command "Go to the operating room on the third floor to deliver medicine", the ring microphone array inside the robot shell captures the sound wave signal in real time, filters out the mechanical noise of the chassis motor rotation through the spectral subtraction noise reduction algorithm, and uses the hidden Markov model to convert the acoustic features into a text sequence. During this process, the semantic parsing engine extracts "go to" as an action intent through dependency parsing. Extract the "operating room" as the target object. The data is mapped to a high-dimensional semantic feature vector; simultaneously, a laser scanning radar positioned on top of the casing rotates and scans at a frequency of 15Hz to acquire three-dimensional point cloud data of the corridor interior; three wide-angle CMOS imaging sensors simultaneously capture image frames containing strong light interference from windows; and the attitude and heading reference system integrated into the chassis uses a high-precision inclinometer to perceive the current road surface in real time. slope.

[0032] In step S3: The mid-range information processing module calls the preset extrinsic parameter calibration matrix. The system affinely aligns the spatial coordinates of obstacles detected by the laser scanning radar with the high-light pixel areas captured by the CMOS imaging sensor. It employs a timestamp alignment mechanism to ensure that speech semantic features and physical environment features are on the same spatiotemporal reference within the same trigger cycle. At this point, the cross-association model uses the semantic feature vector as the query vector and utilizes a self-attention mechanism network to calculate its attention weight distribution in the environmental feature matrix. The principle is that when the semantic intent is "high-speed movement" and the environmental features include "large slope" or "severe light interference", the model will automatically increase the weight of the physical constraint features and generate a fusion feature vector that represents the coupling relationship between the intent and the environment, thereby providing a quantitative correlation basis for subsequent safety assessment.

[0033] In step S4: The system inputs the fused feature vector into the safety assessment neural network model; in the nonlinear mapping layer, the evaluator performs weighted operation logic: first, it calculates the obstacle distance fed back by the current LiDAR. The safe braking distance calculated by the system based on the current speed. proportionality coefficient Subsequently, the physical slope output by AHRS was combined. Factors affecting motor load And the interference coefficient of strong light on positioning accuracy in the visual imaging modality The safety confidence score is calculated by comprehensive calculation. For example, if a pedestrian is detected 2 meters ahead on a downhill section of the road, due to... The value decreases and Fluctuations, the system will The forced offset to the low score range of 0.45 was used to determine that there was a high physical risk in executing the instruction at full speed.

[0034] In steps S5 and S6, the drive adjustment and obstacle avoidance recovery phase: the backend execution feedback module, according to... The score adjusts the driving parameters in real time; when When the slope is 0.45, the system triggers the restricted execution mode, and the obstacle avoidance and recovery logic immediately intervenes, using the dynamic window method to search for the slope change rate in the local point cloud map. The path is smooth; the underlying drive controller sends a corrected control frame to the wheel drive motor through the controller area network bus, and by adjusting the duty cycle of the pulse width modulation signal, the motor output power is reduced to 60% of the original value and the preload of the brake is increased; during this process, the voice synthesis feedback unit drives the speaker to announce through digital-to-analog conversion: "Slope and obstacle detected ahead, automatic switch to safe cruise mode has been made", thereby realizing closed-loop control from semantic intent to physical safety execution.

[0035] All content not described in detail in the specification belongs to the prior art known to those skilled in the art, and the model parameters of each sensor are not specifically limited; conventional equipment can be used. Circuit details not mentioned in this technical solution are not described in detail here as they belong to the prior art.

[0036] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A voice control method for intelligent robots based on environmental perception, characterized in that, Includes the following steps: Step S1: Synchronously collect multimodal raw data of the external environment and user voice interaction signals through a multi-source sensing array integrated on the surface of the robot shell; Step S2: Perform semantic parsing on the collected voice interaction signals to extract semantic feature vectors containing action intentions, target objects, and motion constraints; perform spatiotemporal synchronization processing on the multimodal raw data to extract environmental feature matrices containing spatial distribution of obstacles, ambient light intensity, and physical slope of the road surface. Step S3: Construct a fusion feature space of semantic and environmental features, input the semantic feature vector and environmental feature matrix into the preset cross-association model, and establish the mapping relationship between voice commands at the intent level and the physical environment level. Step S4: Based on the fusion feature space, the executability of the current voice command is dynamically evaluated using a preset security confidence assessment algorithm, and a security confidence score for the current command is calculated and generated. Step S5: Based on the safety confidence score, adjust the model dynamically to calculate the robot's motion control parameters through preset driving parameters, and trigger the obstacle avoidance remediation mechanism for commands with confidence scores below the preset threshold to generate a corrected execution sequence; Step S6: Transmit the motion control parameters and execution sequence to the robot's underlying drive mechanism to execute the corresponding physical actions, and output the execution status information of the current task through the voice feedback module.

2. The method according to claim 1, characterized in that, The multi-source sensing array in S1 includes a laser scanning radar arranged on the top of the housing, a wide-angle CMOS imaging sensor arranged on the front and sides of the housing, a ring microphone array arranged inside the housing, and an attitude and heading reference system arranged in the chassis area.

3. The method according to claim 2, characterized in that, S1 specifically includes the following steps: S101: Laser scanning radar constructs a 3D point cloud around the robot; S102: Wide-angle CMOS imaging sensor captures ambient light intensity and obstacle texture information; S103: Circular microphone array for acquiring voice interaction signals; S104: Real-time data reading from the attitude and heading reference system.

4. The method according to claim 1, characterized in that, The processing of the voice interaction signal in step S2 includes: S201: Apply noise reduction algorithms to remove environmental background noise and convert speech signals into text sequences; S202: Utilize a natural language processing model to perform dependency parsing on the text sequence and extract the action predicate as the action intention. Extract noun phrases as target objects Extracting degree adverbs and locative words as motion constraints ; The processing of the multimodal raw data in step S2 includes: S203: Remove outlier noise points from the laser point cloud and project the point cloud data onto the robot coordinate system; S204: Extract illumination histogram features from the image using a visual recognition algorithm and calculate the ambient light intensity. The robot's current pitch and roll angles are obtained using attitude perception modalities, and the physical slope of the road surface is obtained through gravity vector decomposition. ; S2 further includes: S205: Construct a multidimensional environmental feature matrix.

5. The method according to claim 1, characterized in that, The specific process of constructing the fused feature space in step S3 is as follows: S301: Using the external parameter calibration matrix Spatial alignment of visual pixel coordinates, laser spatial coordinates, and sound source localization coordinates; S302: Employs a timestamp alignment mechanism to ensure that data from each modality is synchronized within a preset time window; S303: Using the semantic feature vector as the query vector and the environmental feature matrix as the key and value vectors, input them into the self-attention mechanism layer to calculate the spatial attention distribution weights between semantic intent and environmental constraints. Generate a fused feature vector.

6. The method according to claim 1, characterized in that, The working logic of the security confidence assessment algorithm in step S4 is as follows: S401: Input the fused feature vector into the preset security assessment neural network model; S402: Calculate obstacle distance Braking distance of the robot proportionality coefficient ; S403: Factors influencing the output torque of the motor due to the slope of the coupled road surface And the interference coefficient of ambient light intensity on visual positioning accuracy ; S404: Performs a non-linear mapping on the fused feature vectors and outputs a security confidence score. ; S405: Determine whether the score is within the safe threshold range.

7. The method according to claim 1, characterized in that, The working logic of the driving parameter adjustment model in step S5 is as follows: S501: When the security confidence score is... When the speed exceeds the first preset threshold, the target speed of the drive motor is calculated according to the original intent corresponding to the semantic feature vector. and steering angle ; S502: When the security confidence score is... When the output power of the drive motor is between the first preset threshold and the second preset threshold, a preset attenuation function is used to reduce the output power of the drive motor. Reduce and increase the sampling frequency of the obstacle avoidance sensor; S503: When the security confidence score is... When the value falls below the second preset threshold, the current voice command execution chain is cut off, triggering an obstacle avoidance and recovery mechanism.

8. The method according to claim 7, characterized in that, The obstacle avoidance and mitigation mechanism includes: using a dynamic window method to search for and avoid obstacles in the local environment map, and considering the slope change rate. The recovery path is determined, and the motion control parameters are updated to control commands that travel along the recovery path. In step S6, motion control parameters are transmitted through the controller local area network bus, and the torque and speed of the motor are controlled by pulse width modulation signals.

9. An environment-aware intelligent robot voice control system, applied to any one of the environment-aware intelligent robot voice control methods described in claims 1-8, characterized in that, include: The data acquisition module is used to acquire voice interaction signals and raw multimodal environmental data through a multi-source sensing array; The feature extraction module is used to perform semantic parsing on speech signals to generate semantic feature vectors and to preprocess environmental data to generate environmental feature matrices. The spatial fusion module is used to construct a fusion feature space of semantic features and environmental features, and to calculate the coupling weights between intent and environment. The security assessment module is used to calculate the security confidence score of the current instruction using a security confidence assessment algorithm. The control and adjustment module is used to dynamically calculate motion control parameters based on the safety confidence score, and to trigger obstacle avoidance recovery logic when the confidence score is insufficient. The execution feedback module is used to drive the actions of the underlying mechanisms and provide voice feedback on the task status.

10. The system according to claim 9, characterized in that, The multi-source sensing array includes a laser scanning radar, a wide-angle CMOS imaging sensor, a ring microphone array, and an attitude and heading reference system; the underlying drive mechanism includes a wheel drive motor, a steering servo, and a brake; the voice feedback module includes a voice synthesis unit, a digital-to-analog converter circuit, and a speaker; the feature extraction module includes a semantic parsing engine and an environmental feature extraction unit; the spatial fusion module includes a spatiotemporal synchronization unit and a cross-correlation model; the safety assessment module includes a pre-trained safety assessment neural network model; and the control and adjustment module includes a drive parameter adjustment model and an obstacle avoidance and recovery logic unit.

Images