Voice data real-time control method and system of interactive toy robot

Abstract

The invention provides a voice data real-time control method and system for an interactive toy robot, and relates to the technical field of data processing, and the method comprises the steps: carrying out the acoustic feature extraction of a voice frame sequence, obtaining voiceprint feature embedding, arranging all voiceprint feature embedding according to a time sequence, and generating a voiceprint feature embedding sequence; inputting the voiceprint feature embedding sequence into a pre-trained voice recognition model for recognition to obtain a recognition text; comparing the recognition text with a preset instruction set to obtain a preliminary control instruction; according to the preliminary control instruction, a three-dimensional space discrete sampling cluster of the current environment is collected; projecting the three-dimensional space discrete sampling cluster to a two-dimensional plane to obtain a two-dimensional projection point set, and delimiting a multi-section constraint boundary on the two-dimensional projection point set; according to the invention, the accuracy, real-time performance and environment adaptability of voice control can be improved.

Description

Real-time voice data control method and system for interactive toy robots Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for real-time voice data control of interactive toy robots. Background Technology

[0002] Currently, interactive toy robots have generally integrated voice interaction and motion control functions, and their core control process is relatively mature: the user's voice signal is collected through a microphone, and after preprocessing, acoustic feature extraction and voice recognition, it is compared with a preset instruction set to generate preliminary control instructions, which are then driven by the controller to complete the corresponding actions. Although the above control process can realize basic voice control functions, there is still room for optimization in actual home use scenarios, and it is difficult to fully adapt to the needs of dynamically changing environments. For example, the main use scenario for interactive toy robots is children's rooms, where the environmental layout is often dynamic. Children's toys, picture books and other items may be temporarily scattered on the floor, and the placement of furniture may be adjusted according to the child's needs. Some areas may also have slight floor slopes or narrow spaces (such as the gap between the child's bed and desk, corner areas, etc.).

[0003] In existing methods, after generating initial control commands, the robot usually directly uses these commands as the final execution basis. The motion parameters (such as speed and turning angle) contained in the commands are mostly preset fixed values, failing to fully incorporate real-time spatial structure information of the current environment for dynamic adjustment. When the user issues control commands such as "forward" or "turn", if there are temporary obstacles such as toys in front of the robot, or if the area to be traversed is narrow, executing the control commands according to fixed parameters may cause the robot to collide with the obstacles, or make it difficult to pass through narrow areas due to mismatched motion parameters, affecting the user experience and potentially causing minor damage to the robot itself or objects in the environment. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for real-time voice data control of interactive toy robots, which can improve the accuracy, real-time performance and environmental adaptability of voice control.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a real-time voice data control method for an interactive toy robot, the method comprising: Step 1, extracting acoustic features from a speech frame sequence to obtain voiceprint feature embeddings, and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence; Step 2, inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model for recognition to obtain recognized text; comparing the recognized text with a preset instruction set to obtain preliminary control instructions; Step 3, collecting three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control instructions; Step 4, processing the three-dimensional spatial discrete sampling clusters... Projecting onto a two-dimensional plane yields a two-dimensional projection point set, and a multi-segment constraint boundary is defined on this set. The multi-segment constraint boundary is then segmented based on the spatial distribution characteristics of each sampling point in the three-dimensional discrete sampling cluster, generating multiple local feature response domains. Each sampling point in the three-dimensional discrete sampling cluster is mapped to its corresponding local feature response domain, and an environmental dynamic adaptation parameter is generated based on the distribution statistics of the sampling points within each local feature response domain. In step 5, the preliminary control command is corrected using the environmental dynamic adaptation parameter to obtain the final control command. The final control command is then sent to the toy robot to enable it to perform the corresponding action.

[0006] Secondly, the real-time voice data control system for the interactive toy robot includes: an extraction module for extracting acoustic features from a speech frame sequence to obtain voiceprint feature embeddings, and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence; a training module for inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model for recognition to obtain recognized text; comparing the recognized text with a preset instruction set to obtain preliminary control instructions; a processing module for acquiring three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control instructions; and a mapping module for projecting the three-dimensional spatial discrete sampling clusters onto a two-dimensional plane. A two-dimensional projection point set is obtained, and a multi-segment constraint boundary is defined on the two-dimensional projection point set. The multi-segment constraint boundary is segmented according to the spatial distribution characteristics of each sampling point in the three-dimensional discrete sampling cluster, generating multiple local feature response domains. Each sampling point in the three-dimensional discrete sampling cluster is mapped to the corresponding local feature response domain, and an environmental dynamic adaptation parameter is generated according to the distribution statistics of the sampling points in each local feature response domain. An execution module is used to modify the preliminary control command using the environmental dynamic adaptation parameter to obtain the final control command. The final control command is sent to the toy robot so that the toy robot can perform the corresponding action.

[0007] Thirdly, a computing device includes: one or more processors; and a storage device for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method.

[0008] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0009] The above-mentioned solution of the present invention includes at least the following beneficial effects: After generating the initial control command, by combining the environmental perception sensor to collect the three-dimensional spatial discrete sampling clusters of the current children's room environment, dividing the local feature response domain, and then generating environmental dynamic adaptation parameters, the voice command and real-time spatial structure information can be effectively integrated; the environmental dynamic adaptation parameters are used to dynamically correct the parameters such as the movement speed and movement direction in the initial control command, so that when the robot executes voice commands such as forward and turning, its movement parameters are no longer limited to preset fixed values, but can be adaptively adjusted according to the actual environment such as temporary obstacles, narrow spaces, and ground undulations in the children's room, thereby effectively reducing the possibility of collisions between the robot and toys, furniture, and other items, and improving the smoothness of the robot passing through narrow areas in a dynamically changing environment; at the same time, the present invention organically combines environmental perception with the voice control process, rather than simply setting up an independent obstacle avoidance module, making the robot's actions more in line with the actual use scenario, improving the safety, stability, and adaptability of the interactive toy robot in the children's room environment, and improving the user experience. Attached Figure Description

[0010] Figure 1 is a schematic flowchart of a real-time voice data control method for an interactive toy robot provided in an embodiment of the present invention.

[0011] Figure 2 is a schematic diagram of the real-time voice data control system for an interactive toy robot provided in an embodiment of the present invention. Detailed Implementation

[0012] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0013] As shown in Figure 1, an embodiment of the present invention proposes a real-time voice data control method for an interactive toy robot. The method includes the following steps: Step 1, extracting acoustic features from a speech frame sequence to obtain voiceprint feature embeddings, and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence; Step 2, inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model for recognition to obtain recognized text; comparing the recognized text with a preset instruction set to obtain preliminary control instructions; Step 3, collecting three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control instructions; Step 4, projecting the three-dimensional spatial discrete sampling clusters onto a two-dimensional space. A 2D projection point set is obtained by constructing a 3D plane, and a multi-segment constraint boundary is defined on the 2D projection point set. The multi-segment constraint boundary is segmented according to the spatial distribution characteristics of each sampling point in the 3D discrete sampling cluster, generating multiple local feature response domains. Each sampling point in the 3D discrete sampling cluster is mapped to the corresponding local feature response domain, and an environmental dynamic adaptation parameter is generated according to the distribution statistics of the sampling points in each local feature response domain. In step 5, the preliminary control command is modified using the environmental dynamic adaptation parameter to obtain the final control command. The final control command is sent to the toy robot so that the toy robot can perform the corresponding action.

[0014] In this embodiment of the invention, after generating initial control commands, the three-dimensional spatial discrete sampling clusters of the current children's room environment are collected by environmental perception sensors, and the local feature response domain is divided to generate dynamic environmental adaptation parameters. This effectively integrates voice commands with real-time spatial structure information. The dynamic environmental adaptation parameters are used to dynamically correct parameters such as motion speed and direction in the initial control commands. This allows the robot's motion parameters to be no longer limited to preset fixed values ​​when executing voice commands such as forward and turning. Instead, they can adaptively adjust according to the actual environment in the children's room, such as temporary obstacles, narrow spaces, and uneven ground. This effectively reduces the possibility of collisions between the robot and toys, furniture, and other items, and improves the robot's smoothness in passing through narrow areas in dynamically changing environments. At the same time, this invention organically combines environmental perception with voice control processes, rather than simply setting up an independent obstacle avoidance module. This makes the robot's actions more in line with actual usage scenarios, improves the safety, stability, and adaptability of the interactive toy robot in the children's room environment, and enhances the user experience.

[0015] In another preferred embodiment of the present invention, the process of acquiring the voice frame sequence may include: Step 001, acquiring the user's voice signal in real time through a microphone and converting it into digitized raw voice data, specifically including: acquiring the user's voice signal in real time through a microphone deployed on the front of the interactive toy robot body. This microphone is a high-sensitivity electret microphone adapted to the environment of a children's room, which can effectively capture voice signals of different volumes and tones emitted by children, while filtering out environmental noises in the children's room, such as the sound of toys colliding and the sound of picture books turning; the acquired analog voice signal will be immediately transmitted to the analog-to-digital converter built into the robot. The conversion process of the analog-to-digital converter specifically involves setting the conversion sampling frequency to 16, which is suitable for children's speech recognition. 000 Hz means that the analog speech signal is sampled 16,000 times per second. During each sampling, the analog-to-digital converter captures the amplitude of the analog speech signal at that moment, and then converts the amplitude into a corresponding digital value through quantization. That is, the quantization process uses a 16-bit binary encoding method to map the dynamic range of the analog signal amplitude to the digital range of 0 to 65535. The analog signal with the smallest amplitude corresponds to the digital value 0, the analog signal with the largest amplitude corresponds to the digital value 65535, and the analog signal with the intermediate amplitude is converted into the corresponding intermediate digital value in a linear proportion. After continuous sampling and quantization, the originally continuously changing analog speech signal is converted into discrete digital raw speech data, and finally forms a continuous digital raw speech data stream.

[0016] Step 002 involves preprocessing the original speech data. This preprocessing includes pre-emphasis, framing, and windowing to obtain a continuous speech frame sequence. Specifically, the preprocessing process includes three operations: pre-emphasis, framing, and windowing, ultimately resulting in a continuous speech frame sequence. The purpose of the pre-emphasis operation is to compensate for the attenuation of high-frequency components in the original speech data. Since the high-frequency components in children's speech are relatively weak and easily attenuated during propagation due to the influence of the children's room environment, the pre-emphasis operation is implemented by constructing a first-order high-pass filter. Specifically, each sampling point in the original speech data is subtracted from the product of the previous sampling point and the pre-emphasis coefficient. The pre-emphasis coefficient is selected as 0.97, which effectively enhances the high-frequency components in the original speech data and improves the recognizability of the speech signal. After pre-emphasis processing, the processed speech data undergoes framing. Framing divides continuous speech data into several short time segments, each segment being a speech frame. Considering the coherence and stability of children's speech, the frame length is set to 25 milliseconds, and the frame shift is set to 10 milliseconds. The frame shift setting avoids information gaps between adjacent speech frames, ensuring the integrity of the speech information. During framing, a 25-millisecond speech segment is extracted every 10 milliseconds in sequential order, forming multiple discrete speech frames. After framing, a windowing operation is performed on each speech frame. The purpose of windowing is... To reduce spectral leakage at both ends of the speech frame, since the signal abrupt changes at both ends of the discrete speech frame will cause spectral distortion during subsequent spectral analysis, affecting the accuracy of subsequent acoustic feature extraction, a Hamming window is used for windowing. Each value of the Hamming window function is multiplied with each sampling point of the corresponding speech frame. The length of the Hamming window function is consistent with the length of the speech frame. Through the multiplication operation, the sampling values ​​at both ends of the speech frame slowly decay to zero, while the middle part remains relatively stable, thereby reducing spectral leakage. After the three operations of pre-emphasis, framing, and windowing, the originally continuous raw speech data is converted into a continuous speech frame sequence.

[0017] In this embodiment, the process of acquiring the speech frame sequence involves collecting children's speech through a high-sensitivity microphone and performing preprocessing, effectively filtering out ambient noise in the children's room, compensating for the attenuation of high-frequency components of the speech, avoiding information gaps and spectral leakage in the speech frame, and improving the success rate of children's speech recognition.

[0018] In a preferred embodiment of the present invention, step 1, extracting acoustic features from the speech frame sequence to obtain voiceprint feature embeddings, and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence, may include: step 101, performing a short-time Fourier transform on each speech frame in the speech frame sequence to obtain the spectral energy distribution corresponding to the speech frame; filtering the spectral energy distribution to obtain the filtered output energy, and taking the natural logarithm of the filtered output energy to generate a logarithmic filter bank energy vector, specifically including: performing a short-time Fourier transform operation sequentially on each speech frame in the speech frame sequence. The purpose of the Short Time Fourier Transform (SFT) is to transform each speech frame from the time domain to the frequency domain, thereby obtaining the spectral energy distribution corresponding to each speech frame. When performing the SFT, each speech frame is first padded with zeros to 512 sampling points. The purpose of zero padding is to improve the resolution of the spectral analysis. The SFT is then performed on the zero-padded speech frame to obtain the complex spectrum of the speech frame. The real and imaginary parts of the complex spectrum are then squared respectively. The two squared results are added together to obtain the spectral energy value corresponding to each frequency point. The spectral energy values ​​of all frequency points are combined to form the spectral energy distribution corresponding to the speech frame.

[0019] After obtaining the spectral energy distribution of each speech frame, the spectral energy distribution is filtered using a Mel filter bank, which consists of 40 Mel filters. Each Mel filter corresponds to a specific frequency range, and the frequency range of the filters is divided according to the Mel scale, making it particularly suitable for capturing the frequency characteristics of children's speech. During the filtering process, the spectral energy distribution of each speech frame is input into the 40 Mel filters. Each Mel filter sums the spectral energy values ​​within its corresponding frequency range to obtain the output energy of the filter. The output energies of the 40 Mel filters are combined to form a filtered output energy vector. The natural logarithm of each element in the filtered output energy vector is taken to compress the dynamic range of the spectral energy, reduce the impact of changes in children's speech volume on feature extraction, and improve the discriminative power of the features. After taking the natural logarithm, a logarithmic filter bank energy vector is generated.

[0020] Step 102 involves performing a discrete cosine transform (DCT) on the logarithmic filter bank energy vector to obtain a transform domain coefficient vector. The transform domain coefficient vector is then weighted, and the mean and variance are normalized to obtain the voiceprint feature embedding. Specifically, this includes performing a DCT on the logarithmic filter bank energy vector. The purpose of the DCT is to reduce the dimensionality of the logarithmic filter bank energy vector, removing redundant information while retaining key acoustic features. During the DCT, the logarithmic filter bank energy vector is multiplied by the DCT matrix. The dimension of the DCT matrix is ​​the same as the length of the logarithmic filter bank energy vector, both being 40 dimensions. After multiplication, a 13-dimensional transform domain coefficient vector is obtained. This 13-dimensional setting retains sufficient acoustic features while effectively reducing the computational load of subsequent processing, adapting to the hardware computing capabilities of the toy robot.

[0021] After obtaining the transform domain coefficient vector, a weighted processing is performed on it. The purpose of this weighting is to highlight the feature components that are more important for speech recognition. The weighting coefficients are cepstral boosting coefficients, which gradually increase from the first component and then gradually decrease, forming a symmetrical boosting curve. Specifically, each component in the transform domain coefficient vector is multiplied by its corresponding cepstral boosting coefficient, thus enhancing the feature components in the middle and appropriately suppressing the feature components at the ends, thereby improving the feature discriminability. After the weighting process, the weighted transform domain coefficient vector is then... Mean normalization and variance normalization are performed. Mean normalization involves first calculating the average value of all elements in the weighted transform domain coefficient vector, and then subtracting this average value from each element in the vector to make the mean of the vector zero. Variance normalization involves calculating the variance of the mean-normalized vector, and then dividing each element in the vector by this variance to make the variance of the vector one. Through mean normalization and variance normalization, the influence of changes in children's speech volume and speech rate can be eliminated, ensuring the consistency of features extracted under different conditions. After the above processing, the voiceprint feature embedding corresponding to the speech frame is obtained.

[0022] Step 103: Arrange all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence. Specifically, this includes: extracting acoustic features for each speech frame in the speech frame sequence according to the above process to obtain the voiceprint feature embedding corresponding to each speech frame; arranging all voiceprint feature embeddings in chronological order according to the speech frames, i.e., the voiceprint feature embeddings corresponding to the speech frames extracted earlier are placed first, and the voiceprint feature embeddings corresponding to the speech frames extracted later are placed last, thus generating a voiceprint feature embedding sequence.

[0023] In this embodiment, the acoustic feature extraction and voiceprint feature embedding sequence generation process highlights the key features of children's speech through multi-step processing, eliminates the influence of changes in volume and speech rate, and the generated voiceprint feature embedding sequence can completely preserve the temporal and acoustic features of speech, improve the distinguishability of features, and further improve the accuracy of speech recognition.

[0024] In a preferred embodiment of the present invention, step 2, inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model for recognition to obtain recognized text; comparing the recognized text with a preset instruction set to obtain preliminary control instructions, may include: step 201, inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model, wherein the speech recognition model performs acoustic forward computation on the voiceprint feature embedding to obtain a posterior probability sequence, specifically including: inputting the generated voiceprint feature embedding sequence into a pre-trained speech recognition model, wherein the speech recognition model is an end-to-end speech recognition model built on the Transformer architecture, specifically trained and optimized for children's speech, and adapted to the usage scenario of children's rooms. The detailed construction, training, and implementation process is as follows: During model construction, a complete Transformer encoder and decoder structure is built. The encoder consists of six cascaded coding layers, each containing two core components: a multi-head attention mechanism and a feedforward neural network. The multi-head attention mechanism is set to eight heads, enabling simultaneous capture of the correlation between features at different positions within the speaker feature embedding sequence, thus improving the comprehensiveness of feature extraction. The feedforward neural network consists of two fully connected layers. The output dimension of the first fully connected layer is set to 2048, and the output dimension of the second fully connected layer is set to 512. The ReLU activation function is used to enhance the model's non-linear fitting ability and better capture the complex features of children's speech. The decoder structure corresponds to the encoder structure, also consisting of six decoding layers. Each decoding layer, in addition to containing the same multi-head attention mechanism and feedforward neural network as the encoder, adds an encoder-decoder attention mechanism to correlate the encoded features output by the encoder with the decoder's own decoded features, ensuring that the model can effectively combine speaker features and language features, thereby improving the accuracy of speech recognition.

[0025] After the model is built, the training phase begins. The entire training process revolves around children's speech characteristics and the use of a children's room to ensure model adaptability. The training dataset uses a dedicated dataset containing a large number of children's speech samples, covering speech from children of different ages, including different speaking speeds, tones, and pronunciation habits. Common background noise samples from a children's room are also included, such as toy collisions, book turning sounds, and slight environmental noise, to improve the model's anti-interference capabilities and adapt it to real-world usage scenarios. Before training begins, all speech samples in the training dataset undergo preprocessing and acoustic feature extraction operations identical to steps 001, 002, and 1, resulting in a voiceprint feature embedding sequence for each speech sample. Each speech sample is also labeled with corresponding text tags, which are related to commonly used robot control commands for children, including forward, backward, left turn, right turn, and stop, ensuring a high degree of match between the tags and robot control requirements. During training, the voiceprint feature embedding sequence of each speech sample is input into the model's encoder. The encoder performs layer-by-layer feature extraction and encoding on the voiceprint feature embedding sequence to obtain... The system first generates an encoded feature vector containing core speech features. This encoded feature vector is then input into a decoder, which, combined with a pre-defined language dictionary, gradually generates the corresponding text sequence. After generating the text sequence, the cross-entropy loss function is used to calculate the loss between the model-generated text sequence and the labeled text labels. The calculation involves multiplying the predicted probability of each text character generated by the model with the true probability corresponding to the labeled text label, then taking the negative value of all multiplication results and calculating the average of all negative results to obtain the loss value for that batch of samples. The model parameters are iteratively updated with a learning rate of 0.001, a batch size of 32, and 100 training rounds. Validation is performed every 10 training rounds, using children's speech samples completely independent of the training dataset. When the speech recognition accuracy on the validation set reaches 95% or higher, model training is stopped, resulting in a trained speech recognition model adapted to the children's room scenario. This model is then deployed to the controller of a toy robot to ensure that the model can respond to speech recognition needs in real time.

[0026] The model implementation process is essentially an acoustic forward computation process. The speaker feature embedding sequence is input into the deployed, trained speech recognition model. The model sequentially performs feature processing and probability generation through an encoder and a decoder. The encoder processes the speaker feature embedding sequence layer by layer. Each coding layer's multi-head attention mechanism first correlates the feature at each position in the speaker feature embedding sequence with features at all other positions, obtaining the attention weight corresponding to each position's feature. The attention weight represents the importance of features at different positions. Then, the attention weight is weighted and summed with the speaker feature embedding sequence to obtain the attention feature that fuses attention information. The attention feature is input into the feedforward neural network of that coding layer. After a non-linear transformation using the ReLU activation function, the output coding feature of that coding layer is obtained. After the six coding layers sequentially complete the above processing, the final coding feature vector is obtained. The final coding feature vector is input into the decoder. The decoder's multi-head attention mechanism further processes the coding feature vector, combining it with pre-defined language dictionary information to generate the predicted probability of each text character. The predicted probabilities of all text characters are arranged sequentially in chronological order, forming a posterior probability sequence.

[0027] After the model is built, it is trained. The training dataset contains a large number of children's speech samples, covering children of different ages and with different speaking speeds, tones, and pronunciation habits. Noise samples from children's rooms are also included to improve the model's anti-interference ability. During training, the speech samples in the training dataset are preprocessed and acoustic features are extracted to obtain the corresponding voiceprint feature embedding sequence. At the same time, each speech sample is labeled with a text label, which is related to robot control commands commonly used by children, such as forward, backward, left turn, right turn, stop, etc. During training, the voiceprint feature embedding sequence is input into the model's encoder. The encoder extracts and encodes the voiceprint feature embedding sequence to obtain the encoded feature vector. The encoded feature vector is input to the decoder, which, combined with a pre-defined language dictionary, generates the corresponding text sequence. The loss value between the model-generated text sequence and the labeled text label is calculated using the cross-entropy loss function. The cross-entropy loss function is calculated by multiplying the predicted probability of each text character generated by the model with the true probability corresponding to the labeled text label, taking the negative of all multiplication results, and then averaging them to obtain the loss value. The model parameters are then updated, with the learning rate set to 0.001, the batch size set to 32, and the training epochs set to 100. Validation is performed every 10 training epochs, using children's speech samples that are different from the training dataset. Training stops when the recognition accuracy on the validation set reaches 95% or higher, resulting in a trained speech recognition model.

[0028] Step 202 involves decoding and searching the posterior probability sequence within the speech recognition model, converting it into candidate paths for word sequences, and adjusting the probabilities of these candidate paths based on pre-stored linguistic constraints. The word sequence with the highest cumulative probability is then selected as the recognized text. Specifically, this includes decoding and searching the obtained posterior probability sequence within the speech recognition model using a beam search algorithm with a beam width of 5. The specific implementation process is as follows: The beam search algorithm starts from the first time step of the posterior probability sequence, sorts the predicted probabilities of all characters within that time step, selects the five characters with the highest predicted probabilities as initial candidate characters, and records the predicted probability corresponding to each initial candidate character. This probability is the initial cumulative probability corresponding to each initial candidate character. Upon entering the second time step, for each initial candidate character, the initial cumulative probability of that initial candidate character is compared with the predicted probability of each character within the second time step. The probabilities are multiplied one by one to obtain the joint probability of each character combined with the initial candidate character. Then, all joint probabilities corresponding to each initial candidate character are sorted, and the five characters with the highest joint probabilities are selected. These five characters are then combined with the corresponding initial candidate characters to form five new candidate paths. At the same time, the cumulative probability of each new candidate path is updated, i.e., the cumulative probability of each path is the value of the corresponding joint probability. The above calculation logic is applied iteratively step by step. For each candidate path formed in the previous time step, the joint probability of the path combined with each character in the current time step is calculated. The joint probability is calculated by multiplying the cumulative probability of the path in the previous time step with the predicted probability of the character in the current time step. Then, the five character combinations with the highest joint probabilities are selected for each path, and the path and its corresponding cumulative probability are updated. This process continues until all time steps of the posterior probability sequence have been traversed, ultimately forming multiple candidate paths containing different word sequences, each path corresponding to a unique cumulative probability.

[0029] The pre-stored language constraint relationships used in the decoding search process are as follows: The core of the language constraint relationships is based on the grammatical rules and semantic associations built on commonly used robot control commands for children. Specifically, forward cannot be directly followed by stop, left turn and right turn cannot appear simultaneously, stop commands cannot appear consecutively with other movement commands, movement commands must conform to children's simple language logic, and there should be no contradictory command combinations. At the same time, it covers the semantic associations of commonly used colloquial control commands for children and adapts to children's language expression habits. The construction process is as follows: First, we sort out commonly used robot control commands for children, clarify the reasonable associations and prohibited associations between various commands, and then count the frequency of command combinations when children use the robot. The adjacent characters corresponding to reasonably associated command combinations are set with a high probability, the adjacent characters corresponding to prohibited command combinations are set with a very low probability, and the adjacent characters corresponding to command combinations with no direct association are set with a medium probability. Based on this, a character adjacency probability matrix is ​​constructed. This probability matrix is ​​the specific carrier of the language constraint relationships and is stored inside the speech recognition model for subsequent probability correction of candidate paths.

[0030] During the probability correction process, each candidate path is processed one by one. The constraint probability in the probability matrix corresponding to the adjacent characters in the path is multiplied by the cumulative probability of the path to obtain the corrected cumulative probability. After all candidate paths are corrected, the word sequence with the highest corrected cumulative probability is selected as the recognized text for this speech recognition.

[0031] Step 203: Calculate the text similarity between the recognized text and each instruction in the preset instruction set. Select the instruction with the highest similarity to the recognized text from the preset instruction set to obtain the preliminary control instructions. Specifically, this includes: calculating the text similarity between the recognized text and the preset instruction set, which is stored in the robot's storage unit. The preset instruction set contains all control instructions supported by the robot. Each instruction corresponds to unique action type information, preset speed parameters, and preset target coordinates. Specific instructions include forward, backward, left turn, right turn, and stop. The forward instruction corresponds to the action type of moving forward, and the preset speed parameters and preset target coordinates are adapted to the children's room environment to avoid moving too fast or too far. The backward instruction corresponds to the action type of moving backward, and the parameter settings are symmetrical to the forward instruction. The left turn and right turn instructions correspond to the action types of turning, and the preset turning angle and speed are adapted to the narrow area of ​​the children's room. The stop instruction corresponds to the action type of stopping all movement, and the preset speed parameter is zero.

[0032] The process of building the preset instruction set first involves combining children's usage needs with the robot's hardware performance to determine the core action types that the robot can perform and formulate corresponding basic control instructions. Then, based on the characteristics of the children's room environment, reasonable preset speed parameters and preset target coordinates are set for each instruction to ensure safe execution of instructions and adaptability to the children's room scenario. All instructions and corresponding parameters are organized and archived to form a preset instruction set, which is stored in the robot's storage unit. At the same time, an instruction index is established to facilitate quick retrieval and similarity comparison later.

[0033] The text similarity calculation uses the cosine similarity method. The specific implementation process is as follows: The identified text and each instruction in the preset instruction set are converted into text vectors. During conversion, the frequency of each character in the identified text and each instruction is first counted. Then, the frequency values ​​of all characters are arranged one by one according to a preset character dictionary order, forming text vectors that correspond one-to-one with the identified text and each instruction. Next, the cosine similarity between the text vector corresponding to the identified text and the text vector corresponding to each instruction in the preset instruction set is calculated. This is done by multiplying the corresponding elements of the two text vectors one by one, and then summing all the multiplication results to obtain the dot product of the two vectors. Then, the magnitude of each text vector is calculated. To calculate the magnitude of each vector, each element in the vector is first squared, and then all the squared results are summed. The square root of the sum is taken to obtain the magnitude of the vector. Finally, the dot product of the two vectors is divided by the product of their magnitudes to obtain the cosine similarity value. The cosine similarity value ranges from 0 to 1; the closer the value is to 1, the higher the similarity between the two texts. After the similarity calculation of all instructions is completed, the instruction with the highest similarity to the recognized text is selected from the preset instruction set. This instruction is the initial control instruction.

[0034] In this embodiment, the initial control command generation process is based on a Transformer architecture speech recognition model specifically trained for children's speech. It combines beam search and decoding correction, and accurately matches preset commands through text similarity calculation. The generated initial control commands can accurately reflect the user's control intentions.

[0035] In a preferred embodiment of the present invention, step 3, collecting three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control command, may include: step 301, parsing the action type information contained in the preliminary control command, and determining the scanning range and sampling density of the environmental perception sensor based on the action type information. Specifically, this includes: parsing the preliminary control command, the parsing process being completed by the robot's controller. The controller extracts the action type information contained in the preliminary control command through command parsing logic. The action type information mainly includes forward, backward, left turn, right turn, stop, etc. Different action types correspond to different movement directions and movement ranges of the robot. Considering the usage scenario of a children's room, for example, when a child issues a forward command, the robot needs to move forward. At this time, it is necessary to focus on collecting environmental information in front to avoid colliding with toys or furniture scattered by the child; when issuing a left turn or right turn command, it is necessary to collect environmental information on the side to ensure that it does not collide with surrounding objects during the turning process. Based on the parsed action type information, the scanning range and sampling density of the environmental perception sensor are determined. The environmental perception sensor is a lidar sensor adapted to the environment of a children's room, installed on the top of the robot body, which can achieve 360-degree all-round scanning. Specifically, when the action type is forward or backward, the scanning range is set to an area 1 to 3 meters in front of or behind the robot, and the sampling density is set to 1,000 sampling points per square meter. This is because the robot travels a relatively long distance when moving forward or backward, requiring denser sampling to accurately capture environmental obstacles in front of or behind. When the action type is left or right turn, the scanning range is set to an area 0.5 to 2 meters to the side of the robot, and the sampling density is set to 800 sampling points per square meter. This is because the robot's range of motion is relatively small when turning, and appropriately reducing the sampling density can reduce the amount of computation and improve the acquisition efficiency. When the action type is stop, there is no need to activate the environmental perception sensor to scan; a stop command can be output directly.

[0036] Step 302: Generate sensor configuration parameters based on the scanning range and sampling density, and send the sensor configuration parameters to the environmental perception sensor; start the environmental perception sensor to scan the current environment according to the sensor configuration parameters to obtain the original three-dimensional space sampling point set; filter the original three-dimensional space sampling point set to obtain a three-dimensional space discrete sampling cluster, specifically including: the sensor configuration parameters include scanning angle, scanning distance, sampling frequency, and number of sampling points. The specific content and range of each parameter are as follows: the scanning angle is determined according to the action type and belongs to the angle parameter, with a range of 180 degrees. When moving forward, the scanning angle is 180 degrees forward, and when moving backward, the scanning angle is 180 degrees backward. The scanning angle is 180 degrees to the rear, 180 degrees to the left when turning left, and 180 degrees to the right when turning right, ensuring that the scanning range accurately covers the key areas of the robot's movement direction; the scanning distance is a distance parameter, corresponding to the defined scanning range, with a scanning distance range of 1 to 3 meters when moving forward and backward, and 0.5 to 2 meters when turning left and right, consistent with the scanning range, ensuring no omissions in the scanning coverage; the sampling frequency is a frequency parameter, fixed at 10 Hz. This parameter range is suitable for the real-time acquisition needs of the children's room environment, ensuring both the timeliness of environmental information acquisition and avoiding excessive computation caused by excessively high sampling frequencies; The number of sampling points is a quantitative parameter, calculated based on the sampling density and scanning range. The calculation method is to multiply the area of ​​the scanning range by the sampling density. For forward and backward movements, the sampling density is 1000 sampling points per square meter, corresponding to a scanning area of ​​3 to 9 square meters, with a sampling point count ranging from 3000 to 9000. For left and right turns, the sampling density is 800 sampling points per square meter, corresponding to a scanning area of ​​0.5 to 4 square meters, with a sampling point count ranging from 400 to 3200, ensuring that the sampling points completely cover the entire scanning range. After generating the sensor configuration parameters, the controller sends these parameters to the environment control via the communication interface. After receiving the configuration parameters, the environmental perception sensor starts scanning. It scans the current children's room environment according to the scanning angle, scanning distance, sampling frequency, and number of sampling points set in the configuration parameters. During the scanning process, the lidar sensor emits laser pulses. The emission frequency of the laser pulses is consistent with the sampling frequency, which is 10 Hz. When the laser pulses encounter obstacles in the environment, they are reflected back. The sensor receives the reflected laser pulses and calculates the distance and coordinates of the obstacles based on the emission and reception times of the laser pulses. Each laser pulse corresponds to a sampling point. All sampling points are combined to form the original three-dimensional spatial sampling point set.

[0037] Due to interference factors such as dust and light reflection in the children's room environment, the original 3D spatial sampling point set will contain some outliers. These outliers will affect the accuracy of subsequent spatial modeling. Therefore, it is necessary to filter the original 3D spatial sampling point set. The filtering process adopts a statistical filtering algorithm. The specific implementation process and parameters involved are as follows: calculate the average distance of other sampling points within a certain range around each sampling point. The range parameter of adjacent sampling points is set to 5, that is, the distance of each sampling point is only calculated to its 5 adjacent sampling points. The calculation process is to calculate the distance between each sampling point and its 5 adjacent sampling points, add these 5 distances together, and then divide by 5 to obtain the average distance of the sampling point. The mean and standard deviation of the average distances of all sampling points are calculated. The mean is the arithmetic mean of the average distances of all sampling points, with no fixed range, and changes dynamically with the distribution of sampling points. The standard deviation is a parameter that measures the dispersion of the average distances of all sampling points, with no fixed range, and changes dynamically with the distribution of sampling points. Its calculation process is to subtract the mean of the average distances of all sampling points from the average distance of each sampling point, square the difference, add all the squared results, divide by the total number of sampling points, and then take the square root of the result to obtain the standard deviation. The average distance of each sampling point is compared with the mean and standard deviation. The threshold parameter for identifying outliers is twice the standard deviation. That is, if the average distance of a sampling point is greater than the mean plus twice the standard deviation, or less than the mean minus twice the standard deviation, then the sampling point is identified as an outlier and deleted from the original three-dimensional space sampling point set. After deleting all outliers, the remaining sampling points are combined together to form a three-dimensional space discrete sampling cluster.

[0038] In this embodiment, the three-dimensional spatial discrete sampling cluster acquisition process dynamically adjusts the sensor scanning range and sampling density according to the action type of the initial control command, which meets the needs of the dynamic environment of the children's room and avoids robot collision problems caused by inaccurate environmental information acquisition.

[0039] In a preferred embodiment of the present invention, step 4 involves projecting a three-dimensional discrete sampling cluster onto a two-dimensional plane to obtain a two-dimensional projection point set, and defining a multi-segment constraint boundary on the two-dimensional projection point set; segmenting the multi-segment constraint boundary according to the spatial distribution characteristics of each sampling point in the three-dimensional discrete sampling cluster to generate multiple local feature response domains; mapping each sampling point in the three-dimensional discrete sampling cluster to the corresponding local feature response domain, and generating an environmental dynamic adaptation parameter based on the distribution statistics of the sampling points in each local feature response domain. This may include: step 401, performing a projection transformation on the sampling points in the three-dimensional discrete sampling cluster to obtain a two-dimensional projection point set; dividing the two-dimensional projection point set to obtain multiple grid cells, each grid cell being enclosed by grid nodes; marking the grid nodes as occupied or idle nodes based on whether there are projection points within a preset neighborhood of the grid nodes, specifically including: projecting the three-dimensional discrete sampling cluster onto a two-dimensional plane to obtain a two-dimensional projection point set; dividing the two-dimensional projection point set to obtain multiple grid cells, each grid cell being enclosed by grid nodes; and marking the grid nodes as occupied or idle nodes based on whether there are projection points within a preset neighborhood of the grid nodes. In the spatial discrete sampling cluster, the spatial coordinates (X-axis, Y-axis, Z-axis) of each sampling point are vertically projected onto the XY plane containing the horizontal ground. The X-axis and Y-axis coordinate information of each sampling point are retained, while the height information in the Z-axis direction is discarded. This transforms all sampling points from three-dimensional space (X,Y,Z) to two-dimensional plane (X,Y), forming a complete two-dimensional projection point set. The two-dimensional projection point set is then divided into regular grids to obtain multiple grid units. Specifically, the X-axis distribution range of the two-dimensional projection point set is determined to be 0 meters to 5 meters, and the Y-axis distribution range is also determined to be 0 meters to 5 meters. Based on this, the grid division start point is set to (0,0) and the end point is set to (5,5). Then, a regular rectangular grid is divided sequentially according to a uniform grid step size of 0.05 meters. Each grid unit is enclosed by four grid nodes: (x,y), (x+0.05,y), (x,y+0.05), and (x+0.05,y+0.05). Based on whether there is a projection point within the preset neighborhood of each grid node, the grid node is marked as an occupied node or an idle node. Specifically, this involves setting a circular neighborhood with a radius of 0.03 meters for each grid node, traversing each grid node, and checking whether there is a two-dimensional projection point within the neighborhood of the node. If there is at least one projection point within the neighborhood, the grid node is marked as an occupied node; if there are no projection points within the neighborhood, the grid node is marked as an idle node.

[0040] Step 402: Boundary grid cells are selected from the grid cells. Boundary segments with the same orientation are extracted from the boundary grid cells and merged with adjacent boundary grid cells to generate multiple constraint boundaries. Specifically, this includes: traversing all grid cells with completed node markings, judging the marking status of the four grid nodes within each grid cell one by one; when both occupied and idle nodes are marked within a grid cell, the grid cell is determined to be at the edge of the environment space and identified as a boundary grid cell; extracting boundary segments from the boundary grid cells, and merging adjacent boundary grid cells... Boundary segments with the same orientation in the grid cells are connected and merged to generate multiple constraint boundaries. Specifically, within each boundary grid cell, the intersection of occupied and vacant nodes is located first. Boundary segments with a length consistent with the grid step size are extracted along the intersection line. Then, all boundary grid cells are traversed in the order of grid arrangement from left to right and from top to bottom. Boundary segments with adjacent spatial positions and an angle of less than 5 degrees are connected end to end in sequence, so that the scattered boundary segments are pieced together into a continuous closed curve, and finally, multiple constraint boundaries that can completely define the spatial range of the children's room environment are generated.

[0041] Step 403: Extract sampling points inside multiple constraint boundaries to form a local sampling point set; calculate the centroid coordinates, second-order central moments, and third-order central moments of the local sampling point set; normalize the second-order and third-order central moments to obtain normalized central moments; calculate seven shape moment parameters based on the normalized central moments and combine them to obtain spatial morphological feature encoding, specifically including: determining whether the two-dimensional projection coordinates of each sampling point in the three-dimensional spatial discrete sampling cluster fall within the closed area enclosed by multiple constraint boundaries, extracting all sampling points whose projection positions are inside the boundaries, as these sampling points can centrally reflect the core spatial structure of the current children's room environment, and combining all extracted sampling points to form a local sampling point set; calculate the centroid coordinates of the local sampling point set, specifically including: summing the X-axis coordinates of all sampling points in the local sampling point set sequentially, and then dividing the sum by the total number of sampling points in the local sampling point set to obtain the X-axis coordinates of the centroid. The Y-axis coordinates of all sampling points in the local sampling point set are summed sequentially, and then the sum is divided by the total number of sampling points in the local sampling point set to obtain the Y-axis coordinate of the centroid. ; and The centroid coordinates that together constitute the local sampling point set Based on the position of each sampling point in the local sampling point set relative to the centroid coordinates, the second and third central moments of the local sampling point set are calculated. Specifically, for each sampling point in the local sampling point set, the X-axis offset is obtained by subtracting the centroid's X-axis coordinate from the X-axis coordinate of that sampling point. The Y-axis offset Δ is obtained by subtracting the Y-axis coordinate of the centroid from the Y-axis coordinate of the sampling point. The second-order correlation value of the X-axis is obtained by squaring the X-axis offset. The second-order correlation value of the Y-axis is obtained by squaring the Y-axis offset. The third-order correlation value of the X-axis is obtained by cubic calculation of the X-axis offset. The third-order correlation value of the Y-axis is obtained by cubic calculation of the Y-axis offset. The second-order central moments of the X-axis are obtained by summing the second-order correlation values ​​of all sampling points and dividing by the total number of sampling points N. Similarly, the second central moment of the Y-axis can be calculated. Third central moment of the X-axis Y-axis third central moment The second and third central moments are normalized to obtain normalized central moments. Specifically, the normalization coefficient of the second central moment is set to the square root of the area S covered by the local sampling point set. The standardized coefficient of the third central moment is the cube root of the area S covered by the local sampling point set. Divide the second central moment of the X-axis by the second-order normalization coefficient to obtain the normalized second central moment of the X-axis. Dividing the second-order central moment of the Y-axis by the second-order normalization coefficient yields the normalized second-order central moment of the Y-axis. Divide the third central moment of the X-axis by the third-order normalization coefficient to obtain the normalized third central moment of the X-axis. Dividing the third central moment of the Y-axis by the third-order normalization coefficient yields the normalized third central moment of the Y-axis. This process unifies the numerical range of all central moments to between 0 and 1, eliminating the influence of the number and distribution range of sampling points. Seven shape moment parameters are calculated based on the normalized central moments, and these seven shape moment parameters are combined to obtain the spatial morphological feature code. Specifically, the seven shape moment parameters are calculated using the normalized central moments according to the following fixed combination relationship: the first parameter is the sum of the normalized second-order central moments of the X-axis and the Y-axis. The first parameter reflects the extent of the environmental space; the second parameter is the difference between the normalized second central moment on the X-axis and the normalized second central moment on the Y-axis. The first parameter reflects the degree of symmetry of the environmental space; the second parameter is the absolute value of the normalized third-order central moment along the X-axis. The first parameter reflects the degree of concentration in the environmental space; the second parameter is the absolute value of the normalized third-order central moment along the Y-axis. The first parameter reflects the degree of spatial offset; the second parameter is the product of the normalized third central moment along the X-axis and the normalized third central moment along the Y-axis. The sixth parameter is the quotient of the normalized second central moment along the X-axis divided by the normalized second central moment along the Y-axis. The first parameter reflects the direction of spatial extension; the seventh parameter is the average of all normalized central moments. This reflects the uniformity of the distribution in the environmental space; the seven shape moment parameters are arranged in the above order to form a one-dimensional sequence of length 7. This sequence is a spatial morphological feature code that can uniquely represent the current spatial morphology of the environment.

[0042] Step 404: Identify the spatial structure type of the current environment based on the spatial morphological feature encoding, determine the number of segments and the segmentation density distribution function of the local feature response domain, and perform segmentation to generate the local feature response domain; allocate the sampling points in the three-dimensional spatial discrete sampling cluster to the corresponding local feature response domain, specifically including: pre-establishing a spatial structure type library, which contains four types: open space, narrow passage, corner space, and irregular obstacle region. Each type corresponds to a set of shape moment parameter feature intervals: the extension degree parameter of open space. >0.8, symmetry parameter <0.1; the extension parameter of the narrow channel is 0.4< <0.8, extension direction parameter >2 or <0.5; Concentration parameter of corner space >0.7, offset degree parameter >0.6; Sharpness parameter of irregular obstacle areas >0.5, Distribution Uniformity Parameter <0.3; Compare the seven parameters in the currently calculated spatial morphological feature encoding with the feature intervals in the type library one by one. The type corresponding to the interval with the highest matching degree is the spatial structure type of the current environment; Determine the number of segments and the segmentation density distribution function of the local feature response domain according to the spatial structure type. Specifically, the number of segments is set according to the spatial structure type: 8 segments for open space, 16 segments for narrow passage, 20 segments for corner space, and 24 segments for irregular obstacle area; The segmentation density distribution function is set as f(L)=k×L+b, where L is the arc length from a point on the multi-segment constraint boundary to the starting point, and k is the density. The coefficients and b are the basic density values, and the coefficients take different values ​​for different spatial structure types: open space k=0.02, b=0.1; narrow passage k=0.05, b=0.2; corner space k=0.08, b=0.25; irregular obstacle area k=0.1, b=0.3. This function is used to describe the segmentation density at each location on the constraint boundary. The larger the function value, the smaller the segmentation interval and the higher the density at that location. According to the number of segments and the segmentation density distribution function, multiple constraint boundaries are segmented to generate multiple local feature response domains. Specifically, starting from the starting point of the multiple constraint boundaries, the segmentation interval of each segment is calculated according to the segmentation density distribution function. According to the interval, segmentation nodes are sequentially marked on the constraint boundary, dividing multiple constraint boundaries into continuous segments with the same number of segments. Each segment extends 1 meter into the boundary to form a closed region, which is a local feature response domain used to describe the spatial features of a local area in the environment. Each sampling point in the three-dimensional spatial discrete sampling cluster is assigned to the corresponding local feature response domain according to its spatial location. Specifically, this includes calculating the two-dimensional projection coordinates of each sampling point one by one. ) to the center of each local feature response domain ( Euclidean distance The sampling point is assigned to the nearest local feature response domain, so that each local feature response domain contains a set of sampling points belonging to its own region.

[0043] Step 405: Count the number of sampling points, average distance, and principal direction angle within each local feature response domain, and fuse these three factors to generate a local response feature vector. Specifically, this includes: counting each sampling point within each local feature response domain to obtain the number of sampling points Nj in that region; traversing all sampling points within that region and calculating the Euclidean distance between each pair of sampling points. (Where p and q are different sampling points within the region), sum all the calculated distance values, and then divide the sum by the total number of distances. The average distance between sampling points is obtained. The distribution directions of all sampling points within the region are statistically analyzed, and the covariance matrix formed by the coordinates of all sampling points is calculated. (in , (Given the mean coordinates of the sampling points in this area), find the direction of the eigenvector corresponding to the largest eigenvalue of the covariance matrix. This direction is the direction in which the extension trend of the sampling points is most obvious. The angle between this direction and the horizontal rightward reference direction is determined as the main direction angle. The angle range is from 0 degrees to 360 degrees. The number of sampling points, average distance, and principal direction angle are weighted and fused to generate a local response feature vector. Specifically, a weighting coefficient of 0.2 is set for the number of sampling points, a weighting coefficient of 0.5 is set for the average distance, and a weighting coefficient of 0.3 is set for the principal direction angle. The number of sampling points is multiplied by 0.2 to obtain the weighted value. Multiply the average distance by 0.5 to obtain the distance-weighted value. Multiply the principal direction angle by 0.3 to obtain the angle weighted value. The quantity weighted value, distance weighted value, and angle weighted value are combined in sequence to form a local response feature vector of length 3. .

[0044] Step 406 involves sequentially concatenating all local response feature vectors and fusing them with spatial morphological feature encoding to obtain environmental dynamic adaptation parameters. Specifically, this includes: according to the spatial arrangement of the local feature response domains from left to right and from top to bottom, sequentially concatenating all local response feature vectors end to end to form an overall feature sequence of length 3 × the number of segments. (n is the number of segments); Encode the overall feature sequence with spatial morphological features of length 7. The segments are concatenated to form a one-dimensional vector with a length of 3 × number of segments + 7. This vector is the environmental dynamic adaptation parameter.

[0045] This embodiment, through projection transformation with a defined parameter range, regular grid division, boundary extraction, shape moment calculation, region segmentation based on specific functions, and feature fusion, can finely extract the global and local spatial features of the dynamic environment of a children's room. The generated environmental dynamic adaptation parameters can accurately quantify and reflect the spatial structure, obstacle distribution, spatial width, and passage difficulty.

[0046] In a preferred embodiment of the present invention, step 5, modifying the preliminary control command using environmental dynamic adaptation parameters to obtain the final control command, and sending the final control command to the toy robot to enable the toy robot to perform the corresponding action, may include: step 501, determining the spatial complexity level of the current environment according to the spatial structure type, and retrieving the corresponding basic correction coefficient from the pre-stored correction coefficient mapping relationship according to the spatial complexity level, specifically including: pre-dividing the spatial structure type into three levels: low complexity, medium complexity, and high complexity: open space corresponds to the low complexity level, and conventional areas correspond to the medium complexity level. The system identifies spatial complexity levels, with narrow passages, corner spaces, and irregular obstacle areas corresponding to high complexity levels. It then iterates through the spatial structure type identification results and matches the corresponding complexity level. Based on the spatial complexity level, it retrieves the corresponding basic correction coefficient from a pre-stored correction coefficient mapping relationship. Specifically, the pre-stored correction coefficient mapping relationship is as follows: low complexity level corresponds to a basic correction coefficient of 0.9 to 1.0, medium complexity level corresponds to a basic correction coefficient of 0.6 to 0.8, and high complexity level corresponds to a basic correction coefficient of 0.3 to 0.5. Finally, based on the spatial complexity level of the current environment, it retrieves the specific basic correction coefficient value from the corresponding range.

[0047] Step 502 involves analyzing the local response feature vectors and spatial morphological feature codes, calculating their feature matching degree, and filtering out local response feature vectors with a feature matching degree higher than a preset threshold as valid local feature vectors. Specifically, this includes: calculating the difference between the quantity weighting value, distance weighting value, and angle weighting value in each local response feature vector and the corresponding extension degree, distribution uniformity, and extension direction parameters in the spatial morphological feature code, respectively; adding the absolute values ​​of the three differences and dividing by 3; and then subtracting the result from 1 to obtain the feature matching degree between the two. The matching degree ranges from 0 to 1, with higher values ​​indicating a higher degree of fit. The preset feature matching degree threshold ranges from 0.7 to 0.8. The feature matching degree of each local response feature vector is compared with this threshold, and local response feature vectors with a feature matching degree greater than the threshold are retained, while invalid features with low matching degrees are removed.

[0048] Step 503: Statistically average the number of sampling points, average distance, and principal direction angle in the effective local feature vectors to obtain the average number, average distance, and average angle. Then, weight the average number, average distance, and average angle with the base correction coefficient to generate the motion amplitude correction, motion direction correction, and obstacle avoidance distance correction. Specifically, this includes: sequentially summing the number of sampling points in all effective local feature vectors and dividing the sum by the total number of effective vectors to obtain the average number; sequentially summing the average distance in all effective local feature vectors and dividing the sum by the total number of effective vectors to obtain the average distance; sequentially summing the principal direction angle in all effective local feature vectors and dividing the sum by the total number of effective vectors to obtain the average angle; and then weighting the average number, average distance, and average angle... The mean value and the base correction coefficient are weighted to generate the motion amplitude correction, motion direction correction, and obstacle avoidance distance correction. Specifically, the weights of the mean value, distance, and angle are set to 0.4 and 0.2 respectively. The mean value is multiplied by 0.4 and then by the base correction coefficient to obtain the motion amplitude correction, which is used to adjust the robot's movement speed and has a value range of 0.1 to 1.0. The mean angle is multiplied by 0.2 and then by the base correction coefficient to obtain the motion direction correction, which is used to adjust the robot's movement angle and has a value range of -30 degrees to 30 degrees. The mean distance is multiplied by 0.4 and then by the base correction coefficient to obtain the obstacle avoidance distance correction, which is used to set the safe distance between the robot and obstacles and has a value range of 0.1 meters to 0.5 meters.

[0049] Step 504: Calculate the motion amplitude correction amount with the preset speed parameters in the initial control command to obtain the corrected motion speed parameters; calculate the motion direction correction amount with the preset target coordinates in the initial control command to obtain the corrected target position coordinates. Specifically, the preset speed parameters in the initial control command range from 0.1 m / s to 0.5 m / s; multiply the preset speed parameters by the motion amplitude correction amount to obtain the corrected motion speed parameters, so that the speed parameters are adaptively adjusted according to the complexity of the environment: the more complex the environment, the smaller the motion amplitude correction amount, and the lower the corrected speed; the more open the environment, the larger the motion amplitude correction amount, and the higher the corrected speed, ultimately obtaining the corrected motion speed parameters suitable for the current environment. The motion direction correction is vector-superimposed with the preset target coordinates in the initial control command to obtain the corrected target position coordinates. Specifically, the preset target coordinates in the initial control command are in the range of 0 meters to 5 meters on the X-axis and 0 meters to 5 meters on the Y-axis. The horizontal component of the motion direction correction is added to the X-axis component of the preset target coordinates, and the vertical component of the motion direction correction is added to the Y-axis component of the preset target coordinates. This vector superposition operation shifts the target position in a safe and unobstructed direction, preventing the robot from moving toward obstacles or narrow areas, and finally obtaining the corrected target position coordinates.

[0050] Step 505: Using the obstacle avoidance distance correction amount as the safety distance threshold, perform path planning verification on the corrected motion speed parameters and the corrected target position coordinates to generate the final motion path. Specifically, this includes: setting the safety distance threshold as the obstacle avoidance distance correction amount, with a value ranging from 0.1 meters to 0.5 meters; generating an initial straight path starting from the robot's current position and ending at the corrected target position coordinates; discretizing this straight path at 0.05-meter intervals to obtain a series of discrete points, with adjacent points spaced 0.05 meters apart, the starting and ending points being the robot's current position and the corrected target position, respectively; calculating the shortest straight-line distance between each discrete point and surrounding obstacles, and determining whether this distance is greater than the set safety distance threshold; if all discrete points are greater than the safety distance threshold, the path is deemed to have no collision risk, and this straight-line path is directly used as the final motion path; if at least one point is less than the safety distance threshold, the path is considered to have no collision risk. To meet safety distance requirements, if a collision risk is identified on the path, a path detour algorithm is used to adjust the path. Specifically, an environmental grid map is constructed in 0.05-meter units. The presence of obstacles within each grid is marked, and the distance between the grid center point and an obstacle is checked against the safety distance threshold. Grids meeting these conditions are marked as passable, while those not are marked as impassable. Using the robot's current grid as the starting grid and the corrected target grid as the target grid, the robot traverses all passable grids surrounding the starting grid, calculating the straight-line distance from each passable grid to the target grid. The nearest passable grid to the target grid is selected as the next path node. This process is repeated until the target grid is reached, forming a continuous sequence of path nodes. All path nodes are located in passable grids, and the distance between each node and an obstacle is greater than the safety distance threshold. The path formed by connecting these path node sequences is the final motion path.

[0051] Step 506: Combine the corrected motion speed parameters, corrected target position coordinates, and final motion path to generate the final control command; send the final control command to the toy robot, and drive the robot's actuator to complete the corresponding action according to the final control command. Specifically, this includes: organizing the corrected motion speed parameters in a fixed command format of speed parameters, target coordinates, and path point sequence, with the parameter value ranging from 0.1 m / s to 0.5 m / s; corrected target position coordinates, with coordinate values ​​ranging from 0 m to 5 m in the horizontal direction and 0 m to 5 m in the vertical direction; and a discrete point sequence of the final motion path, with adjacent points spaced 0.05 m apart, forming the final control command containing complete motion information; and sending the final control command to the actuator's control unit via the robot's built-in serial communication link, with the communication baud rate set to 1152. The communication process maintains a stable transmission rate of 00bps to ensure the integrity and losslessness of command data. After receiving the command, the control unit parses the structured information in the command, extracting the corrected motion speed parameters, corrected target position coordinates, and the point sequence of the final motion path one by one, clarifying the type of action, motion speed, target orientation, and specific trajectory that the robot needs to perform. Based on the parsed motion speed parameters, the target speed of the drive motor is calculated. The specific calculation process is as follows: the fixed diameter of the robot wheel is pre-calibrated, with a value range of 0.06 meters to 0.1 meters, and 0.08 meters is actually selected. The wheel circumference is calculated by multiplying the wheel diameter by pi (3.1416), and then the corrected motion speed parameters are divided by the wheel circumference to obtain the number of revolutions per second of the wheel. This value is multiplied by 60 seconds to convert it into speed per minute, which is the target speed of the motor.

[0052] The motor steering control logic is planned according to the point sequence of the final motion path. The directional offset angle from the current point to the next point is calculated sequentially, with the offset angle ranging from -30 degrees to 30 degrees. Based on the offset angle, a motor rotation angle control signal is generated. The steering action is achieved by adjusting the speed difference between the left and right drive motors. For example, when the robot needs to shift to the right from the current point to the next point, the control unit will reduce the speed of the right motor and maintain the speed of the left motor, so that the robot can smoothly turn along the preset path. After the drive motors start, the control unit obtains the robot's position information in real time through the encoder at the bottom of the robot body and collects attitude information through the built-in gyroscope. The preset position deviation range is 0 to 0.02 meters, and the preset angle deviation range is 0 to 1 degree. The real-time collected position information is compared with the point sequence of the final motion path to calculate the position deviation. At the same time, the real-time attitude information is compared with the preset motion attitude to calculate the angle deviation. If the position deviation exceeds 0.02 meters or the angle deviation exceeds 1 degree, the control unit immediately generates a fine-tuning signal to adjust the motor speed or angle to correct the deviation and ensure that the robot always moves along the final motion path.

[0053] Throughout the entire action execution process, the control unit continuously compares the robot's current position with the distance to surrounding obstacles and the safe distance threshold. The safe distance threshold is determined by the obstacle avoidance distance correction amount, which ranges from 0.1 meters to 0.5 meters. The corresponding safe distance threshold range is also from 0.1 meters to 0.5 meters. If the detected distance is less than the safe distance threshold, the motor speed is immediately reduced or the steering adjustment is initiated to avoid collisions between the robot and toys, furniture, and other items in the children's room, until the robot smoothly reaches the corrected target position coordinates and completes the control command issued by the user.

[0054] This embodiment, by clearly defining the specific ranges of various parameters, preset values, and thresholds, makes the generation and execution of final control commands more standardized and operable, ensuring the accuracy of robot action execution; combined with the dynamic environment characteristics of children's rooms, by accurately calculating motor speed, correcting deviations in real time, and monitoring safe distances, the robot can smoothly adapt to different spatial scenarios, effectively avoid collision risks, and improve the reliability and safety of action execution.

[0055] As shown in Figure 2, embodiments of the present invention also provide a real-time voice data control system for an interactive toy robot, comprising: an extraction module for extracting acoustic features from a speech frame sequence to obtain voiceprint feature embeddings, and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence; a training module for inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model for recognition to obtain recognized text; comparing the recognized text with a preset instruction set to obtain preliminary control instructions; a processing module for collecting three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control instructions; and a mapping module for projecting the three-dimensional spatial discrete sampling clusters onto a map. A two-dimensional plane is used to obtain a two-dimensional projection point set, and a multi-segment constraint boundary is defined on the two-dimensional projection point set. The multi-segment constraint boundary is segmented according to the spatial distribution characteristics of each sampling point in the three-dimensional discrete sampling cluster, generating multiple local feature response domains. Each sampling point in the three-dimensional discrete sampling cluster is mapped to the corresponding local feature response domain, and an environmental dynamic adaptation parameter is generated according to the distribution statistics of the sampling points in each local feature response domain. An execution module is used to modify the preliminary control command using the environmental dynamic adaptation parameter to obtain the final control command. The final control command is sent to the toy robot so that the toy robot can perform the corresponding action.

[0056] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0057] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0058] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0059] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for real-time voice data control of an interactive toy robot, characterized in that, The method includes: Step 1, extracting acoustic features from the speech frame sequence to obtain voiceprint feature embeddings, and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence; Step 2, inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model for recognition to obtain recognized text; comparing the recognized text with a preset instruction set to obtain preliminary control instructions; Step 3, collecting three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control instructions; Step 4, projecting the three-dimensional spatial discrete sampling clusters onto a two-dimensional plane to obtain a two-dimensional projection point set, and defining a multi-segment constraint boundary on the two-dimensional projection point set; segmenting the multi-segment constraint boundary according to the spatial distribution characteristics of each sampling point in the three-dimensional spatial discrete sampling cluster to generate multiple local feature response domains; mapping each sampling point in the three-dimensional spatial discrete sampling cluster to the corresponding local feature response domain, and generating an environmental dynamic adaptation parameter according to the distribution statistics of the sampling points in each local feature response domain; Step 5, using the environmental dynamic adaptation parameter to correct the preliminary control instructions to obtain final control instructions; sending the final control instructions to the toy robot to make the toy robot perform corresponding actions.

2. The real-time voice data control method for an interactive toy robot according to claim 1, characterized in that, The process of acquiring the speech frame sequence is as follows: real-time acquisition of the user's speech signal and conversion into digitized raw speech data; preprocessing of the raw speech data, including pre-emphasis, framing, and windowing, to obtain a continuous speech frame sequence.

3. The real-time voice data control method for an interactive toy robot according to claim 2, characterized in that, Acoustic feature extraction is performed on the speech frame sequence to obtain voiceprint feature embeddings. All voiceprint feature embeddings are arranged in chronological order to generate a voiceprint feature embedding sequence. This includes: performing a short-time Fourier transform on each speech frame in the speech frame sequence to obtain the spectral energy distribution corresponding to the speech frame; filtering the spectral energy distribution to obtain the filtered output energy, and taking the natural logarithm of the filtered output energy to generate a logarithmic filter bank energy vector; performing a discrete cosine transform on the logarithmic filter bank energy vector to obtain a transform domain coefficient vector; weighting the transform domain coefficient vector, and then normalizing the mean and variance of the weighted transform domain coefficient vector to obtain the voiceprint feature embeddings; and arranging all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence.

4. The real-time voice data control method for an interactive toy robot according to claim 3, characterized in that, The voiceprint feature embedding sequence is input into a pre-trained speech recognition model for recognition, and the recognized text is obtained. The process involves comparing the recognized text with a preset instruction set to obtain preliminary control instructions. This includes: inputting the voiceprint feature embedding sequence into a pre-trained speech recognition model; the speech recognition model performing acoustic forward computation on the voiceprint feature embedding to obtain a posterior probability sequence; decoding and searching the posterior probability sequence within the speech recognition model to convert it into candidate paths for word sequences; adjusting the probability of the candidate paths based on pre-stored language constraints; selecting the word sequence with the highest cumulative probability from the candidate paths as the recognized text; and calculating the text similarity between the recognized text and each instruction in the preset instruction set, selecting the instruction with the highest similarity to the recognized text from the preset instruction set to obtain the preliminary control instructions.

5. The real-time voice data control method for an interactive toy robot according to claim 4, characterized in that, Based on the initial control command, the system collects discrete sampling clusters of the current environment in three dimensions, including: parsing the action type information contained in the initial control command; determining the scanning range and sampling density of the environmental perception sensor based on the action type information; generating sensor configuration parameters based on the scanning range and sampling density, and sending the sensor configuration parameters to the environmental perception sensor; starting the environmental perception sensor to scan the current environment according to the sensor configuration parameters to obtain the original three-dimensional spatial sampling point set; and filtering the original three-dimensional spatial sampling point set to obtain discrete sampling clusters in three dimensions.

6. The real-time voice data control method for an interactive toy robot according to claim 5, characterized in that, Step 4 includes: performing a projection transformation on the sampling points in the three-dimensional spatial discrete sampling cluster to obtain a two-dimensional projection point set; dividing the two-dimensional projection point set to obtain multiple grid cells, each grid cell being enclosed by grid nodes; marking grid nodes as occupied or idle nodes based on whether projection points exist within their preset neighborhood; selecting boundary grid cells from the grid cells; merging boundary segments with the same orientation between the boundary grid cells and adjacent boundary grid cells to generate multiple constrained boundaries; extracting sampling points within the multiple constrained boundaries to form a local sampling point set; calculating the centroid coordinates of the local sampling point set and the second and third central moments of the local sampling point set; and calculating the second central moment... The moments and third-order central moments are normalized to obtain normalized central moments. Seven shape moment parameters are calculated based on the normalized central moments and combined to obtain spatial morphological feature codes. The spatial structure type of the current environment is identified based on the spatial morphological feature codes, and the number of segments and segmentation density distribution functions of the local feature response domains are determined. Segmentation is performed to generate local feature response domains. Sampling points in the three-dimensional spatial discrete sampling clusters are assigned to the corresponding local feature response domains. The number of sampling points, average distance, and principal direction angle in each local feature response domain are counted and fused to generate local response feature vectors. All local response feature vectors are concatenated in sequence and fused with the spatial morphological feature codes to obtain environmental dynamic adaptation parameters.

7. The real-time voice data control method for an interactive toy robot according to claim 6, characterized in that, Step 5 includes: determining the spatial complexity level of the current environment based on the spatial structure type; retrieving the corresponding basic correction coefficient from the pre-stored correction coefficient mapping relationship based on the spatial complexity level; analyzing the local response feature vector and the spatial morphology feature code, calculating the feature matching degree between the two, and selecting local response feature vectors with a feature matching degree higher than a preset threshold as effective local feature vectors; statistically averaging the number of sampling points, average distance, and main direction angle in the effective local feature vectors to obtain the average number, average distance, and average angle; and weighting the average number, average distance, and average angle with the basic correction coefficient to generate the motion amplitude correction amount and motion direction correction amount. The system calculates the corrected motion speed parameters by combining the motion amplitude correction with the preset speed parameters in the initial control command. It also calculates the corrected target position coordinates by combining the motion direction correction with the preset target coordinates in the initial control command. Using the obstacle avoidance distance correction as a safety distance threshold, the system performs path planning verification on the corrected motion speed parameters and the corrected target position coordinates to generate the final motion path. Finally, it combines the corrected motion speed parameters, the corrected target position coordinates, and the final motion path to generate the final control command. The final control command is then sent to the toy robot, which drives the robot's actuators to complete the corresponding actions based on the final control command.

8. A real-time voice data control system for an interactive toy robot, the system implementing the method as described in any one of claims 1 to 7, characterized in that, include: The extraction module is used to extract acoustic features from the speech frame sequence, obtain voiceprint feature embeddings, and arrange all voiceprint feature embeddings in chronological order to generate a voiceprint feature embedding sequence. The training module is used to input the embedded sequence of voiceprint features into a pre-trained speech recognition model for recognition, thereby obtaining the recognized text; the recognized text is compared with a preset instruction set to obtain preliminary control instructions; the processing module is used to collect three-dimensional spatial discrete sampling clusters of the current environment according to the preliminary control instructions; the mapping module is used to project the three-dimensional spatial discrete sampling clusters onto a two-dimensional plane to obtain a two-dimensional projection point set, and delineate a multi-segment constraint boundary on the two-dimensional projection point set; the multi-segment constraint boundary is segmented according to the spatial distribution characteristics of each sampling point in the three-dimensional spatial discrete sampling cluster, generating multiple local feature response domains; Each sampling point in the discrete sampling cluster in three-dimensional space is mapped to the corresponding local feature response domain, and an environmental dynamic adaptation parameter is generated based on the distribution statistics of the sampling points in each local feature response domain. The execution module is used to modify the preliminary control command using the environmental dynamic adaptation parameter to obtain the final control command. The final control command is sent to the toy robot so that the toy robot can perform the corresponding action.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images