Voice data real-time control method and system for AI toy accompanying robot

Abstract

The invention provides a voice data real-time control method and system for an AI toy accompanying robot, and relates to the technical field of automatic voice recognition, and the method comprises the steps: 1, collecting a user voice instruction, and obtaining a voice feature sequence; step 2, inputting the speech feature sequence into an end-to-end speech recognition model of AI to perform streaming preliminary decoding, generating a candidate text sequence, and performing semantic adjustment on the candidate text sequence to obtain an initial recognition text; and step 3, performing multi-dimensional stability evaluation on the initial recognition text, establishing a credibility screening mechanism based on an evaluation result, and processing the initial recognition text through the credibility screening mechanism to obtain a credible text. According to the invention, the real-time performance and accuracy of voice instruction recognition are improved, and the interaction continuity of the robot is enhanced.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of automatic speech recognition, in particular to a voice data real-time control method and system for an AI toy companion robot. BACKGROUND

[0002] In the field of AI toy companion robots, although the traditional voice data real-time control technology has realized the basic voice instruction response function, there are still some defects in the actual interactive scene for children users. In the traditional technology, part of the voice recognition scheme often adopts the mode of complete voice collection and then decoding, and the streaming processing capability is not fully optimized. When the voice instruction is long or the environment has slight interference, decoding delay is prone to occur, which causes the robot to fail to respond to the child's instruction in time.

[0003] For example, in the scene of a child playing a story chain game, the child may continuously say to the robot, "put the teddy bear doll on the second layer of the bookshelf, and then take a picture book for me". The traditional technology needs to wait for the child to finish all the content before starting feature extraction and decoding on the complete voice data. The whole process may take 1.5-2 seconds. However, after the child issues the instruction, the child usually expects the robot to act immediately. The long delay time may make the child lose patience, and even repeat the instruction, further causing confusion of the instruction and affecting the interactive coherence. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a voice data real-time control method and system for an AI toy companion robot, to improve the real-time performance and accuracy of voice instruction recognition, to optimize the adaptability and safety of mechanical arm path planning, and to enhance the interactive coherence of the robot.

[0005] To solve the above technical problems, the technical solutions of the present application are as follows: In a first aspect, a voice data real-time control method for an AI toy companion robot, the method comprising: Step 1, collecting a user's voice instruction to obtain a voice feature sequence; Step 2, inputting the voice feature sequence into an end-to-end speech recognition model of AI for streaming preliminary decoding to generate a candidate text sequence, and performing semantic adjustment on the candidate text sequence to obtain an initial recognition text; Step 3, performing multi-dimensional stability evaluation on the initial recognition text, establishing a credibility screening mechanism based on the evaluation result, and processing the initial recognition text through the credibility screening mechanism to obtain a credible text; Step 4, dynamically adapting the voice input blocking strategy according to the credible text, synchronously adjusting the cooperative working mechanism between connection temporal classification decoding and attention re-scoring, and generating a real-time semantic analysis result; Step 5, the real-time semantic analysis result is matched with a preset action library to identify the user intention, and an improved multi-strategy fusion fast random expansion tree algorithm is adopted to perform real-time obstacle avoidance planning, so that a mechanical arm movement path is generated; Step 6, based on the mechanical arm movement path, delay data and misoperation events are collected in the action execution process, and feedback information is used to dynamically update acoustic parameters of a voice recognition model and path planning strategies, so that a continuously improved adaptive control mechanism is constructed.

[0006] In a second aspect, a voice data real-time control system of an AI toy companion robot comprises: A collection module is configured to collect user voice instructions and acquire a voice feature sequence; An adjustment module is configured to input the voice feature sequence into an end-to-end voice recognition model of AI for stream-based preliminary decoding to generate a candidate text sequence, and perform semantic adjustment on the candidate text sequence to obtain an initial recognition text; A screening module is configured to perform multi-dimensional stability evaluation on the initial recognition text, establish a credibility screening mechanism based on the evaluation result, and process the initial recognition text through the credibility screening mechanism to obtain a credible text; A coordination module is configured to dynamically adapt a voice input blocking strategy according to the credible text, synchronously adjust a coordination working mechanism between connection time sequence classification decoding and attention re-scoring to generate a real-time semantic analysis result; A planning module is configured to match the real-time semantic analysis result with a preset action library to identify the user intention, and adopt an improved multi-strategy fusion fast expansion random tree algorithm to perform real-time obstacle avoidance planning, so that a mechanical arm movement path is generated; An updating module is configured to collect delay data and misoperation events in the action execution process based on the mechanical arm movement path, and use feedback information to dynamically update acoustic parameters of a voice recognition model and path planning strategies, so that a continuously improved adaptive control mechanism is constructed.

[0007] In a third aspect, a computing device comprises: One or more processors; A storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method.

[0008] In a fourth aspect, a computer readable storage medium stores a program, when the program is executed by a processor, the method is implemented.

[0009] The above scheme of the present application at least has the following beneficial effects: The user voice instruction is collected, the voice information can be captured, the interference of environmental noise on voice collection is reduced, the voice feature sequence is accurately acquired, the voice feature sequence is input into an AI end-to-end speech recognition model for stream preliminary decoding, the rapid processing of the voice instruction can be realized, the recognition delay is shortened, the robot responds to the user demand faster, and meanwhile, the semantic adjustment is carried out on the candidate text sequence, the semantic deviation that may exist in the preliminary decoding can be corrected, the accuracy of the initial recognition text is improved, and the robot is closer to the real instruction intention of the user.

[0010] The multi-dimensional stability evaluation is carried out on the initial recognition text, and a credibility screening mechanism is established, the text content with unstable recognition and low credibility can be filtered out, it is ensured that the final obtained credible text has high reliability, the deviation of robot action execution caused by recognition error is avoided, the voice input blocking strategy is dynamically adapted according to the credible text, different lengths and different complexities of voice instructions can be flexibly coped with, whether it is a short instruction or a complex long instruction, the high-efficiency processing can be realized, the collaborative working mechanism of the connection timing classification decoding and the attention rescore is synchronously adjusted, the semantic analysis process is optimized, the accuracy of the real-time semantic analysis result is improved, and it is ensured that the robot accurately understands the deep demand behind the user instruction; the real-time semantic analysis result is matched with a preset action library, the user intention can be quickly and accurately identified, and the robot is clear about the execution direction; the improved multi-strategy fusion fast random expansion tree algorithm is adopted to execute real-time obstacle avoidance planning, the robot can accurately avoid obstacles such as tables, chairs and toys in a complex environment such as a family, the safety in the movement process of the robot arm is ensured, the success rate of the robot in completing the user instruction is improved, and the robot always maintains a good accompanying interaction effect. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of a voice data real-time control method of an AI toy accompanying robot provided by an embodiment of the present application.

[0012] Figure 2 is a schematic diagram of a voice data real-time control system of an AI toy accompanying robot provided by an embodiment of the present application. DETAILED DESCRIPTION

[0013] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0014] As Figure 1 shown, an embodiment of the present application proposes a voice data real-time control method of an AI toy accompanying robot, the method comprises the following steps: Step 1, collect user voice instructions, and obtain a voice feature sequence; Step 2, input the voice feature sequence into an end-to-end speech recognition model of AI for streaming preliminary decoding to generate a candidate text sequence, and perform semantic adjustment on the candidate text sequence to obtain an initial recognition text; Step 3, perform multi-dimensional stability evaluation on the initial recognition text, establish a credibility screening mechanism based on the evaluation result, and process the initial recognition text through the credibility screening mechanism to obtain a credible text; Step 4, dynamically adapt the voice input blocking strategy according to the credible text, and synchronously adjust the cooperative working mechanism between connection time sequence classification decoding and attention re-scoring to generate real-time semantic analysis results; Step 5, match the real-time semantic analysis results with a preset action library to identify user intent, and use an improved multi-strategy fusion fast random expansion tree algorithm to perform real-time obstacle avoidance planning to generate a robot motion path; Step 6, based on the robot motion path, collect delay data and misoperation events during the action execution process, and use feedback information to dynamically update the acoustic parameters of the voice recognition model and the path planning strategy to build a continuously improved adaptive control mechanism.

[0015] In the embodiment of the application, the user voice instructions are collected to capture voice information, reduce environmental noise interference on voice collection, accurately obtain a voice feature sequence, input the voice feature sequence into an end-to-end speech recognition model of AI for streaming preliminary decoding, which can realize fast processing of voice instructions, shorten recognition delay, and make the robot respond to user demand faster. At the same time, the candidate text sequence is adjusted in semantics to correct possible semantic deviations in preliminary decoding, improve the accuracy of the initial recognition text, and make the robot closer to the user's real instruction intent.

[0016] The multi-dimensional stability evaluation of the initial identified text and the establishment of the credibility screening mechanism can filter out the text content with unstable identification and low credibility, ensure that the final reliable text has high reliability, avoid the deviation of the robot action execution caused by the identification error, dynamically adapt the voice input blocking strategy according to the reliable text, and flexibly cope with voice instructions of different lengths and different complexities. Whether it is a short instruction or a complex long instruction, it can be efficiently processed; the synchronous adjustment of the cooperative working mechanism of the connection timing classification decoding and the attention re-scoring optimizes the semantic analysis process and improves the accuracy of the real-time semantic analysis result, ensuring that the robot accurately understands the deep demand behind the user instruction; the matching of the real-time semantic analysis result and the preset action library can quickly and accurately identify the user's intention, so that the robot can clearly execute the direction; the improved multi-strategy fusion fast random expansion tree algorithm is used to execute real-time obstacle avoidance planning, which can make the robot accurately avoid obstacles such as tables, chairs and toys in complex environments such as homes, ensure the safety of the robot during the movement process, and improve the success rate of the robot in completing the user's instruction, so that the robot can always maintain good accompanying interaction effect.

[0017] In a preferred embodiment of the present application, the above step 1, collecting user voice instructions and obtaining voice feature sequences, can include: In the embodiment of the present application, when the user speaks to the robot, the sound collecting device carried by the robot receives the sound waves propagating in the air. These sound waves are essentially the vibration of air molecules. The device converts this physical vibration into a corresponding electrical signal through the internal sensing components, such as a vibrating diaphragm. At this time, the obtained electrical signal is a continuous changing analog signal, and its waveform fluctuation directly corresponds to the change of sound height, strength, such as the user's speaking pitch, which reflects the difference in electrical signal frequency, and the volume corresponds to the change in electrical signal amplitude. Next, the analog electrical signal is preliminarily purified. Since there may be interference in the environment, such as background noise and device current sound, it is necessary to remove these irrelevant signals through filtering processing, such as filtering out low-frequency current noise (usually below 20Hz) and high-frequency sharp noise (usually above 20000Hz), and only keeping the signal within the frequency range commonly used by human voice (about 300Hz-3400Hz), so that the processing can be more focused on effective speech.

[0018] Then the digital conversion of the signal is carried out, because feature extraction needs to be based on digital information, so the purified analog electrical signal needs to be converted into a computer-recognizable digital signal. This process is achieved by sampling the analog signal at fixed time intervals, measuring the strength of the electrical signal every few microseconds, and converting the result of each measurement into a specific numerical value. In this way, the originally continuous waveform is split into a series of discrete numbers, forming a digital signal sequence. After that, the digital signal is processed by framing. Since the speech signal changes dynamically over time, but its characteristics are relatively stable within a very short period of time (such as 20-30 milliseconds), the long digital signal sequence is divided into short segments (called frames). The length of each frame is usually between 10-30 milliseconds, and there is some overlap between adjacent frames (the overlap rate is about 50%-75%). This is done to avoid losing the continuity of the signal due to segmentation and to ensure that subtle changes in speech can be captured.

[0019] Finally, feature extraction is performed to form a feature sequence. For each divided frame, the speech characteristics it contains are analyzed, such as calculating the energy of the frame signal (reflecting the strength of the sound), the frequency distribution (reflecting the pitch, and the frequency characteristics of different pronunciations are different), and the trend of change over time (such as the feature change when transitioning from one sound to another). These features extracted from each frame (such as energy values, key frequency points, etc.) are arranged in chronological order to form a speech feature sequence that can represent the characteristics of the original speech command.

[0020] In a preferred embodiment of the present application, step 2 above, inputting the speech feature sequence into the end-to-end speech recognition model of AI for streaming preliminary decoding to generate a candidate text sequence, and performing semantic adjustment on the candidate text sequence to obtain the initial recognition text, can include: In the embodiment of the present application, step 220, the speech feature sequence is input into the streaming decoder for real-time parsing to generate an intermediate decoding result including multiple candidate words and time boundaries; specifically including: determining the composition of the speech feature sequence, which is composed of Mel Frequency Cepstral Coefficients (MFCC) extracted from the child speech signal, each frame of features corresponding to a 20-millisecond speech segment, with a 10-millisecond overlap between frames, for example, when a child says "the robot puts the teddy bear on the bookshelf", the speech signal is divided into multiple frames, and 13-dimensional MFCC features are generated for each frame to form a continuous speech feature sequence; then the core architecture of the AI streaming decoder is constructed, a lightweight streaming decoding network based on Connectionist Temporal Classification (CTC) is adopted, which includes 3 one-dimensional convolution layers (convolution kernel size 3x3, used to extract local time-frequency features and capture high-frequency details of the child's high-pitched speech), 2 Bi-LSTM layers (hidden unit number 128, used to capture the time sequence dependence of the speech and handle the pauses and repetitions of the child's pronunciation), and 1 fully connected layer (output dimension is the size of the vocabulary, the vocabulary includes high-frequency words in the child interaction scene, such as robot, teddy bear, doll, bookshelf, etc., a total of 500 core words, excluding adult professional words).

[0021] The real-time parsing process adopts a streaming processing mode of decoding while inputting, the speech feature sequence is input into the decoder frame by frame, and the decoder performs a local decoding every 10 frames (corresponding to 200 milliseconds of speech) to output the top 5 candidate words (each candidate word is accompanied by a probability value, which is the prediction confidence of the word at the current time sequence position) corresponding to the segment of speech, and calculate the time boundary corresponding to each candidate word, the start time of the time boundary is the start time of the current 10 frames of speech, and the end time is the end time of the current 10 frames of speech, for example, the 1st-10th frame of speech corresponds to 0-200 milliseconds, and the decoding candidate words are robot (probability 0.92), machine (probability 0.05), doll (probability 0.01), toy (probability 0.01), and doll (probability 0.01), then the time boundary of the robot is 0-200 milliseconds, and the time boundaries of the remaining candidate words are the same.

[0022] The above process is repeated continuously until the speech feature sequence is input, and all local decoding results are integrated in chronological order to form an intermediate decoding result including candidate words-probability values-time boundaries.

[0023] At step 221, based on the intermediate decoding result, a candidate text sequence including multiple candidate paths is formed through time window alignment and path merging processing. Specifically, first, the time window parameters are set, and the average pause interval of the child's speech is combined to set the time window length to 0.4 seconds and the sliding step to 0.2 seconds. During the time window alignment process, the candidate words in the intermediate decoding result whose time boundaries overlap with the time window by more than 50% are matched based on the start time and end time of the time window. For example, when the time window is 300-700 milliseconds, the candidate words in the intermediate decoding result whose time boundaries overlap with the window by more than 200 milliseconds are matched, i.e., put (200-350 ms), bear (350-550 ms), cat (350-550 ms), and doll (550-700 ms), to ensure that the candidate words in each time window can be completely included in the alignment range.

[0024] After alignment, path generation is performed, and in time window units, the candidate words in each time window are concatenated in time order to form preliminary path segments. For example, the path segment of the first time window (0-400 ms) is robot and put, the path segment of the second time window (200-600 ms) is put and bear, put and cat, and the path segment of the third time window (400-800 ms) is bear, doll and put, cat and put, and doll and put. Then, path merging is performed, and the overlap degree (number of overlapping words ÷ number of words in the previous window path segment) of adjacent time window path segments is calculated. If the overlap degree exceeds 50%, the paths are merged into one path. After merging, the total probability of each complete path is calculated, and the top 3 paths with the highest total probability are retained as candidate paths. Each candidate path corresponds to a continuous text. For example, the first ranked path text is robot put bear doll put bookshelf (total probability 0.65), the second ranked path text is robot put cat doll put bookshelf (total probability 0.08), and the third ranked path text is robot put bear toy put bookshelf (total probability 0.05). The three paths together form the candidate text sequence.

[0025] Step 222, input the candidate text sequence into the pre-trained semantic understanding model for context analysis, identify syntax errors and semantic ambiguity fragments, and generate annotation results; Specifically, first build a pre-trained semantic understanding model, use a lightweight DistilBERT architecture (adapt to real-time processing requirements, reduce computational delay), and train a child voice command-specific corpus (contains 100,000 instructions from children aged 3-8 in toy interaction scenarios, such as putting a teddy bear on a bookshelf, getting a drawing book, and building a house with blocks, etc. Each instruction is annotated with syntax structure and semantic integrity), and focus on optimizing the understanding ability of children's commonly used sentence patterns (putting words, verb-object sentences) during training. For example, by adjusting the attention weight, the sensitivity of the semantic understanding model to the put+object+action+location structure is increased by 30%. The input of the semantic understanding model is each path text in the candidate text sequence, and the output contains two parts of the analysis results. One is syntax error identification. The semantic understanding model identifies syntax errors such as reversed order, missing components, and repeated words by comparing the similarity of text sentence patterns with correct sentence patterns in training data (number of correct sentence pattern features ÷ total number of text sentence pattern features), and annotates error positions and types. For example, the robot bookshelf puts the teddy bear doll, which is annotated as syntax error, and the bookshelf and teddy bear doll position error, with the error position at the 3rd-4th word. The second is semantic ambiguity identification. The semantic understanding model identifies ambiguities such as unclear reference and incomplete information by analyzing the integrity of the core elements (action, object, location) in the text, and annotates ambiguity types and missing elements. For example, putting it on the bookshelf is annotated as semantic ambiguity, with unclear reference and missing object elements. The syntax error annotation and semantic ambiguity annotation of each candidate text are integrated to form the annotation results. For example, the candidate text robot puts the teddy bear is annotated as syntax correct; semantic ambiguity information is incomplete, and the location element is missing; the candidate text robot bookshelf puts the teddy bear doll is annotated as syntax error, syntax reversal, bookshelf and teddy bear doll position error; and semantic integrity.

[0026] Step 223, based on the annotation results, replace, delete or reorder the error and ambiguity fragments in the candidate text sequence to form an adjusted text that is grammatically correct and semantically coherent; Specifically, first select the text to be adjusted from high to low according to the total probability of the candidate text sequence, for example, prioritize adjusting the robot puts the teddy bear (total probability 0.65), and then adjust the robot puts the cat doll (total probability 0.08); For syntax error adjustment, if it is annotated as reversed order, reorder according to the action subject+put+object+action+location structure of the commonly used put word sentence of children, exchange the positions of the bookshelf (location) and the teddy bear doll (object), and adjust it to the robot puts the teddy bear doll on the bookshelf; If it is annotated as repeated words, delete the repeated put, and adjust it to the robot puts the teddy bear on the bookshelf; If it is annotated as missing components, supplement the missing preposition put, and adjust it to put the teddy bear on the bookshelf.

[0027] For semantic ambiguity adjustment, if it is annotated as unclear reference, it is replaced with the candidate word in the intermediate decoding result with the same time boundary, and the common object in the child interaction scene, and adjusted to put the teddy bear on the bookshelf; if it is annotated as incomplete information, the missing elements are inferred from the context, such as the candidate word bookshelf (800-1000 ms) in the intermediate decoding result after 700-800 ms, and the child often puts the teddy bear on the bookshelf, and the bookshelf is supplemented, and adjusted to put the teddy bear on the bookshelf.

[0028] During the adjustment process, the child's expression habits need to be followed, such as the child often omits the doll, if it is annotated as semantic ambiguity, the object attribute is missing, the doll is supplemented, and adjusted to put the teddy bear on the bookshelf; when the child says put on the bookshelf, the upper part can be omitted, and no additional supplement is needed during adjustment, keeping the simple expression of putting on the bookshelf. After adjustment, each candidate text corresponds to an adjusted text, such as robot puts teddy bear and robot puts teddy bear on the bookshelf, and robot puts teddy bear on the bookshelf.

[0029] Step 224, language fluency verification and format regularization are performed on the adjusted text to obtain an initial recognized text that meets the natural language expression specification; specifically, first, language fluency verification is performed, and the verification dimensions include lexical coherence, sentence integrity, and expression naturalness. Lexical coherence verification is performed by calculating the co-occurrence probability of adjacent words (the number of times two words appear simultaneously in the child corpus ÷ the total number of times the previous word appears), if the co-occurrence probability is less than 0.1, it is determined to be incoherent, for example, robot puts teddy bear in the table, and the co-occurrence probability of the table is 0.08, which needs to be adjusted to put the bookshelf; sentence integrity verification checks whether the text contains action subject + action + object + target four elements, if any element is missing, it is supplemented, for example, put teddy bear on the bookshelf lacks the action subject robot, after supplementing, it becomes robot puts teddy bear on the bookshelf; expression naturalness verification compares the similarity of the text and similar instructions in the child corpus (number of identical words ÷ total number of text words), if the similarity is less than 0.7, it is adjusted, for example, robot puts teddy bear on the bookshelf has a similarity of 0.6, and is adjusted to robot puts teddy bear on the bookshelf.

[0030] After the fluency check passes, the format is regularized, and the text format is unified as action subject + core action (put words / sentence-objective sentence) + supplementary information. The action subject is fixed as a robot (to avoid ambiguity caused by children omitting the subject); the core action prefers to put words, and if it is a sentence-objective sentence, it is retained; the supplementary information is arranged in the order of quantity + attribute + object; finally, the text with the highest score (score = coherence score x 0.4 + integrity score x 0.3 + naturalness score x 0.3, full score 1) in fluency check is selected from all adjusted texts as the initial recognition text. For example, the robot puts the teddy bear on the bookshelf. The coherence score is 0.95, the integrity score is 1.0, the naturalness score is 0.98, and the total score is 0.96, which is higher than other texts, and is determined as the initial recognition text.

[0031] The time window alignment and path merging processing fully considers the common corrections and repeated expressions in children's speech, retains multiple possible text paths, reduces the recognition deviation caused by one-time decoding errors, improves the understanding ability of children's incoherent expressions, and the pre-trained semantic understanding model is specially constructed for children's language characteristics. It can accurately identify children's specific grammar habits and ambiguous expressions, combined with targeted adjustment operations, reduce the problem of low accuracy in understanding children's non-standard speech instructions, and language fluency check and format regularization further ensure the standardization and coherence of the initial recognition text, and enhance the smoothness of the robot and children's interaction.

[0032] In a preferred embodiment of the present application, step 3, the above-mentioned multi-dimensional stability evaluation of the initial recognition text, based on the evaluation results, establishes a credibility screening mechanism, and processes the initial recognition text through the credibility screening mechanism to obtain a reliable text, which can include: In the embodiment of the present application, step 330, the initial recognition text is segmented to obtain a plurality of text segments, and confidence evaluation is performed based on the text segments to generate an evaluation sequence; specifically comprising: first determining the specific manner of segmentation, considering that children often have obvious pauses between each action instruction when expressing, therefore taking the pause duration detected in the speech recognition process, such as a pause of more than 0.5 seconds, as the main basis, and combining punctuation marks (commas, periods, exclamation marks, etc.) and semantic units (each complete action instruction, such as picking up a toy car and putting it on the table) in the text for comprehensive segmentation; for example, when the initial recognition text is robot, first build a small house with blocks, and then tell a story to a teddy bear, it will be segmented into two text segments, robot, first build a small house with blocks and then tell a story to a teddy bear, according to the pause and semantics, then confidence evaluation is performed, for each text segment, first extract the original recognition probability of each word in the speech recognition process, add the recognition probabilities of all words in the segment, divide by the total number of words in the segment to obtain the average recognition probability of the segment, then adjust according to the characteristics of children's speech, such as children often say small house as small square, the error rate of this kind of easily confused pronunciation in historical data is 15%, so when small square appears in the segment, subtract the corresponding value of 15% from the average recognition probability; if there is no common confused pronunciation in the segment, the average recognition probability remains unchanged, finally the adjusted confidence of each text segment is arranged in the original order to form an evaluation sequence.

[0033] In step 331, the text segments are used for context coherence analysis, the semantic correlation between adjacent segments is calculated, coherence analysis results are generated, and syntax structure detection is performed on the text segments to identify syntax errors and generate error marking information. Specifically, when performing context coherence analysis, first extract the core semantic elements of each text segment, including action elements, object elements, target elements, and modification elements. For two adjacent segments, first count the total number of core semantic elements of the first segment and the total number of core semantic elements of the second segment, and then count the number of core semantic elements common to both segments. Divide the number of common core semantic elements by the sum of the total number of core semantic elements of the two segments to obtain the semantic correlation of the adjacent segments. For example, the core elements of the previous segment are build, blocks, house, and first (a total of 4), and the core elements of the next segment are tell story, bear, and again (a total of 3). There are no common elements, so the semantic correlation is 0 divided by the sum of 4 and 3, which is 0. If the next segment is to put the bear on the blocks, the core elements are put, bear, blocks, and again (a total of 4), and there is a common element blocks, so the semantic correlation is 1 divided by the sum of 4 and 4, which is 0.125. Arrange all the semantic correlations of adjacent segments in order to form coherence analysis results. When performing syntax structure detection, check the structure of each text segment according to the basic sentence patterns commonly used by children, such as checking whether there is an action element, whether there is an action object, and whether there is a reversed word order. For each problematic segment, clearly mark the error type and specific location to form error marking information.

[0034] Step 332, input the evaluation sequence, the coherence analysis result and the error marking information into the dynamic threshold calculation rule to generate the confidence threshold value adapting to the current text features; specifically including: the dynamic threshold calculation rule first sets the basic threshold value as 0.7, then adjusts the basic threshold value according to the evaluation sequence, first calculates the average confidence of all segments in the evaluation sequence, if the average confidence is higher than 0.8 (preset high confidence benchmark), it indicates that the overall recognition quality is better, and the basic threshold value is reduced by 10% (i.e. 0.7 multiplied by 0.9, 0.63); if the average confidence is lower than 0.5 (preset low confidence benchmark), it indicates that the overall recognition quality is poor, and the basic threshold value is increased by 10% (i.e. 0.7 multiplied by 1.1, 0.77); if the average confidence is between 0.5 and 0.8, the basic threshold value remains unchanged; then further adjust in combination with the coherence analysis result, calculate the average semantic correlation degree of all adjacent segments in the coherence analysis result, if the average semantic correlation degree is higher than 0.3 (preset high correlation benchmark), it indicates that the text coherence is good, and the threshold value is reduced by 5% (e.g. 0.63 multiplied by 0.95, 0.5985) based on the threshold value adjusted before; if the average semantic correlation degree is lower than 0.1 (preset low correlation benchmark), it indicates that the text coherence is poor, and the threshold value is increased by 5% (e.g. 0.77 multiplied by 1.05, 0.8085) based on the threshold value adjusted before; if the average semantic correlation degree is between 0.1 and 0.3, the current threshold value remains unchanged, finally the third adjustment is made by referring to the error marking information, the proportion of the number of segments with syntax errors to the total number of segments is calculated, if the proportion is higher than 30% (preset high error benchmark), it indicates that there are more text syntax problems, and the threshold value is increased by 5% (e.g. 0.8085 multiplied by 1.05, 0.8489) based on the threshold value adjusted before; if the proportion is lower than 10% (preset low error benchmark), it indicates that the text syntax is relatively standard, and the threshold value is reduced by 5% (e.g. 0.5985 multiplied by 0.95, 0.5686) based on the threshold value adjusted before; if the proportion is between 10% and 30%, the current threshold value remains unchanged, the value obtained after the three adjustments is the confidence threshold value adapting to the current text features.

[0035] Step 333, screening the text segments based on the credibility threshold, retaining the segments meeting the threshold requirements to form the screened text; specifically including: extracting the confidence of each text segment in the evaluation sequence one by one, comparing it with the credibility threshold generated in step 332, if the confidence of a segment is greater than or equal to the credibility threshold, the segment is retained; if the confidence of a segment is less than the credibility threshold, the segment is discarded; for example, the credibility threshold is 0.6, the confidence of a segment is 0.65, the segment is retained; the confidence of a segment is 0.58, the segment is discarded. In the screening process, if two or more consecutive segments are discarded, check whether there is a semantic break between the retained segments before and after the discarded segments, i.e. the end of the previous retained segment and the beginning of the next retained segment are irrelevant in semantics, if there is a semantic break, record the position of the break, i.e. the end index of the previous retained segment and the beginning index of the next retained segment, so as to focus on processing when repairing, arrange all retained segments in their original order in the initial identified text to form the screened text.

[0036] Step 334, repairing the semantic integrity of the screened text, restoring the semantic coherence by supplementing necessary vocabulary and adjusting the word order, and performing boundary smoothing processing on the repaired text to generate a credible text meeting the semantic and grammatical specifications; specifically including: when repairing the semantic integrity, first check whether there is a semantic break or information missing in the screened text, for the semantic break, supplement the connecting content in combination with the common action sequence and context information in children's instructions, for example, the screened text is to build a small house with blocks and tell a story to a bear, there is a break in the middle, then it is supplemented as a connection; if the screened text is to build a small house and tell a story to a bear, the subject block is missing in the previous segment, then the block is supplemented; for the common inverted word order of children, adjust the word order according to the correct sentence structure, change the bear to give the bear; after completing the semantic repair, perform boundary smoothing processing to check whether the connection between adjacent segments is natural, add appropriate connecting words according to the semantic relationship between the segments to make the overall text smooth, for example, the repaired segment is robot first builds a small house with blocks and tells a story to a bear, add again to connect, form robot first builds a small house with blocks and then tells a story to a bear, finally check whether the overall text meets the common expression habits and grammatical rules of children's instructions to ensure no ambiguity and no grammatical errors, generate the final credible text.

[0037] The accurate splitting of children's possibly lengthy or intermittent instructions avoids information confusion caused by overall processing, multi-dimensional confidence evaluation combined with children's speech characteristics accurately judges the reliability of each segment, reduces misjudgment caused by non-standard pronunciation, dynamic threshold setting makes the screening mechanism flexible to adapt to different quality of voice input, semantic repair and boundary smoothing processing specifically solves the problem of incomplete expression and reversed sequence of children, ensures the final text coherent and consistent with semantic logic, improves the understanding ability of children's complex instructions, reduces the overall misunderstanding caused by inaccurate identification of part of the segment, shortens the response time of the robot, so that children can obtain more timely and accurate feedback in the interaction, and enhance the coherence and interest of the interaction.

[0038] In a preferred embodiment of the present application, the above step 4, dynamically adapting the voice input blocking strategy according to the trusted text, synchronously adjusting the cooperative working mechanism between the connection timing classification decoding and the attention re-scoring, generating real-time semantic analysis results, can include: In the embodiment of the present application, step 440, the block size and the block overlap rate of the voice input are dynamically adapted according to the text length and the semantic complexity of the trusted text to generate an adapted voice block sequence; specifically, first, the text length of the trusted text is counted, and the text length is divided into three intervals according to the number of Chinese characters, that is, the number of Chinese characters of short text is less than 15, the number of Chinese characters of medium-long text is between 15 and 30, and the number of Chinese characters of long text is greater than 30; then the semantic complexity of the trusted text is determined by counting the number of core actions and the number of modifying information in the text, if the number of core actions is 1 and the number of modifying information is less than 3, it is low complexity; if the number of core actions is 2 and the number of modifying information is between 3 and 5, it is medium complexity; if the number of core actions is greater than 2 or the number of modifying information is greater than 5, it is high complexity; then the block size is determined according to the text length and the semantic complexity, when the text is short and the complexity is low, the block size is set to 10 Chinese characters; when the text is short and the complexity is medium or high, or the text is medium-long and the complexity is low, the block size is set to 15 Chinese characters; when the text is medium-long and the complexity is medium or high, or the text is long and the complexity is low, the block size is set to 20 Chinese characters; when the text is long and the complexity is medium or high, the block size is set to 25 Chinese characters; then the block overlap rate is determined, the overlap rate of low complexity text is set to 10% (i.e. the number of overlapping Chinese characters between each block and the next block is the block size multiplied by 10%, the result is an integer); the overlap rate of medium complexity text is set to 15%; the overlap rate of high complexity text is set to 20%, finally, the text corresponding to the original voice input is blocked according to the determined block size and overlap rate, for example, the trusted text is a robot, put the pink teddy bear doll on the third layer of the living room bookshelf, and then take a picture book with a small rabbit cover, a total of 32 Chinese characters, 2 core actions and 4 modifying information, which belongs to long text and medium complexity, the block size is set to 20 Chinese characters, and the overlap rate is 15% (i.e. 3 Chinese characters), the first block is a robot, put the pink teddy bear doll on the third layer of the living room bookshelf (20 Chinese characters), the second block is the third layer, and then take a picture book with a small rabbit cover (20 Chinese characters are extracted from the 18th Chinese character of the first block), and the two blocks together form an adapted voice block sequence.

[0039] At step 441, based on the adapted speech chunk sequence, a connection time sequence classification decoding process is performed to generate an initial decoding text sequence; specifically, the connection time sequence classification decoding process is performed on each chunk in the order of the speech chunk sequence, and when processing each chunk, the speech signal features corresponding to the chunk are first extracted, then the candidate words corresponding to each time node are determined according to the features and the preset speech-word mapping relationship, and then the word combination that meets the language expression logic is screened out through time sequence coherence judgment to form the decoding text segment of the chunk; for example, for the chunk of putting the pink teddy bear into the third layer of the living room bookshelf, after extracting the speech features, the word combination of robot, put, pink teddy bear, into, the third layer of the living room bookshelf is determined to meet the time sequence logic through time sequence matching to form the decoding text segment of the chunk, and when processing adjacent chunks, the decoding result of the overlapping part is focused on, if the decoding text of the overlapping part of the previous chunk is the third layer of the living room bookshelf and the decoding text of the overlapping part of the next chunk is the third layer of the shelf, the complete and semantically correct third layer of the living room bookshelf is used as the criterion to correct the decoding result of the overlapping part of the next chunk, and the text segments of all the chunks after decoding and overlapping correction are arranged in the original order of the chunk sequence to form the initial decoding text sequence, for example, in the above example, the decoding text segments of the two chunks are robot, put the pink teddy bear into the third layer of the living room bookshelf, and then bring another book with a small rabbit cover, and the combination is the initial decoding text sequence.

[0040] At step 442, the participation coefficient of the attention re-scoring mechanism is calculated according to the semantic consistency between the initial decoding text sequence and the trusted text, and the participation degree of the attention re-scoring mechanism in the decoding process is adjusted based on the participation coefficient; including comparing the initial decoding text sequence with the trusted text in terms of semantic units, identifying the text units with semantic differences between the two, and recording the number and distribution characteristics of the text units with semantic differences; based on the number and distribution characteristics of the text units with semantic differences, the semantic consistency quantitative score is calculated through the preset evaluation rule; according to the semantic consistency quantitative score, the corresponding relationship table of the participation coefficient is queried to obtain the corresponding attention re-scoring participation coefficient; according to the specific value of the attention re-scoring participation coefficient, the participation degree of the attention re-scoring mechanism is determined combined with the preset configuration strategy; specifically including: first, extracting the core semantic elements of the trusted text and the initial decoding text sequence, the core semantic elements including action elements, object elements, and location elements, counting the total number of core semantic elements of the trusted text, and then counting the number of elements in the initial decoding text sequence that completely match the core semantic elements of the trusted text, dividing the number of matched elements by the total number of core semantic elements of the trusted text to obtain the semantic consistency quantitative score, for example, the trusted text has 5 core semantic elements (actions "put" and "take", objects "pink teddy bear" and "book with small rabbit cover", location "third layer of living room bookshelf"), and the initial decoding text sequence matches 4 of them, then the semantic consistency quantitative score is the result of 4 divided by 5, i.e. 0.8, then the preset participation coefficient corresponding relationship table is queried, the corresponding rule of the semantic consistency quantitative score and the participation coefficient in the table is that when the score is greater than 0.8, the participation coefficient is 0.2; when the score is between 0.6 and 0.8, the participation coefficient is 0.5; when the score is between 0.4 and 0.6, the participation coefficient is 0.8; when the score is less than 0.4, the participation coefficient is 1.0; according to the semantic consistency quantitative score 0.8 in the above example, the participation coefficient is obtained as 0.2, and finally the participation degree is adjusted according to the participation coefficient, the value of the participation coefficient directly corresponds to the participation proportion of the attention re-scoring mechanism in the decoding process, i.e. when the participation coefficient is 0.2, the attention re-scoring mechanism only re-scores and calibrates 20% of the content in the initial decoding text sequence; when the participation coefficient is 0.5, 50% of the content is calibrated; when the participation coefficient is 0.8, 80% of the content is calibrated; when the participation coefficient is 1.0, 100% of the content is calibrated.

[0041] Step 443, the initial decoding text sequence is processed by the adjusted attention re-scoring mechanism for semantic calibration, and an adjusted decoding text sequence is generated; Specifically, first, determine the text content range that needs to be calibrated according to the participation degree, if the participation degree is 20%, select the 20% content with the lowest semantic consistency (usually the core element matching is incomplete or there is ambiguity) from the initial decoding text sequence as the calibration object; If the participation degree is 50%, select 50% of the content with lower semantic consistency as the calibration object, and so on; For example, when the participation degree is 20%, the picture book with a small rabbit cover matched in the initial decoding text sequence is a reliable text, and the third layer of the living room bookshelf is not completely decoded (only decoded as the living room bookshelf), so the living room bookshelf segment is taken as the calibration object, then the semantic re-scoring is performed on the calibration object, the differences between the calibration object and the corresponding core semantic elements in the reliable text are compared, the missing information in the calibration object is supplemented, and the error information is corrected, for example, the living room bookshelf is supplemented to the third layer of the living room bookshelf, and then the calibrated text segment and the uncalibrated part of the initial decoding text sequence are integrated in the original order to form the adjusted decoding text sequence, for example, in the above example, the integrated result is: the robot puts the pink teddy bear in the third layer of the living room bookshelf, and then takes a picture book with a small rabbit cover.

[0042] Step 444, real-time semantic analysis is performed on the adjusted decoding text sequence to generate a real-time semantic analysis result; Specifically, first, analyze the adjusted decoding text sequence segment by segment, extract the corresponding core semantic information of each segment, including action type, action object, action target, and action sequence, for example, for the segment "the robot puts the pink teddy bear in the third layer of the living room bookshelf", the action type is "put", the action object is "pink teddy bear", and the action target is "the third layer of the living room bookshelf"; For the segment "then take a picture book with a small rabbit cover", the action type is "take", the action object is "a picture book with a small rabbit cover", the action target is "the user's location", and the action sequence is "after the put action"; Then, the extracted core semantic information is arranged according to the logical structure of "action sequence-action type-action object-action target" to ensure that the information is complete and clear, and finally a structured real-time semantic analysis result is formed, which clearly includes the execution requirements of each action, for example, action 1 (sequence: 1): the action type is "put", the action object is "pink teddy bear", and the action target is "the third layer of the living room bookshelf"; Action 2 (sequence: 2): the action type is "take", the action object is "a picture book with a small rabbit cover", and the action target is "the user's location".

[0043] By dynamically adapting the chunking strategy according to the text length and semantic complexity of the child's instruction, the delay caused by the excessive processing load of long instructions under the fixed chunking mode is reduced, and the reasonable overlap rate ensures the semantic coherence between chunks, which is consistent with the characteristics of children's coherent expression. The real-time processing mode of connection time sequence classification decoding allows each chunk to quickly generate decoding results, reducing the overall waiting time. The dynamic participation of the attention re-scoring mechanism ensures processing speed when the semantics are consistent and accurately calibrates when the semantics deviate, balancing real-time performance and accuracy, and improving interactive coherence and children's user experience.

[0044] In a preferred embodiment of the present application, the step 5 of matching the real-time semantic analysis result with the preset action library to identify the user's intention, and using the improved multi-strategy fusion fast random expansion tree algorithm to perform real-time obstacle avoidance planning to generate the motion path of the robot arm can include: In an embodiment of the present application, step 550, the real-time semantic analysis result is processed by word segmentation to obtain a word sequence, and action keywords are extracted based on the word sequence to identify the core action elements of the user's instruction. Specifically, first, determine the word segmentation rules, combine the expression habits of children's instructions (common simple words, short sentences, and less complex grammar), and perform word segmentation based on natural semantic pauses (such as speech pauses corresponding to commas and periods) and common word boundaries (such as robot, teddy bear, and other non-detachable noun phrases). For example, the real-time semantic analysis result is action 1, the action subject is robot, the action type is placing, the action object is a pink plush teddy bear, and the action target is the third layer left grid of the living room bookshelf; action 2, the action subject is robot, the action type is taking, the action object is a bedtime picture book with a small rabbit cover, and the action target is the user's hand. After word segmentation, the word sequence obtained is action 1, the action subject is robot, the action type is placing, the action object is a pink plush teddy bear, the action target is the third layer left grid of the living room bookshelf, action 2, the action subject is robot, the action type is taking, the action object is a bedtime picture book with a small rabbit cover, and the action target is the user's hand.

[0045] Next, the action keywords are extracted. First, a children's common action word library is established, and the words in the word sequence that match the word library are selected as action keywords, such as placing and taking in the above sequence. Then, based on the action keywords, the core action elements are identified, which include three types: action type, action object, and action target. The action type directly corresponds to the extracted action keyword. The action object is extracted from the word segment containing the action object, which must completely contain the attribute description of the object. The action target is extracted from the word segment containing the action target, which must completely contain the location or recipient information. Finally, each action corresponds to a set of core action elements, such as the core action elements of action 1, which are action type placing, action object pink plush teddy bear, and action target third layer left grid of the living room bookshelf.

[0046] Step 551, similarity matching of core action elements and instruction structure stored in preset action library is performed to determine target action instruction, and corresponding robot target position and motion parameters are obtained according to the target action instruction; specifically, first, the instruction structure of the preset action library is determined, each instruction structure contains three parts of action type + action object type + action target type, and is associated with corresponding robot target position parameters such as coordinate range and motion parameters such as motion speed, clamping force, and motion accuracy; for example, the instruction structure of the preset action library is placed + plush toy + bookshelf grid, and the associated robot target position parameters of the bookshelf grid center coordinates (X: 0.8 meters, Y: 0.6 meters, Z: 1.5 meters) ± 0.05 meters, and the motion parameters are motion speed 0.2 meters / second, clamping force 5 Newton, and motion accuracy ± 0.02 meters; the instruction structure of taking + drawing book + user's hand is associated with the robot target position parameters of the user's hand commonly used area coordinates (X: 0.3 meters, Y: 0.4 meters, Z: 0.9 meters) ± 0.05 meters, and the motion parameters are motion speed 0.25 meters / second, clamping force 3 Newton, and motion accuracy ± 0.02 meters.

[0047] Then, the similarity of the core action elements and each instruction structure is calculated, the similarity calculation method is that the action type of the core action elements matches the action type of the instruction structure for 1 point, and does not match for 0 point; the action object of the core action elements matches the action object type of the instruction structure for 1 point, and does not match for 0 point; the action target of the core action elements matches the action target type of the instruction structure for 1 point, and does not match for 0 point; the total similarity score is the sum of the three scores, and the total score is 3 points, for example, the core action elements of action 1 are placed + pink plush bear toy + bookshelf third layer left grid, and the three scores of the instruction structure of placed + plush toy + bookshelf grid are all 1 point, and the total similarity score is 3 points; the scores of other instruction structures may be 1 point or 2 points.

[0048] The instruction structure with the highest total similarity score is selected as the matching result, if the highest score is unique, the corresponding instruction is the target action instruction; if there are multiple highest scores, the specific attributes of the action object are further compared to determine the unique target action instruction, after the target action instruction is determined, the robot target position parameters and motion parameters associated with the instruction structure are extracted, and the target position parameters are fine-tuned combined with the specific information of the core action elements, for example, the action target of action 1 is the bookshelf third layer left grid, and the bookshelf grid center coordinates associated with the preset instruction structure are (0.8 meters, 0.6 meters, 1.5 meters), according to the specific position of the third layer left, the target position parameters are adjusted to (0.78 meters, 0.58 meters, 1.52 meters) ± 0.05 meters, and the motion parameters remain unchanged, finally the corresponding robot target position and motion parameters of the action are obtained.

[0049] At step 552, based on the target position of the mechanical arm and the real-time data collected by the environmental sensor, a current environment map including obstacle spatial distribution information is constructed; specifically including: first, determine the coordinate system of the environment map, take the center of the robot base as the origin, the horizontal forward direction as the positive direction of the X axis, the horizontal right direction as the positive direction of the Y axis, and the vertical upward direction as the positive direction of the Z axis, the range of the map covers the reachable area of the mechanical arm motion (such as X: 0-1.2 meters, Y: -0.5-0.5 meters, Z: 0.3-1.8 meters), and is consistent with the child interaction scene, such as the toys and furniture within 1.2 meters around the robot in the living room; then collect the real-time data of the environmental sensor, the sensor includes distance sensor to collect the distance between the robot and the surrounding objects and contour sensor to collect the approximate contour size of the object, the collection frequency is 10 times per second, and the average value of 5 continuous collections is taken to reduce the error; for example, the distance sensor collects that there is an object at the position of X: 0.8 meters, Y: 0.6 meters, Z: 1.4 meters, and the distance from the robot base is 0.9 meters; the contour sensor collects the contour size of the object as 0.2 meters long, 0.15 meters wide, and 0.1 meters high, which is judged as a possible obstacle, all collected obstacle data is labeled according to the coordinate system, the center coordinates and contour size of each obstacle are recorded, and the obstacle spatial distribution information is formed; finally, the target position of the mechanical arm is labeled in the coordinate system, and the obstacle spatial distribution information is integrated to form the current environment map, the coordinate range of the target position of the mechanical arm, the center coordinates and contour size of all obstacles, and the initial position of the mechanical arm are clearly labeled in the map, so that the map can completely reflect the space and obstacle distribution that may be involved in the motion path of the mechanical arm.

[0050] At step 553, based on the target position of the mechanical arm, the motion parameters and the current environment map, a modified multi-strategy fusion fast random expansion tree algorithm is used for path planning; specifically including: first determining the multi-strategy composition of the improved algorithm, including the guided sampling strategy, the dynamic expansion strategy and the path pruning strategy, which cooperates to realize path planning. The guided sampling strategy means that when generating the sampling points, not only some sampling points are randomly generated, but also the sampling points are preferentially generated towards the target position of the mechanical arm. The number of preferentially generated sampling points accounts for 60% of the total number of sampling points, and the randomly generated accounts for 40%. The dynamic expansion strategy means that starting from the initial position of the mechanical arm, the sampling point closest to the current expansion tree is selected each time, and it is calculated whether the path between the sampling point and the expansion tree node meets the motion parameter requirements, such as the motion time corresponding to the path length is not more than 1.2 times the motion distance ÷ motion speed, to ensure compliance with the speed. If it meets the requirements, the sampling point is added to the expansion tree to form a new node. If it does not meet the requirements, the position of the sampling point is adjusted and recalculated until a sampling point meeting the motion parameters is found or it is determined that there is no feasible path in this direction. The path pruning strategy means that when the expansion tree extends to the vicinity of the target position of the mechanical arm, which is less than 0.1 meters away from the target position, the formed path is simplified, and redundant nodes in the path are deleted, such as the approximate three adjacent nodes on the same straight line, and the distance between the middle node and the straight line is less than 0.03 meters, to reduce the number of turns of the path and ensure smooth movement of the mechanical arm.

[0051] In the planning process, in combination with the spatial distribution information of obstacles in the current environment map, when generating a sampling point or expanding a path each time, it is checked whether the sampling point is in the obstacle area, whether the coordinates of the sampling point fall within the size range of the obstacle contour, whether the path passes through the obstacle area, and whether the coordinates of any point on the path fall within the size range of the obstacle contour. If there is, the sampling point or path is abandoned and reselected. For example, when planning the path of action 1, if the sampling point falls within the size range of the contour of obstacle 1 (center coordinates 0.78 meters, 0.45 meters, 1.4 meters) (X: 0.68-0.88 meters, Y: 0.375-0.525 meters, Z: 1.35-1.45 meters), the sampling point is abandoned. If the path is from (0.5 meters, 0.5 meters, 1.0 meters) to (0.7 meters, 0.6 meters, 1.4 meters) and passes through the Z-axis range 1.35-1.45 meters of obstacle 1, the path is abandoned. Through multi-strategy fusion, a feasible path framework from the initial position of the mechanical arm to the target position is gradually constructed.

[0052] Step 554, in the path planning process, the sampling strategy is dynamically adjusted according to the environmental complexity, an initial motion path is generated, and the initial motion path is subjected to collision detection to identify potential collision risk areas to obtain a collision detection result; specifically including: first, judge the environmental complexity, the environmental complexity is calculated by the number of obstacles x the average volume of obstacles, wherein the number of obstacles is the total number of obstacles in the current environment map, and the average volume of obstacles is the sum of the volumes (length x width x height) of all obstacles divided by the number of obstacles; if the calculation result is less than 0.01 cubic meters, such as 2 obstacles, each with a volume of 0.004 cubic meters, 2x0.004=0.008 cubic meters, then it is a low complexity environment; if the result is between 0.01-0.03 cubic meters, it is a medium complexity environment; if the result is greater than 0.03 cubic meters, it is a high complexity environment.

[0053] Then adjust the sampling strategy according to the environmental complexity, in a low complexity environment, set the total number of sampling points to 50, the sampling points generated preferentially to the target position account for 60% (30), the random sampling points account for 40% (20), and the sampling interval is 0.1 meters; in a medium complexity environment, set the total number of sampling points to 80, the sampling points generated preferentially to the target position account for 70% (56), the random sampling points account for 30% (24), and the sampling interval is reduced to 0.08 meters; in a high complexity environment, set the total number of sampling points to 120, the sampling points generated preferentially to the target position account for 80% (96), the random sampling points account for 20% (24), and the sampling interval is further reduced to 0.05 meters, by increasing the number of sampling points, increasing the target-oriented sampling ratio, and reducing the sampling interval, more obstacle avoidance paths can be found in a high complexity environment.

[0054] After the path expansion based on the adjusted sampling strategy, the continuous nodes from the initial position to the target position in the expansion tree are sequentially connected to form an initial motion path. Then, the initial motion path is subjected to collision detection. The path is divided into several small segments at intervals of 0.02 meters, and the start and end points of each segment are detection points. The distance between each detection point and all obstacles (the spatial distance between the coordinates of the detection point and the coordinates of the center of the obstacle) is calculated. If the distance is less than the maximum radius of the obstacle (half of the maximum value of the length, width, and height of the obstacle) + the safety distance of the robot arm (0.03 meters), then the region corresponding to the small segment path at the detection point is a collision risk region. For example, the maximum radius of obstacle 1 is 0.1 meters (half of the length of 0.2 meters), the distance between the detection point (0.7, 0.6, 1.4) and the center of obstacle 1 is 0.08 meters, and 0.08 meters is less than 0.1 meters + 0.03 meters = 0.13 meters. Therefore, the region corresponding to the small segment path (0.65 meters, 0.55 meters, 1.38 meters) to (0.75 meters, 0.65 meters, 1.42 meters) is a collision risk region. The position range of all collision risk regions is recorded to form the collision detection result.

[0055] At step 555, based on the collision detection result, the risk region is subjected to trajectory adjustment to generate a safe motion path, and it is verified whether the safe motion path meets the motion parameter requirements to generate a final robot arm motion path. Specifically, first, a trajectory adjustment scheme is developed for each collision risk region. The adjustment principle is to avoid the risk region and increase the path length by no more than 20% of the original path length. The adjustment method is to generate 2-3 new nodes between the previous and next nodes of the risk region. The coordinates of the new nodes need to satisfy a distance greater than the safety distance of the robot arm + the maximum radius of the obstacle, and the connection between the new nodes is smooth. For example, in the initial path, node 3 (0.5, 0.3, 1.1) and node 4 (0.78, 0.58, 1.52) pass through a collision risk region. New nodes 3a (0.6, 0.2, 1.3) and 3b (0.7, 0.4, 1.45) are added between them. It is checked that the distances between nodes 3a and 3b and the risk region are both greater than 0.13 meters, and the angle deviations of nodes 3, 3a, 3b, and 4 are all less than 30 degrees. The adjusted path segment is nodes 3, 3a, 3b, and 4.

[0056] After all the risk area adjustments are completed, the adjusted nodes are connected in order to form a safe motion path, and then it is verified whether the safe motion path meets the motion parameter requirements, the motion speed is verified, the total length of the safe motion path (the sum of the spatial distances between adjacent nodes) is calculated, and the preset motion time (total length ÷ motion speed) is divided to check whether the actual required time is within the reasonable range corresponding to the preset motion speed; the clamping force is verified, the posture change of the mechanical arm in the path is checked to ensure that the posture change is gentle and the clamping force can be stably maintained within the preset value (such as 5N) ± 0.5N; the motion accuracy is verified, and it is checked whether the end point coordinates of the safe motion path are within the range of the target position parameters of the mechanical arm; if the verification is passed, the safe motion path is the final mechanical arm motion path; if the verification is not passed, the trajectory is adjusted again, and the verification is performed again until all the motion parameter requirements are met, and finally the mechanical arm motion path is generated.

[0057] By extracting the core action elements that fit the characteristics of children's instructions, the operation intention of children can be accurately captured, the intention misjudgment caused by the simple expression and limited vocabulary of children can be avoided, the similarity matching mechanism of the preset action library is combined with the instruction structure of the common interactive scene of children to quickly determine the target action instruction, the intention recognition time is shortened, the obstacle characteristics of the activity space of children are fully considered when the environment map is constructed, it is ensured that the map can truly reflect the interactive environment, the continuity and interest of the interaction are improved, and the children lose patience due to waiting or robot misoperation are avoided.

[0058] In a preferred embodiment of the present application, the above step 6 collects delay data and misoperation events during the motion execution based on the motion path of the mechanical arm, and dynamically updates the acoustic parameters of the voice recognition model and the path planning strategy by using the feedback information to build a continuously improved adaptive control mechanism, which can include: In the embodiment of the present application, step 660, the actual running condition of the mechanical arm executing the planned motion path is monitored to obtain motion trajectory information and path execution deviation information with accurate time marking; specifically, first, a multi-dimensional monitoring system is built, in addition to mechanical arm motion monitoring, voice signal acquisition monitoring is additionally supplemented, a 3-microphone array is used, original audio data at the time of issuing a voice instruction is synchronously acquired, audio sampling rate, audio duration, and environmental noise data at the time of acquisition are recorded; when determining the key nodes and parameters to be monitored, on the basis of the original mechanical arm nodes, key nodes in the voice recognition stage are newly added, voice data transmission completion T01, acoustic feature extraction completion T02, and decoding text generation completion T03, forming a voice acquisition-recognition-mechanical arm action full-link time marking, accurate to milliseconds; when acquiring motion trajectory information, the three-dimensional coordinates of the mechanical arm end are recorded at a frequency of 20 times per second, associated with the time stamp; at the same time, the time consumption of each stage of voice recognition is acquired, included in the time analysis dimension of the trajectory information, when acquiring path execution deviation information, in addition to calculating the difference (deviation value = real-time coordinate - preset coordinate) between the real-time coordinate and the preset coordinate, the confidence of the voice recognition result also needs to be associated, if the deviation exceeds the threshold value (0.03 meters) and the corresponding recognition confidence is less than 0.8, it is marked as recognition associated deviation.

[0059] Step 661, analyze the motion trajectory information to obtain the delay time statistical result from the issuance of the voice instruction to the mechanical arm reaching each key action node; at the same time, analyze the path execution deviation information to obtain the classified and sorted misoperation type statistical result; specifically, when analyzing the delay time, in addition to calculating the delay of each node of the mechanical arm (start-up delay = T1-T0, completion delay = T3-T0), the delay analysis in the voice recognition stage is newly added, the total recognition delay = T03-T01, which is decomposed into feature extraction delay (T02-T01) and decoding delay (T03-T02), for the same action frequently used by children, 10 execution data are continuously collected, the average value (average value = sum of 10 delays ÷ 10) and the proportion (certain stage delay ÷ total delay × 100%) of each stage delay are calculated, if the recognition stage delay proportion exceeds 40%, it is marked as recognition dominant delay.

[0060] When analyzing the misoperation type, in addition to the original three types of misoperations, recognition associated misoperation is newly added, which refers to the confidence of the voice recognition result being less than 0.8 and the mechanical arm action deviation exceeding the threshold value (0.05 meters), such as a child saying to take a blue drawing book, recognizing it as taking a green drawing book, resulting in the mechanical arm taking the wrong object, when counting each type of misoperation, voice related dimension data needs to be supplemented, such as recording the misrecognized word and the corresponding acoustic feature in the recognition associated misoperation, finally counting the occurrence number, proportion, and associated voice recognition confidence, acoustic feature mean value of each type of misoperation, forming a more comprehensive misoperation statistical result.

[0061] Step 662, integrate the delay time statistics and the misoperation type statistics to construct a comprehensive analysis data set reflecting the association characteristics of delay and misoperation; specifically including: adding voice recognition related dimensions to the core field, such as voice recognition confidence, feature extraction time consumption, decoding time consumption, recognition error vocabulary, acoustic characteristics, and child voice age; when correlating data, a three-level association between voice features, recognition results, and misoperations needs to be established, such as a voice feature of a certain execution ID being a speech rate of 9 words per second, a tone of 480 Hz, and a signal-to-noise ratio of 25 dB, a recognition result of taking a green picture book (confidence of 0.72), and a misoperation type of recognition associated misoperation; then bind the three records, supplement the recognition result and action library matching time consumption in the path feature, and supplement the child-robot distance in the environment information; when the data set accumulates to 50, group them by child age and misoperation type, such as a recognition associated misoperation group for 3-5 year old children and a position deviation misoperation group for 6-8 year old children.

[0062] Step 663, based on the misoperation type and voice feature mapping relationship in the comprehensive analysis data set, adjust and update the acoustic feature extraction parameters of the voice recognition model; specifically including: first, construct the basic framework of the voice recognition model in detail, use a CNN-LSTM hybrid network, the input is acoustic features, the CNN layer (3 layers, convolution kernel size 3x3) is responsible for extracting the local time-frequency features of the voice, the LSTM layer (2 layers, hidden unit number 256) is responsible for capturing the time sequence dependence of the voice, and the output is the phoneme probability distribution; the training data set uses a child interaction scene special voice library; use an improved n-gram model (3-gram), the vocabulary library contains child commonly used words and colloquial expressions, and excludes adult words; optimize the text generation logic by statistical child corpus co-occurrence probability; use a weighted finite state transducer WFST to combine the phoneme probability of the hybrid architecture acoustic model and the vocabulary probability of the language model to generate the final text sequence; in the initial decoding parameters, the weight of the hybrid architecture acoustic model is set to 0.6, and the weight of the language model is set to 0.4, which prioritizes the phoneme accuracy rate of child voice.

[0063] Then build the mapping relationship between misoperation type and voice feature, group the 50 data by misoperation type, calculate the distribution interval of acoustic features in each group, the recognition associated misoperation group, the speech rate > 9 words per second (60%), the tone > 460 Hz (3-5 year old children, 75%), the signal-to-noise ratio < 25 dB (55%), the pause number > 2 times (45%), and the recognition confidence < 0.8 (90%); the position deviation misoperation group, the speech rate 8-9 words per second (50%), the tone 400-460 Hz (6-8 year old children, 60%), the signal-to-noise ratio 25-30 dB (50%), and the recognition confidence 0.8-0.9 (70%).

[0064] Through the control variable method test, it is found that when the speech rate is > 9 words per second, the MFCC feature frame shift of the hybrid architecture acoustic model is too large (originally 50%), resulting in insufficient frame overlap, missing the details of syllables with fast pronunciation, and the recognition error rate rising by 30%; when the pitch is > 460 Hz (3-5 year-old children), the number of original MFCC mel filter banks (24) is insufficient to capture the details of the high frequency band (4000-6000 Hz), resulting in confusion between xiong and neng, and the error rate rising by 25%; when the signal-to-noise ratio is < 25 dB, the original pre-emphasis coefficient (0.97) is insufficient to suppress low-frequency noise, the feature signal-to-noise ratio decreases, the bias of the decoding mode phoneme selection increases, and the error rate rises by 20%.

[0065] Then the acoustic feature extraction parameters and the speech recognition model structure are adjusted accordingly. For fast speech rate (> 9 words per second), the MFCC frame shift ratio is increased from 50% to 70% (frame shift = sampling rate x frame shift ratio, 16 kHz x 70% = 11.2 ms), increasing the frame overlap degree to ensure that adjacent syllable frames with fast pronunciation are not lost. For example, when a child says "naibu" (speech rate 10 words per second, syllable interval 100 ms), a frame overlap of 70% can capture 3-4 frame features for each syllable. For high pitch (> 460 Hz), the number of mel filter banks is increased from 24 to 32 to expand the filter coverage of the high frequency band (4000-8000 Hz) and enhance the feature discrimination of children's high pitch. For example, the MFCC feature difference between xiong (450 Hz fundamental frequency) and neng (480 Hz fundamental frequency) is expanded from 0.1 to 0.3. For low signal-to-noise ratio (< 25 dB), the pre-emphasis coefficient is adjusted from 0.97 to 0.95 to enhance the suppression of 200-500 Hz low-frequency noise. An adaptive noise cancellation (ANC) mode is added to generate reverse sound waves to cancel noise in real time based on environmental noise samples collected by the microphone array, increasing the feature signal-to-noise ratio by 10-15 dB.

[0066] The CNN-LSTM network is incrementally trained using 50 pieces of children's speech data in the comprehensive analysis dataset (grouped by age), with a learning rate of 0.001, 10 training rounds, the first 2 layers of CNN frozen, and the 3rd layer and LSTM layer fine-tuned to adapt the speech recognition model to the adjusted acoustic features. For example, after training the high-pitch data of 3-5 year-old children, the phoneme recognition accuracy of the speech recognition model for 450-500 Hz pitch is improved by 28%. To address the low confidence problem in the recognition association error operation, the hybrid architecture acoustic model weight is increased from 0.6 to 0.7 to enhance the influence of phoneme probability on text generation and reduce language over-intervention. At the same time, the number of decoding candidate texts is increased from 5 to 8 to improve the probability of capturing the correct text.

[0067] Step 664, according to the corresponding relationship between the delay time and the motion path characteristics in the comprehensive analysis data set, the motion trajectory smoothness and the execution efficiency balance criterion of the path planning algorithm are adaptively adjusted; Specifically, it includes: first, analyze the corresponding relationship between the delay time and the path characteristics, the first step is to extract the path characteristics and the completion action delay time of each record in the data set, calculate the average delay time under different path characteristic combinations, such as path total length <1 meter, turning number ≤3 times, average curvature ≤0.1, the average delay time is 0.8 seconds; path total length 1-1.5 meters, turning number 4-6 times, average curvature 0.1-0.2, the average delay time is 1.2 seconds; path total length >1.5 meters, turning number >6 times, average curvature >0.2, the average delay time is 1.8 seconds; The second step is to determine the path characteristic threshold of delay exceeding, combined with the patience of children, the path characteristics corresponding to the average delay time >1 second are defined as high delay path characteristics, such as path total length >1.2 meters, turning number >4 times, average curvature >0.15; The third step is to analyze the current balance state of smoothness and efficiency, the original balance criterion gives priority to smoothness, the average curvature is controlled within 0.1, and the turning number is ≤3 times, but it causes the delay of some long paths (>1.2 meters) to exceed 1 second; And if you pursue efficiency too much, reduce the turning number, it will cause the average curvature to increase (>0.2), and the action will be harsh.

[0068] Then adjust the balance criterion and develop a hierarchical adjustment strategy, first, for high delay path characteristics, allow the turning number to be reduced by 1-2 times under the premise of ensuring the average curvature ≤0.18, shorten the delay by reducing the deceleration-acceleration process when turning; Second, for low delay path characteristics, maintain the average curvature ≤0.1, prioritize smoothness to ensure gentle action and meet the children's interactive scene; Third, set a dynamic curvature threshold, when the path passes through the children's active area, the average curvature is forced to be ≤0.12 to avoid the action amplitude being too large to hit the children; When the path is away from the children's area, the average curvature can be relaxed to 0.2 to improve efficiency, write the adjusted balance criterion into the parameter configuration file of the path planning algorithm, and automatically adapt according to the real-time path characteristics during planning.

[0069] Step 665, integrate the adjusted updated acoustic feature extraction parameters and the adaptively adjusted path planning balance criterion to form a continuously improved adaptive control mechanism; Specifically, it includes: first, establish an associated storage structure of parameters and criteria, store the adjusted acoustic feature extraction parameters and path planning balance criterion according to the update batch, each batch is associated with the corresponding comprehensive analysis data set, which is convenient for tracing the adjustment basis.

[0070] Then an automatic update triggering mechanism is constructed, and update triggering conditions are set. One is that the cumulative execution times reach 20 times, and 20 times of execution corresponds to 1-2 hours of interaction in a child use scenario, which can quickly capture the recent use rules. Two is that the occurrence rate of a certain type of misoperation increases by more than 10% compared with the previous batch, such as the position deviation misoperation increasing from 10% to 25%. Three is that the average value of the completion action delay time increases by more than 200 ms compared with the previous batch. When any condition is met, the data reanalysis-parameter adjustment-criterion update process is automatically triggered. First, steps 660-662 are re-executed to collect the latest 20 execution data and update the comprehensive analysis data set. Then, steps 663-664 are executed to adjust the acoustic parameters and balance criteria based on the new data set. Finally, the new parameters and criteria are marked as a new batch, and the old configuration is overwritten.

[0071] Finally, an effect verification link is established. After each update, 5 test actions are continuously executed to verify whether the misoperation occurrence rate decreases and the delay time shortens. If the verification is passed, the new configuration takes effect formally. If not, the configuration of the previous batch is rolled back, the data set is reanalyzed, the adjustment deviation is found, and the acoustic parameters and path criteria are integrated through the closed loop process of storage-triggering-updating-verification to form a continuously improved adaptive control mechanism, which ensures that the performance can be dynamically optimized as the child use scenario changes.

[0072] By monitoring the mechanical arm running data in real time, the core causes of delay and misoperation in the child interaction scenario are accurately captured, and the mapping relationship between misoperation and voice features is constructed. The acoustic parameters can be optimized, the recognition deviation caused by the child's voice characteristics and environmental interference can be reduced, the misoperation such as position deviation can be reduced, the robot can better understand the child's instructions, and the interaction continuity is improved.

[0073] As shown in Figure 2 The embodiment of the application also provides a voice data real-time control system of an AI toy companion robot, which comprises: A collection module is configured to collect user voice instructions and obtain a voice feature sequence. An adjustment module is configured to input the voice feature sequence into an end-to-end voice recognition model of AI for stream-based preliminary decoding to generate a candidate text sequence, and perform semantic adjustment on the candidate text sequence to obtain an initial recognition text. A screening module is configured to perform multi-dimensional stability evaluation on the initial recognition text, establish a credibility screening mechanism based on the evaluation result, and process the initial recognition text through the credibility screening mechanism to obtain a credible text. A coordination module is configured to dynamically adapt the voice input blocking strategy according to the credible text, synchronously adjust the coordination working mechanism between connection time sequence classification decoding and attention re-scoring, and generate a real-time semantic analysis result. The planning module is used for matching the real-time semantic analysis result with a preset action library to identify a user intention, and performing real-time obstacle avoidance planning by using an improved multi-strategy fusion fast extended random tree algorithm to generate a robot arm motion path. The updating module is used for collecting delay data and misoperation events in an action execution process based on the robot arm motion path, and dynamically updating acoustic parameters of a speech recognition model and path planning strategies by using feedback information to construct a continuously improved adaptive control mechanism.

[0074] It should be noted that the system is a system corresponding to the above method, and all the implementation manners in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0075] Embodiments of the application also provide a computing device, comprising a processor and a memory storing a computer program, wherein the computer program is executed by the processor to perform the method described above. All the implementation manners in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0076] Embodiments of the application also provide a computer readable storage medium storing instructions, wherein the instructions are executed on a computer to make the computer perform the method described above. All the implementation manners in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0077] The above is the preferred embodiment of the application, and it should be noted that for those skilled in the art, without departing from the principles of the application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the application.

Claims

1. A voice data real-time control method for an AI toy companion robot, characterized by, The method comprises: Step 1, collecting user voice instructions, and obtaining a voice feature sequence; Step 2, inputting the voice feature sequence into an end-to-end speech recognition model of AI for stream-based preliminary decoding to generate a candidate text sequence, and performing semantic adjustment on the candidate text sequence to obtain initial recognition text; Step 3, performing multi-dimensional stability evaluation on the initial recognition text, establishing a credibility screening mechanism based on the evaluation results, and processing the initial recognition text through the credibility screening mechanism to obtain credible text; Step 4, dynamically adapting the voice input blocking strategy according to the credible text, synchronously adjusting the cooperative working mechanism between connection time sequence classification decoding and attention re-scoring, and generating real-time semantic analysis results; Step 5, matching the real-time semantic analysis results with a preset action library to identify user intent, and using an improved multi-strategy fusion fast random expansion tree algorithm to perform real-time obstacle avoidance planning to generate a robot arm movement path; Step 6, collecting delay data and misoperation events during action execution based on the robot arm movement path, and using feedback information to dynamically update the acoustic parameters of the speech recognition model and the path planning strategy to build a continuously improved adaptive control mechanism. 2.The voice data real-time control method of the AI toy companion robot according to claim 1, wherein, The method comprises: inputting the voice feature sequence into a stream-based decoder for real-time analysis to generate intermediate decoding results including multiple candidate words and time boundaries; based on the intermediate decoding results, performing time window alignment and path merging processing to form a candidate text sequence including multiple candidate paths; inputting the candidate text sequence into a pre-trained semantic understanding model for context analysis to identify syntax errors and semantic ambiguity segments and generate annotation results; based on the annotation results, performing replacement, deletion or reordering operations on the error and ambiguity segments in the candidate text sequence to form adjusted text that is grammatically correct and semantically coherent; performing language fluency verification and format regularization on the adjusted text to obtain initial recognition text that conforms to natural language expression specifications. 3.The voice data real-time control method of the AI toy companion robot according to claim 2, characterized in that, The method comprises: segmenting the initial recognition text to obtain multiple text segments, and performing confidence evaluation based on the text segments to generate an evaluation sequence; performing context coherence analysis using the text segments to calculate the semantic correlation between adjacent segments to generate coherence analysis results, and performing syntax structure detection on the text segments to identify syntax errors and generate error marker information; inputting the evaluation sequence, coherence analysis results and error marker information into a dynamic threshold calculation rule to generate a credibility threshold that adapts to the current text features; based on the credibility threshold, screening the text segments to retain segments that meet the threshold requirements to form screened text; The semantic integrity of the screened text is repaired by supplementing necessary words and adjusting the word order to restore semantic coherence, and the repaired text is subjected to boundary smoothing processing to generate a reliable text that conforms to semantic and grammatical norms. 4.The voice data real-time control method of the AI toy companion robot according to claim 3, characterized in that, According to the reliable text, the speech input block strategy is dynamically adapted, and the cooperative working mechanism between the connection time sequence classification decoding and the attention re-scoring is synchronously adjusted to generate real-time semantic analysis results, including: According to the text length and semantic complexity of the reliable text, the block size and overlap rate of the speech input are dynamically adapted to generate an adapted speech block sequence; Based on the adapted speech block sequence, connection time sequence classification decoding processing is performed to generate an initial decoding text sequence; According to the semantic consistency between the initial decoding text sequence and the reliable text, the participation coefficient of the attention re-scoring mechanism is calculated, and the participation degree of the attention re-scoring mechanism in the decoding process is adjusted based on the participation coefficient; The initial decoding text sequence is subjected to semantic calibration processing through the adjusted attention re-scoring mechanism to generate an adjusted decoding text sequence; The adjusted decoding text sequence is subjected to real-time semantic analysis to generate real-time semantic analysis results. 5.The voice data real-time control method of the AI toy companion robot according to claim 4, characterized in that, According to the semantic consistency between the initial decoding text sequence and the reliable text, the participation coefficient of the attention re-scoring mechanism is calculated, and the participation degree of the attention re-scoring mechanism in the decoding process is adjusted based on the participation coefficient, including: The initial decoding text sequence and the reliable text are compared in terms of semantic units to identify the text units with semantic differences between them, and the number and distribution characteristics of the semantic difference text units are recorded; Based on the number and distribution characteristics of the semantic difference text units, the semantic consistency quantitative score is calculated through the preset evaluation rules; According to the semantic consistency quantitative score, the corresponding attention re-scoring participation coefficient is obtained by querying the preset participation coefficient correspondence table; According to the specific value of the attention re-scoring participation coefficient, the participation degree of the attention re-scoring mechanism is determined based on the preset configuration strategy. 6.The voice data real-time control method of the AI toy companion robot according to claim 5, characterized in that, The real-time semantic analysis results are matched with the preset action library to identify user intent, and an improved multi-strategy fusion fast random expansion tree algorithm is used to perform real-time obstacle avoidance planning to generate a robot arm motion path, including: The real-time semantic analysis results are subjected to word segmentation processing to obtain a word sequence, and the action keywords are extracted based on the word sequence to identify the core action elements of the user instruction; The core action elements are matched with the instruction structures stored in the preset action library to determine the target action instruction, and the corresponding robot arm target position and motion parameters are obtained according to the target action instruction; Based on the robot arm target position and the real-time data collected by the environment sensor, a current environment map including obstacle spatial distribution information is constructed; Based on the robot arm target position, motion parameters and current environment map, an improved multi-strategy fusion fast random expansion tree algorithm is used for path planning; During path planning, the sampling strategy is dynamically adjusted according to the environmental complexity to generate an initial motion path, and collision detection is performed on the initial motion path to identify potential collision risk areas to obtain collision detection results; Based on the collision detection result, the trajectory of the risk area is adjusted, the safe motion path is generated, and whether the safe motion path meets the motion parameter requirement is verified, and the final mechanical arm motion path is generated. 7.The voice data real-time control method of the AI toy companion robot according to claim 6, characterized in that, Based on the mechanical arm motion path, delay data and misoperation events are collected during the action execution process, and the acoustic parameters of the speech recognition model and the path planning strategy are dynamically updated using feedback information to build a continuously improving adaptive control mechanism, including: Monitoring the actual running condition of the mechanical arm executing the planned motion path, obtaining motion trajectory information with accurate time marking and path execution deviation information; Analyzing the motion trajectory information to obtain delay time statistical results from the issuance of the voice instruction to the arrival of the mechanical arm at each key action node; and analyzing the path execution deviation information to obtain classified and sorted misoperation type statistical results; Integrating the delay time statistical results and the misoperation type statistical results to build a comprehensive analysis data set reflecting the correlation characteristics of delay and misoperation; Based on the mapping relationship between the misoperation types in the comprehensive analysis data set and the voice features, adjusting and updating the acoustic feature extraction parameters of the speech recognition model; According to the corresponding relationship between the delay time in the comprehensive analysis data set and the motion path features, adaptively adjusting the motion trajectory smoothness and execution efficiency balance criteria of the path planning algorithm; Integrating the adjusted and updated acoustic feature extraction parameters and the adaptively adjusted path planning balance criteria to form a continuously improved adaptive control mechanism.

8. A voice data real-time control system for an AI toy companion robot, the system implementing the method of any one of claims 1 to 7, characterized in that, It includes: A collection module for collecting user voice instructions and obtaining voice feature sequences; An adjustment module for inputting the voice feature sequence into an AI end-to-end speech recognition model for stream-based preliminary decoding to generate a candidate text sequence, and performing semantic adjustment on the candidate text sequence to obtain an initial recognition text; A screening module for performing multi-dimensional stability evaluation on the initial recognition text, establishing a credibility screening mechanism based on the evaluation results, and processing the initial recognition text through the credibility screening mechanism to obtain a credible text; A coordination module for dynamically adapting the voice input blocking strategy according to the credible text, synchronously adjusting the coordination working mechanism between connection time sequence classification decoding and attention re-scoring to generate real-time semantic analysis results; A planning module for matching the real-time semantic analysis results with a preset action library to identify user intent, and executing real-time obstacle avoidance planning using an improved multi-strategy fusion rapid expansion random tree algorithm to generate a mechanical arm motion path; An update module for collecting delay data and misoperation events during the action execution process based on the mechanical arm motion path, and dynamically updating the acoustic parameters of the speech recognition model and the path planning strategy using feedback information to build a continuously improving adaptive control mechanism.

9. A computing device, comprising: It includes: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program is executed by the processor to implement the method in any one of claims 1 to 7.

Images