A classroom interaction-oriented educational robot semantic understanding method

Abstract

The application discloses a classroom interaction-oriented educational robot semantic understanding method, relates to the technical field of classroom interaction semantic understanding, and comprises the following steps: a terminal first performs hardware clock strong latching on state discrete codes corresponding to environmental audio and teaching display interface teaching plan nodes, and generates binding records; then, the state discrete codes are transformed into pre-constraint penalty tensors and injected into a semantic model hidden layer to inhibit candidate intentions inconsistent with the current teaching plan node; when a high-confidence intention has a spatial entity slot defect, a control action is suspended and an abnormal defective suspension signaling stack package carrying a multi-source space arrival polar angle matrix is generated; subsequently, a space perception probe group is woken up to position a target entity of the teaching display interface and backfill the spatial entity slot, the control action is released, and a half-life damping control constant is updated according to a positioning cost. The method can improve classroom semantic attribution accuracy, reduce the risk of misexecution, and enhance the interaction stability in the classroom scene.

Description

Technical Field

[0001] This invention relates to the field of semantic understanding technology for classroom interaction, specifically to a semantic understanding method for educational robots oriented towards classroom interaction. Background Technology

[0002] Classroom interactive semantic understanding is typically set up in environments using electronic whiteboards, teaching display terminals, and classroom service robots. It receives voice input from teachers or students and drives interactive processes such as display interfaces, annotation windows, courseware page turning, content retrieval, or classroom auxiliary responses.

[0003] The existing main technical approach is to acquire voice data from the terminal, convert the voice into text commands, and then call the display interface or terminal application to perform operations based on the converted text commands, which mainly focuses on voice command recognition and interface function invocation.

[0004] Chinese patent document CN107153499A discloses a voice control method and apparatus for an interactive whiteboard device. The interactive whiteboard device described in this document comprises one or more processors and a non-transient computer-readable medium storing instructions. During the operation of the interactive whiteboard device, when the annotation window is running, the device detects voice input sent by the user, records it as an audio packet, and sends the audio packet to a speech-to-text service. The speech-to-text service outputs a transcribed command string corresponding to the voice input. The terminal parses the command string according to the voice pattern command processing in the command processor to obtain an executable command that can be executed by the interactive whiteboard device, and then calls the application to execute the executable command. Therefore, the processing flow of this prior art revolves around audio acquisition, text transcription, command recognition, and application execution; it is an interactive control scheme based on voice command-driven display device functionality.

[0005] While the aforementioned existing technologies can facilitate smooth human-computer interaction in general voice command control scenarios, they exhibit significant limitations in classroom interaction. Classroom scenarios are characterized by frequent changes in teaching pages, multiple simultaneous voices, complex environmental noise sources, and spatial referential phrases such as "this," "that," and "this side" in student voices. However, existing technologies merely identify executable commands based on the transcribed command string, lacking a stable relationship with the state of the teaching interface at the moment of speech, and lacking further confirmation of the specific display object pointed to by the statement. Mechanistically, speech-to-text processing compresses continuous speech into a linear text sequence. The command processor directly triggers application execution based on this text sequence. When the page state changes, the speaker's location shifts, or the statement itself relies on spatial referencing, the text command can easily become disconnected from the actual teaching object. If this issue is not addressed promptly, the terminal may experience semantic attribution problems, incorrect execution objects, or interrupted interactive responses, affecting the continuity and reliability of the classroom interaction.

[0006] Therefore, how to improve the semantic accuracy of voice input during page switching, noise interference, and spatial reference in classroom interaction has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides a semantic understanding method for educational robots in classroom interaction. The method transforms discrete state codes into pre-constraint penalty tensors and injects them into the hidden layer of the semantic model, suppressing candidate intentions inconsistent with the current lesson plan node. When a high-confidence intention has incomplete spatial entity slots, the control action is suspended, and an abnormal incomplete suspension signaling stack packet carrying a multi-source spatial arrival polar angle matrix is ​​generated. Subsequently, the spatial perception probe group is activated to locate the target entity on the teaching display interface and fill the spatial entity slots, releasing the control action and updating the half-decay debouncing control constant based on the location cost. This method improves the accuracy of classroom semantic attribution, reduces the risk of erroneous execution, and enhances the stability of interaction in classroom scenarios; it also solves the technical problems described in the background art.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A semantic understanding method for educational robots oriented to classroom interaction, executed by a terminal device, includes: concurrently collecting environmental audio and obtaining a discrete code representing the current lesson plan node of the teaching display interface, and generating a binding record based on the hardware clock when the time stamp difference between the two falls within the first reference time tolerance window;

[0012] The state discrete code is transformed into a pre-constraint penalty tensor consistent with the hidden layer topology corresponding to the ambient audio; the pre-constraint penalty tensor is injected when performing hidden layer operations of the semantic model based on the ambient audio to suppress candidate intentions inconsistent with the state discrete code.

[0013] When a high-confidence intent contains a missing spatial entity slot, the control action corresponding to the high-confidence intent is suspended and an abnormal incomplete suspension signaling stack packet carrying a multi-source spatial arrival polar angle matrix is ​​generated. In response to the abnormal incomplete suspension signaling stack packet, the spatial perception probe group is awakened, the target entity on the teaching display interface is located according to the multi-source spatial arrival polar angle matrix, the spatial entity slot is backfilled, and the control action is released.

[0014] Furthermore, generating the binding record includes: extracting the first frame timestamp of the ambient audio, extracting the received timestamp of the status discrete code, and writing the first frame timestamp, received timestamp, session identifier, and status discrete code into the high-speed register area; when the timestamp difference exceeds the first reference time tolerance window, the corresponding status discrete code is discarded.

[0015] Furthermore, transforming the state discrete code into a pre-constraint penalty tensor includes: performing node order mapping and one-hot encoding on the state discrete code, performing smoothing processing on the one-hot encoding result based on the half-decay dejitter control constant, and broadcasting extension along the time step dimension and hidden layer dimension of the ambient audio to generate a pre-constraint penalty tensor consistent with the hidden layer topology of the semantic model.

[0016] Furthermore, the injection of the pre-constraint penalty tensor includes: before the semantic model performs hidden layer operations, performing element-wise suppression processing on the pre-constraint penalty tensor and the hidden layer feature vector at the corresponding time step, and sending the processed hidden layer feature vector into the normalization and intent aggregation link to output a high-confidence intent.

[0017] Furthermore, the suspension control action includes: detecting spatial pronoun items in the current high-confidence intent based on the fuzzy spatial generic pronoun blocking dictionary, and freezing the control action when the spatial entity slot is empty; generating an abnormally incomplete suspension signaling stack packet including writing a deadlock flag field, a snapshot time field, a multi-source spatial arrival polar angle matrix, and an empty slot identifier.

[0018] Furthermore, the state discrete code is obtained through MQTT lightweight messages. The payload of the MQTT lightweight message adopts the JSON object representation format and includes at least a session identifier field and a node status field. The terminal device performs session consistency verification between the state discrete code and the ambient audio based on the session identifier field.

[0019] Furthermore, the abnormally incomplete suspended signaling stack packet adopts a JSON object representation format and is written to a high-privilege memory area, and at least includes a deadlock flag field, a snapshot time field, a sound source orientation field corresponding to the multi-source spatial arrival polar angle matrix, a space entity slot vacancy identifier, and a frozen action sequence identifier.

[0020] Furthermore, in the silent testing scenario, after suspending the control action, the terminal device sends an encrypted UDP message to the teacher's central control terminal. The encrypted UDP message contains at least a session identifier, an empty slot identifier, and a snapshot time field, and disables the output of prompts from the local speaker.

[0021] Furthermore, the abnormal and incomplete suspended signaling stack packets are stored consecutively in the high-privilege memory area in the header segment, slot segment, and orientation segment. The header segment stores the deadlock flag field and the snapshot time field, the slot segment stores the empty slot identifier, the orientation segment stores the multi-source spatial arrival polar angle matrix, and a check field is set in the header segment.

[0022] Furthermore, the first reference time tolerance window is dynamically adjusted based on the bus input / output load occupancy rate; the terminal device reads the bus input / output load occupancy rate and the proportion of recent misalignment records in each adjustment cycle, and adjusts the first reference time tolerance window based on the results of dual closed-loop proportional-integral-derivative control.

[0023] Furthermore, the half-decay de-jitter control constant is dynamically updated based on the ambient reverberation time; the terminal device obtains the ambient reverberation time through the speaker retrace signal on the teaching display interface, and adjusts the tail length of the smoothing process based on the ambient reverberation time.

[0024] Furthermore, the spatial perception probe group includes a millimeter-wave radar antenna array and a forward-looking vision lens; the positioning of the target entity of the teaching display interface based on the multi-source spatial arrival polar angle matrix includes: obtaining the skeleton ray vector of the speaker along the corresponding direction of the multi-source spatial arrival polar angle matrix, determining the two-dimensional hit coordinates of the teaching display interface based on inverse perspective intersection, and then determining the target entity of the teaching display interface based on the two-dimensional hit coordinates.

[0025] Furthermore, after backfilling the spatial entity slot, the terminal device measures the addressing delay cost and multipath offset cost of this round of positioning process, updates the half-decay debouncing control constant based on the addressing delay cost and multipath offset cost, and writes the updated half-decay debouncing control constant into the local parameter area for use in the next round when generating the pre-constraint penalty tensor.

[0026] (III) Beneficial Effects

[0027] This invention provides a semantic understanding method for educational robots oriented towards classroom interaction, which has the following beneficial effects:

[0028] By binding the ambient audio to the discrete state code corresponding to the current lesson plan node on the teaching display interface within a first reference time tolerance window, the spatiotemporal correspondence between the speech and the teaching process is completed before semantic processing, reducing semantic attribution offsets caused by page or bus delays or stray codes. By converting the discrete state code into a pre-constraint penalty tensor consistent with the hidden layer topology of the ambient audio and injecting it into the hidden layer operations of the semantic model, branches inconsistent with the current lesson plan node can be compressed before candidate intents diffuse, and high-confidence intent feature clusters converge around the teaching content.

[0029] Control actions are frozen at incomplete spatial entity slots, generating abnormally incomplete suspended signaling stack packets with multi-source spatial arrival polar angle matrices. This transforms semantically unclosed states into intermediate states that can continue processing, preventing the educational robot from blindly executing actions when the object is unknown. The spatial perception probe group locates the target entity on the teaching display interface based on the multi-source spatial arrival polar angle matrix, and the location results are filled into the spatial entity slots. This compensates for the spatial disambiguation link, which is insufficient to be completed by voice alone, allowing previously suspended control actions to be released only after an object is identified.

[0030] By writing back the time delay cost and multipath offset cost formed in this round of positioning to the half-decay debouncing control constant, the subsequent manufacturing process of the pre-constraint penalty tensor is kept in touch with the classroom conditions. The semantic entry is corrected in reverse by the blinding result. The first reference time tolerance window, the pre-constraint penalty tensor, the abnormal and incomplete suspended signaling stack packet and the spatial perception probe group are connected in series into a continuous control chain of forward convergence and backward blinding. This increases the pertinence of the understanding of strong classroom semantics and the stability of the educational robot in handling fuzzy references, page switching and object missing situations. Attached Figure Description

[0031] Figure 1 This is a diagram illustrating the overall architecture of the classroom interaction closed-loop system for educational robots in this embodiment of the invention.

[0032] Figure 2 This is a flowchart illustrating the strong latch binding of the initial audio feature sequence and the state discrete code in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram illustrating the generation of a pre-constraint penalty tensor from a state discrete code in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of injecting pre-constraint penalty tensors into hidden layers and performing exclusive hard collapse pruning in an embodiment of the present invention;

[0035] Figure 5 This is a flowchart of the business slot closure determination and abnormal incompleteness suspension processing in an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of cross-modal addressing and spatial entity backfilling after abnormal suspension in an embodiment of the present invention;

[0037] Figure 7 This is a closed-loop diagram illustrating the blinding cost feedback and the reverse rewriting of preceding parameters in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figures 1-7 This invention provides a semantic understanding method for educational robots oriented towards classroom interaction, comprising:

[0040] Step one of this implementation is executed by the educational robot terminal, with the pickup array front end, bypass listening port, real-time clock (RTC), clock latch register, and exception discard routine working together. It first pushes the initial audio feature sequence and the discrete state code corresponding to the current lesson plan node on the electronic whiteboard into the same hardware time base, and then hands it over to the subsequent step two for topology expansion and constraint propagation.

[0041] Step 1: Place the initial audio feature sequence collected in the classroom and the discrete state code played on the electronic whiteboard in the same hardware time coordinate to complete strong latch binding, thus eliminating the temporal ambiguity of whether the audio belongs to the current page or the previous page.

[0042] In classroom scenarios, when students verbally ask questions like "How do I solve this?" or "Why do I need to rearrange terms on the blackboard?", the content of the speech alone cannot determine the object being referred to. What truly determines semantic attribution is not the word itself, but rather the current teaching plan node the electronic whiteboard is at the moment of the sound. If the whiteboard has already switched from explaining an example problem to classroom practice, but the voice acquisition still uses the state of the previous node, then even using a complex neural network will only result in a fine calculation of the incorrect physical scenario. Therefore, step one first addresses the problem of aligning the physical starting points of heterogeneous control flows: one is a continuous acoustic flow, and the other is a discrete teaching node control flow. The former comes from the front end of the microphone array, and the latter comes from the bypass listening port; the two are inherently asynchronous, and network congestion, whiteboard broadcast jitter, and robot local thread scheduling can all cause misalignment between them.

[0043] This implementation uses the real-time clock (RTC) on the motherboard as the sole time base. Its principle does not rely on upper-layer software timestamps; instead, it directly uses an incrementing counter register driven by a quartz crystal oscillator to record the arrival time of events. Software thread switching widens the recording time for the same event, while the RTC's writing of data to the clock latch register occurs near the interrupt entry point, resulting in a short recording path. This allows the first audio frame and the status discrete code to be pressed onto the same physical scale.

[0044] After the educational robot terminal is powered on, the front end of the microphone array and the bypass listening port are started by the same startup routine.

[0045] In a preferred embodiment, the microphone array is positioned within an arc-shaped shell at the front edge of the robot's head. It employs four to six MEMS (Micro-Electro-Mechanical Microphones) soldered onto a fiberglass epoxy board (FR-4). The center-to-center distance between adjacent microphones is 20 to 35 millimeters, and the arc-shaped arrangement ensures that the students and the podium are both within the main pickup sector. The microphone output first undergoes analog anti-aliasing filtering before entering an analog-to-digital converter to form a digital audio stream. This digital audio stream is then framed, windowed, and beamformed in a digital signal processing (DSP) routine. Subsequently, the first effective sound segment is extracted based on the speech initiation gating, generating an initial audio feature sequence. The initial audio feature sequence is preferably a framed Mel-filtered energy sequence, but it can also be replaced with a logarithmic spectral sequence, as long as its first frame retains a clear start time.

[0046] Meanwhile, the bypass listening port does not participate in whiteboard control, but intercepts external electronic whiteboard broadcast messages without altering the existing teaching network link. Preferably, the electronic whiteboard communicates with the classroom IoT bus using the Message Queuing Telemetry Transport Protocol (MQTT), with the message payload being a JSON object (JavaScript object notation), which fixedly includes a session identifier field and a node status field. The bypass listening port operates in promiscuous reception mode, only copying messages and not writing them back; once it parses the session identifier field consistent with the current session, it sends the corresponding node status field as a status discrete code into the shared circular buffer. If the terminal uses a wired bus, mirroring interception can also be performed on the Ethernet physical layer transceiver (PHY) side; if the terminal uses a real-time operating system (RTOS) motherboard, the interrupt service routine directly writes the parsing result into the statically allocated buffer.

[0047] For example, when the teacher switches to the fractional equation solving page, the electronic whiteboard broadcasts a new node status field; the student then immediately explains why a common denominator is needed. At this moment, the microphone array front end first receives the student's voice at the leading edge of the head, and the bypass listening port reads the node status broadcast by the whiteboard almost simultaneously. The two records are not directly handed over to the natural language recognition link yet, but are first sent to the same clock latch register to await further verification.

[0048] Furthermore, the subsequent step two receives not isolated speech, but all speech from the fractional equation solution node. If the student hasn't spoken yet, the whiteboard page is turned, and the bypass listening port only caches the discrete state code without generating a separate strong latch binding record. Once the first valid speech frame appears, the subsequent range check begins.

[0049] When the microphone array front end writes the voice start gating to the first frame, the interrupt entry reads the real-time clock (RTC) to obtain the audio start time. After the bypass listening port parses the discrete state code and performs a session consistency check, it simultaneously reads the real-time clock (RTC) to obtain the IoT receiving time. .

[0050] Both then enter the range verification routine, whereby the state discrete code can be strongly latched and bound to the initial audio feature sequence only after verification. Their determination relationship is as follows:

[0051]

[0052] In the formula, the audio start time is... The timestamp written to the clock latch register when the front end of the pickup array completes the sealing of the first effective frame, with a value range of non-negative microsecond counts that can be represented by the real-time clock (RTC) within the current teaching session;

[0053] IoT receiving time The timestamp written to the clock latch register after the bypass listening port completes the status discrete code parsing, with a value range corresponding to the audio start time. same;

[0054] First reference time tolerance window : The maximum time difference that allows audio start-point and status discrete codes to enter the same binding batch, with a value range of positive microsecond window between the preset lower limit and the preset upper limit, used to block network late packets, thread queued packets and cross-page residual packets;

[0055] Absolute value operator It retains only the amplitude of the time difference without distinguishing the order of arrival, and is used to uniformly handle the two common classroom orders: speech arrives first and state discrete code arrives last, and state discrete code arrives first and speech arrives last.

[0056] If the relationship is not satisfied, the exception discard routine immediately marks the direct memory access (DMA) descriptor containing the corresponding state discrete code as invalid and unattaches the record from the shared circular buffer, preventing it from entering the subsequent binding table. If the relationship is satisfied, a record to be bound is written from the clock latch register, which contains the initial audio feature sequence index, the state discrete code index, and two time values. The advantage of this approach is that subsequent step two no longer deals with the fuzzy relationship of which speech corresponds to which node, but rather with a deterministic relationship that has already been filtered by the physical time base.

[0057] To avoid misclassifying legitimate discrete codes as misaligned packets during short-term bus congestion, this implementation does not fix the first reference time tolerance window as a single constant. Instead, it is tuned in each adjustment cycle based on the bus occupancy status and recent misalignment accumulation. The outer loop reads the bus input / output occupancy rate, and the inner loop reads the misalignment error ratio; both determine the expansion / contraction direction of the first reference time tolerance window. The preferred tuning relationship is as follows:

[0058]

[0059] In the formula, the current period tolerance window : No. The first baseline time tolerance window for actual input range verification within each adjustment cycle is a positive value between the minimum and maximum window limits; the baseline tolerance window... The initial tolerance window written into the parameter table when the system is shipped or deployed, with a value range of positive microsecond window limits; load factor. Bus I / O occupancy rate is the weighting factor applied to the tolerance window, and its value range is a positive real number; bus occupancy rate : No. The busy-idle ratio of the bus where the bypass listening port is located within each adjustment cycle, with a value range of 0 to 1, is used to characterize the degree of network congestion.

[0060] Cumulative coefficient The recent misalignment error ratio is the weighting factor applied to the tolerance window, with a value range of positive real numbers. It is used to avoid frequent false positives of valid discrete codes during continuous jitter. : No. The proportion of candidate binding records discarded due to range windowing within a single adjustment period, with a value ranging from 0 to 1 within a closed interval; sensitivity coefficient. The response weight of the change in the ratio of misalignment error between two adjacent adjustment cycles, with a value range of positive real numbers, is used to slow down the interception slope in time when the misalignment suddenly increases and to tighten the tolerance window in time when the misalignment falls back quickly.

[0061] Looking back at the length The misalignment error ratio is calculated cumulatively to cover the number of historical periods, with a value range of positive integers, used to control the inner loop memory depth; minimum window limit. The tolerance window is allowed to shrink to a lower bound, with a value range of positive microseconds; the maximum window limit. The tolerance window is allowed to be magnified to its upper limit, and its value range is a positive microsecond window limit that is greater than the minimum window limit. Cutting operator Limit the tuning result to between the minimum and maximum window limits;

[0062] In this scenario, when multiple terminals simultaneously request resources from the whiteboard within the classroom, the bypass monitoring port observes an increase in bus occupancy, and the first reference time tolerance window is appropriately widened. As congestion subsides and the misalignment error ratio decreases, the first reference time tolerance window gradually returns to a tighter range. This preserves the valid discrete code while preventing the delayed broadcast of the previous lesson plan node from being incorporated into the current audio segment. Its beneficial effects correspond point-by-point to the parameters in the above formula: bus occupancy... The decision was made to relax the direction, and the misalignment error ratio The cumulative adjustment is determined by the variable term, which determines the rate of adjustment, while the boundary term is responsible for keeping the tuning result within the stable range.

[0063] In actual classrooms, whiteboards sometimes display two discrete state codes consecutively within a very short period, for example, when the teacher flips through a page and immediately expands the annotations. In this case, simply knowing that a certain discrete state code has entered the first reference time tolerance window is not enough; it is also necessary to select the record closest to the start time of the audio from within the window.

[0064] This implementation performs the most recent time selection on the candidate state discrete code set within the window under the same session identifier:

[0065]

[0066] In the formula, the target state discrete code The final state discrete code bound to the initial audio feature sequence takes values ​​within the set of all valid state discrete codes in the current session, and is used to generate the unique physical node entry point for the subsequent step two; candidate set. The set of all discrete codes with consistent session identifiers within the first reference time tolerance window, whose value range is a subset of discrete codes verified by range within the current binding period; candidate time. Candidate set The IoT receiving time corresponding to any state discrete code, and the value range of the IoT receiving time. same;

[0067] Once the target state discrete code Once selected, a strong latch binding record is generated in the clock latch register. This binding record consists of four parts: the initial audio feature sequence index, the target state discrete code index, the audio start time, and the IoT reception time. After this record is written to the binding table, subsequent step two only reads the binding table and does not revisit the original message. This ensures that the interface between steps remains unambiguous; step two always receives input indicating that a certain audio segment belongs to a specific lesson plan node, rather than a loose relationship of an audio segment with several possible nodes.

[0068] Continuing with the aforementioned classroom scenario, if the teacher switches from the fractional equation solution page to the annotation page immediately afterward, the whiteboard will briefly broadcast two discrete state codes. When a student's question falls between these two broadcasts, the nearest-moment selection will retain the discrete state code that is closer to the moment the student speaks, thus ensuring that the subsequent semantic chain is based on the whiteboard content that the student is facing at the moment of asking the question, rather than the annotation overlay that appears later.

[0069] When the IoT link is disconnected and the bypass listening port cannot obtain the status discrete code, this implementation method initiates a parallel alternative path.

[0070] The wide-angle camera mounted on the chest or head of the educational robot terminal performs fixed-frequency snapshots of the electronic whiteboard under a fixed exposure period. The camera preferably uses a global shutter complementary metal-oxide-semiconductor CMOS sensor, with a lens focal length of 2.8 mm to 4 mm to cover the main screen area of ​​the podium. The edge character recognition (OCR) routine extracts the text identifier of the current node from the image, and then converts the node text identifier into a state discrete code using a local mapping table; this state discrete code is no longer used with the IoT receiving time. Instead, the exposure time is written to the clock latch register and then enters the same range verification link mentioned above.

[0071] For example, during the maintenance of the local network in the teaching building, the robot did not receive the whiteboard broadcast. The camera could only be pointed at the title of Newton's First Law at the top of the screen. The edge character recognition OCR routine read the title and mapped it to the corresponding discrete state code. The clock latch register was aligned with the first frame of the student's voice at the time of this exposure. Therefore, visual substitution cannot change the goal of step one, nor can it change the object read in the subsequent step two; it only changes the source of the discrete state code.

[0072] When both the bypass listening port and the wide-angle camera fail simultaneously, a fallback path is entered. The educational robot terminal downloads a static configuration file of the teaching timeline from local non-volatile memory, preferably a read-only jsOn file, which stores the teaching session identifier, the state discrete code corresponding to the teaching time period, and the chapter order. Based on the current running time of the real-time clock (RTC), the system searches for the current time period in the static configuration file of the teaching timeline and extracts the corresponding state discrete code for binding. This path does not require external network or visual input, but uses a chain of terms: state discrete code - time - strong latch binding record.

[0073] Specifically, the output interfaces of the main path, visual substitution path, and static blind induction path are consistent, and there is no need to set entry points based on different sources. The following step two is completed by the educational robot terminal, and the input objects are the initial audio feature sequence output from step one and the state discrete code strong latch binding record.

[0074] Step 2: Transform the single-dimensional discrete state code released in Step 1 into a pre-constraint penalty tensor that is topologically isomorphic to the hidden layer of the initial audio feature sequence, so that what is received in Step 3 is not an isolated state code, but a continuous constraint base that can be directly pressed down on non-associated intention branches.

[0075] The discrete state code output in step one is suitable for page switching and device linkage, but not for directly feeding into the hidden layer entry in step three. The reason is that the discrete state code is abruptly changing single-valued square wave, while the initial audio feature sequence is a multi-timestep acoustic vector obtained by continuously dividing the frame. If directly connected, the former lacks structural positions for element-wise connection, while the latter cannot withstand hard boundaries without transition. When a student asks a question the instant they turn a page, the natural connection between the previous and next nodes is disrupted.

[0076] Step two involves transforming the discrete state code into a continuous shape acceptable to the hidden layer. The processing chain expands unidirectionally: first, the discrete state code is changed to a fixed position using the node sequence table; then, basis vectors are generated by one-hot projection; finally, a Gaussian smooth landing is controlled by a half-decay debouncing control constant, and a pre-constraint penalty tensor is formed by broadcasting and stretching along the time step length and the hidden layer dimension.

[0077] The edge computing board of the educational robot terminal is connected to the clock latch register area in step one via an on-chip bus. After the clock latch register area submits a strong latch binding record, the node sequence correction routine first reads the state discrete code. Then, the corresponding node position is retrieved from the node sequence table in non-volatile memory. The node sequence table preferably stores three types of fields: session identifier, node status, and node sequence index, arranged in segments according to the teaching session. The node sequence correction routine first locks the index segment based on the session identifier field, and then finds the corresponding node sequence index based on the node status field.

[0078] After locating the node, the one-hot projection routine writes the result to the state vector cache page. The cache pages are preferably arranged in contiguous address blocks, with each element stored in either a 16-bit or 32-bit fixed-point format for smooth sequential reading in subsequent processing. The projection relationship is written as:

[0079]

[0080] In the formula, the unique heat vector The basic state vector generated in step two is in the... The values ​​at each node position are within a discrete value set. , used to rewrite a single-valued discrete code into a vector shape that can participate in subsequent smoothing processing;

[0081] Node position : The position number of the current teaching session node in the sequence table, with a value range of positive integers. ;

[0082] Mapping function The node sequence correction routine performs a lookup-based sequence mapping on the state discrete code, with the state discrete code as input. The output is the node position. This function is used to convert hexadecimal discrete state codes into vector positions; in practice, this mapping function is implemented using a read-only index table lookup or a hash lookup.

[0083] State Discrete Code Step 1 outputs a single-dimensional physical state marker that is verified by the first reference time tolerance window. The value range is the set of legal node codes for the current teaching session.

[0084] Total number of nodes : The total number of lesson plan nodes registered in the current teaching session node sequence table, with values ​​ranging from positive integers;

[0085] For example, when the teacher switches the electronic whiteboard to the node of example two in the fractional equation, the discrete state code submitted in step one falls into the seventh node position after looking up the table. The one-hot projection routine writes the value 1 in the seventh position of the state vector cache page, and writes the value 0 in the other positions. Furthermore, node order correction ensures that the discrete state code and position index are consistent within the same teaching session, and one-hot projection sets the next calculation object as a fixed-length vector, leaving a structural entry point for step three to access the dimension by dimension.

[0086] The one-hot vector is still too sharp, with no buffer zone between adjacent nodes. When the teacher turns the page, erases, or inserts annotations, student questions often fall within a very short segment before and after the node switch; if the hidden layer constraints still only allow all-or-nothing, step three will result in excessive shearing when suppressing non-associative branches. To address this, step two concatenates a Gaussian smoothing landing routine after the one-hot projection, keeping the target node with the highest weight, while adjacent nodes decrease in a controlled manner along the node distance.

[0087] The Gaussian smooth landing routine directly reads the state vector cache page and generates a smooth state vector point by point. The half-decay debouncing control constant... When the value is small, the decrease near the target node is steeper; the half-decay debouncing control constant When the value is large, the decrease is more gradual. Its smoothing relationship can be written as:

[0088]

[0089] In the formula, the smooth state vector Gaussian smooth landing routine in the first The smoothed value output at each node position takes the value within a closed interval. Normalized real numbers within the range are used to transform single-point activations into a node distribution with gently descending edges; node positions The meaning, value range, and function of the aforementioned node positions are related to... Maintain consistency; mapping function The meaning, value range, and function of the aforementioned mapping function are the same. Maintain consistency; state discrete code The meaning, value range, and function are the same as those of the aforementioned discrete state codes. Maintain consistency;

[0090] Half-life debouncing control constant : A positive real number parameter that controls the spread width of the smooth state vector, and its value range is the range of positive real numbers. Used to determine the width of the transition zone around the target node; summation index The node traversal position in the normalized denominator, with values ​​ranging from positive integers. ;

[0091] For example, when a teacher transitions from explaining an example problem to a classroom exercise, a student might immediately ask why the expression needs a common denominator. The smoothed value at the target node is maximized, while adjacent nodes retain their residual values, decreasing sequentially with distance from the target node. (Half-decay debouncing control constant) Used to control the descent speed at the edges, smoothing the state vector. For the transitional distribution of continuous nodes, the combination of the two can offset the problem of discrete node jumps directly impacting the hidden layer.

[0092] The smoothed state vector is still a one-dimensional vector and cannot be directly used in step three because step three accesses a tensor entry organized by batch, time step, and hidden layer dimensions. Therefore, step two continues to perform broadcast stretching, ensuring that the same physical node a priori covers all time steps of the initial audio feature sequence, and then writes irrelevant positions as fixed negative extreme values. Furthermore, within the entire initial audio feature sequence corresponding to the same strongly latched binding record, the physical node remains consistent; therefore, the smoothed state vector is not re-estimated at each time step but is directly copied.

[0093] In one implementation, the edge computing board copies the smoothed state vector to contiguous memory blocks, with the memory order being batch-then-step, step-next-step, and then step-next-dimensional. If the inference engine uses a floating-point path, the pre-constraint penalty tensor is loaded using a 32-bit floating-point array; if the inference engine uses a low-precision path, it is loaded using a 16-bit floating-point array, and negative extreme values ​​are saturated and truncated before being written. The tensor writing relationship is as follows:

[0094]

[0095] In the formula, the pre-constraint penalty tensor The final output of step two, the 3D tensor ready to be passed to step three, is located in the batch position. Time step position Hidden layer location The values ​​are taken from the range of negative real numbers and the range of real numbers close to zero; batch position. Sample numbers in the same batch of inference tasks, with values ​​ranging from positive integers. Total number of batches The number of samples processed in parallel during the same round of inference, with values ​​ranging from positive integers; the time step position. : The frame position in the initial audio feature sequence, with values ​​ranging from positive integers. ;

[0096] Time step length : The total number of frames in the initial audio feature sequence, with values ​​ranging from positive integers; hidden layer position. : The feature dimension position in the hidden layer of the target, with values ​​ranging from positive integers. Hidden dimension The target hidden layer width connected in step three takes a value in the range of positive integers and satisfies the following conditions: Used to determine the length of the third dimension of the pre-constraint penalty tensor; smoothing the state vector. The aforementioned smoothed state vector in the th... The value at each node position has the following applicable scope: Used to provide a gradient penalty to the hidden layer location associated with the current physical node; protection constant. A positive real number protection term is added before the logarithmic mapping, with values ​​ranging from positive real numbers less than 1 to greater than 0; this blocks negative extreme values. Write a positive real constant that is independent of the tail dimension, with a value range that is sufficient to suppress the non-associative branches in step three;

[0097] In class, students continuously explain why fractions are simplified in the same area on the blackboard, and whether the denominator can be further simplified. Although the entire audio segment is divided into multiple time steps, the corresponding physics lesson plan nodes remain unchanged. Broadcast stretching therefore copies the same node priors to each time step position, rather than re-evaluating frame by frame.

[0098] Furthermore, the length of the time step With hidden layer dimensions Bidirectional expansion ensures that the pre-constraint penalty tensor corresponds to the entry shape in step three, while the protection constant guarantees the stability of the logarithmic mapping and blocks negative extrema. Irrelevant tail dimensions are pre-assigned to the strong penalty region.

[0099] In addition, step two includes three paths: the enhancement path, the parallel alternative path, and the degradation bearing path. All three paths expand the same object's pre-constraint penalty tensor without altering the reading semantics of step three. The enhancement path first utilizes the environmental reverberation time. Ambient reverberation time The degree of sound field trailing on node boundaries is determined by using short-pulse calibration sounds when the classroom is empty, the attenuation segment of the teacher's voice, or the sweepback signal from the whiteboard's built-in speaker. Subsequently, the half-decay de-jitter control constant is no longer fixed and is determined by the ambient reverberation time. and the first reference time tolerance window in step one Adjustment:

[0100]

[0101] In the formula, the updated half-life debouncing control constant is... The half-decay de-jitter control constant used in the Gaussian smooth landing routine under the enhanced path, with a value range of a closed interval. The positive real number within is used to jointly adjust the transition band width based on the sound field trailing and the release conditions in step one; the reference half-decay de-jitter control constant. : The initial half-decay debouncing control constant written into the parameter table during deployment, with a value range of positive real numbers; gain coefficient Ambient reverberation time Update the half-life debouncing control constant The influence weight, with a range of positive real numbers; environmental reverberation time The time parameter corresponding to the sound energy attenuation characteristics in the classroom sound field, with values ​​ranging from positive real numbers;

[0102] Coupling coefficient First reference time tolerance window Update the half-life debouncing control constant The influence weights, with values ​​ranging from positive real numbers; the first reference time tolerance window. Step 1 determines the tolerance window for whether the audio start time and the IoT reception time can be bound; the value range is positive real numbers; lower limit parameter. With upper limit parameter These represent the updated half-life debouncing control constants. The minimum and maximum allowable values ​​are both positive real numbers within the range and satisfy the following conditions: Cutting operator Limit the result within the parentheses to the lower bound parameter. With upper limit parameter Between these spaces, to prevent the node edges from spreading too narrowly or too wide;

[0103] In the parallel alternative path, if the terminal uses a low-end microcontroller motherboard without a configured floating-point concurrency unit, a complete three-dimensional array is not generated in the dynamic memory area. Instead, a lookup table of node sequence, page address start point, page offset, and blocking negative extreme value is pre-built in the read-only program memory. The node sequence correction routine reads the page address start point and page offset from the lookup table and submits this lookup relationship to the adaptation layer. The adaptation layer then outputs an address view equivalent to the pre-constraint penalty tensor logic to step three.

[0104] In the degraded bearing path, when memory usage is large, a complete Gaussian smooth landing is no longer performed. Instead, the nearest neighbor filling routine is used: instead of retaining the primary position corresponding to the node order, a fixed number of positions are copied to the secondary positions at the same level and written sequentially to the pre-constraint penalty tensor.

[0105] Enhance the path to increase environmental reverberation time With the first reference time tolerance window Integrate them together to ensure that steps one and two correspond to each other in terms of boundary scale; parallel alternative paths ensure that the low-end motherboard still contains constraint objects with the same semantics; the degraded bearing path only keeps the output port unchanged when memory is congested.

[0106] By employing node order correction, one-hot projection, Gaussian smooth landing, and broadcast stretching, the discrete state code submitted in step one is reconstructed into a pre-constraint penalty tensor. The pre-constraint penalty tensor conforms to the shape of the entry point in step three, leaves a gradual descent region on the boundary, and pre-sets blocking negative extrema in the irrelevant tail dimension. Thus, while representing the current physical lesson plan node, it can also rewrite the abrupt square wave into a continuous constraint shape that can be accepted by the hidden layer.

[0107] Step 3 below is executed by the educational robot terminal, and the executing entities include the edge computing board, on-chip static memory, attention computing unit, junction temperature probe, sound pressure monitoring register area and unmasked interrupt controller.

[0108] Step 3: Convert the pre-constraint penalty tensor Deep-dive injection of hidden layer deduction channels enables state discrete codes. Inconsistent candidate intent branches are first suppressed at the physical computing level, and then taken over by the hardware anti-damage link when the junction temperature and noise floor exceed the limits.

[0109] Step 2 delivery of the pre-constraint penalty tensor Topology expansion has been completed, but if the tensor is merely kept in the input buffer without entering the hidden scoring layer, non-associated word branches will still spread along the model's original global search path. Common classroom noises such as chair legs dragging, book turning, and whispered conversations with classmates create acoustically attached components accompanying the question fragments. If these attached components are not severed at the hidden layer entry point, step four will not receive the converged business intent, but rather a candidate set mixed with miscellaneous branches. Therefore, step three does not remedy this at the output end, but rather by applying the pre-constraint penalty tensor before attention or fully connected layer computation begins. By incorporating a scoring surface, the search space is initially narrowed, and then further progress is made through subsequent business analysis.

[0110] Specifically, the physical lesson plan nodes determined in step one and the continuous constraint shapes generated in step two are brought down to the energy allocation stage of the hidden layer. This can be understood as: instead of selecting answers after the model has finished calculating, it tells the model which channels should be darkened pre-emptively before it is ready to allocate attention. In this way, what the subsequent step four sees is a candidate intent that has already undergone spatial constraint, rather than a pre-open global intent pool.

[0111] The edge computing board of the educational robot terminal preferably adopts a heterogeneous motherboard integrating a central processing unit and a neural network accelerator. A copper-based heat sink is press-fitted onto the motherboard, with a preferred thickness of 1 mm to 3 mm. A graphite thermal conductive sheet with a preferred thickness of 50 μm to 200 μm is laid between the copper-based heat sink and the chip package. The purpose is to ensure that the continuous inference heat during the subsequent hard collapse pruning stage is promptly transferred to the vicinity of the junction temperature probe. Pre-constraint penalty tensor After writing to the on-chip static memory in step two, the hidden layer entry alignment routine first adjusts the batch size according to the total number of batches. Time step length and hidden layer dimensions The tensor shape is read and then rearranged to the same address step size as the current hidden layer scoring cache. In a preferred embodiment, the hidden layer scoring cache adopts a linear arrangement with batch priority, time step priority, and header position priority, and the tensor is pre-constrained and penalized. They are also moved in the same order, thus avoiding address conversion when injecting elements one by one.

[0112] In the software pipeline, the inference execution routine runs in an embedded Linux environment or a deterministic timing-based operating system environment with tensor instruction sets. The hidden entry alignment routine first receives the pre-constraint penalty tensor from the shared memory area in step two. Then, the tensor is written to the accompanying cache page before attention scoring; if the current model is at the entry point of a fully connected layer, the tensor is written to the accompanying cache page corresponding to the fully connected weight mapping table. The key to this is not the number of copies, but that the tensor is always penalized by the pre-constraints. It shares the same index axis as the current vector to be scored.

[0113] For example, while the teacher is explaining the derivation of Ohm's Law, the whiteboard nodes remain unchanged. A student asks why the resistor is written in the denominator, and at the same time, a student in the back row whispers that they'll play a song after class. The pre-constraint penalty tensor submitted in step two... The derivation of nodes based on Ohm's law has already been completed. Step three involves pasting this tensor into the accompanying cache page used by the current attention head, so that the hidden layer has a background color that only surrounds the current node before it begins to allocate attention.

[0114] Specifically, the coaxial rearrangement of the on-chip static memory ensures that the injection operation does not disrupt the original time step order, and the bit-by-bit loading of the cache pages ensures that subsequent element-by-element operations do not require additional table lookups, thus providing a definite entry point for the hard collapse pruning in the next technical point. After completing the hidden layer entry alignment, step three begins to load the pre-constraint penalty tensor. The data is directly fed into the scoring unit. In a preferred embodiment, if the model is currently performing self-attention calculation, the query matrix, key matrix, and value matrix are first obtained, and then the pre-constraint penalty tensor is superimposed on the scaling component. If the model is currently performing fully connected layer computation, then first take the hidden layer weight vector, and then use the Hadamard dot product to convert the pre-constraint penalty tensor. Transformed into a dimensionally repressive factor

[0115] Furthermore, in the attention path, step three preferably adopts the following scoring relationship:

[0116]

[0117] In this embodiment, the pre-constraint penalty tensor generated in step two corresponds one-to-one with the hidden layer feature vector of the current time step in the batch dimension, time step dimension, and hidden layer dimension. Step three does not rearrange the pre-constraint penalty tensor into a two-dimensional scoring matrix of query length multiplied by key length; instead, it performs element-wise correction on the base scores for the same batch, the same time step, and the same hidden layer position. The hidden layer score value is determined by the following formula:

[0118]

[0119] In the formula, the hidden layer score is... Step 3 yields pre-normalized scores at batch position, time step position, and hidden layer position. The score range is the real number range and is used to carry the superposition result of the original semantic relevance and physical node penalty.

[0120] Its query component This represents the query value at the current batch, current time step, and current hidden layer position, obtained by linear projection of the hidden layer feature vector at the current time step; key components The key value at the corresponding position is obtained by linear projection from the hidden layer feature vectors at the same time step; dimensionality scale The dimension constant used to scale the product of the query component and the key component is obtained from the implicit layer configuration preset of the semantic model; the pre-constraint penalty tensor This represents the constraint value output from step two; hidden layer dimension. The hidden layer width of the current layer of the semantic model is obtained by the model structure preset.

[0121] Subsequently, step three does not directly assign a score to the hidden layer. Instead of normalization, logarithmic settling and amplitude limiting are performed first, and then the pruning probability is obtained:

[0122]

[0123] In the formula, the pruning probability The retention probability after amplitude limiting and normalization takes values ​​in an open interval. Used to determine the weight of each hidden layer location in subsequent intent aggregation; amplitude limiting operator This is a saturation mapping that restricts input values ​​to a lower and upper limit. Specifically, it outputs the lower limit when the input value is less than the lower limit, outputs the upper limit when the input value is greater than the upper limit, and retains the original value in other cases. The lower limit parameter... : Negative boundary constant of the limiting operator, taking values ​​in the range of positive real numbers; upper limit parameter : The positive boundary constant of the limiting operator, taking values ​​in the range of positive real numbers; summation position The hidden positions traversed in the normalized denominator, with values ​​ranging from positive integers. ;

[0124] In the fully connected path, step three can use the Hadamard suppression method, which is consistent with the attention path logic described above: from the pre-constraint penalty tensor The output result is obtained by subtracting the decay factor from the exponential mapping and multiplying it element-wise with the fully connected weights. The hidden layer score combines the physical node constraints with the original semantic score, and the lower bound parameter... Reduce irrelevant branches, upper limit parameter Make strongly correlated branches exclusive, pruning probability Transform these effects into probabilistic shapes that can be directly used in subsequent intent aggregation.

[0125] The acoustic environment in a classroom is not constant. Fans, footsteps in the hallway outside the window, and tapping on the desk can cause the signal-to-noise ratio of the input audio to fluctuate within a given question. If hard collapse pruning is consistently applied with the same intensity, segments with poor acoustic quality are easily suppressed. Therefore, step three introduces a two-state adjustment on top of hard collapse pruning: when the input audio remains relatively clear, the pre-constraint penalty tensor... The process follows the aforementioned deep settling path; when a significant increase in background noise occurs in the input audio, the system does not revoke the pre-constraint penalty tensor. Instead, it transitions from absolute hard cutoff to soft damping penalty with residual throughput.

[0126] The adjustment first involves the front-end digital signal processing routine writing the instantaneous acoustic energy and quiet zone noise floor into the sound pressure monitoring register, and then step three calculates the signal-to-noise pulling coefficient:

[0127]

[0128] In the formula, the signal-to-noise traction coefficient The mixed weights during the transition from hard collapse pruning to soft damping penalty, with values ​​ranging from an open interval. Used to adjust the pre-constraint penalty tensor according to the current acoustic conditions. The intensity of suppression; slope parameter The steepness of the signal-to-noise ratio curve, with values ​​ranging from positive real numbers; the current signal-to-noise ratio. The ratio of signal energy to noise floor energy in the current speech segment, with values ​​ranging from positive real numbers; the baseline signal-to-noise ratio. : The switching reference point written into the parameter table during deployment, with a value range of positive real numbers;

[0129] Obtain the signal-to-noise traction coefficient Then, step three uses its tensor of pre-constraint penalty to... A hybrid transformation is performed, changing the negative settlement from rigid deep settlement to gentle settlement that retains the residual value of the narrow slit. In this way, when a student asks a question from a position directly exposed to a fan, and the sentence contains some ambiguous syllables, the system still maintains the dominance of the current physical node, but does not flatten out weakly related but valuable word clues.

[0130] In the parallel paths of low-end motherboards, if the device uses a microcontroller without a configured floating-point concurrency unit, step three does not perform full exponential normalization. Instead, it converts the pre-constraint penalty tensor logical view from step two into a page table masking constant and writes it to the non-associative intent dictionary index table in the read-only memory. Subsequently, the page table management routine disables read enable on the page containing the non-associative intent dictionary, allowing the decoding routine to retrieve candidate intents only in reserved pages.

[0131] Specifically, signal-to-noise traction coefficient To ensure that changes in classroom background noise do not directly interrupt the target intent, the low-end motherboard path reproduces the same principle of narrowing the search space by means of physical page isolation.

[0132] Step 3 not only needs to complete the hidden layer pruning, but also needs to prevent the pruning itself from pushing the edge computing board into an overheated state during long-term high noise and long-term multi-head concurrency.

[0133] To this end, the educational robot terminal places junction temperature probes near the motherboard package and writes their sampled values ​​to the junction temperature monitoring daemon process via the system interface. In a Linux environment, the daemon process polls the processor temperature monitoring value through the thermal management interface. In a deterministic timing operating system environment, the on-chip temperature sensor interrupt routine writes the sampled values ​​to the shared register area. Simultaneously, the sound pressure monitoring register area continuously records the instantaneous noise floor transitions output by the digital signal processing routine. Only when both exceed their limits does the non-maskable interrupt controller take over step three.

[0134] Furthermore, the preferred circuit breaker triggering relationship is written as follows:

[0135]

[0136] In the formula, the junction temperature value The junction temperature sample value of the current chip core on the edge computing board, with a range of positive real temperature values, is used to reflect the level of heat accumulation caused by the continuous inference in step three; junction temperature threshold. : The upper bound of the junction temperature calculated by hard collapse pruning is allowed to be retained, and the value range is positive real temperature; noise floor transition value The instantaneous increase in background noise recorded in the sound pressure monitoring storage area, with values ​​ranging from positive real sound pressure levels; background noise threshold. : Allows the retention of the upper bound of the noise floor calculated by hard collapse pruning, with values ​​ranging from positive real sound pressure levels; Logic AND sign Junction temperature value With noise floor transition value The circuit breaker will only be triggered if both out-of-bounds conditions are met simultaneously.

[0137] When this relationship is established, the non-maskable interrupt controller initiates a highest-priority interrupt, and the hidden entry alignment routine immediately withdraws the pre-constraint penalty tensor in the accompanying cache page. Then, it reverts to a pure listening state. In the pure listening state, step three only retains the first segment of the voice buffer, no longer performs large-scale hard collapse pruning, and no longer activates multi-head scoring units. When metal desks and chairs collide with each other during class or when students pile up physics experiment equipment, the noise floor transition and chip heat accumulation increase simultaneously. The terminal's behavior is that the robot stops continuing to perform in-depth inference, only performs basic listening, and no longer outputs new high-confidence intent feature clusters.

[0138] Therefore, the junction temperature value Characterizing whether the chip is in a dangerous hot zone, the noise floor transition value. Representational contexts are sufficient to cripple sustained reasoning, logic and symbolic representation. Simultaneously, thermal and acoustic risks are linked as preventative actions, making step three a continuous source of damage to the hardware. This is achieved through pre-constraint penalty tensors. Alignment loading, hidden layer scoring value Superimposed settlement and pruning probability Hard collapse forming, signal-to-noise traction coefficient Dual-state regulation and junction temperature With noise floor transition value Cross-fusing, step three changes the current physics lesson plan node from an external prior to an exclusive passage rule within the hidden layer.

[0139] Step four below is executed by the educational robot terminal. The execution entities include the business slot parsing routine, the main control thread, the speech synthesis driver, the servo driver board, the sound source location extraction routine, and the high-privilege memory area.

[0140] Step 4: Divide the high-confidence intent feature cluster formed in Step 3 into two types of processing results: basic response instructions that can be directly issued and spatial incomplete states that need to be suspended and supplemented, thereby avoiding the terminal from blindly acting when the object is unknown.

[0141] Step three has already suppressed the non-related intent branches, but the high-confidence intent feature cluster it outputs still only represents what the current intent is more like; it doesn't inherently equate to having all the objects required for an executable action. In a classroom setting, students often say things like "how to solve this," "turn the page back," or "why subtract two from that side of the blackboard." The verbs and tones of these statements are clear enough, and even the intent categories are quite explicit. However, the key to truly driving the interaction between the motor, speech synthesizer, or electronic whiteboard lies not in the intent category itself, but in whether the necessary slots in the action template are closed. If the spatial entity slots are empty, but the robot directly turns its head, turns a page, or speaks an explanation, it will pull the current physics classroom setting and the terminal action towards the wrong object.

[0142] Therefore, step four does not directly execute the method with the highest recognition result, but first lets the business slot parsing routine check the slot status item by item according to the action template table, and then decides whether to release the basic response instruction or freeze the main control thread and resident the abnormal incomplete suspension signaling stack packet based on the degree of spatial reference incompleteness.

[0143] The business slot parsing routine of the educational robot terminal runs in the user-mode process of the edge computing board, while the main control thread runs in the kernel-mode scheduling queue. The two are connected through a shared slot page and a semaphore. The shared slot page is preferably divided into an intent category area, an action slot area, an object slot area, a spatial entity slot area, and a sequence identifier area; each area uses a fixed-length recording unit, so that after the business slot parsing routine is written, the main control thread does not need to parse the original text again.

[0144] Once the high-confidence intent feature cluster arrives, the business slot parsing routine first extracts the set of required slots corresponding to the current intent based on the action template table, and then checks each slot item by item to see if it already has a usable value. To avoid prematurely releasing an action based on only one slot hit, this implementation introduces a slot closure coefficient. :

[0145]

[0146] In the formula, the slot closure coefficient is... : The degree of closure of the current intention under the action template table, with a value range of closed intervals. This is used to uniformly indicate whether all required slots are available; slot number. : The slot number in the action template table that participates in the closure judgment, with a value range of positive integers. Total number of required slots : The number of slots that must exist in the action template table corresponding to the current intent, with values ​​ranging from positive integers; slot weight. : No. The importance of each slot in the closure determination, with values ​​ranging from positive real numbers; slot state variables. : No. Whether each slot has been filled with an executable value. The value is 1 when the slot has an executable value and 0 when the slot is empty or contains only a fuzzy pronoun. This is used to convert the slot check results into an additive discrete state.

[0147] When the slot closure coefficient When the closure threshold is met and the spatial entity slot is not empty, the main control thread releases the basic response command. If the intent is a speech interpretation type intent, the main control thread sends a speech frame containing the intent type, object name, and sequence identifier to the speech synthesis driver; if the intent is a mechanical rotation type intent, the main control thread sends an action frame to the servo drive board, which includes the axis number, target angle, action sequence identifier, and check bit.

[0148] In a preferred embodiment, the servo drive board is mounted on the robot's thoracic support frame, which is made of aluminum alloy plate. The voice speaker is installed in the acoustic cavity of the lower housing, which is made of a flame-retardant blend of acrylonitrile-butadiene-styrene copolymer and polycarbonate.

[0149] Preferably, the basic response command adopts a fixed frame format. The frame header includes an action type field, a target object field, and a sequence identifier field, and the frame tail includes a short integer check bit. After the service slot parsing routine writes the closed slot value into the fixed frame, the main control thread first sends the fixed frame to the loopback verification port, and then sends it to the speech synthesis driver or servo driver board. During on-site verification, it is only necessary to observe whether the fixed frame in the driver log is consistent with the closed value in the shared slot page to determine whether step four releases the action according to the established link.

[0150] Furthermore, although the speech synthesis driver and servo driver board are located in the same physical location, they obtain a closed-loop business instruction from the main control thread and no longer provide separate semantics. Slot closure coefficient. First, determine whether to convert it to a uniform quantity, and then decide on the release method. This way, the execution side sees a complete action that can be issued instead of a finished product without semantics.

[0151] When the high-confidence intent feature cluster indicates that the action category has converged, but the business slot parsing routine hits the fuzzy spatial generic pronoun blocking dictionary in the word stream, and the spatial entity slot has not yet been filled, step four does not regard the sentence as a normal recognition failure, but regards it as a dangerous state that has the impulse to act but lacks a physical anchor.

[0152] Among them, the fuzzy spatial generic pronoun blocking dictionary is preferentially stored in the read-only area, with a minimum of four entries: "this," "that," and "this side." Compound entries such as "that on the blackboard" and "the formula in front" can also be added based on the school's subject. After the business slot parsing routine segments the words, it compares the word sequence and dictionary entries sequentially. If the word matches and the spatial entity slot is empty, a freeze judgment is performed.

[0153] Furthermore, if term hits and slot vacancies are placed on the same triggering scale, a spatial incompleteness coefficient is added. :

[0154]

[0155] In the formula, the spatial incompleteness coefficient : Whether the current intent has entered a space-deficient state that requires freezing the main control thread, the value range is a closed interval. This is used to compress the hit rate of generic pronouns in fuzzy space and the degree of slot unclosure into the same decision metric; slot closure coefficient. The meaning, value range, and function are the same as those of the aforementioned slot closure coefficient. Maintain consistency;

[0156] Mixing coefficient Fuzzy space refers to the weighting parameter between pronoun hit rate and slot unclosure degree, with a value range of closed intervals. Fuzzy hit count : The strength value of the hit in the fuzzy space generic pronoun blocking dictionary in the current lexical sequence, with a value range of a closed interval. The fuzzy hit rate is calculated using the following formula:

[0157]

[0158] When the total number of current lexical units is 0, the fuzzy hit count is 0.

[0159] Among them, the number of hit words This represents the number of lexical units in the current lexical unit sequence that match the fuzzy space generic pronoun blocking dictionary. It is obtained by comparison after word segmentation in the business slot parsing routine; total number of lexical units. This represents the total number of lexical units in the current lexical unit sequence, obtained from the same word segmentation routine; fuzzy hit count. The ambiguity of the spatial reference of the current statement is indicated; the mixing coefficient represents the relative weight of the ambiguity hit rate and the slot closure coefficient in the freeze determination.

[0160] When the spatial incompleteness coefficient When the freeze threshold is reached, the main control thread does not destroy the current statement. Instead, it uses a kernel semaphore to freeze the action queue item corresponding to that statement and simultaneously suspends the downstream action delivery port. For example, when a student says to circle something on the blackboard, the verb "circle" is sufficient to indicate a labeling intent, but the specific object is not given. The visual result for both the teacher and the student is that the robot does not immediately raise its arm or rush to answer; the main control thread remains paused just before the action is released, waiting for further completion.

[0161] Specifically, fuzzy hit rate Slot closure coefficient is used to capture the referential ambiguity on the surface of a statement. Used to capture structural gaps within the motion template, both are processed using a spatial incompleteness coefficient. After merging, it can distinguish between states that originally intended to move but whose objects were unclear and ordinary recognition noise.

[0162] After the main control thread is frozen, step four immediately involves a high-privilege exception routine creating a fragmented suspension signaling stack packet in the high-privilege memory area. The high-privilege memory area is preferably allocated in a separate page of on-chip static random access memory, not sharing an address segment with ordinary reply cache pages, to prevent subsequent ordinary sessions from overwriting the exception context. The fragmented suspension signaling stack packet is carried using a JavaScript object representation JSON object, but its writing is performed by a kernel-mode routine; user-mode processes can only read, not overwrite. This signaling stack packet preferably includes at least the following fields: deadlock flag field, snapshot time field, sound source location field, session identifier field, intent identifier field, vacant slot field, and frozen action sequence field.

[0163] In this embodiment, the sound source orientation field is not inferred from the text, but is obtained by re-performing beamforming and direction of arrival inversion on the current sound segment using a microphone array from the same source as in step one. In a preferred embodiment, the microphone array inside the robot's head housing is installed at equal intervals along the arc surface. The beamforming routine first calculates the candidate sound source orientation based on the arrival time difference of adjacent channels, and then compresses multiple orientation samples from the same sound segment into a multi-source spatial polar angle matrix. .

[0164] Multi-source space arrival polar angle matrix This can be represented as a set of pairs of pitch and yaw angles, each pair corresponding to the direction of a sound source that exists simultaneously or sequentially. The purpose of this processing is not to immediately determine where the student is pointing, but to preserve the underlying physical clue of who is speaking from which direction in the abnormally incomplete suspended signaling stack packet, so that the entire audio segment does not need to be replayed when step five takes over.

[0165] Preferably, the abnormally incomplete suspended signaling stack packet is arranged continuously in the high-privilege memory area according to the header segment, slot segment, and orientation segment. The header segment stores the deadlock flag field, snapshot time field, session identifier field, and verification field; the slot segment stores the currently closed slot value, the empty slot identifier, and the frozen action sequence; the orientation segment stores the multi-source spatial arrival polar angle matrix. And its corresponding sound source sequence number. The high-privilege exception routine first freezes the queue pointer of the main control thread during writing, then locks the current shared slot page, and then completes the encapsulation in the order of header segment-slot segment-location segment. Finally, the verification field is filled back to the end of the header segment. The direct benefit of this approach is that during step five, reading does not require scanning scattered memory blocks again, but can sequentially obtain the deadlock situation, the semantically incomplete location, and the sound source location base.

[0166] During the on-site implementation, if a student asks why the appointment was canceled, the robot will not answer; simultaneously, an abnormally incomplete and suspended signaling stack packet record is added to the high-privilege memory area. Observing the memory image reveals the current deadlock flag field, snapshot time field, and multi-source spatial arrival polar angle matrix. The high-privilege memory area isolates abnormal scenarios from being overwritten by ordinary replies. The sound source orientation field removes pure acoustic spatial clues from the semantic layer, reserving a physical foundation for cross-modal addressing in step five.

[0167] Step four, in addition to the main path, can set up enhanced paths, parallel paths, and degraded paths. This allows for the use of the same term across different scenarios. Enhanced paths are used for scenarios where the motion is already closed and the robot is ready to rotate. Since the robot's own speaker and servo rotation introduce new echoes and structural noise, an echo cancellation residual evaluation routine is inserted before deployment to obtain the residual noise ratio. Then adjust the original motion angle Perform damping correction:

[0168]

[0169] In the formula, the angle of the correction is... The final mechanical motion angle after residual noise ratio correction, with a value range of non-negative real angles, is used to mitigate self-excitation noise caused by sudden servo rotation; the original motion angle... The business slot parsing routine generates the uncorrected target angle based on the current closure intention; the value range is a non-negative real angle quantity; damping coefficient. The residual noise ratio affects the reduction strength of the motion angle, and its value range is a closed interval. Residual noise ratio The ratio of echo cancellation residue to the current effective speech energy, with a value within a closed interval. ;

[0170] Parallel paths are suitable for serious, silent testing scenarios. In this case, the local speaker no longer emits error messages; instead, the network sending routine transmits a digest of the abnormal, incomplete, suspended signaling stack packet via encrypted UDP protocol to the teacher's central control terminal. A red dot and session digest appear on the teacher's central control terminal screen, while the student-side robots remain silent.

[0171] The degradation path is applicable when an abnormally incomplete suspended signaling stack packet remains resident for an extended period without being taken over in step five. The high-privilege exception routine continuously records the duration of the suspended status. :

[0172]

[0173] In the formula, the duration of stay is displayed. : The duration from the formation of the abnormally incomplete suspended signaling stack packet to the current check time, with a value range of non-negative real numbers, used to determine whether to return from the suspended state to the local fallback prompt; Current check time The high-privilege exception routine reads the system timestamp during this inspection, with a value range of non-negative real numbers representing time; snapshot time. : The snapshot timestamp written when the abnormally incomplete suspended signaling stack packet is created, with a value range of non-negative real number time values;

[0174] When the dwell time is suspended When the residency limit is exceeded and no takeover event occurs, the high-privilege exception routine destroys the incomplete suspended signaling stack packet and switches to the fixed prompt playback path of the local sound card. The prompt content is preferably "Please ask again with the full name".

[0175] Specifically, adjust the angle of motion. Responsible for suppressing robot-generated noise, encrypting user datagram protocol paths, fulfilling alarm requirements for silent classrooms, and suspending dwell time. This ensures that abnormal states will not remain in the high-privilege memory area indefinitely.

[0176] Step four further decomposes the high-confidence intent feature cluster into immediately executable basic response commands and spatial incomplete states that can be subsequently filled in. For questions like "that," "there," and "here," which carry strong action intent but lack spatial anchors, execution will not be blind, nor will the statement be discarded; instead, an abnormal situation with a sound source orientation base will be left. Furthermore, step four separates semantic convergence and object localization into two judgments: the former relies on the high-confidence intent feature cluster, and the latter relies on spatial entity slots and sound source orientation bases, ensuring that action release and abnormal incompleteness suspension are both based on evidence.

[0177] Step five is executed by the educational robot terminal, and the executing entities include the high-privilege exception takeover routine, millimeter-wave radar antenna array, forward vision lens, inverse perspective intersection engine, slot backfilling routine, parameter write-back routine, and watchdog controller.

[0178] Step 5: Transform the incomplete state left by the suspension in Step 4 into a closed control state that can be located, backfilled, and reversed, so that the terminal can fill the entity before releasing the action when the object is unknown, and backflow the cost of filling the blind spot back to the preceding link.

[0179] The abnormally incomplete suspended signaling stack packet output in step four already indicates that the semantics themselves have not collapsed; what is truly missing is the spatial entity slot. If we continue to repeatedly question within the voice domain, the main control thread frozen in step four will occupy the action queue for an extended period. If we directly release the action, the risk of object errors will be transformed into device malfunction.

[0180] Step five therefore no longer searches for answers in the word group, but instead takes the multi-source space preserved in step four and reaches the polar angle matrix. As a spatial fuse, it awakens the peripheral spatial sensing probe group, searches for the limb rays of the speaker along the direction of the sound source, and then uses two-dimensional interface topology to complete the intersection and collision calculation.

[0181] Furthermore, step five does not consider the completion of the blind spot repair as the end point. This is because each blind spot repair requires time to initiate a radar cold start, and multipath interference can cause offsets; the more severe the obstruction, the more blind spots need to be repaired. Step five converts the delay and offset of this addressing into feedback weights, and then modifies the first reference time tolerance window in step one in reverse. And the half-decay debouncing control constant in step two .

[0182] The high-privilege exception takeover routine exists only in the kernel-mode task queue of the edge computing board and does not normally occupy the power supply branches of the millimeter-wave radar antenna array and the forward-looking vision lens. Only the exception incomplete suspension signaling stack packet written in step four contains the deadlock flag field, snapshot time field, and multi-source spatial arrival polar angle matrix. At this time, the power supply gate transistor will be in the on state, and the millimeter-wave radar antenna array and forward vision lens will be powered on.

[0183] Preferably, the millimeter-wave radar antenna array is fixed behind the wave-transparent window on the robot's chest, and the wave-transparent window is made of polypropylene sheet or polytetrafluoroethylene composite sheet; the array substrate is made of low-loss copper-clad laminate. The forward-looking vision lens is mounted on the head.

[0184] The high-privilege exception takeover routine first unblocks the multi-source space in the exception incomplete suspended signaling stack packet to reach the polar angle matrix. Then, the pitch and yaw angles of each set are passed to the beam scheduling routine. This is because the multi-source space arrival polar angle matrix... Often including students speaking simultaneously, echo sidelobes, and temporary noise sources, step five does not simultaneously consider all directions as the primary direction. Instead, it first selects the primary addressing direction based on the snapshot time field in the abnormally incomplete suspended signaling stack packet and the corresponding speech period in the frozen action sequence of step four. :

[0185]

[0186] In the formula, the main addressing direction The set of spatial polar angles of arrival selected as the main beam pointing to the millimeter-wave radar antenna array, with values ​​ranging from the multi-source spatial polar angle matrix. Any directional element in the matrix is ​​used to focus subsequent detections on the direction of the sound source most consistent with the anomalous snapshot time; multi-source spatial arrival polar angle matrix Step four encapsulates multiple sets of pitch and yaw angles into the abnormally incomplete suspended signaling stack packet, with values ​​ranging from a finite-dimensional set of directions; snapshot time. : Snapshot timestamp in the abnormally incomplete suspended signaling stack packet, with a value range of non-negative time values; direction time Multi-source space arrival polar angle matrix Central element The corresponding arrival time, with a value range of non-negative time quantities, is used to measure the distance between this direction and the snapshot time. The degree of closeness;

[0187] Among them, the power supply gate transistor restricts the high-power probe to being awakened only in abnormal scenarios, and the main addressing direction Subsequent probes will be concentrated within a single spatial sector. Additionally, the main addressing direction... After selection, the millimeter-wave radar antenna array transmits a narrow beam along this direction to receive the echoes from the target student's upper limbs and hands. The skeleton ray extraction routine is obtained by filtering through the range gate, velocity gate, and angle gate.

[0188] In a preferred embodiment, the skeleton ray extraction routine first separates the three main scattering clusters of shoulder, elbow, and wrist from the radar point cloud, and then fits the direction from the wrist to the fingertip into a skeleton ray vector. If the fingertip scattering in the radar point cloud is insufficient, the hand outline is supplemented by the forward-looking vision lens, and the connection between the wrist and fingertip is completed by the joint tracking routine.

[0189] Wherein, is to convert the skeleton ray vector Mapped onto the electronic whiteboard interface, the inverse perspective intersection engine reads the calibration parameters from the robot's body coordinate system to the electronic whiteboard interface coordinate system, and sequentially performs rotation, translation, and inverse perspective projection to obtain the two-dimensional hit coordinates. :

[0190]

[0191] In the formula, the two-dimensional hit coordinates Skeleton ray vector The two-dimensional landing point obtained after inverse perspective intersection with the electronic whiteboard interface takes values ​​in the two-dimensional coordinate domain of the electronic whiteboard interface and is used as the direct input for subsequent interface entity collision detection.

[0192] Inverse perspective matrix : Projection transformation operator from 3D body coordinates to 2D interface coordinates, with a value range of a fixed-dimensional matrix; rotation matrix The rotation relationship between the robot's body coordinate system and the electronic whiteboard's reference coordinate system, with values ​​ranging from a set of orthogonal matrices; skeleton ray vectors. The speaker's pointing ray, extracted jointly by a millimeter-wave radar antenna array and a forward-looking vision lens, has a value range in a three-dimensional vector space; the translation vector... The displacement relationship between the robot's origin and the electronic whiteboard's reference origin, with values ​​ranging in a three-dimensional vector space, is used to represent the skeleton ray vectors. Move the electronic whiteboard into the reference coordinate system;

[0193] The inverse perspective matrix Rotation matrix Translation vector The coordinates of these coordinates are obtained during the terminal installation and calibration phase. During calibration, at least four preset reference points are displayed on the teaching display interface. The forward-looking vision lens acquires the coordinate positions of these preset reference points in the image. The terminal establishes a correspondence between the image coordinate positions and the known coordinates of the preset reference points on the teaching display interface to obtain the inverse perspective matrix. The terminal then generates the rotation matrix and translation vector based on the fixed installation relationship between the body coordinate system and the teaching display interface coordinate system, and writes the inverse perspective matrix, the rotation matrix, and the translation vector into the local parameter area.

[0194] Among them, the inverse perspective matrix is ​​used to project 3D points onto the teaching display interface plane; the rotation matrix is ​​used to represent the attitude relationship between the body coordinate system and the teaching display interface coordinate system; the translation vector is used to represent the positional relationship between the origins of the two coordinate systems; and the skeleton ray vector is extracted by the millimeter-wave radar antenna array and the forward vision lens.

[0195] Calculate the two-dimensional hit coordinates Then, the interface topology matching routine reads the primitive boundary table of the current electronic whiteboard page and matches the two-dimensional coordinates. Perform collision and intersection calculation; once the boundary of a primitive is hit, step five writes the physical entity identifier corresponding to the primitive into the spatial entity slot left in step four, and rewrites the frozen action sequence to the filled state.

[0196] For example, when a student asks why the force analysis diagram points to the left, the skeleton ray vector extracted by the millimeter-wave radar antenna array... After reverse perspective, it falls into the two-dimensional hit coordinates The interface topology matching routine then hits the boundary of the target primitive, and finally writes the physical entity identifier of the primitive into the spatial entity slot left in step four.

[0197] Specifically, skeleton ray vector The direction of the sound source is refined into hand pointing, two-dimensional hit coordinates. Then map the spatial orientation to a physical entity identifier that can be written to the business slot.

[0198] After completing step five of the spatial entity slot backfilling, the physical cost of this round of blind patching begins to be measured. The measured cost includes at least the addressing delay from the high-privilege exception takeover routine to the exception incomplete suspension signaling stack packet and the physical entity identifier output by the interface topology matching routine, which is called the addressing cost. skeleton ray vector The offset between the inverse perspective and the center of the final hit primitive is called the offset cost. The parameter write-back routine obtains the feedback weights through these two types of costs. :

[0199]

[0200] In the formula, the feedback weight Step 5: The reverse correction strength generated based on the cost of this round of blind patching is within a closed interval. Positive real numbers within the range are used to uniformly handle latency costs. With offset cost Backflow force of preceding parameters; benchmark feedback weight : Initial feedback strength set during the deployment phase, with values ​​ranging from positive real numbers, used to provide a write-back baseline without additional cost; gain coefficient Latency Cost Feedback weights The influence weight, with a range of positive real numbers;

[0201] latency cost This refers to the duration from the unpacking of the abnormal and incomplete suspended signaling stack packet to the confirmation of the physical entity identifier. The value is a non-negative time value, used to measure the time consumption level of cross-modal blinding; standard latency. The addressing reference time set during deployment, with a value range of non-negative time, is used as a latency cost. Comparison baseline; gain coefficient Offset cost Feedback weights The influence weight, with values ​​ranging from positive real numbers; offset cost 2D hit coordinates The displacement between the final hit primitive reference point and the reference point, with a value range of non-negative length;

[0202] Standard Offset The offset reference value set during deployment, with a value range of non-negative length, is used as the offset cost. Comparison baseline; lower limit parameter With upper limit parameter These represent the feedback weights. The minimum and maximum allowable values ​​are both positive real numbers within the range and satisfy the following conditions: This is used to define stable boundaries for reverse write-back operations;

[0203] Obtain feedback weights Then, the parameter write-back routine selects the rewrite target based on the current exception type. If the main cost of this round is excessive latency, the first baseline time tolerance window in step one is rewritten first. If the main cost in this round is pointing offset and edge drift, then the half-fading debouncing control constant in step two should be rewritten first. .

[0204] In one implementation, these two types of write-back follow the following formula:

[0205]

[0206]

[0207] In the formula, update window The first reference time tolerance window after the write-back in step five, used in the next round of step five, has a closed interval. The positive real numbers within are used to adjust the spacetime lock storage scale according to the difficulty of this round of blind patching; the original window Step 1: The first effective reference time tolerance window, with values ​​ranging from positive real numbers; tightening coefficient. Feedback weights For the original window The tightening strength, taking values ​​in the range of positive real numbers; lower limit of the window. With window limit These represent updating the window. The lower and upper bounds of the value are both positive real numbers and satisfy the following conditions: Update constants The half-decay debouncing control constant used in the next round of step two after the write-back in step five has a value range of a closed interval. Positive real numbers within;

[0208] original constant Step 2: The currently effective half-decay debouncing control constant, with values ​​ranging from positive real numbers; expansion coefficient. Feedback weights For the original constant The expansion strength, taking values ​​in the range of positive real numbers; the lower limit of the constant. With upper limit of constant They represent the update constants respectively. The lower and upper bounds of the value are both positive real numbers and satisfy the following conditions: This is used to prevent smooth boundaries from being too narrow or too wide;

[0209] Specifically, feedback weight It is fixed as a front-end parameter and updates the window. With update constant This is implemented at the entry scale of Step One and Step Two, respectively. The selection of the write-back target is performed according to the following logic: First, calculate the latency excess and offset excess:

[0210]

[0211] When the time delay excess is greater than or equal to the offset excess, the first reference time tolerance window is updated; when the time delay excess is less than the offset excess, the half-fading debouncing control constant is updated.

[0212] Among them, latency excess This represents the portion of the addressing delay exceeding the standard delay, obtained by subtracting the addressing delay cost of this round from the standard delay and taking the result that is not less than zero; Offset Excess. The portion of the address offset exceeding the standard offset in this round is obtained by subtracting the offset cost of this round from the standard offset and taking the result that is not less than zero; the standard delay and standard offset are written into the local parameter area when the terminal is deployed; the first reference time tolerance window and the half-fading debouncing control constant are respectively written back according to the original update formula.

[0213] When millimeter-wave radar antenna arrays experience severe obstruction in areas with high-density student seating, relying solely on the main addressing direction... If the speaker's limbs cannot be stably separated, step five enters the enhancement path. The terminal first wakes up the ultra-wideband (UWB) substations deployed in the four corners of the classroom or near the podium, and each substation transmits the time difference of arrival (TDOA) data; the forward-looking vision lens simultaneously outputs the optical flow attitude vector within the same time period. The fusion routine sends the TDOA data and the optical flow attitude vector to the posterior density solver to obtain the coordinates of a single entity, and then maps these coordinates onto the electronic whiteboard interface.

[0214] The posterior density solver calculates the posterior probability of candidate entity coordinates based on ultra-wideband positioning observations and optical flow attitude observations, specifically according to the following relationship:

[0215]

[0216] The coordinates of the candidate entity that maximizes the posterior probability are then taken as the output.

[0217]

[0218] Among them, candidate entity coordinates Indicates the location of the target to be determined within the classroom; ultra-wideband positioning observation. Represents the time difference of arrival data transmitted back from each ultra-wideband substation; optical flow attitude observations. Represents the optical flow attitude vector output by the forward-looking vision lens; prior probability The spatial sector is given by the multi-source spatial arrival polar angle matrix in the abnormally incomplete suspended signaling stack packet; the conditional probability terms represent the probability of the occurrence of ultra-wideband positioning observations and optical flow attitude observations when the candidate entity coordinates are given; output coordinates This is used for subsequent mapping to the teaching display interface.

[0219] When the application scenario does not require active electromagnetic irradiation, step five enters the silent alternative path. The micro inertial unit (IMU) beacon worn on the student's wrist periodically receives three-axis displacement and attitude logs. The terminal locally constructs a three-dimensional bounding box based on the wearer's initial position relative to the electronic whiteboard. The attitude log is converted into a pointing vector and intersected by a volume projection with the boundary of the electronic whiteboard primitives.

[0220] Regardless of whether it's the main path, the enhanced path, or the silent alternative path, step five is controlled by the watchdog power supply controller. The watchdog power supply controller continuously compares the current addressing duration with the temperature rise of the power supply branch. When the temperature rise exceeds the boundary, it disconnects the power supply to the millimeter-wave radar antenna array or auxiliary substation, destroys the current blind spot buffer, and sends a fallback warning to step four, triggering a flag.

[0221] Specifically, it enhances path coverage in occlusion scenarios, silently replaces path coverage in privacy-sensitive scenarios, and uses a watchdog power controller to block overheating and prolonged operation. Step five achieves closure at three levels: object closure, thread closure, and parameter closure, transforming external resistance into front-end spatiotemporal latching scale and topology smoothing scale correction.

[0222] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0223] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0224] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0225] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A semantic understanding method for educational robots oriented towards classroom interaction, executed by a terminal device, characterized in that: include: Concurrently collect environmental audio and obtain the discrete code representing the current lesson plan node on the teaching display interface. Generate a binding record based on the hardware clock when the time stamp difference between the two falls within the first reference time tolerance window. The state discrete code is transformed into a pre-constraint penalty tensor consistent with the hidden layer topology corresponding to the ambient audio; the pre-constraint penalty tensor is injected when performing hidden layer operations of the semantic model based on the ambient audio to suppress candidate intentions inconsistent with the state discrete code. When a high-confidence intent contains a missing spatial entity slot, the control action corresponding to the high-confidence intent is suspended and an abnormal missing suspension signaling stack packet carrying a multi-source spatial arrival polar angle matrix is ​​generated. In response to an abnormally incomplete suspended signaling stack packet, the spatial awareness probe group is activated, and the target entity on the teaching display interface is located based on the multi-source spatial polar angle matrix and the spatial entity slot is filled, and the control action is released.

2. The semantic understanding method for educational robots according to claim 1, characterized in that: The process of generating binding records includes: extracting the first frame timestamp of the ambient audio, extracting the received timestamp of the status discrete code, and writing the first frame timestamp, received timestamp, session identifier, and status discrete code into the high-speed register area; when the timestamp difference exceeds the first reference time tolerance window, the corresponding status discrete code is discarded.

3. The semantic understanding method for educational robots according to claim 2, characterized in that: Transforming the state discrete code into a pre-constraint penalty tensor involves: performing node order mapping and one-hot encoding on the state discrete code, smoothing the one-hot encoding result based on the half-fading dejitter control constant, and broadcasting and expanding along the time step dimension and hidden layer dimension of the ambient audio to generate a pre-constraint penalty tensor consistent with the hidden layer topology of the semantic model.

4. The semantic understanding method for educational robots according to claim 3, characterized in that: The injection of the pre-constraint penalty tensor includes: before the semantic model performs hidden layer operations, the pre-constraint penalty tensor and the hidden layer feature vector at the corresponding time step are subjected to element-wise suppression processing, and the processed hidden layer feature vector is sent into the normalization and intent aggregation link to output a high-confidence intent.

5. The semantic understanding method for educational robots according to claim 4, characterized in that: The suspension control actions include: detecting spatial pronoun items in the current high-confidence intent based on the fuzzy spatial generic pronoun blocking dictionary, and freezing the control action when the spatial entity slot is empty; generating an abnormally incomplete suspension signaling stack packet including writing a deadlock flag field, a snapshot time field, a multi-source spatial arrival polar angle matrix, and an empty slot identifier.

6. The semantic understanding method for educational robots according to claim 2, characterized in that: The state discrete code is obtained through MQTT lightweight messages. The payload of the MQTT lightweight message adopts the JSON object representation format and includes at least a session identifier field and a node status field. The terminal device performs session consistency verification between the state discrete code and the ambient audio based on the session identifier field.

7. The semantic understanding method for educational robots according to claim 5, characterized in that: The abnormally incomplete suspended signaling stack packet is represented in JSON object format and written to a high-privilege memory area. It contains at least a deadlock flag field, a snapshot time field, a sound source orientation field corresponding to the multi-source spatial arrival polar angle matrix, a void identifier for the spatial entity slot, and a frozen action sequence identifier.

8. The semantic understanding method for educational robots according to claim 5, characterized in that: In the silent test scenario, after suspending the control action, the terminal device sends an encrypted UDP message to the teacher's central control terminal. The encrypted UDP message contains at least a session identifier, an empty slot identifier, and a snapshot time field, and disables the output of prompts from the local speaker.

9. The semantic understanding method for educational robots according to claim 5, characterized in that: The abnormal and incomplete suspended signaling stack packet is stored continuously in the high-privilege memory area in the header segment, slot segment and orientation segment; the header segment stores the deadlock flag field and the snapshot time field, the slot segment stores the empty slot identifier, the orientation segment stores the multi-source spatial arrival polar angle matrix, and a check field is set in the header segment.

10. The semantic understanding method for educational robots according to claim 1, characterized in that: The first reference time tolerance window is dynamically adjusted based on the bus input / output load occupancy rate; the terminal device reads the bus input / output load occupancy rate and the proportion of recent misalignment records in each adjustment cycle, and adjusts the first reference time tolerance window based on the results of dual closed-loop proportional-integral-derivative control.

11. The semantic understanding method for educational robots according to claim 3, characterized in that: The half-decay de-jitter control constant is dynamically updated based on the ambient reverberation time; the terminal device obtains the ambient reverberation time through the speaker retrace signal on the teaching display interface, and adjusts the tail length of the smoothing process based on the ambient reverberation time.

12. The semantic understanding method for educational robots according to claim 1, characterized in that: The space perception probe group includes a millimeter-wave radar antenna array and a forward-looking vision lens; The method for locating the target entity of the teaching display interface based on the multi-source spatial arrival polar angle matrix includes: obtaining the skeletal ray vector of the speaker along the direction corresponding to the multi-source spatial arrival polar angle matrix, determining the two-dimensional hit coordinates of the teaching display interface based on inverse perspective intersection, and then determining the target entity of the teaching display interface based on the two-dimensional hit coordinates.

13. The semantic understanding method for educational robots according to claim 12, characterized in that: After backfilling the spatial entity slot, the terminal device measures the addressing delay cost and multipath offset cost of the current positioning process, updates the half-decay debouncing control constant based on the addressing delay cost and multipath offset cost, and writes the updated half-decay debouncing control constant into the local parameter area for use in the next round when generating the pre-constraint penalty tensor.

Images