Student abnormal state recognition method based on multi-modal behavior data fusion

Abstract

This invention discloses a student abnormal state recognition method based on multimodal behavioral data fusion, relating to the field of classroom behavior recognition technology. The method includes: first, acquiring video, audio, and posture data and performing pre-blinding determination; during blinding, redirecting the source address of the damaged visual channel to a zero-based physical block to form a fragmented feature stream; then, using audio and posture information to fill in and dampen the fragmented feature stream to obtain a steady-state multi-class distribution; generating prediction uncertainty based on this distribution and writing back the front-end admission threshold out-of-band; when the threshold reaches its lowest point and the uncertainty remains high, suspending the fragmented latent vector slice and deadlock beacon, obtaining a low-rank residual patch across domains and hot-loading it to the breakpoint operator page, and then raising the admission threshold according to the uncertainty change before and after patch loading. This method can reduce invalid computation and heat accumulation, maintain the continuous operation of the abnormal recognition chain, and unlock the front-end hardware dead zone.

Description

Technical Field

[0001] This invention relates to the field of classroom behavior recognition technology, specifically a method for identifying abnormal student states based on multimodal behavior data fusion. Background Technology

[0002] Multimodal perception and video behavior recognition are typically applied in scenarios where continuous monitoring is possible, such as classrooms, examination rooms, study rooms, and other fixed seating areas. Front-end devices must simultaneously receive signals from cameras, microphones, and auxiliary sensing units, and perform anomaly detection and reporting locally, while also considering engineering challenges such as prolonged power-on operation, localized heat dissipation, and bandwidth limitations. Current technologies mostly employ a multi-source acquisition-anomaly detection-correlation analysis-strategy handling approach. This involves first determining the anomaly results separately, and then deciding whether to suppress, enhance, or shorten the output of a particular modality based on the correlation between different modalities, thereby reducing false alarms and maintaining the monitoring chain.

[0003] US Patent document US10665251B1 discloses a multimodal anomaly detection system. This system consists of a main sensor, several auxiliary sensors, a server, and a policy rule module. The main sensor is an acoustic monitor, while the auxiliary sensors can be video cameras, presence sensors, or door magnetic sensors. The server comprises a data storage device and an anomaly dependency graph. The system first acquires anomaly data from the main and auxiliary sensors, and uses the anomaly dependency graph to describe the dependencies between anomalies of different modalities. The policy rule module determines whether a target modality needs to be suppressed, shut down, strengthened, or confirmed based on the anomaly type, dependency relationship, and temporal correlation. The system further states that anomalies of each modality can be viewed as dual-attribute time series. Correlation is calculated by comparing the start, end, and duration intervals of anomaly clusters, thereby determining the duration of suppression of the target modality before and after anomaly occurrence, or improving the reliability of the target modality's anomaly after the occurrence of a supporting anomaly.

[0004] The shortcomings of the above solution lie in its reliance on the continuous extraction and correlation of anomalies across modalities. While visual channel output may exhibit anomalies under conditions such as continuous occlusion, strong reflective coverage, or foreign objects attached to the lens, these anomalies correspond to input quality degradation, not actual behavior. If the system continues to input such degraded images into subsequent recognition and correlation calculations, edge devices will still need to continuously perform data transfer, feature extraction, and multimodal comparison, leading to a continuous accumulation of computational and thermal loads over time. Furthermore, this solution only suppresses or enhances the target modality through software-level correlation and policy actions, failing to address the long-term imbalance between the front-end perception threshold and the back-end recognition state. When long-tail contamination occurs, the front-end threshold can easily remain within a range unsuitable for the current input, causing continuous fluctuations in anomaly judgment and difficulty in converging local feedback. In severe cases, this can even lead to a stalemate between continuous triggering and continuous suppression in the front-end perception chain.

[0005] Therefore, the key technical problem that this solution needs to solve is how to lock the front-end threshold in a local closed loop for a long time under the conditions of poor edge computing power and heat dissipation and the possibility of long-term distortion of visual input, by reducing ineffective visual computing and continuous output of the continuous multimodal recognition chain. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] To address the shortcomings of existing technologies, this invention provides a student anomaly state recognition method based on multimodal behavioral data fusion. It utilizes audio and posture information to fill in and dampen incomplete feature streams, obtaining a steady-state multi-class distribution. Based on this distribution, a prediction uncertainty is generated and an out-of-band write-back of the front-end admission threshold is applied. When the threshold reaches its lowest point and the uncertainty remains high, the incomplete latent vector slice and deadlock beacon are suspended. A low-rank residual patch is obtained across domains and hot-loaded to the breakpoint operator page. The admission threshold is then raised based on the uncertainty change before and after patch loading. This method can maintain the continuous operation of the anomaly recognition chain and can unlock the front-end hardware dead zone; thus solving the technical problems described in the background section.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A student abnormal state identification method based on multimodal behavioral data fusion includes acquiring video frames, audio segments, and pose packets, and generating blinding labels by a digital signal coprocessor based on the spatial structure response of the video frames before the video frames enter the neural network processor.

[0011] In response to the blinding mark, the DMA source address of the damaged visual channel is redirected to a preset all-zero physical block in the kernel interrupt context, and a fragmented feature stream and a vent execution mark are generated according to the alignment rules.

[0012] The current original distribution is generated based on the audio segment, attitude packet, incomplete feature stream and empty execution marker, and then damped and stitched with the steady-state distribution of the previous time step to obtain the current steady-state multi-class distribution.

[0013] The prediction uncertainty is generated from the current steady-state multi-class distribution and then written back to the front-end admission threshold via the inter-integrated circuit bus.

[0014] If the admission threshold does not reach the lower boundary, the spatial structure response generation continues; if the admission threshold reaches the lower boundary and the prediction uncertainty is higher than the confusion threshold within the preset continuous window, the incomplete latent vector slice and deadlock beacon are suspended, a suspended payload without video plaintext frames is sent and the low-rank residual patch returned by the remote parsing node is received, the low-rank residual patch is hot-written into the breakpoint operator page, and the admission threshold is pulled up according to the change in prediction uncertainty before and after the patch loading.

[0015] Furthermore, the digital signal coprocessor performs spatial structure response calculations on the luminance plane of the video frame and writes a blinding flag when the response value crosses the blinding determination threshold. After receiving the blinding flag, the edge recognition device enters the interrupt context, replaces the source address field of the damaged visual channel with a preset all-zero physical block address, and writes a zero-page short pass flag in the same descriptor header, keeping the target video memory address field, channel length field and timestamp field unchanged.

[0016] Furthermore, the incomplete feature flow includes visual occupant segments, audio feature segments, and posture feature segments spliced ​​together in the original channel order; the edge recognition device first generates visual distribution branches based on the visual occupant segments, then generates supplementary distribution branches based on the audio feature segments and posture feature segments, and synthesizes the current original distribution according to the compensation ratio, and then dampens and stitches it with the steady-state distribution of the previous moment to generate the current steady-state multi-class distribution.

[0017] Furthermore, the edge recognition device calculates the prediction uncertainty corresponding to the steady-state multi-class distribution in each sampling period, and simultaneously checks within a continuous window whether the current admission threshold has reached the lower boundary, whether the average prediction uncertainty of the window is higher than the deadlock threshold, and whether the write-back confirmation code remains unchanged; when all three conditions are met, the deadlock beacon is set and the incomplete latent vector slice is frozen; when any one condition is not met, the out-of-band write-back chain continues to run; the deadlock beacon includes a breakpoint layer number field, a breakpoint page offset field, a start channel index field, and an end channel index field.

[0018] Furthermore, after setting the deadlock beacon, the edge recognition device generates a suspended payload, which includes at least a fragmented latent vector slice, a deadlock beacon, an admission threshold, a prediction uncertainty range, and a steady-state multi-class distribution. After receiving the patch payload page, the edge recognition device generates a write gate matrix based on the breakpoint layer number field, breakpoint page offset field, start channel index field, and end channel index field in the deadlock beacon, and performs hot writing of low-rank residual patches only on the corresponding breakpoint operator pages.

[0019] Furthermore, the edge recognition device first performs an integrity check on the patch payload page; when the check results are consistent, it performs a hot write of the low-rank residual patch; when the check results are inconsistent, it keeps the current breakpoint operator page and the previous patch version participating in inference, and writes the current patch payload page into the cache page to be retransmitted.

[0020] Furthermore, when performing hot writing of low-rank residual patches, the edge recognition device first writes the low-rank residual patch to a shadow page that is different from the current breakpoint operator page; when the shadow page writing is completed and the page verification is consistent, the active page index is switched; when the page verification is inconsistent, the current active page index is kept unchanged and the shadow page write mark is cleared.

[0021] Furthermore, the patch payload page includes a cluster identifier, a breakpoint layer number, and an admission threshold interval code. After receiving the patch payload page, the edge recognition device compares the local deadlock beacon, the breakpoint layer number, and the current admission threshold interval code. When the three are consistent, the low-rank residual patch is loaded. When the three are inconsistent, the patch payload page is cached without performing a hot write.

[0022] Furthermore, when the edge recognition device fails to obtain a valid patch payload page in multiple consecutive communication windows, it restores the breakpoint operator page to the valid initial page saved during the installation and calibration phase and writes a reset code to the front-end register; when a valid patch payload page is obtained in multiple consecutive communication windows, it keeps the current breakpoint operator page and the current patch version participating in inference.

[0023] Furthermore, the header of the suspended payload includes at least a timestamp field, a seat number field, a deadlock state field, a slice length field, a slice dimension field, a quantization bit width field, a layout field, and a verification field. The payload body of the suspended payload includes incomplete latent vector slices and quantization results of the admission threshold, prediction uncertainty range, and steady-state multi-class distribution.

[0024] Furthermore, the header of the patch payload page includes at least the breakpoint layer number field, the video memory page offset field, the start channel index field, the end channel index field, the left factor row number field, the right factor column number field, the release suggestion threshold field, and the integrity check field. The payload body of the patch payload page includes the left factor matrix and the right factor matrix corresponding to the low-rank residual patch.

[0025] Furthermore, when setting the deadlock beacon, the edge recognition device writes a timestamp, seat number, breakpoint layer number, current admission threshold code, current prediction uncertainty code, and deadlock status bit to the local status log page; when releasing the deadlock beacon, it appends the release status bit, updated prediction uncertainty code, and reverse release threshold code to the local status log page.

[0026] Furthermore, after the deadlock beacon is set, the edge recognition device outputs a suspension repair status code, seat number, and breakpoint layer number on the status output interface; after the deadlock beacon is released, it outputs a threshold release status code, update operator page version number, and current admission threshold code on the status output interface.

[0027] (III) Beneficial Effects

[0028] This invention provides a method for identifying abnormal student states based on multimodal behavioral data fusion, which has the following beneficial effects:

[0029] By triggering source address redirection through polarization-induced blinding interrupt markers, the damaged visual channel is completely emptied before entering the neural network processor, preventing dirty data from continuously occupying video memory, bus, and multiply-accumulate operations. This reduces the computational power consumption, heat accumulation, and link congestion of edge recognition devices from the source. By connecting the incomplete feature stream, the empty execution marker, the audio sampling segment, and the pose sampling packet, the classification chain continues to run even when the visual channel is cut off, reducing high-frequency jumps, misjudgment propagation, and output instability caused by visual tortuosity. The multimodal fusion chain will not fail due to visual failure.

[0030] The steady-state multi-class distribution is further transformed into the prediction uncertainty range, and the front-end admission threshold is directly written back through the out-of-band control link. The back-end identification status and the front-end perception threshold achieve closed-loop linkage. The admission boundary drifts synchronously with the change of continuous pollution intensity and is no longer fixed at a static value. When the local closed loop fails, incomplete latent vector slices and deadlock beacons are attached to make the deadlock intermediate state undigestable and transform it into a transferable and reusable structured breakpoint material. This not only blocks the invalid cycle on the edge side, but also provides triggering conditions, breakpoint locations and state basis for subsequent cross-domain repair.

[0031] Based on incomplete latent vector slicing and deadlock beacons, low-rank adaptive residual patches are generated, enabling breakpoint topology repair without requiring plaintext-aware data. This fully considers bandwidth constraints, computing power limitations, and post-patch adaptation requirements in edge scenarios. The low-rank adaptive residual patches are hot-written into the breakpoint operator page, and the admission threshold is released in reverse based on the change in the range of prediction uncertainty before and after patch loading, forming a closed loop that connects front-end interception, local steady-state operation, breakpoint suspension, cross-domain repair, and hardware dead zone removal. 4 Attached Figure Description

[0032] Figure 1This is a schematic diagram of the overall architecture of the edge recognition device and the remote cloud parsing node of the present invention to collaboratively perform student abnormal state recognition;

[0033] Figure 2 This is a schematic diagram of the visual channel bypass pre-assessment, blinding loss judgment and polarization blinding interruption marker generation process of the present invention;

[0034] Figure 3 This is a schematic diagram illustrating the isolation between video channel source address redirection and underlying memory addressing based on polarization-blinding interrupt markers according to the present invention.

[0035] Figure 4 This is a schematic diagram of the process for generating cross-modal complementation, inertial damping, and steady-state multi-class distribution based on incomplete feature flow according to the present invention.

[0036] Figure 5 This is a schematic diagram of the out-of-band threshold write-back and deadlock suspension control process based on steady-state multi-class distribution of the present invention;

[0037] Figure 6 This is a schematic diagram of the process of throwing up incomplete latent vector slices, solving low-rank adaptive residual patches in the cloud, and hot-loading and reviving at the edge in this invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figures 1-6 This invention provides a method for identifying abnormal student states based on multimodal behavioral data fusion.

[0040] The following actions are all performed by an edge recognition device installed at the front end of the teaching area. This edge recognition device includes a video acquisition unit, an audio acquisition unit, a posture acquisition unit, a digital signal coprocessor, a main control processor, an interrupt controller, and a frame buffer memory. (Video sampling frames) Audio sampling segment With attitude sampling package First, the data is written to its respective buffer. Then, the digital signal coprocessor performs a bypass pre-evaluation before the video enters the subsequent fusion calculation to determine whether the visual channel is blinded and to provide a trigger for step two.

[0041] Step 1: In the video sampling frame Before entering fusion computing, an independent bypass pre-evaluation chain is established using a digital signal coprocessor to identify the blinding state of the visual channel and output a polarization blinding interruption flag for direct takeover in step two. .

[0042] When student anomaly detection is deployed on edge recognition devices, the video channel bears the heaviest texture interpretation task. However, the same channel is also most susceptible to close-range occlusion, malicious attachment, strong light oblique scanning, and specular reflection. If such damaged videos are directly fed into subsequent fusion operators, the subsequent links will first consume memory transfer, convolution multiplication and addition, and cross-modal alignment, instead of first determining whether the video still has the basic qualifications to be recognized. The result is not a delayed recognition conclusion, but rather that limited computing power is wasted on distorted data at the very beginning. To avoid this upfront waste, this implementation moves the admission decision from the neural network entry point to the frame start buffer exit point, and allows the digital signal coprocessor to complete the decision in an independent clock domain; the digital signal coprocessor only reads the luminance information and register state, without rewriting the original video sampling frame. Therefore, its task is not to repair the image, but to give an admission decision with clear physical meaning.

[0043] Among them, the main control processor will sample video frames. Upon reaching the frame start buffer, subsequent inference queues are suspended. The digital signal coprocessor directly extracts the brightness plane from the frame start buffer, and a third-order SObel convolution kernel obtains the first-order edge response. Horizontal convolution kernels then extract... The vertical convolution kernel is transposed; the first-order edge response is subjected to neighborhood quadratic difference within the same on-chip storage page to obtain the local Hessian matrix.

[0044] The local Hessian matrix does not pursue image semantics, but rather characterizes whether the brightness texture has collapsed into a single plane in space. When the local Hessian matrix simultaneously loses curvature concentration and directional differences over a large area, it indicates that the current image is not ordinary blur, but rather closer to a planarized result after being completely covered by an occluded object or washed by polarized light spots. Then, the audio sampling segment... With attitude sampling package As a threshold for cross-exclusion evidence, it avoids misjudging a complete lighting change or a group standing as visual impairment. Finally, the interrupt controller encapsulates the blinding conclusion into a polarization-induced blinding interruption marker that can be read from step two. It also includes the damaged channel number, frame buffer page number, and timestamp.

[0045] In a preferred embodiment, the digital signal coprocessor reads only video sample frames at a time. The luminance plane is read, but the chrominance plane is not read, and the weights of the convolutional neural network are not adjusted. The frame start buffer adopts a two-page alternation method, with the previous page being read by the camera interface and the next page being read by the digital signal coprocessor, to avoid contention on the same physical address between bypass pre-evaluation and acquisition write.

[0046] Digital signal coprocessor Sobel convolution kernels are used to obtain the brightness gradient, and the gradient results are then differentially analyzed to obtain the local Hessian matrix. Calculate the spatial entropy response value :

[0047]

[0048] Where: Spatial entropy response value : Indicates the current video sampling frame The availability of spatial curvature is represented by a non-negative real number; the smaller the value, the more planar the luminance texture tends to be. (Video frame width) : Represents the horizontal pixel count of the brightness plane, preferably taken as the actual output width of the monitoring terminal; video frame height. : Represents the vertical number of pixels in the brightness plane, preferably taken as the actual output height of the monitoring terminal;

[0049] Local Hessian matrix : Represents pixel coordinates The second-order transformation matrix at the location has diagonal terms derived from the horizontal and vertical quadratic differences, and off-diagonal terms derived from the cross differences; curvature coupling coefficients The weights used to balance the determinant and trace terms are preferably between 0.2 and 0.8; (function) : Represents the natural logarithm, used to compress the dominance of local maxima on the overall frame result;

[0050] Operator : Represents the determinant operator, used to extract local Hessian matrices. The degree of curvature concentration; operator : Represents the trace operator, used to extract the local Hessian matrix. The total change;

[0051] For example, in a standard classroom, an edge recognition device is installed above the podium, with the camera facing the first three rows of seats. When a student presses their exam paper cover close to the camera, the luminous plane rapidly loses the edges of the desk, facial contours, and seat boundaries within several consecutive frames. The digital signal coprocessor reads a large area of ​​monotonous luminous patch instead of a texture structure that can be subsequently recognized. At this point, the spatial entropy response value... The original video sample frames will be extracted before any high-level classification actions. The image remains unchanged in the frame-first buffer, without pixel patching, interpolation, or semantic inference. This ensures that the determination of whether the image is still worth entering the subsequent links is fixed before semantic recognition.

[0052] Furthermore, the bypass interception action and the video acquisition action are physically isolated, eliminating the need for meaningless transfer of data to the subsequent recognition chain; the brightness plane and the local Hessian matrix ensure that the blinding determination occurs at the curvature collapse physical layer, rather than in higher-level categories that are susceptible to scene semantics; the double-page frame first buffer ensures that camera writing and bypass reading are not blocked, allowing the output of step one to be passed to step two.

[0053] Only spatial entropy response value It is still not enough to distinguish between the camera being blocked and the classroom lights being switched.

[0054] Therefore, within the same sampling period, the main control processor also reads the automatic exposure register and automatic white balance register of the image signal processor to obtain the illuminance drop slope. Simultaneously from the audio sampling segment Extracting sound continuity from the attitude sampling packet The motion continuity of the seating area is extracted and synthesized into a multimodal consistency value. Subsequently, the baseline threshold obtained during installation calibration was used. Starting from this point, construct the entry threshold. :

[0055]

[0056] Where: entry threshold : Represents the spatial entropy response value of the current sampling period. The lower limit for loss determination is a non-negative real number; the baseline threshold... : Indicates the initial threshold of the edge recognition device under normal classroom lighting and unobstructed conditions, written into the register during the installation and calibration phase; Illuminance drop slope : Represents the fallback rate of the auto exposure register and auto white balance register in consecutive frames, with values ​​normalized to 0 to 1; a larger value indicates a closer approximation of ambient light changes rather than occlusion; multimodal consistency value : Indicates an audio sampling segment With attitude sampling package The degree of support for the visual anomaly originating from channel damage rather than a global event is normalized to 0 to 1; the larger the value, the more consistent the visual collapse is with the stable state of other modalities.

[0057] Illuminance coupling coefficient : Used to suppress false triggering caused by light switching, preferably 0.1 to 0.5; consistency coupling coefficient : Used to improve the blinding sensitivity after cross-modal exclusion, preferably 0.2 to 0.7;

[0058] During the installation and calibration phase, the edge recognition device first collects initial samples under three unobstructed conditions: an empty classroom, normal teaching, and a teacher moving around. The main control processor then processes the spatial entropy response values ​​corresponding to each initial sample. A calibration sequence is formed, and the lower envelope, which is unaffected by projection switching, is written into the read-only calibration area as the baseline threshold. The source of the writing.

[0059] Step one focuses on the lower bound of visual usability, rather than the average state of classroom semantics; as long as the baseline threshold is met... Align with the lower boundary where the edge is still discernible, and observe the subsequent slope of the illuminance drop. and multimodal consistency value The linkage provides a stable fulcrum.

[0060] During classroom teaching, if the teacher switches the projector screen, causing the front row area to brighten momentarily, the automatic exposure register will provide a noticeable drop in illuminance within a few frames, with the illuminance drop slope being... The entry threshold has been raised. It then shifts downwards; at this point, even the spatial entropy response value A brief decline will not immediately lead to a blinding decision.

[0061] Conversely, if the lens is blocked by a book for an extended period, the automatic exposure register will still change, but the audio sampling segment... Maintain the continuity of the teacher's speech, and use gesture sampling packages. The system still shows that the main location of the classroom has not shifted, and the multimodal consistency value remains unchanged. The entry threshold has been raised. Instead, it shifts upward, thus causing the spatial entropy response value to... It falls below the stop-loss threshold more quickly. Once the judgment is made, the interrupt controller does not modify the video sampling frame. It is not itself, but only the polarization blinding interruption marker is set. .

[0062] Furthermore, the slope of the illuminance decline Excluding scene lighting prevents environmental transitions from being equated with lens damage; multimodal consistency value audio sampling segment and attitude sampling package By introducing the same adjudication path, evidence of visual abnormalities was obtained; entry requirements. Composed of installation calibration data and field sampling data, the polarization blinding interruption markers were read from the subsequent step two. This is how the source came about.

[0063] When the spatial entropy response value Less than the entry threshold When a bypass evaluation window is continuously covered, the interrupt controller issues an interrupt request line pulse and writes a polarization blinding interrupt flag into the shared register. Polarization-induced blinding interruption marker It should contain at least four types of fields: the Damaged Channel field indicates the video channel rather than the audio channel; the Buffer Page field points to the buffer page at the beginning of the corresponding frame when the trigger occurs; the Timestamp field and the audio sample segment. and attitude sampling package Subsequent alignment; the threshold source field records whether the illuminance drop slope was introduced in this determination. Multimodal consistency value .

[0064] To prevent the digital signal coprocessor from occupying the on-chip bus for an extended period, this implementation sets a watchdog counting window on the interrupt controller. The watchdog counting window is preferably set to one-half to two-thirds of the single-frame acquisition cycle. Once a timeout occurs, the digital signal coprocessor stops evaluating the current page, directly uses the threshold source field of the previous page, and clears the cache page field to prevent bypass evaluation itself from slowing down the acquisition chain.

[0065] Furthermore, the polarization blinding interruption flag output in step one... It is not an abstract alarm, but a physical trigger that can be directly used in step two to rewrite the underlying memory addressing.

[0066] Furthermore, the interrupt message field directly connects to the underlying addressing action in step two, avoiding subsequent semantic conversion; the watchdog counting window limits the bypass pre-evaluation to a controlled time, preventing low-power paths from preempting the main path; the shared register only transmits the flag and address, not the original image, thus making the interface between steps simple and stable.

[0067] In a preferred operating environment, the main processor runs a trimmed-down embedded system, and the interrupt controller polarizes blind interrupt flags. Write to the shared memory page header. This page header sequentially contains a synchronization header, a damaged channel field, a cached page field, a timestamp field, a threshold source field, and a checksum field. The synchronization header is used in step two to quickly locate valid packets. The checksum field uses cyclic redundancy check to filter out damaged packets when on-chip bus jitter occurs. If the checksum field read in step two is invalid, the previously valid polarization blinding interrupt flag is reused. Instead of performing isolation actions on the current page, this avoids triggering error interception by noisy packets.

[0068] On a motherboard without a dedicated digital signal coprocessor, step one uses the image signal processor register proxy path. The main control processor directly extracts the step size changes of the automatic exposure register and automatic white balance register in consecutive frames, and uses the absolute difference between this change sequence and the center region of the brightness plane to form a substitute loss judgment value, and then outputs a polarization blinding interrupt flag with the same name. This path does not change the subsequent terminology system; it only changes the local Hessian matrix. The extraction responsibility has been changed from the digital signal coprocessor to the image signal processor register proxy, therefore step two still receives the polarization blinding interrupt flag with the same name. .

[0069] In another enhanced implementation, the audio sampling segment It not only provides a single judgment on whether it is continuous, but also first obtains the frequency band occupancy map through short-window Fourier transform, and then the main control processor extracts the main peak continuity state of the teacher's voice region. Only when the main peak continuity state is maintained and the spatial entropy response value is... The multimodal consistency value is only increased when the collapse occurs. In the ultra-low power guarantee implementation, the register proxy of the digital signal coprocessor and the image signal processor is simultaneously turned off. The main control processor only performs absolute difference determination on the center blocks of two consecutive frames. If the center block approaches zero as a whole within the consecutive window, a polarization blinding interrupt flag is directly generated. .

[0070] By using edge recognition devices to complete the admission decision before the video enters the subsequent fusion calculation, damaged images that have lost spatial curvature are first identified by bypass and then encapsulated into an executable polarization-blinding interruption marker. Spatial entropy response value Entry requirements Illuminance drop slope and multimodal consistency value Together, they form a complete mapping chain from the physical screen to the interrupt trigger. Based on this, step two can directly take over the underlying memory addressing without having to reinterpret the source of the judgment in the previous step.

[0071] All of the following actions are performed by an edge recognition device installed at the front end of the teaching site. The edge recognition device includes a main control processor, a digital signal coprocessor, an interrupt controller, a direct memory access controller, a neural network processor, an on-chip bus, a power management chip, a frame buffer memory, and a kernel-dedicated memory area.

[0072] Step 2: Based on the polarization blinding interruption marker from Step 1 In kernel mode, the original source address of the damaged visual channel is replaced with a physical empty block of all-zero constants, while maintaining the tensor length, stride, and boundary unchanged. The neural network processor does not perform any actual multiplication or addition, and at the same time, the incomplete feature stream is submitted to step three. .

[0073] When the camera remains obstructed and step one has confirmed that the visual channel has lost its access qualification, if the conventional algorithm layer masking is still used, the main control processor still needs to send the damaged frame to the video memory first, and then the neural network processor performs a large number of zero multiplications in the channel dimension. At this time, the heat comes from data transfer and invalid emission, not from the recognition itself. Therefore, step two moves the intervention position down to the descriptor layer before the direct memory access controller. Instead of discussing ignoring this path of data in the computation graph, it changes the source of reading to a read-only kernel-dedicated memory area with all-zero constant physical empty blocks before the physical transfer begins. This allows the subsequent hardware to see a placeholder input that still has a valid size but no longer carries the damaged image content. Furthermore, the original reading source is cut off from the damaged frame buffer, so that the invalid transfer chain is broken before it enters the neural network processor.

[0074] First, the interrupt controller marks the polarization-blind interrupt. The interrupt service routine of the main controller is sent to the main controller. Subsequently, the main controller locks the descriptor ring entry corresponding to the damaged channel in kernel mode, and reads the original source address, burst length, and channel stride. Then, the direct memory access controller replaces the original source address with the base address of an all-zero constant physical empty block, and writes a zero-page short pass bit to the same descriptor header, enabling the tensor unpacking unit at the front end of the neural network processor to recognize that the channel retains only its shape and does not expand multiply-accumulate. Next, boundary alignment is corrected through adjacent padding byte pools to prevent misalignment in width, stride, or page boundaries during subsequent multimodal concatenation. Finally, the main controller generates a release execution flag. The fused input, after being occupied by equal-shaped elements, is encapsulated into a fragmented feature stream. The process is then handed over to step three for cross-modal inertial compensation. Therefore, step two does not delete the visual channel, but rather transforms it from a high-cost, damaged real load into a low-cost, uniformly bounded no-load.

[0075] The empty execution flag It includes at least the damaged channel number field, the descriptor ring entry index field, the zero-page short transmission confirmation field, and the timestamp field. The incomplete feature stream... It is formed by splicing visual placeholder segments, audio feature segments, and pose feature segments in the existing channel order. The dimension of the visual placeholder segment is consistent with the output dimension of the original visual channel, and all values ​​come from all-zero constant physical empty blocks. The audio feature segments and pose feature segments use the sampling time scale and existing dimensions from step one, without rearranging them. Therefore, step three involves reading the incomplete feature stream. At that time, there is no need to re-determine which segment comes from the visual channel after it has been emptied.

[0076] In a preferred embodiment, the main control processor runs a modified embedded system, and the direct memory access descriptors of the damaged visual channels are stored in a continuous circular descriptor table. Each circular entry includes at least the original source address field, the target video memory address field, the channel length field, the channel step size field, and a check field. Polarization blinding interrupt flag. Upon arrival, the interrupt service routine first locates the ring entry by the damaged channel number, then freezes the commit bit corresponding to that ring entry to prevent the camera interface from continuing to overwrite during the rewriting process. At this point, no deep learning framework is entered, nor is an algorithm layer mask generated; instead, a completely zero-constant physical empty block is directly selected from the kernel's dedicated memory area. and the original source address Replace with the redirected source address pointer :

[0077]

[0078] Where: source address pointer : Indicates the source address to be written to the direct memory access descriptor after rewriting, and its value is an integer address within the physical address range accessible by the edge detection device; redirection threshold. : Indicates whether to perform source address rewriting, with a value of 0 or 1; when polarization blinding interrupt flag is executed. The value is 1 if the corresponding damaged channel number is the same as the current loop term and the verification field is true; otherwise, the value is 0.

[0079] Original source address : Represents the starting physical address of the damaged visual channel in the frame buffer memory, and its value is an integer address within the frame buffer mapping area; all-zero constant physical empty block. : Represents the base address of a read-only zero page reserved in the kernel's dedicated memory area, and its value is a physical address that can be read by the direct memory access controller;

[0080] All-zero constant physical empty blocks are set as contiguous physical pages with page sizes ranging from 4KB to 16KB and page header addresses aligned to 64B or 128B, because the size is sufficient to complete the first burst access of a single vision channel block read without occupying too much on-chip storage.

[0081] If the edge recognition device adopts Bit address bus, then all-zero constant physical empty block Write to the low address mapped segment; if the edge recognition device uses a 64-bit address bus, then use an all-zero constant physical empty block. Write the constant source address mode to the direct memory access controller. After the source address is rewritten, the main controller immediately flushes the descriptor cache and invalidates the old address translation entries, then reopens the commit bit, causing the direct memory access controller to follow the new source address pointer. Initiate the transfer. To prevent mixed memory accesses (half old, half new) within the rewrite window, the check field of the ring descriptor table preferably uses cyclic redundancy check, with a ring depth of 16 to 64 entries. When the check field fails, the interrupt service routine does not commit the current ring entry but instead continues to output audio and gesture sampling segments, thus preventing the damaged visual channel from sending incorrect addresses to the neural network processor during switching. During board assembly, the memory page containing the kernel's dedicated memory area and the frame buffer page are preferably placed in different memory banks. The signal ground between the camera module cable, the main control processor, and the neural network processor uses a star-shaped return path. This is because if the same high-frequency return path is shared during an interrupt, the descriptor latch edge is susceptible to crosstalk, leading to board-level mismatch where the damaged channel number is correct but the source address is written late. Furthermore, the ring descriptor table should be placed near the main control processor, and the frame buffer memory should be placed near the camera interface, making the address rewrite path shorter than the original video transfer path, thus facilitating the compression of the control actions in step two within the interrupt window.

[0082] For example, during an exam, a student used a notebook to cover the camera lens; step one provides a polarization blinding interruption marker. Then, the main control processor first stops one of the descriptor ring entries corresponding to that channel. The video block on the screen that was originally intended to be sent to the neural network processor no longer points to the frame buffer page, but instead points to a physical empty block of all zero constants in the kernel's dedicated memory area. The visible result was not that the image was repaired, but that the visual channel no longer carried the texture content of the laptop surface from the next segmentation onwards, and subsequent links retained the size but lost the burden of image processing.

[0083] Furthermore, the interrupt service routine directly locates the descriptor ring entry; polarization blinds the interrupt flag. One-to-one correspondence with the underlying transport markers; source address pointer The code was rewritten to lower the intervention point to the transport entry point, eliminating meaningless masking within the computation graph; all-zero constant physical empty blocks. Use consecutive pages and fixed alignment rules.

[0084] Source address pointer After the rewriting is completed, if the zero page is sent directly to the neural network processor, the multimodal splicing position will still be offset due to the inconsistency between the channel length, the block header and the bus burst boundary.

[0085] Therefore, step two involves a zero constant physical empty block. Set up a padding byte pool at adjacent positions, and use the load length of the damaged channel. Descriptor header length and alignment granularity Calculate the number of padding bytes :

[0086]

[0087] Where: number of padding bytes : Indicates the number of trailing bytes appended to the padding byte pool to meet the current channel transport boundary; the value is an integer greater than or equal to 0; Damaged channel load length : Indicates the theoretical byte length of the damaged visual channel in this transfer, with a value that is a positive integer; Descriptor header length : Indicates the fixed header length of this channel before it is fed into the neural network processor; the value is a positive integer; Alignment granularity : Indicates the minimum alignment unit required for the joint operation of the on-chip bus and neural network processor, preferably 64, 128 or 256 bytes;

[0088] Padding bytes After writing, the direct memory access controller no longer initiates memory access according to the complete span of the original damaged frame. Instead, it uses a combination of constant source address and tail padding to complete a single transmission. At the same time, the zero-page short pass bit in the descriptor header is sent to the tensor unpacking unit at the front end of the neural network processor. After seeing the zero-page short pass bit, the tensor unpacking unit does not allocate a subsequent multiply-accumulate emit queue for the visual channel. Instead, it only retains the channel boundary, channel index, and timestamp so that the audio sampling segment and the pose sampling segment can still enter the fusion unit in the original order.

[0089] In other words, step two does not shut down the entire neural network processor, but rather maintains the damaged visual channel in a state where the boundaries remain intact, the content is empty, and the emission converges. This allows the incomplete feature stream received in step three to... The channels can still be arranged in the original order, but the visual channel section has been replaced with equal-boundary placeholder segments.

[0090] In a more specific board-level implementation, the frame buffer memory uses a low-power double data rate memory, and other low-power dynamic memory with equivalent functionality is included as an equivalent device in this implementation; the direct memory access controller is connected to the neural network processor via an advanced scalable interface bus, and the burst length is set to 4 to 16 cycles; byte pools and all-zero constant physical blocks are padded. Arranged on adjacent physical pages, the padding byte pool can cover both empty block reads and tail padding in a single address translation, reducing secondary descriptor switching. If board-level resources are tighter, the padding byte pool is shrunk into a single-page circular area and rolled back to the beginning of the page after each rewrite; if board-level resources are wider, the padding byte pool is expanded into a multi-page sequential area to alleviate bus contention caused by frequent rollbacks in high-concurrency classrooms.

[0091] Then, pad the number of bytes. The original step size and stitching boundary of the damaged visual channel replaced by zero pages are not disrupted by the ordering of audio and pose sampling segments; zero-page short pass enables the neural network processor front end to converge the emission queue of the corresponding channel, propagating it from the transport layer to the execution layer; and fills in the byte pool and all-zero constant physical empty blocks. Adjacent paths ensure that the rewritten memory access path has a stable engineering form.

[0092] In the implementation, the source address pointer The rewriting and buck triggering of the power management chip are completed serially within the same interrupt window. Before reopening the descriptor commit bit, the main control processor sends a level-to-together signal to the power management chip via general-purpose input / output pins, causing the neural network processor to switch down from its current operating frequency to a lower frequency. Then, the direct memory access controller commits the rewritten loop term. The reason for this arrangement is that after the damaged visual channel is physically unloaded, the effective load of the neural network processor has already decreased. Switching down the power supply frequency in advance can further shorten the ineffective oscillation time of the unloaded channel on the on-chip power grid. If the motherboard uses a discrete aluminum heat sink, the connection between the general-purpose input / output pins and the power management chip is preferably placed on the short side of the motherboard to shorten the signal line length. If the motherboard uses a small integrated metal casing, the connection is preferably kept away from the camera module cable to avoid additional crosstalk within the interrupt window.

[0093] In another parallel implementation, if the deployment environment restricts kernel write permissions, the main control processor cannot directly rewrite the source address pointer. Then, a negative infinity constant sparse matrix with the same shape as the damaged visual channel is constructed in the input layer of the deep learning framework, and fed into the pre-activation buffer, so that the exponential normalization stage merges the corresponding channel into zero-value placeholders. This path still uses polarization blinding interruption markers. Incomplete feature flow and empty execution mark This set of terms simply shifts the focus of low-level memory addressing isolation from the kernel descriptor layer to the frame input layer. In another degraded implementation, if the on-chip bus is congested, the main processor calls the video driver's frame dropping interface to directly truncate the currently damaged frame buffer pages, and then combines the truncated result with the audio sampling segment and the attitude sampling segment to form a shortened, incomplete feature stream. This is for reading in step three. The former is suitable for software environments with restricted permissions, while the latter is suitable for hardware environments with extremely tight bus loads, allowing damaged vision channels to be removed from high-cost handling.

[0094] Specifically, the preferred evaluation metrics for step two are the number of descriptor rewrite hits, the number of zero-page short pass bit triggers, and the number of padding bytes. Landing point distribution and empty execution marker Interruption markers for polarization-induced blinding The timing correspondence should be observed. For board-level verification, observe the timing sequence of general-purpose input / output pins, power management chip interrupt pins, and the first burst address of the advanced scalable interface bus on a logic analyzer; for system-level verification, record the incomplete feature stream. Does the channel sequence remain unchanged when reaching step three?

[0095] Furthermore, source address rewriting and buck conversion share an interrupt window, and the transfer of discharge and power supply are closed-loop; parallel alternative paths can use the same principle simultaneously under different permissions and buses; the evaluation metrics are descriptor, padding and timing, which correspond to the hardware actions in step two.

[0096] When using it, through the source address pointer Rewrite and pad with bytes The calculation and zero-page short pass are linked to activate the polarization blinding interrupt flag output in step one. This is transformed into physical actions that can be executed by both direct memory access controllers and neural network processors. Because the damaged visual channel retains its original channel order, timestamps, and boundary information, the incomplete feature stream received in step three... No splicing misalignment will occur, thus enabling cross-modal compensation to continue without reverting to the original frame.

[0097] Step 3: Empty the visual channel and use the audio sampling segment. Attitude sampling package Empty execution flag For incomplete feature flow Implementing cross-modal complementation and inertial damping, the output boundary is a continuous steady-state multi-class distribution. This process is all completed by edge recognition devices installed at the front end of the teaching venue.

[0098] Step two, by rewriting the damaged visual channel to an empty load, relieves the pressure on computing power and heat. However, another type of disruption immediately occurs in the fusion link: visual texture disappears, but the classifier still retains the category output. As a result, the conclusions at adjacent time points can easily swing rapidly between "writing while seated," "abnormal departure," and "continuous absence." This swing does not stem from abrupt changes in actual behavior, but from the information gap caused by the deliberate severing of visual evidence. Therefore, the task of step three is not to restore the image, but to reconstruct a continuous decision trajectory around this information gap, so that the edge recognition device can still provide a propagable and adjustable classification output without re-ingesting dirty data.

[0099] Among them, the main control processor reads the incomplete feature stream. Corresponding timestamp, audio sampling segment Extract acoustic continuity relationships and attitude sampling packages. Extract the skeleton anchor point continuation relationship, and the fusion unit performs an execution flag based on the void execution. Generate the filler proportions and reorganize the current original distribution using the filler proportions. Next, the main control processor uses a one-dimensional state register chain retrieved from the on-chip static random access memory to retrieve the current original distribution. and the steady-state multi-class distribution at the previous time step Damped stitching yields the steady-state multi-class distribution at this moment. If the visual hole persists, the serial peripheral interface wakes up the external microphone array, and the main pickup lobe redirects to the uncovered seating area, further increasing the audio sampling segment. The directional effectiveness. Therefore, step three rewrites the empty channels removed in step two as filler conditions, rather than converting the empty channels into identification endpoints.

[0100] In a preferred embodiment, the main control processor freezes the fusion cache page corresponding to the current timestamp, starting from the audio sampling segment. Calculate audio continuity From the pose sampling packet Calculate attitude continuity Audio continuity Continuous amplitude values ​​of writing friction noise, page turning noise, and ambient noise in the seat area are taken for consecutive short windows, not single-point amplitude values; posture continuity. Take the continuous amplitude values ​​of head and shoulder anchor points, forearm swing amplitude, and torso direction in consecutive frames, and do not take the single frame attitude labels.

[0101] Subsequently, the fusion unit adjusts the audio continuity. Attitude continuity With empty execution flag Write the percentage of compensation :

[0102]

[0103] Where: the proportion of compensation : Indicates that the current moment requires an audio sampling segment With attitude sampling package The degree to which visual voids are taken over, with values ​​ranging from [value range missing]. Audio continuity : Indicates an audio sampling segment The degree to which events are preserved within consecutive short windows, with values ​​ranging from [value range missing]. Attitude continuity : Indicates attitude sampling packet The degree to which skeleton anchor points are maintained across consecutive frames, with values ​​ranging from [value range missing]. ; Empty execution flag : Indicates whether physical venting of the visual channel has been performed in step two, with a value of 0 or 1;

[0104] Audio coupling coefficient : Indicates audio continuity The weights, ranging from positive real numbers, are preferably 1 to 3; the attitude coupling coefficient. : Indicates attitude continuity The weights, ranging from positive real numbers, are preferably 1 to 3; the coupling coefficient is empty. : Indicates empty execution flag The effect of increasing the fill ratio is a positive real number, preferably greater than the audio coupling coefficient. Coupling coefficient with attitude ;

[0105] In terms of structural implementation, the audio front-end preferably consists of four microelectromechanical microphones (MEMS), mounted on a 1.0mm thick copper-clad fiberglass substrate with an adjacent center-to-center distance of 18mm. The posture front-end is preferably positioned beside the main camera module, outputting only the skeleton anchor point coordinates and torso direction index. The fusion unit uses this information to extract the incomplete feature stream. This is considered a placeholder segment that retains the channel position but loses visual content, combined with the compensation ratio. Recombination yields the current original distribution Regarding the time window setting, the audio sampling segment... The short window length is preferably 24ms to 40ms, attitude sampling packet The length of consecutive frames is preferably 3 to 7 frames; the main control processor pushes the two into the same fusion cache page with the same timestamp, writes the seat number, timestamp and hole source field at the beginning of the cache page, and writes the check word at the end of the cache page.

[0106] Therefore, step three reads not scattered features, but rather a stream of incomplete features. Evidence of alignment compensation includes, for example, students using textbooks to block their view, visual aisle not empty, but continued pen-scratching sounds from the seating area, and forearm movement indexes at the forehand; the proportion of compensation in this situation. The original distribution has been raised. Dominated by visual texture, audio sampling segment With attitude sampling package Jointly driven.

[0107] Furthermore, the proportion of compensation Set the empty execution flag in step two. Directly connect to step three; audio continuity and attitude continuity It focuses on continuous rather than instantaneous values, which can fit the occluded scene; the arrangement of four microphones and the pose front end.

[0108] Current original distribution After generation, the main control processor does not immediately output it, but first quantizes it relative to the steady-state multi-class distribution of the previous time step. The transition strength is then determined. Therefore, the main control processor calculates the decision transition strength. :

[0109]

[0110] Where: decision transition strength : Represents the current original distribution Compared to the steady-state multi-class distribution at the previous time step The total jump variable takes values ​​of non-negative real numbers.

[0111] Calculating the decision transition strength Previously, the fusion device first determined the compensation ratio. Generate the current original distribution :

[0112]

[0113] in, For incomplete feature flow The initial distribution obtained from the original classification head, For audio sampling segment With attitude sampling package The initial distribution obtained after supplementing the sub-branch. The main control processor... and First, perform same-dimensional normalization, then mix according to the above formula to obtain the current original distribution where the sum of each component is 1. .

[0114] Number of categories : Indicates the number of classification categories used in the task, and the value is a positive integer greater than or equal to 2; component : Represents the current original distribution In the The components in each category have a value range of . ; quantity : Represents the steady-state multi-class distribution at the previous time step In the The components in each category have a value range of . ;symbol : Represents the summation operator, used to aggregate jump variables of all categorical components;

[0115] Subsequently, the main control processor sets the compensation ratio. With judgment transition strength Write the values ​​into the one-dimensional state register chain to generate the damping gate. Then use the damping threshold stitching together the current original distribution Compared with the steady-state multi-class distribution at the previous time step :

[0116]

[0117] Where: Damping threshold : Represents the current original distribution The proportion directly adopted during suturing, with a value range of [value missing]. ;

[0118] Compensation ratio : Indicates an audio sampling segment Attitude sampling package and empty execution mark The degree of complementation formed by the joint process has a range of values. ; transition coupling coefficient : Indicates the strength of the decision transition Damping threshold The inhibitory effect is measured by values ​​ranging from positive real numbers, preferably 1 to 4; the stability constant... : Represents a small positive number that prevents the denominator from being zero, and its range is 1. ;

[0119]

[0120] Where: steady-state multi-class distribution : Represents the output probability vector after damping stitching, with the sum of its components being 1; damping threshold. : Represents the current original distribution The adoption rate, with a value range of 100%. Current original distribution : Represents incomplete feature flow The instantaneous probability vector obtained after recombination with the replacement channel; the steady-state multi-class distribution at the previous time step. : Represents the steady-state probability vector that has been output and written into the state register chain in the previous time step;

[0121] The one-dimensional state register chain is preferentially stored in on-chip static random access memory, and each category component is stored in fixed-point format. Each category component in the state register chain is preferentially stored in 16-bit fixed-point format, and its depth-first storage is used to store the most recent 8 sampling periods, updating the steady-state multi-class distribution each time. Once completed, first write to the current page, then move the previous page to the steady-state multi-class distribution from the previous time step. A reserved page is used to prevent historical tracks from being overwritten by the same page. If the edge recognition device has a floating-point unit, the damping threshold is... Update directly according to the closed expression above; if the edge recognition device does not have matrix inversion capability, update according to the same closed expression without high-dimensional inversion. This implementation path corresponds to a one-dimensional Kalman inertial chain, and the final implementation is the gated stitching of a writable register.

[0122] For example, within several sampling periods after the lens is obstructed, the current original distribution The audio sampling segments successively exhibited behaviors such as head-down stagnation, persistent absence, and getting up and leaving their positions. The writing friction sound continues to be emitted, and the posture sampling package is still being used. The forearm swing is still not interrupted; at this time, the damping threshold is... It will not allow the current original distribution to be Directly overturn the steady-state multi-class distribution of the previous moment Instead, it pulls the output back to a trajectory more consistent with continuous motion.

[0123] Furthermore, the judgment transition strength To provide quantifiable inputs for the flickering problem; damping threshold Replacing high-cost matrix inversion with closed-gating; steady-state multi-class distribution It directly becomes the input for step four.

[0124] Furthermore, when the empty execution flag is executed... Continuous setting and audio continuity While still valid, the main control processor wakes up the external microphone array via the serial peripheral interface. The external microphone array is preferably mounted on a 2.0mm thick aluminum alloy bracket, which is fixed below the main camera module via a dual-axis rotating frame, corresponding to the horizontal and pitch axes respectively. Based on the seat index retained in step two and the last valid visual coordinates from step one, the main control processor deflects the main pickup lobe to the seat area that has lost visual coverage, and then writes the updated directional acoustic results back to the audio continuity. The generation chain is such that step two first empties the vision, and step three uses directional acoustics to take over the hole in a cooperative relationship, rather than isolated acoustic enhancement. After the main lobe is locked, the master processor writes the locked seat number and the start and end times of the lock back to the fusion cache page, so that step four can determine whether the hole belongs to a continuous homogeneous event during subsequent threshold callbacks. This field is not written to plaintext audio.

[0125] In a parallel alternative implementation, if the floating-point capability of the edge recognition device is insufficient to continuously update the damping threshold... Then the main control processor switches the one-dimensional state register chain to a hidden Markov state chain. The hidden Markov state chain defines being seated writing, looking down and pausing, looking left and right, getting up and leaving, and continuous absence as discrete hidden states, and uses audio sampling segments... With attitude sampling package The continuation label is defined as the observation symbol, and the state transition table pre-written in the read-only memory is used to complete the step-by-step recursion. This alternative path does not change the incomplete feature flow. Empty execution flag and steady-state multi-class distribution The terminology system simply changes the calculation method of damping stitching from closed-gate control to discrete-state recursion.

[0126] In the downgraded backup implementation, if the audio sampling segment With attitude sampling package Simultaneously, the continuity is lost, and the main control processor suspends the neural network processing chain, reading the historical normal motion dictionary for this seat from the motherboard flash memory. Each record in the historical normal motion dictionary preferably includes a torso angle range, head pitch range, forearm swing range, and writing rhythm label. The main control processor loads the corresponding record according to the seat number and timestamp index, and then writes the baseline state generated by the record into a steady-state multi-class distribution. The low-confidence slots are used to ensure that there is still parseable input in step four.

[0127] Among them, the evaluation index selected is the decision transition strength. Time series, damping threshold Write register counts, steady-state multiclass distribution The number of category switching times, microphone array main lobe locking times, and historical normal action dictionary trigger times. The order of dual-axis rotary table control pulses, serial peripheral interface wake-up timing, and state register chain update timing; during system-level verification, incomplete feature streams. Empty execution flag and steady-state multi-class distribution The loop is closed at the same timestamp; then, the external microphone array transforms the visual void into a target area for directional acoustic blinding, the hidden Markov state chain provides an equivalent path for the low floating-point platform; the historical normal action dictionary preserves the minimum boundary for continuous output for extreme degradation conditions.

[0128] Step 4: Calculate the steady-state multi-class distribution output from Step 3. Converted into an out-of-band hardware write-back signal, and used to reverse-adjust the front-end admission threshold in step one. When a local closed-loop failure occurs, the incomplete latent vector slice is suspended. and deadlock beacon All of the above actions are performed by edge recognition devices installed at the front end of the teaching venue.

[0129] Step three has already utilized the audio sampling segment. Attitude sampling package and incomplete feature flow Stitching visual voids into a steady-state multi-class distribution However, this steady-state multi-class distribution The physical sensitivity of the front-end sensor entry point remains unchanged. If the front-end access threshold... If the edge detection device remains stuck at old values ​​for an extended period, it will continuously send similar long-tailed contaminants into step one, causing steps two and three to repeatedly enter a dissipation loop. Therefore, step four no longer aims to retrain the classifier, but instead focuses on establishing a steady-state multi-class distribution. The level of confusion is transcribed into an out-of-band control chain, and the front-end hardware threshold in step one is modified in reverse.

[0130] Among them, the main control processor first reads the steady-state multi-class distribution. With damping threshold The prediction uncertainty range is extracted; then, the prediction uncertainty range is fed into a first-order Markov damping chain to generate an out-of-band drive level; the quantized threshold code is then written back to the admission threshold register at the front end of step one via the I2C inter-integrated circuit bus, causing the interception boundary of step one to drift; subsequently, a long-tail catastrophe watchdog continuously monitors the admission threshold. Whether the prediction is pushed to the physical lower boundary and whether the prediction uncertainty remains high; if the above state continues to cover the preset window, the main control processor stops local closed-loop write-back and instead extracts incomplete latent vector slices from the cross-modal fusion breakpoint. And simultaneously set the deadlock beacon. .

[0131] In a preferred embodiment, the main control processor is distributed from a steady-state multi-class distribution. Extract all category components, then extract the damping threshold. As a historical dependency compensation term. Due to the damping threshold. The smaller the value, the more the current moment relies on the historical steady-state chain to maintain continuity. Therefore, in step four, the entropy value is not considered as the degree of confusion alone, but rather in conjunction with the damping threshold. The margin is included together with the prediction uncertainty range. .in:

[0132]

[0133] Where: the prediction uncertainty range : Represents the current steady-state multi-class distribution The combined result of the degree of confusion and the inertial drag margin has a value range of non-negative real numbers; components : Represents a steady-state multi-class distribution In the The components in each category have a value range of . Number of categories : Represents the total number of categories in the classification task, and its value is a positive integer greater than or equal to 2; stability constant. : Represents a small positive number to prevent the logarithmic term from becoming unstable, with a range of values ​​of 1000. ;coefficient : Indicates the damping threshold Margin affects the range of prediction uncertainty The compensation weight, with a value ranging from a positive real number, is preferably between 0.2 and 1.5; the damping threshold... : Indicates the adoption ratio when stitching the current original distribution with the previous steady-state distribution in step three, with a value range of . ;

[0134] In board-level implementation, steady-state multi-class distribution The damping gate is located in the on-chip static random access memory to the right of the main control processor. The timestamp is written to the cache page header along with the cache page header, facilitating a single memory access in step four. Low-power motherboards use fixed-point logarithmic lookup tables, while motherboards with hardware floating-point functionality use floating-point paths; both share the same prediction uncertainty range. .

[0135] For example, after the test paper has been attached to the front of the lens for a long time, the category components are similar and the damping threshold is... It will decrease; the prediction uncertainty read by the main control processor is extremely poor. Raise.

[0136] Furthermore, the prediction uncertainty is extremely poor. Simultaneously absorbs steady-state multi-class distribution Entropy information and damping threshold Historical dependencies; the on-chip cache layout ensures continuous parameter passing from step three to step four.

[0137] Prediction uncertainty range After generation, the main control processor does not directly write it back to the front-end register. Instead, it first sends it to a first-order Markov damping chain to ensure that the out-of-band drive level changes in the same direction as the thermal hysteresis of the edge recognition device. If the drive level has a very poor instantaneous prediction uncertainty... Too sensitive, front-end access threshold It will oscillate back and forth over several consecutive sampling periods, thus tearing the interception boundary in step one into a jagged shape. To avoid this phenomenon, the main control processor adjusts the uncertainty based on the difference between the out-of-band drive level of the previous moment and the prediction uncertainty of the current moment. The weighted combination of the cost of the external drive level at any moment :

[0138]

[0139] Where: external drive level : This represents the normalized drive quantity generated by the damping chain in step four and prepared to be sent to the bus write-back path. Its value range is [value range missing]. ; previous moment's external drive level : Represents the normalized driving quantity written to the state register chain in the previous sampling period, with a value range of . ; forecast uncertainty range : Represents the current steady-state multi-class distribution The result of the confusion level synthesis is a non-negative real number.

[0140] Damping coefficient : Indicates the external drive level at the previous moment For the current out-of-band drive level The degree of retention, with a value range of . The preferred value is 0.75 to 0.95; normalization constant : indicates the range of prediction uncertainty The scaling constant compressed to the normalized interval takes the value of a positive real number, preferably a fixed constant of the same order of magnitude as the median value of confusion during the installation and calibration stage;

[0141] Subsequently, the main control processor sets the external drive level. Converted to the front-end admission threshold in step one The write-back amount, truncated according to the upper and lower boundaries:

[0142]

[0143] Where: Admission threshold : Represents the normalized threshold value that the front-end admission threshold register should reach in the current sampling period in step one, with a value range of . The previous admission threshold : Represents the normalized threshold value of the front-end admission threshold register in step one for the previous sampling period, with a value range of . ;coefficient : Indicates the external drive level Alignment threshold The compressive strength, taking values ​​in the range of positive real numbers, is preferably between 0.05 and 0.3;

[0144] coefficient : Indicates the damping threshold Alignment threshold The tensile strength, taking values ​​in the range of positive real numbers, preferably less than the coefficient. Lower boundary : Represents the physical minimum threshold of the front-end admission threshold register, corresponding to the upper edge of the register level dead zone; upper boundary : Represents the physical maximum threshold of the front-end admission threshold register, corresponding to the upper limit value when the front-end admission chain maintains open access;

[0145] In terms of hardware connectivity, the main control processor connects to the front-end sensor board via the I2C inter-integrated circuit bus. The front-end sensor board is equipped with an admission threshold register, a status register, and a write-back confirmation register. The write-back message includes a start bit, device address, register address, threshold code, checksum byte, and stop bit. After writing, the main control processor immediately reads the write-back confirmation register. If the returned value matches the quantized admission threshold... If the results are consistent, the write-back is complete. If the results are inconsistent, the data will be retransmitted once within the same sampling period, and the bus alarm bit will be recorded.

[0146] Furthermore, the main control processor writes the timestamp, threshold, and acknowledgment to the same status page after each write operation, continuously tracing steps one through four, rather than mixing the front-end register state and the back-end distribution state. For example, if the device is repeatedly obstructed by similar elements at the front end of the examination room, the main control processor calculates a higher out-of-band drive level. Re-quantify the admission threshold Write to the front-end register, and before the next batch of video sampling frames has completed step one, the interception boundary has already moved forward.

[0147] Furthermore, out-of-band drive level Make steady-state multi-class distribution Able to be converted into a write-back level through a damping chain; admission threshold The updated formula takes the damping gate value from step three. Include pullback terms to avoid one-way collapse of the threshold.

[0148] In long-tail disaster scenarios, the front-end admission threshold Despite continuous downward pressure, steady-state multi-class distribution It still cannot return to the low confusion range, at which point the local closed loop is approaching its limit. To prevent steps one through four from continuously dissipating in the same infinite loop, the edge recognition device is equipped with a long-tail catastrophe watchdog. The long-tail catastrophe watchdog checks three conditions simultaneously in each sampling period: first, the admission threshold. Has it been pressed to the lower boundary? Secondly, the prediction uncertainty is extremely poor. Whether it remains high within a continuous window; and third, whether the write-back confirmation register has continuously returned the same threshold code without any further valid changes.

[0149] If all three conditions are met, the main controller processor will stop local out-of-band write-back and switch step four to suspend mode.

[0150] In suspended mode, the main control processor extracts incomplete latent vector slices from the cross-modal fusion breakpoint. Incomplete latent vector slices Select a continuous tensor slice from the end of the fusion unit and before the classifier input in step three. The value format is a 16-bit fixed-point array. The beginning of the array contains the timestamp, seat number, threshold code, and channel hole source field, and the end contains the verification field. The main control processor simultaneously sets the deadlock beacon. Deadlock The label must include at least a dead zone lock field, a bus health field, and a write-back stall field to indicate to step five that the local closed loop will no longer continue.

[0151] To ensure that deadlock detection has executable rules, the main control processor first uses a rule of length [missing information]. Calculate the window-averaged prediction uncertainty within a continuous window:

[0152]

[0153] in, Determine the window length for deadlock. This represents the range of prediction uncertainties for each sampling period within the window. If the following conditions are met simultaneously: the current admission threshold... The lower boundary has been reached. Window average prediction uncertainty Not lower than the preset deadlock threshold If the write-back confirmation register continuously returns the same threshold code within the same window, then the deadlock beacon is set. And extract incomplete latent vector slices .

[0154] Deadlock Beacon It also includes breakpoint layer number field, breakpoint page offset field, start channel index field, and end channel index field. The breakpoint layer number field is used to identify the operator level corresponding to the current deadlock, the breakpoint page offset field is used to identify the breakpoint page location in video memory, and the start channel index field and end channel index field are used to limit the local weight block range of the patch writing.

[0155] In one example of on-site action, the camera was covered by a textbook for an extended period, affecting the front-end access threshold. It also repeatedly writes back and pushes to the lower boundary, confirming that the register continuously writes back the same threshold code, and the main control processor continues to read the high-order prediction uncertainty extreme difference. The edge recognition device stops writing back, and then copies the continuous latent vectors from the cross-modal fusion breakpoint to the suspend buffer, setting the deadlock beacon in the status page. .

[0156] In the enhanced implementation, the damping coefficient In conjunction with the thermal recovery constant of the passive heat sink aluminum plate, the passive heat sink aluminum plate is preferably made of anodized aluminum plate with a thickness of 2.0mm to 3.0mm, so as to enable the out-of-band drive level The rate of change is in the same direction as the heat dissipation rhythm of the aluminum plate. In the parallel alternative implementation, if the flexible cable is damaged, causing the I2C bus between integrated circuits to fail, the main control processor switches to the parameter mapping file path at the operating system level to periodically overwrite the parameter nodes exposed by the front-end driver instead of writing back the hardware bus, while maintaining the admission threshold. Incomplete latent vector slices With deadlock beacons The terminology system remains unchanged.

[0157] In the degradation-assurance implementation, when the main control processor load approaches its limit, step four stops all entropy paths and only reads the steady-state multi-class distribution. The difference between the largest and second-largest class components is calculated and written back to the front-end register with a fixed step size; when the degradation path also triggers the long-tail disaster watchdog, the incomplete latent vector slice is directly suspended. And deadlock beacons.

[0158] Among them, the evaluation index is generally selected as the range of prediction uncertainty. Time series, out-of-band drive level Write-back count, admission threshold Code lines, incomplete latent vector slices Number of hangs, deadlock beacons Number of bit set. Examine the timing relationships of the I2C start bit, threshold code, and acknowledgment readback on the inter-integrated circuit bus; during system-level verification, examine the steady-state multi-class distribution. Admission threshold Incomplete latent vector slices and deadlock beacon Whether they are linked on the same timestamp chain.

[0159] Furthermore, the long-tail disaster watchdog defines the endpoint for local closed loops; incomplete latent vector slices Provide intermediate materials for step five; deadlock beacon Transform the threshold dead zone locking fact into a cross-step interface.

[0160] Because the long-tail disaster watchdog suspends the incomplete latent vector slice when the local loop closure fails. With deadlock beacons Step five accepts not fragmented logs, but available intermediate states with timestamps, threshold codes, and breakpoint semantics.

[0161] Step 5: Slice the incomplete latent vectors that were suspended in Step 4. With deadlock beacons The algorithm is converted to a cross-domain low-rank adaptive residual patch, and the operator topology is reshaped after hot-loading the patch on the edge side. Then, it is based on the new prediction uncertainty range. Reverse release step four crushed admission threshold All of the above actions are performed through edge recognition devices installed at the front end of the teaching venue and remote cloud parsing nodes.

[0162] Steps one through four have blocked dirty data at the front end, stitched visual holes into a steady-state distribution, and then converted the steady-state distribution into a hardware write-back level. However, the front-end admission threshold... It has fallen to the physical lower boundary and the prediction uncertainty is extremely poor. When the level remains high, the local closed loop has lost its ability to explain new pollution patterns. Continuing to loop within the edge recognition device at this point will only repeatedly trigger the interception in step one, the release in step two, the compensation in step three, and the downward pressure in step four, without generating new structural knowledge. Therefore, step five treats the fragments that the local closed loop cannot explain as entry points for higher-level solutions, causing the edge recognition device to stop consuming data locally and instead utilize remote cloud parsing nodes to slice the incomplete latent vectors. Perform topology-level reconstruction.

[0163] Among them, the edge recognition device first slices the incomplete latent vector. Deadlock beacon Prediction uncertainty range Admission threshold and the steady-state multi-class distribution output in step three The payload is compiled into a suspended payload; the communication processor de-identifies and encapsulates the suspended payload using Advanced Encryption Standard (AES) Block Link Authentication Mode and sends it to the remote cloud resolution node via a wide area link.

[0164] The remote cloud parsing node generates a slice of incomplete latent vectors based on the frozen base operator topology object. The generated low-rank adaptive residual patch; after receiving the low-rank adaptive residual patch, the edge recognition device uses a mutex write lock to write the patch to the breakpoint operator in the video memory; after the new operator topology is generated, the main control processor recalculates the updated prediction uncertainty range. and put by Towards The falling vector is converted into a reverse release level to raise the locked admission threshold. Therefore, step five transforms the suspended state of step four into a cross-domain closed loop, instead of remaining in a local infinite loop.

[0165] In a preferred embodiment, the communication processor of the edge recognition device first reads the incomplete latent vector slices sequentially from the pending buffer. Deadlock beacon Prediction uncertainty range Admission threshold and steady-state multi-class distribution Write a continuous payload page, the first part of which consists of a timestamp field, a seat number field, a dead zone lock field, a step number field, and a check field, and the middle part stores the incomplete latent vector slice. A fixed-point array, followed by the storage of steady-state multi-class distributions. With the range of prediction uncertainty Compressed representation.

[0166] The header of the continuous payload page also contains a slice length field, a slice dimension field, a quantization bit width field, and a layout field. The slice length field records the effective byte length of the incomplete latent vector slice; the slice dimension field records the dimension order and length of each dimension of the incomplete latent vector slice; the quantization bit width field records the bit width format used by the middle fixed-point array; and the layout field records the arrangement order of the fixed-point array in the continuous payload page. The middle fixed-point array is written according to the order displayed in the layout field, and the tail check field records all bytes from the header and middle sections to ensure that the remote cloud parsing node can recover the tensor form of the incomplete latent vector slice and perform patch solving.

[0167] The communication processor encapsulates the ciphertext page by page using Advanced Encryption Standard (AES) Block Link Authentication Mode (BLOCK) and then performs byte compression to reduce the amount of data transported over wide area links. Upon receiving the ciphertext, the remote cloud parsing node first performs authentication and verification, then decapsulates the payload pages from the isolation container to recover the incomplete latent vector slice. Deadlock beacon Steady-state multi-class distribution The correspondence is as follows. Remote cloud parsing nodes are distributed within independent container clusters. Container entry points can only receive consecutive payload pages and cannot receive plaintext images, plaintext audio, or raw pose sequences. Within the container, the parsing context is first obtained by seat number and timestamp. Then, the most recently suspended windows from the same terminal are connected into a time chain to separate occasional noise and deadlocked paths. However, this is not written back to the edge; it only participates in the patch matrix. Please provide a solution.

[0168] During the cloud-based solution phase, the remote cloud parsing node freezes the base operator topology. Starting from this point, the left and right factors of the low-rank adaptive residual patch are... and Perform the solution and constrain the patch to the vicinity of the breakpoint level. The patch body is written as follows:

[0169]

[0170] Where: patch matrix : Indicates that the remote cloud parsing node slices the incomplete latent vector. The obtained low-rank adaptive residual patch takes the value of a real matrix; left factor : Represents the patch factor of the low-rank adaptive residual patch in the column space, and its value is a real matrix;

[0171] Right factor : Represents the patch factor of the low-rank adaptive residual patch in the row space, with values ​​being real matrices; Freeze the base operator topology. : Represents the basic operator topology currently used by the edge recognition device, with values ​​taking the form of a fixed set of weights; incomplete latent vector slices : Represents the fusion breakpoint tensor fragment suspended in step four, with values ​​either fixed-point or floating-point arrays; steady-state multi-class distribution : Represents the probability vector output from step three, with values ​​being a distribution vector where the sum of its components is 1; deadlock beacon : This indicates the dead zone locking and write-back stall combination flag set in step four, with a value of structured beacon;

[0172] Remote cloud parsing nodes based on incomplete latent vector slicing Distribution shape, steady-state multi-class distribution Category preferences and deadlock beacons The locked positions are then used to remove patch components unrelated to deadlock, retaining only low-rank increments pointing to breakpoint operator pages. Thus, the patch matrix... It does not cover the entire model domain, but rather converges the topology around the breakpoint after the suspension.

[0173] Remote cloud parsing nodes are solving for left factors. With right factor First, slice based on the incomplete latent vector. The dimensional information is read from the number of input and output channels of the breakpoint layer, making the left factor... The number of rows is consistent with the number of input channels in the breakpoint layer, making the right factor... The number of columns is consistent with the number of output channels of the breakpoint layer, and the common dimension of the two factors is a low-rank dimension. When solving, first fix the right factor. Update left factor Then fix the left factor Update right factor The patch matrix is ​​generated iteratively using an alternating update method. When the patch matrix corresponds to two consecutive iterations. The patch solution is terminated when the category order no longer changes after the breakpoint layer is applied, or when the difference between the patch matrices of two consecutive iterations is lower than a preset stopping threshold. The patch solution uses only incomplete latent vector slices. Steady-state multi-class distribution With deadlock beacons It does not call plaintext images, plaintext audio, or the original pose sequence.

[0174] In one example of real-world action, a terminal in the front row of a classroom repeatedly enters the dead zone lock-on. The edge recognition device uploads not plaintext, but incomplete latent vector slices with timestamps and seat numbers. After the remote cloud resolution node is unblocked, the left and right factors are directly derived. and Low-rank adaptive residual patching prevents the uploading of original video to edge recognition devices.

[0175] Continuous load pages ensure the integrity of the intermediate state suspended in step four; low-rank factor solving only sets small volume patches on the solution object, without touching the weights of the complete model; patch solving absorbs incomplete latent vector slices. Steady-state multi-class distribution And deadlock beacons, ensuring that patches and deadlocks are not from the same source. After the remote cloud parser node solves, the left and right factors are... and The patch is encapsulated as a patch payload page and transmitted back to the edge identification device via a wide area link. The communication processor of the edge identification device first verifies the patch before sending it to the video memory hot-loading queue. The header of the patch payload page contains the breakpoint layer number, video memory page offset, and checksum. The main control processor uses this header to locate the breakpoint and calculate the subpage, thus preventing the patch from falling into a non-deadlock layer.

[0176] To avoid contention between the current inference thread and the patch writing thread for the same memory block during hot loading, the main processor first wakes up the mutex write lock, then freezes and deadlock beacons. The corresponding breakpoint operator queue is then used; subsequently, the low-rank adaptive residual patch is written to the breakpoint operator page in the video memory, and cache consistency is refreshed page by page. After the patch is written, the edge recognition device does not restart the entire model, but only the deadlock beacon. The breakpoint operator it points to performs bypass replacement. Its operator topology update method is as follows:

[0177]

[0178] In the formula: update operator topology : Indicates the actual operator topology that takes effect after hot-loading the patch for the edge recognition device; freezes the base operator topology. : Represents the basic operator topology before patch writing; write gate matrix : Indicates based on deadlock beacon The generated write location selection matrix takes values ​​that are identical to the gate matrix of the patch; left factor : Represents the patch factor of the low-rank adaptive residual patch in the column space; right factor : Represents the patch factor of the low-rank adaptive residual patch in the row space; operator : Indicates the element-wise gated product operator, used to restrict the patch to only apply to the breakpoint operator page;

[0179] The main control processor is based on the deadlock beacon The breakpoint layer number field, breakpoint page offset field, start channel index field, and end channel index field in the data are used to generate and freeze the base operator topology. A zero matrix of the same shape is then set to 1 only within the corresponding breakpoint layer, the corresponding breakpoint page, and the corresponding channel range to obtain the write gate matrix. The remaining positions remain 0. This ensures the patch matrix... Only write the local weight block corresponding to the deadlock path, without spreading it to unrelated layers.

[0180] In terms of hardware implementation, the memory hot-loading queue is preferably located in an independent queue page near the neural network processor, with the queue page and the current inference page located in different banks. The main control processor first locks the patch write page using a mutex write lock, and then releases the inference page, enabling the edge recognition device to complete patch import and operator switching within the same runtime cycle. If the motherboard uses a passive heat dissipation aluminum plate, a thermal pad is preferably placed above the memory area, with a thickness of 0.5mm to 1.5mm, to reduce the impact of local hot spots on memory stability during hot loading. For example, after a terminal receives a patch payload page, the screen does not restart; the main control processor only freezes one memory page corresponding to the breakpoint operator, and resumes scheduling immediately after writing; subsequently, a new batch of payloads with the same seat number enters the update operator topology. It will no longer proceed along the old deadlock path.

[0181] Then, write the gate matrix The patch is only loaded into the deadlock-related breakpoint operator page; the hot-load path keeps the inference chain uninterrupted; the combination of thermal pads and independent queue pages reduces the local thermal shock during patch writing.

[0182] Update operator topology After taking effect, the main control processor immediately uses the same batch of inputs to obtain a new prediction uncertainty range. When the new prediction uncertainty is extremely poor Lower than the original prediction uncertainty range This indicates that the patch has changed the way the edge recognition device interprets long-tail contamination patterns. Step five does not stop here, but further converts this decrease into a reverse release, used to remove the admission threshold imposed by step four. The admission threshold after reverse release is denoted as... :

[0183]

[0184] Where: Reverse release threshold This indicates that step five involves writing back the new threshold value to the front-end register from step one after the patch takes effect. The value range is... Admission threshold : Indicates that step four has reached the current state's front-end threshold value, with a value range of . The original prediction uncertainty range was extremely poor. : Represents the composite level of confusion before patch loading; the range of uncertainty in the new prediction. : Indicates the amount of confusion synthesized after the patch is loaded; Release coefficient : This indicates that the decrease in confusion level is mapped to the reverse release threshold. The gain strength, whose value ranges from positive real numbers;

[0185] lower boundary : Represents the physical minimum boundary of the front-end admission threshold register in step one; upper boundary : Represents the physical maximum boundary of the front-end admission threshold register in step one; operator : This indicates a constraint operator that cuts off the threshold between the upper and lower boundaries;

[0186] Among them, the new prediction uncertainty range The prediction is recalculated from the same batch of inputs that were cached before the patch was loaded, without mixing inputs from subsequent time steps, to ensure a very low range of uncertainty in the new prediction. The difference in uncertainty from the original prediction Comparability. Release coefficient. During the installation and calibration phase, based on the upper boundary... and lower boundary The threshold span and the prediction uncertainty range within the most recently suspended window of this terminal The dimensional ratios are fixed and remain unchanged within the same model version. Reverse release threshold. After the calculation is completed, the value is quantized into a register write value according to the existing threshold code encoding rules in step four, and then written back to the front-end register via I2C.

[0187] The main control processor determines the reverse release threshold. Generate a new bus write-back message and transmit the reverse release threshold via the I2C inter-integrated circuit bus. Write back to the front-end register from step one. Front-end register acknowledge value and reverse release threshold. When consistent, deadlock beacons Switching from the locked position to the unlocked position, the edge recognition device reopens the front-end interception chain of step one. In the enhanced implementation, the remote cloud parsing node clusters multiple terminals from the same region, the same time window, and with consistent deadlock beacon patterns; if multiple terminals correspond to incomplete latent vector slices... If nodes fall into the same cluster in the potential space, the remote cloud resolution node will simultaneously distribute the same low-rank adaptive residual patch to the healthy terminals in that area to form a pre-release path for early protection.

[0188] In a parallel alternative implementation, if the wide-area link is interrupted, the edge recognition device enables the local point-to-point discovery protocol to slice the incomplete latent vector. The solution is sent to a sibling terminal on the same network segment that is not locked. The idle computing power of the sibling terminal performs a scaled-down patch solution and then returns the left and right factors. and .

[0189] In the downgraded backup implementation, if multiple consecutive communication windows do not receive the patch payload page, the edge recognition device restores the breakpoint operator page to the valid initial page saved during the installation and calibration phase, and sends a reset pulse to the front-end register; this path is only used to exit the current locked state and is not used as the main basis for restoring recognition capability.

[0190] Specifically, the evaluation metrics selected include the number of patch payload page back propagations, the duration of mutex write lock holding, and the update operator topology. Page write count, reverse release threshold write-back count, deadlock beacon The number of times the function is released. The timing relationship between the video memory page write pulse, the I2C inter-integrated circuit bus write-back message, and the front-side register acknowledgment value; in system-level verification, checking incomplete hidden vector slices. Update operator topology The new prediction uncertainty range and the reverse release threshold are on the same timestamp chain.

[0191] The reverse release threshold directly transmits the patch effect to the front-end hardware threshold; the group pre-release path prevents deadlocks on similar terminals one by one; the factory constant matrix overwrite preserves a recovery entry point for extreme disconnection scenarios, through left and right factors. and Low-rank solution and update operator topology The hot loading and reverse release threshold write-back transform the intermediate state suspended in step four into a cross-domain self-rescue chain.

[0192] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A student abnormal state identification method based on multimodal behavioral data fusion, characterized in that: include, Video frames, audio segments, and pose packets are acquired, and before the video frames enter the neural network processor, the digital signal coprocessor generates blinding labels based on the spatial structure response of the video frames. In response to the blinding mark, the DMA source address of the damaged visual channel is redirected to a preset all-zero physical block in the kernel interrupt context, and a fragmented feature stream and a vent execution mark are generated according to the alignment rules. The current original distribution is generated based on the audio segment, attitude packet, incomplete feature stream and empty execution marker, and then damped and stitched with the steady-state distribution of the previous time step to obtain the current steady-state multi-class distribution. The prediction uncertainty is generated from the current steady-state multi-class distribution and then written back to the front-end admission threshold via the inter-integrated circuit bus. If the admission threshold does not reach the lower boundary, the spatial structure response generation continues; if the admission threshold reaches the lower boundary and the prediction uncertainty is higher than the confusion threshold within the preset continuous window, the incomplete latent vector slice and deadlock beacon are suspended, a suspended payload without video plaintext frames is sent and the low-rank residual patch returned by the remote parsing node is received, the low-rank residual patch is hot-written into the breakpoint operator page, and the admission threshold is pulled up according to the change in prediction uncertainty before and after the patch loading.

2. The student abnormal state identification method according to claim 1, characterized in that: The digital signal coprocessor performs spatial structure response calculations on the luminance plane of the video frame and writes a blinding flag when the response value crosses the blinding determination threshold; After receiving the blinding mark, the edge recognition device enters the interrupt context, replaces the source address field of the damaged visual channel with a preset all-zero physical block address, and writes a zero-page short pass flag in the same descriptor header, while keeping the target video memory address field, channel length field and timestamp field unchanged.

3. The student abnormal state identification method according to claim 2, characterized in that: The incomplete feature flow includes visual occupant segments, audio feature segments, and posture feature segments spliced ​​together in the original channel order. The edge recognition device first generates visual distribution branches based on the visual occupant segments, then generates supplementary distribution branches based on the audio feature segments and posture feature segments, and synthesizes the current original distribution according to the compensation ratio. Then, it is damped and stitched with the steady-state distribution of the previous moment to generate the current steady-state multi-class distribution.

4. The student abnormal state identification method according to claim 3, characterized in that: The edge recognition device calculates the prediction uncertainty corresponding to the steady-state multi-class distribution in each sampling period, and simultaneously checks whether the current admission threshold has reached the lower boundary, whether the average prediction uncertainty of the window is higher than the deadlock threshold, and whether the write-back confirmation code remains unchanged within a continuous window. When all three conditions are met, the deadlock beacon is set and the incomplete latent vector slice is frozen. When any one condition is not met, the out-of-band write-back chain continues to run. The deadlock beacon includes the breakpoint layer number field, the breakpoint page offset field, the start channel index field, and the end channel index field.

5. The student abnormal state identification method according to claim 4, characterized in that: After setting the deadlock beacon, the edge recognition device generates a suspended payload, which includes at least a fragmented latent vector slice, a deadlock beacon, an admission threshold, a prediction uncertainty range, and a steady-state multi-class distribution. After receiving the patch payload page, the edge recognition device generates a write gate matrix based on the breakpoint layer number field, breakpoint page offset field, start channel index field, and end channel index field in the deadlock beacon, and performs hot writing of low-rank residual patches only on the corresponding breakpoint operator pages.

6. The student abnormal state identification method according to claim 5, characterized in that: The edge recognition device first performs an integrity check on the patch payload page; if the check results are consistent, it performs a hot write of the low-rank residual patch; if the check results are inconsistent, it keeps the current breakpoint operator page and the previous patch version participating in inference, and writes the current patch payload page into the cache page to be retransmitted.

7. The student abnormal state identification method according to claim 5, characterized in that: When performing hot writing of low-rank residual patches, the edge recognition device first writes the low-rank residual patch to a shadow page that is different from the current breakpoint operator page; when the shadow page writing is completed and the page verification is consistent, the active page index is switched; when the page verification is inconsistent, the current active page index is kept unchanged and the shadow page write mark is cleared.

8. The student abnormal state identification method according to claim 5, characterized in that: The patch payload page includes a cluster identifier, a breakpoint layer number, and an admission threshold interval code. After receiving the patch payload page, the edge recognition device compares the local deadlock beacon, the breakpoint layer number, and the current admission threshold interval code. When the three are consistent, the low-rank residual patch is loaded. When the three are inconsistent, the patch payload page is cached without performing a hot write.

9. The student abnormal state identification method according to claim 5, characterized in that: When the edge recognition device fails to obtain a valid patch payload page in multiple consecutive communication windows, it restores the breakpoint operator page to the valid initial page saved during the installation and calibration phase and writes a reset code to the front-end register; when a valid patch payload page is obtained in multiple consecutive communication windows, it keeps the current breakpoint operator page and the current patch version to continue participating in inference.

10. The student abnormal state identification method according to claim 5, characterized in that: The header of the suspended payload includes at least the timestamp field, seat number field, deadlock state field, slice length field, slice dimension field, quantization bit width field, layout field, and verification field. The payload body of the suspended payload includes the incomplete latent vector slices and the quantization results of the admission threshold, prediction uncertainty range, and steady-state multi-class distribution.

11. The student abnormal state identification method according to claim 5, characterized in that: The header of the patch payload page includes at least the breakpoint layer number field, the video memory page offset field, the start channel index field, the end channel index field, the left factor row number field, the right factor column number field, the release suggestion threshold field, and the integrity check field. The payload body of the patch payload page includes the left factor matrix and the right factor matrix corresponding to the low-rank residual patch.

12. The student abnormal state identification method according to claim 4, characterized in that: When setting a deadlock beacon, the edge recognition device writes a timestamp, seat number, breakpoint layer number, current admission threshold code, current prediction uncertainty code, and deadlock status bit to the local status log page; when releasing the deadlock beacon, it appends the release status bit, updated prediction uncertainty code, and reverse release threshold code to the local status log page.

13. The student abnormal state identification method according to claim 12, characterized in that: After the deadlock beacon is set, the edge recognition device outputs a suspension repair status code, seat number, and breakpoint layer number on the status output interface; after the deadlock beacon is released, it outputs a threshold release status code, update operator page version number, and current admission threshold code on the status output interface.

Images