System and method for live monitoring of embedded inference modules

Abstract

This invention provides a system and method for monitoring inference performed by a microcontroller (MC) in real time. [Solution] The inference module embedded in the MC20 analyzes the dataset, generates a judgment value, then transmits a judgment data payload representing the judgment value, and transmits the dataset to the host system 10, which is a monitoring subsystem. The inference module is an AI module that can be trained on datasets previously transmitted from the MC. The monitoring subsystem can visually represent the judgment value on a user interface, generate validation values ​​to test the accuracy of the judgment value, and store the judgment data payload in memory for future improvements of the inference module.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims the benefit of priority based on U.S. Patent Application No. 13 / 303,222, filed on August 18, 2025, and claims the benefit of priority based on U.S. Provisional Patent Application No. 63 / 685,390, filed on August 21, 2024. The content of each is hereby incorporated by reference in its entirety.

[0002] This disclosure relates to the development of modules embedded within a product. More particularly, this disclosure relates to the development, monitoring, testing, and debugging of embedded AI / ML inference modules, or other analytics modules, in - field on a product.

Background Art

[0003] Microcontrollers (MCs) are becoming commonly used in a wide range of modern electronic devices, from small devices to automobiles. These MCs provide compact, low - power, and cost - effective computing, along with firmware (software permanently programmed) that provides the core functionality of the product. MCs typically include only the components necessary for their specific function, have limited connectivity and capabilities, and are much more inconvenient to program than general - purpose computers.

[0004] The MC's functions, as part of controlling the device, may require making decisions based on inputs from various sources. As some non-limiting examples, a device may need to self-diagnose based on internal sensor inputs and determine whether a situation justifies shutting down an ongoing function to prevent hazardous situations such as overheating or power surges. A device in operation may need to use inputs from cameras, LiDAR, and similar sensors to identify obstacles and decide whether to change course. The device may include biometric sensors that need to distinguish between active and inactive users and activate functions only for the former. To make these decisions, analysis modules can be included in the MC firmware. These modules are configured to receive inputs and interpret them as part of the decision-making process.

[0005] In simple, predictable scenarios with only a few input variables, engineers can code the decision structure of an analysis module using, for example, if-then-else loops, while-do loops, and other basic programming techniques. However, analyzing more complex scenarios may require an inference module. While such modules are not exclusive, they are typically generated through an artificial intelligence or machine learning (AI / ML) training process using a training set of data examples with known correct outcomes. The training process allows the module to make decisions based on new data collected during operation, such as classifying inputs (classifiers) or assigning values ​​(regressions). Examples of AI / ML decision structures include, but are not limited to, neural networks (NNs), deep learning, support vector machines (SVMs), random forests, decision trees, logistic regression, and K-nearest neighbors.

[0006] As is well understood in this field, inference modules are less deterministic and more susceptible to unpredictable errors than handcrafted algorithms. Therefore, thorough testing of the accuracy of these modules is essential to maintain product quality. [Overview of the project]

[0007] One aspect of the disclosed system and method is to enable in-situ monitoring, testing, and debugging of software on a computer system that, due to its limited capabilities and connectivity, is not easily accessible or modified, such as a microcontroller embedded in an end product.

[0008] Another aspect of the disclosed system and method is to intuitively present the output and other data from the monitored software.

[0009] Another aspect of the disclosed system and method is to collect training data for an AI / ML inference module that accurately reflects the state of data received by a module within the final product, intended to be embedded within the final product.

[0010] These and other embodiments can be achieved in systems and methods for live monitoring of embedded inference modules.

[0011] According to one embodiment of the present invention, a system is provided for monitoring inference performed by a microcontroller (MC) in real time. The system includes an MC communicatively coupled to an electronic device and a monitoring subsystem located outside the MC and communicatively coupled to the MC. The MC includes at least one MC processor, an MC communication stack configured to send and receive signals, and an MC memory embedded with MC program code executable by at least one MC processor. The MC program code includes an inference module configured to cause at least one of the at least one MC processors to analyze a dataset and generate a corresponding decision value. The electronic device selectively performs at least one operation based on the decision value. The MC program code further includes a payload module configured to cause at least one of the at least one MC processors to generate a decision data payload representing a decision value for each decision value generated by the inference module and to transmit the decision data payload through the MC communication stack. The monitoring subsystem includes at least one monitoring processor, a monitoring communication stack configured to send and receive signals, a display, and a monitoring memory encoding monitoring program code executable by at least one monitoring processor. The monitoring program code includes a monitoring module configured to cause at least one of the at least one monitoring processors to process multiple decision data payloads received through the monitoring communication stack and originating from the MC communication stack. The monitoring program code further includes an interface module configured to cause at least one of the at least one monitoring processors to visually render a user interface on a display. The user interface visually represents the decision value of at least one of the multiple decision data payloads.

[0012] Another embodiment of the present invention provides a method for monitoring inference performed by an MC in real time. This method includes establishing an MC, which includes at least one MC processor, an MC communication stack configured to send and receive signals, and an MC memory embedding an inference module. The method further includes establishing a monitoring subsystem located outside the MC and communicably coupled to it. The monitoring subsystem includes at least one monitoring processor, a monitoring communication stack configured to send and receive signals, monitoring memory, and a display. The method further includes acquiring a plurality of datasets by at least one MC processor. The method further includes, for each dataset of the plurality of datasets, analyzing the dataset to generate at least one corresponding decision value, generating a decision data payload representing the decision value, and transmitting the decision data payload through the MC communication stack, all performed by at least one MC processor. The method further includes receiving a plurality of decision data payloads originating from the MC communication stack through the monitor communication stack, all performed by at least one monitor processor. The method further includes generating a user interface on a display representing the decision value of each processed decision data payload of the received plurality of decision data payloads, all performed by at least one monitor processor.

[0013] According to yet another embodiment of the present invention, a method is provided for training and improving an artificial intelligence (AI) module of a multimedia controller (MC). The method includes establishing an MC, which includes at least one MC processor, an MC communication stack configured to send and receive signals, and an MC memory. The method further includes establishing a trained AI module, which further includes embedding the trained AI module in the MC memory. The method further includes establishing a test subsystem located outside the MC and communicatively coupled to the MC. The test subsystem includes at least one test processor, a test communication stack configured to send and receive signals, a test memory, and a display. The method further includes acquiring a plurality of test datasets by at least one MC processor, which further includes, for each of the plurality of test datasets, analyzing the test dataset to generate at least one corresponding test decision value, generating a test data payload representing the test decision value, and transmitting the generated test data payload through the MC communication stack. The method further includes receiving a plurality of test data payloads originating from the MC communication stack through the test communication stack by at least one test processor. The method further includes generating a user interface on a display that represents at least one test decision value for each processed test data payload of a plurality of received test data payloads, using at least one test processor.

[0014] Additional embodiments are described below, may become apparent from the description, or may be realized by implementing the embodiments presented in this disclosure.

[0015] The above and other aspects and features of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0016] [Figure 1] Figure 1 illustrates a hardware environment in which various embodiments of the subject system and method can be implemented. [Figure 2] Figure 2 shows an exemplary hardware configuration of a microcontroller that can be monitored by various embodiments of the subject system and method. [Figure 3A-3B] Figures 3A and 3B are block diagrams illustrating the abstracted relationships between functional code modules in a system for monitoring embedded analysis modules according to one embodiment of the present disclosure. [Figure 4] Figure 4 is a flowchart illustrating an exemplary process for configuring an embedded analysis module to be monitored, according to an embodiment of the present disclosure. [Figures 5A-5C] Figures 5A to 5C are flowcharts illustrating an exemplary flow of the inference decision process in a classifier-type analysis module according to a particular embodiment of the present disclosure. [Figure 6A] Figure 6A is a flowchart illustrating an exemplary process flow for an embedded application for providing analytical data, according to an embodiment of the present disclosure. [Figure 6B] Figure 6B is a flowchart illustrating an exemplary process for an embedded application to reconfigure its operation according to an embodiment of the present disclosure. [Figure 7] Figure 7 is a sequence diagram illustrating an exemplary operational flow for controlling the operation of an embedded payload module from a host system, according to one embodiment of the present disclosure. [Figure 8] Figure 8 illustrates an exemplary implementation of a graphical user interface for a data collection application module according to an embodiment of the present disclosure. [Figure 9A-9L] Figures 9A–9L illustrate exemplary implementations of a graphical user interface for a monitoring application module according to one embodiment of the present disclosure. [Figure 10]FIG. 10 is a flowchart showing an exemplary flow of a process for receiving and presenting analysis data in a monitoring application module according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a flowchart showing an exemplary flow of a process for preparing and configuring a conventional embedded analysis module. [Figure 12] FIG. 12 is a block diagram showing an exemplary computer system for program and / or hardware implementation of various embodiments and aspects of the present invention.

Embodiments for Carrying Out the Invention

[0017] Here, the present disclosure refers in detail to exemplary embodiments shown in the accompanying drawings, and like reference numerals indicate like elements throughout. The embodiments are described below to explain the disclosed systems and methods with reference to the figures shown in the drawings for selected exemplary embodiments and their sampling applications.

[0018] The present disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit implementation to the exact forms disclosed herein. Modifications and variations are possible in view of the present disclosure and can be obtained through implementation of the embodiments. Furthermore, one or more features or components of one embodiment may be incorporated into another embodiment or combined with (or with one or more features of another embodiment). Additionally, it should be understood that in the flowcharts and descriptions of operations provided below, one or more operations may be omitted or added, or the operations may be executed simultaneously (at least partially) or reconfigured in their order.

[0019] Flowcharts and block diagrams of the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of one or more executable instructions for implementing the specified logical function. The methods, computer systems, and computer-readable media may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those depicted in the drawings. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the drawings. For example, two blocks shown in succession may actually be executed simultaneously, or substantially simultaneously, or sometimes in the reverse order, depending on the functions involved.

[0020] It will be apparent that the systems and methods described herein may be implemented in various forms of hardware, firmware, or a combination of hardware and software. For example, each block shown in the provided figures and flowcharts may be implemented by a dedicated hardware-based system that performs the specified function, either alone or in combination with other things, or by a combination of dedicated hardware and computer instructions. The particular dedicated control hardware or software code used to implement these systems and methods does not limit the scope of the implementation. Thus, the operations and behaviors of the systems and / or methods described herein are presented without reference to specific software code, as it is understood that software and hardware can be designed to implement the systems and / or methods based on this description.

[0021] Certain combinations of features are described in the claims and / or disclosed herein, but these combinations are not intended to limit the range of possible implementations. Conversely, many of these features can be combined in ways not expressly described in the claims and / or disclosed herein. Each dependent claim listed below may depend directly on only one other claim, but the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim setting.

[0022] Any elements, actions, or instructions used herein should not be construed as important or essential unless expressly defined so. Furthermore, the articles “a” and “an” should be construed as encompassing one or more items and may be used interchangeably with “one or more.” When only one item is intended, the term “one” or similar language should be used. Furthermore, the terms “has,” “have,” “having,” “include,” “including,” or similar terms should be construed as open-ended. Furthermore, the phrase “based on” should be construed as meaning “at least partially based on” unless specifically stated otherwise. Furthermore, expressions such as “at least one of [A] and [B]” or “at least one of [A] or [B]” should be construed as including implementations having A only, B only, both A and B, and / or any of their variations.

[0023] Figure 11 is a flowchart illustrating a typical process flow for developing artificial intelligence / machine learning (AI / ML) inference modules and deploying them to a microcontroller (MC). Briefly, the process operates as follows: (1101) Train the AI / ML inference module with the training data. (1102) Test the AI / ML inference module with test data. (1103) Check if the module's precision is sufficient. If not, repeat steps 1101 and 1102. (1104) Generate an embedded code version of the AI / ML inference module. (1105) Combine the embedded code version with other application code. (1106) Compile the combined code and flash it to the MC. (1107) Observe the system operation of the entire module and test its performance on-site on the MC.

[0024] Operations 1101, 1102, and 1103 are typically performed using training and testing tools that run either on a host system such as a desktop workstation or on a server, depending on the scale of the data. Operations 1104 and 1105 are typically performed on a host system, or in some embodiments, on cloud-based software. Therefore, the majority of development and testing takes place outside of the MC.

[0025] The host system allows for detailed examination of the AI / ML module's behavior, including both performance accuracy and internal behavior. However, in operation 1107, after the module is embedded in the MC, typically only the overall system behavior is visible. This makes it difficult to understand and troubleshoot issues with the AI / ML module occurring within the context of the intended system.

[0026] There are several reasons why the behavior of AI / ML modules differs in MC compared to the original host system:

[0027] (1) The data itself is different. A set of pre-recorded data, such as that used during the initial training and testing, is inherently limited by natural, real-world variability. As the system is deployed, the setting and context of its operation can lead to data differences that are not adequately represented in the pre-recorded training and testing data. Furthermore, in some cases, the transfer of data samples (data channels) and changes in the process may be inadvertently introduced between the original training data collection process and the subsequent live operation of the AI / ML module.

[0028] (2) Embedded systems often require lower computational resolution and higher resolution data quantization. AI / ML modules running on computer workstations can typically leverage high-precision mathematics (e.g., 64-bit floating-point). In contrast, MCs typically operate at reduced resolution (e.g., 32-bit floating-point, 16-bit fixed-point, or even 8-bit integers). Lower resolution allows for faster processing within the MC's native functions, but the trade-off is the introduction of rounding and truncation errors that reduce the precision of the AI / ML module.

[0029] (3) Other modules operating on the MC within the complete system may introduce differences. When other functions compete for resources within the MC, the behavior of any system component may change due to other complexities, such as interrupt handling, RAM usage, and resource sharing.

[0030] (4) The physical system in which the MC operates can affect the data reaching the module. Noise, systematic voltage problems, sensor vibrations, and similar factors can distort the data signal, which has a greater impact on the MC components than on the larger system and cannot be easily simulated outside of the MC system.

[0031] The testing process typically relies on programmatic debugging tools within an integrated development environment (IDE). These tools allow engineers to set breakpoints and inspect internal data values ​​while the machine learning module (MC) is running. However, these tools are highly general-purpose and, metaphorically, allow users to see the trees precisely, but not the forest precisely. Because AI / ML requires understanding numerous interconnected components simultaneously to grasp operational problems, such traditional debugging tools are often cumbersome and time-consuming when used to debug AI / ML modules.

[0032] Other measurements include testing the behavior of embedded code in simulations or host systems, or flashing custom applications to the MC. Results from such tests on the MC typically include measurements of accuracy and operating speed. While these measurements provide a basic evaluation of the functionality of AI / ML modules on the MC, they have significant limitations for several reasons.

[0033] Firstly, these test measurements typically use dedicated "toy" applications running on the MC, designed solely to execute AI / ML modules and communicate with the host. Since such applications lack the complete user MC software or full context of the product system, they cannot address points (3) and (4), and can only partially address point (1). These applications primarily focus on point (2) by testing the fundamental functionality of embedded systems that are subject to quantization.

[0034] These measurements also provide limited visibility into real-time operation. Many embedded systems simply act on the output of AI / ML. Designers typically have no motivation to configure the system to share this information with external observers during production. Direct monitoring of AI / ML itself is usually not considered because extra code in embedded projects requires space, processing time, and time to develop. Even systems that provide monitoring generally do so through abstracted notifications (e.g., a change in the color of a warning light or a message indicating a failure of a particular system component). Rarely, engineers may manually build monitors that provide current states of AI / ML output, such as the current "class" of a classifier, sometimes with limited score information. However, even this does not provide a history showing the context and stability of decisions, and in particular lacks a record of internal state or source data, which could be significantly useful for identifying and debugging the problem. In some situations, an AI / ML module can perform a large number of inferences per second, severely limiting observation of only instantaneous output.

[0035] In short, exemplary embodiments of this disclosure provide methods and systems that allow analytical modules, such as AI / ML inference modules, to be thoroughly tested and debugged while embedded in the intended product.

[0036] The disclosed embodiments of the methods and systems offer a significant improvement over the ad-hoc deployments described above. They address all of points (1), (2), (3), and (4) by enabling users to test live within the context of the embedded product. Furthermore, they allow for the collection of training and test data that more closely matches the data the module would encounter during actual use. They also provide a detailed history of state and internal data, which can be used to understand and extend the operation of the AI / ML module and the entire product. Moreover, they provide a very convenient way to quickly demonstrate and evaluate the operation of such modules in place without requiring additional effort from the engineering team.

[0037] As described above and to be further established, certain principles disclosed herein have particular advantages in the context of training and testing artificial intelligence and machine learning inference modules. For explanatory purposes, this disclosure focuses on embodiments in that particular context. However, it will be understood by those skilled in the art that the same principles can be extended to be applied to monitoring and debugging other firmware and embedded systems, and that these other situations, in particular, but not exclusively, other forms of analysis and inference modules, continue to provide at least some of the same advantages. Thus, the systems and methods of the subject matter can also be adequately implemented in other categories of systems, and are not limited to those expressly disclosed.

[0038] Furthermore, for simplicity, this disclosure uses the descriptors “artificial intelligence” and “machine learning” interchangeably to refer to the field of training software algorithms that use existing training data having predetermined correct results, and the resulting inference modules that are trained by this process to provide correct results in response to new data.

[0039] Figure 1 shows an exemplary hardware environment in which the subject system and method may be implemented.

[0040] The circuit board 200 housing the microcontroller (MC) 20 may be installed within or otherwise coupled to any electronic device, and the device may be instructed to selectively perform operations by transmitting control signals to appropriate components. In one exemplary example, if the circuit board 200 is coupled to the motor control system of an apparatus, it may be able to instruct various motor components to increase or decrease their activity in response to user input, readouts from internal motor control states, outputs from additional sensors such as electrical instruments, encoders, accelerometers, and thermometers, and / or other factors.

[0041] The circuit board 200 is communicatively coupled to the host system 10 via a connector 30. The connector 30 may be a dedicated debugger connection, UART, USB, Ethernet®, SPI, I2C, or other communication cable, or a combination thereof. In some embodiments, a wireless connection such as Wi-Fi or Bluetooth may be used.

[0042] Various applications and software modules can run on the host system 10. These may include, for example, an IDE (Integrated Development Environment) 17 that combines editing, debugging, and compiler toolchains. Through the IDE, engineers can write source code, compile the code into binaries, flash the binaries onto the target MC 20 for embedded execution, and run standard debugger functions to monitor code execution. The host system 10 may also be connected to other systems 40 via a network to access remote application services, for example, development tools used to create and optimize modules.

[0043] Embodiments of the disclosed systems and methods further provide a monitoring application module 11 (more simply, a “monitor”) on the host system 10. The monitor 11 may be integrated as a plug-in to the IDE 17 or may reside in a separate application. By operating the monitor 11, the host system functions as a monitoring subsystem of the MC 20.

[0044] Figure 2 shows an exemplary hardware configuration for a microcontroller such as the MC20. The components are typically implemented as hardware on a silicon die and are electrically connected to power and external peripherals.

[0045] The core element of the MC is the 201 microprocessor, which typically operates at a lower clock speed and with lower capabilities than general-purpose processors found in computer workstations and servers.

[0046] The microprocessor 201 is coupled to random access memory (RAM) 202, which is typically used for dynamic storage of information. A portion of RAM 202 can target different forms of storage, such as general-purpose memory, tightly coupled memory (TCM), and cache.

[0047] The microprocessor 201 is also coupled to flash memory 203, which is typically used for program code and persistent data storage. Flash memory 203 stores information that remains available even after the processor is powered off or restarted. For this reason, program code stored in flash memory 203 is sometimes referred to as "firmware" rather than "software."

[0048] Data and code are typically written to flash memory 203 via an external programming interface subsystem 204 that receives data from an external host and programs the device. Communication between the external programming interface subsystem 204 and the external host may be via any suitable connection known in the art, such as a dedicated connection like a UART, USB, or J-Link device. Figure 2 assumes the external host is the host system 10 in Figure 1, but it may be another type of dedicated device. For example, a system used in mass production of MCs can "burn" firmware onto devices on the production line. Furthermore, different external hosts can be used at different times.

[0049] The MC 20 also typically has onboard peripherals 206 through which it can communicate. These may include one or more external connectors 209 such as UART, USB, Ethernet®, SPI, i2C, or other communication cables, or a combination thereof. The peripherals 206 may also include sensor channels such as AD converters. The MC 20 typically uses such connectors and channels to operate functions and electrically control other physical elements of the circuit board 200 or the entire product system in which the MC 20 resides.

[0050] The MC 20 may also include processing components that work in conjunction with the microprocessor 201 to accelerate specific operations. These processing components may include a floating-point unit (FPU) 208 to accelerate floating-point operations. More advanced MCs may include a neural processing unit (NPU) 207 designed to accelerate specific AI / ML-specific operations. Other possible components not shown may include a digital signal processing unit (DSP). These processing units are optional, and the simplest MC can use only the microprocessor 201, which is optimized for high speed, low power, and integer operations. In such cases, any floating-point operations, AI / ML operations, or DSP-type operations may instead be implemented via software.

[0051] An alternative to the implementation shown in Figure 1 is that the basic functions of the MC 20 can be incorporated onto a single physical chip component, along with additional external peripheral devices and interfaces, as detailed with reference to Figure 2. This implementation is commonly referred to as a system-on-a-chip (SoC). The SoC implementation simplifies the physical design on a standard circuit board 200, as shown in Figure 1, but otherwise operates in the same general manner.

[0052] The disclosed system can operate in any context in which an MC or similar device is used, including on a development or evaluation board, within a SoC, or on a manufacturer's own custom board. It can also be used when embedded operation is simulated on a host computer. This degree of flexibility is not present in many other systems that function only in the context of a specific evaluation board and cannot be used with "production" hardware.

[0053] Using an exemplary environment established for context, this disclosure describes how data analysis is performed within that environment.

[0054] Figure 3A is a block diagram illustrating an abstract relationship between a functional code module on the MC 20 and the host system 10 in the context of data analysis, according to one embodiment of the present disclosure. Figure 3A assumes that a core application 28 has been prepared and flashed to the MC 20. In the illustrated system, the core application 28 includes application logic 24 supported by an analysis module 23. The analysis module 23 may, but is not limited to, an artificial intelligence / machine learning (AI / ML) module or other inference module that generates inferences based on received data.

[0055] One or more external data sources 21a, such as sensors, are coupled via peripheral connections to a data acquisition buffer module (DC) 22 embedded on the MC 20. The external data sources 21a provide external data 211a to the DC 22, which temporarily stores it in a buffer. Peripheral connections may allow for additional buffering, which may be necessary in some cases to capture small frames of data from the source. For example, the device as a whole may use "direct memory access" (DMA) to write small arrays of information into the buffer, which is then serviced by interrupts to supply these to the rest of the system. In other cases, the external data 211a may be provided by value.

[0056] In the illustrated system, an internal data source 21b is also provided, which provides internal data 211b to the DC 22. The internal data 211b may include variables or values ​​generated by other processes within the MC 20, in contrast to external data 211a which arrives directly from an external source 21a. For example, if the MC 20 is part of a motor control circuit, an algorithm may generate internal parameter values ​​representing the motor state, and a sequence of internal values ​​from the algorithm may form the basis of the data vector 211b provided to the DC 22.

[0057] Various embodiments may use any number of external data sources 21a and / or internal data sources 21b that are suitable for the needs of the entire system.

[0058] Data 211a and / or 211b may be received as frames or individual values. DC 22 may be configured to organize data 211a and / or 211b into longer frames or arrays, which may be called “data segments” or “data sets” by aggregation and / or segmentation. Each data segment is formatted, sized and otherwise configured according to the configuration of the analysis module 23 for analysis by which it is performed. Each data segment has a window length and may have one or more data channels, forming an array of input data for the analysis module 23. The data within a data segment may include, but is not limited to, visual, auditory, vibration, electrical, RF, and internal variables, and may have one or more channels of any combination of signals, each divided into time-based segments for evaluation and analysis. Thus, source data 211a and 211b can be thought of as time-segmented signals, although the data may initially be received in a different form.

[0059] Data segments do not need to be independent. In some embodiments, it is advantageous to overlap one data segment with a previous data segment. As one example, in a particular audio analysis algorithm, each data segment retains 50% of the previous data segment while filling the other 50% with new data. This approach allows the analysis module 23 to generate more decisions per unit time without waiting for new signal data and may also demonstrate the benefit of better handling of transitions between output states. This configuration can also be used in conjunction with "smoothing" (described later) to reduce instantaneous errors in the output.

[0060] In some embodiments, the analysis module 23 can receive the data 211a and / or 211b more directly and segment it as part of its own process. However, for convenience, this disclosure assumes that the segmentation is performed in DC 22.

[0061] Once DC 22 segments the data, it sequentially provides the resulting data segments 221 to the analysis module 23. The analysis module 23 may be configured to analyze each data segment and generate a decision value based on it. The decision value may also be called a predictive value or forecast and may represent a class, detected event, numerical value, or other possible output known in the art, depending on the nature of the analysis process. For each data segment, a unique new analysis result may be generated.

[0062] For convenience, only one analysis module 23 is shown, but multiple analysis modules, each generating its own determination values, are within the scope of this disclosure. Thus, the illustrated system can be extended by duplication for each additional such instance in the system.

[0063] The output of the analysis module 23 is provided to and acted upon by the application logic 24 of the core application 28. The application logic 24 encapsulates the larger operation of the embedded system so that it directs the operation of the entire device.

[0064] The details of the application logic 24 vary considerably depending on the needs of the device. As an example, a vibrating mechanical system such as a washing machine is briefly described. In this exemplary device, the analysis module 23 may be configured to detect a particular type of abnormal behavior. The data source 21a may be an accelerometer, and the analysis module 23 may output one of two classes, "normal" or "abnormal," as a determined value. Based on the "abnormal" output, the application logic 34 may perform one or more activations, among other possibilities, adjusting the system activation to reduce vibration, completely stop activation, or trigger an alarm.

[0065] In this context and other contexts, the analysis module 23 is a component used to determine or detect the conditions under which the rest of the system, such as the application logic 24, can operate. While the operation of the application logic 24 is the “ultimate goal” of the analysis module 23 and the device itself, this disclosure focuses on monitoring the analysis module 23 and therefore does not discuss the application logic 24 further.

[0066] The core application 28 is shown in Figure 3A in a manner intended to highlight the operation of the analysis module 23. It will be understood that the illustrated functional block does not necessarily have to be simple sequential program code and could be implemented in an interrupt-mediated process or through the application of multiple processes running on the MC 20 under a real-time operating system (RTOS), among other possibilities known in the art.

[0067] A data shipping module (DS) 25 is provided to support monitoring of the core application 28, more specifically the analysis module 23. The DS 25, sometimes called a payload module, aggregates information from the core application 28 being monitored. The DS 25 captures the output 231 of the analysis module 23 (e.g., decision values), as well as preferably internal analysis data 232 (e.g., one or more intermediate values ​​generated during intermediate steps in the analysis and used to arrive at the final decision 231). When each decision 231 is output by the analysis module 23, the DS 25 packages the corresponding information into a decision data payload or data packet 251 that is sent to the host system 10 via the communication stack 26.

[0068] Optionally, a raw data segment 221 from DC 22 may also be collected by DS 25 and included in payload 251. This additional data allows for a comparison between decisions made on MC 10 (i.e., decisions 231) and decisions made on higher-resolution systems such as host system 10, as described later. It also allows for the recording and reuse of data as part of further offline testing or training, an example of which is further illustrated herein with reference to Figure 3B. Both are very powerful tools for monitoring. However, data segment 221 is typically significantly larger than the internal analysis data 232 and output 231. Therefore, packaging data segment 221 and transmitting the resulting larger payload 251 can substantially impact the performance of MC 20. Preferably, including the raw data segment 221 in payload 251 is configurable and can be enabled or disabled as needed by the analyst.

[0069] The communication stack 26 may comprise buffers, firmware, and other processes used by the MC 20 to transmit data via a known communication channel 30. The communication channel 30 may include, but is not limited to, UART, USB, J-Link, SPI, I2S, Bluetooth, or any other suitable connection known in the art that is supported by the MC 20.

[0070] Data (e.g., payload 251) passing through the communication channel 30 is received by a corresponding communication stack 16a operating within the host system 10, which may include buffers, firmware, and other processes. For convenience and brevity, the communication stack 26 on the MC, the communication channel 30, and the corresponding communication stack 16a on the host 10 may collectively be referred to as the “communication bus” in this specification.

[0071] The monitoring application module 11, running on the host system 10, can then receive data, which is buffered by the monitoring buffer module 111, and as a result can be reviewed or saved and supplied to the monitoring graphical user interface (GUI) 112 for display. Exemplary implementations of the monitor GUI 112 are further described herein with reference to Figures 9A to 9L.

[0072] Although not shown, monitor 11 may also include a test module. In an embodiment of the system in which the decision payload 251 provided by DS 25 includes raw data segments 221, the test module can perform a separate analysis of each data segment to generate separate decision values, which may be called validation values. The test module can then compare its validation values ​​with the decision values ​​generated by the analysis module 23 for the same data frame. The results of this test may be used to identify and diagnose errors in the decisions of the analysis module 23, and potentially to refine the analysis module 23 or its operating parameters, thereby improving accuracy.

[0073] The host system 10 typically has other applications 13, which may include an IDE for programming the core application 28, as discussed earlier. The host system 10 can also store configuration data 12 for the monitored analysis module 23. Furthermore, the host system 10 may have an additional communication stack 16b that can communicate with other external systems 40 over the Internet or other networks. In other embodiments, a single communication stack (e.g., communication stack 16a) can be used to communicate with both the MC 20 and other external systems 40.

[0074] In particular, in the case of AI / ML inference, smaller modules, such as those required for embedded MC systems, often depend heavily on the finer details of the received data. In this regard, even slight differences in the data channels from which the data is obtained can alter the data enough to cause significant deviations in the results. For this reason, the data collected by the host system 10 to initially train the AI / ML-based inference module 23 differs from the data collected by the MC 20 itself during the actual operation of the same module 23, reducing the accuracy of inference and decision-making. However, the illustrated system addresses this challenge.

[0075] Briefly, Figure 3B is a block diagram illustrating an abstract relationship between a functional code module on MC 20 and the host system 10 in the context of data collection prior to training the analysis module, according to one embodiment of the present disclosure. Most of the illustrated components are the same as those in Figure 3A, and their descriptions do not need to be repeated.

[0076] In the context of collecting data for training, a data acquisition and storage module (DST) 14 operating on the host system 10 may receive data from the MC 20 via the communication stack 16a and package it into a data acquisition payload 252 by the DS 25. The DST 14 may buffer the data using a monitoring buffer module 141, allowing it to be reviewed or stored in a short-term data storage device (not shown), such as random access memory. Long-term local storage of the collected data may be to a data storage device or memory 15, which may be the same storage device containing the configuration data 12, or a different storage device.

[0077] DST 14 may also supply data to a data collection graphical user interface (GUI) 142 for operations such as display, review, and labeling. Briefly, one exemplary implementation of the collection GUI 142 is shown in Figure 8. Details of this GUI are not central to this disclosure and will not be described in detail.

[0078] DST 14 can also upload data to one or more external systems 40, such as AI / ML training and test development tools, which are configured to train AI / ML analysis modules for later embedding in MC 20. Module training may alternatively be performed on the host system 10 itself.

[0079] The DST 14 in Figure 3B and the monitor 11 in Figure 3A can operate simultaneously as needed, enabling the capture and analysis of raw data frames along with the output of the analysis module 23. More specifically, in an embodiment of a system in which the data decision payload 251 provided by the DS 25 includes raw data frames 221, these payloads 251 can also serve as data acquisition payloads 252, and the DST 14 can process them in the same manner as described above.

[0080] Comparing Figure 3B with Figure 3A, we can see that the raw data frames are captured by the same sources 21a and 21b using the same data channels, the same hardware, and the same segmentation module (i.e., DC 22) as those used by the analysis module 23, and then routed through the same MC 20, while avoiding unnecessary interaction with the application logic 24. In other words, the illustrated systems in Figures 3A and 3B standardize the data flow across context. By using the same data flow for data acquisition as for the analysis itself, the potential for accuracy degradation due to differences in data channels is significantly reduced, and the training and testing of the analysis module 23 are improved.

[0081] The data collected through the data flow shown in Figure 3B can also be used to analyze various forms of MC behavior. For example, the collected data can be used to test the results of the analysis module before it is embedded, using the expected data format. Also, if the analysis module is not working as expected, it may be useful to review the collected data itself in detail, and the analyst may determine that this is due to unexpectedly high noise levels, a faulty sensor, or some other influence on the data.

[0082] Figure 4 is a flowchart illustrating an exemplary process for configuring an embedded analysis module to be monitored, according to an embodiment of the present disclosure.

[0083] In step 401, the analysis module is prepared. In embodiments where the analysis module is an AI / ML inference module, it is prepared through an interactive training and testing process, which is repeated iteratively until the accuracy of its decisions on new data is satisfactory, as described with reference to operations 1101–1103 of the method shown in Figure 11. In other embodiments, alternative forms of the analysis module may be prepared during this operation by a process appropriate to the respective module.

[0084] In 402, the prepared analysis module is typically translated into embedded code and associated parameter data, usually in C or C++, enabling efficient implementation of the module's operation on a microprocessor or microcontroller.

[0085] In 403, an executable analysis module, whether in source or object form, is linked to an embedded project. This embedded project (also called the “embedded application”) is the overall codebase for the MC software to achieve its design goals. As mentioned earlier, an analysis module is typically only a part of the overall functionality of MC, taking data as input and producing a decision as output, with the rest of the embedded software acting on that output as needed as part of its execution.

[0086] If not already included in the embedded project, in 403, a data delivery module (DS) (e.g., DS 25) is also linked to the project. The DS is configured to deliver data from an electronic device (e.g., a device including MC 20) to an external monitoring workstation or similar device (e.g., host 10). The impact of this addition to the embedded project is minimal. This combined embedded project may, at the user's discretion, be in the final production code or development testing phase.

[0087] Once the embedded project is fully configured, it is compiled using the appropriate toolchain and flushed to the MC with a 404 error. This completes the programming to the MC.

[0088] In 405, the MC may be connected to the host system via a communication channel (for example, a communication stack 26 and a data channel 30 as shown in Figure 3A).

[0089] In 406, the embedded operation of the project is usually initiated by powering on the MC, which automatically starts up and runs its firmware. At this stage, the embedded project is functioning normally. During operation, the analysis module periodically analyzes the received data according to its program, and the DS package transmits operational data corresponding to these analyses via the communication channel. In 407, this operational data is received and monitored by a monitoring application on the host system, allowing the user to monitor aspects of the embedded application's operation that were previously invisible.

[0090] For context, this disclosure briefly describes the nature of this operation data with reference to Figures 5A to 5C.

[0091] Figure 5A shows a basic AI / ML inference process in a classifier type analysis module according to one embodiment of the present disclosure.

[0092] A dataset contains one or more data points collected from a data source. If the data points in a set were collected over the same period (and therefore likely to reflect the same object or event), the set is also called a data frame.

[0093] The dataset is provided as input to the module in 501, and features are extracted in 502. Feature extraction can be performed using statistical computation or explicit and closed forms such as FFT, or it can be a learned behavior as seen in convolutional neural networks (CNNs). Each of these extraction processes is known in the art.

[0094] The extracted features are fed into a decision structure such as SVM, NN, random forest, or other suitable structure known in the art. In 503, the structure applies the learned mathematical transformation to determine the most likely class of the input data frame, and in 504, it outputs this result.

[0095] Figure 5B shows operation 503 of Figure 5A in more detail. In a typical AL / ML inference classifier, operation 503 may be divided into two parts: namely, determining the score for each class through the inference process in 5031, and selecting a “winner” by examining and comparing these scores in 5033. Depending on the embodiment, the scores may undergo additional processing, such as weighting, before comparison.

[0096] Optionally, individual class scores can be normalized in 505 and a second output can be generated in 506 that provides ranked comparisons. The normalized range may be 0 to 1 (or equivalently, 0% to 100%).

[0097] Normalized scores can differ from the raw individual scores underlying a classifier, as not all AI / ML decision architectures naturally produce scores within a normalized range. For example, many neural networks (NNs) output scores in the range of 0 to 1 due to their activation features, while others can use the range of -1 to +1. Also, many forms of SVMs output scores in the range of 0 to -infinity. However, appropriate algorithms for normalizing scores are well understood in this art.

[0098] The systems and methods disclosed are not limited to these, but a preferred normalization algorithm is the Softmax formula. Softmax has the additional advantage of normalizing the scores, which, when summed up, produce a total of 1. Thus, Softmax-normalized scores can be interpreted as confidence estimates or as probability values ​​assigned to each class.

[0099] Preferably, both the classification result output in 504 and the normalized score output in 506 are included in the payload sent by the data shipper module. The normalized scale provides a convenient score comparison, as relative scores are more important than absolute values ​​in determining the winner. Thus, having both outputs enables important diagnostics for live debugging and testing, as will be discussed further.

[0100] Figure 5C extends the process shown in Figure 5A by adding a smoothing operation 508. The output at 504 is fed into a smoothing buffer, which "smooths" the most recent N samples at 507, and outputs the smoothed result at 508. Smoothing reduces the possibility of instantaneous output errors by considering recent history, thereby suppressing intermittent errors and improving overall accuracy. More specifically, smoothing examines a window of recent history and applies the function to a set of past predictions to improve the accuracy of the current prediction. Unlike many other aspects of the AI / ML inference module, smoothing parameters such as buffer length and smoothing type can be easily reconfigured even after the module has been embedded to further refine the accuracy of its inference. These parameters are discussed further herein.

[0101] The above operation is described for AI / ML classifiers, but it will be obvious to those skilled in the art that the operation for detectors or regression outputs is similar. A detector essentially outputs two classes, true or false, and optionally a score, for the presence of a learned object. Regression predicts a numerical value that can essentially be treated the same as a denormalized score value for processing purposes, without the need to resolve the class.

[0102] Figure 6A is a flowchart illustrating an exemplary process flow for an embedded application for providing analytical data, according to an embodiment of the present disclosure.

[0103] In the embedded application, the main application logic is continuously operated in 601. The logic can operate the device's functions and provide user interaction, etc. However, in embodiments or configurations where the user is primarily interested in the analysis operation itself, the logic may be a simple loop. Nevertheless, operation 601 continues until the device is stopped or shut down.

[0104] In 602, it is checked whether an analysis event has occurred. It will be understood by those skilled in the art that such branch-on events may occur on an "if-then" statement, an interrupt, a callback function, or any other suitable equivalent means known in the computer art.

[0105] When an analysis event occurs, in 603, the output (e.g., class or regression value) is captured along with the score data, if appropriate. Optionally, the data may include both raw (pre-smoothed) and final (smoothed) outputs, as described above. In various embodiments and configurations, the captured data may instead include only one or more subsets of these elements.

[0106] In 604, the captured data is organized into a payload by the DS, and in 605, the payload is transmitted to the host system via the communication bus as one or more packets.

[0107] The embedded system then completes part of the process and returns to operation 601, with any resources being returned to the main application logic until the next analysis event. Thus, live data is captured and sent to the host system with little to no interference to the internal operation of the embedded application.

[0108] In certain embodiments, the execution and data sharing processes and parameters for an embedded application may be fixed at compile time by setting certain values, flags, or specific code. In other embodiments, the configuration of the embedded application may be externally adjustable and modified during live operation.

[0109] Figure 6B is a flowchart illustrating an exemplary process for an embedded application to reconfigure its operation according to one embodiment of the present disclosure.

[0110] Similar to Figure 6A, the embedded application continues to run its main application logic at 601. However, the application checks for incoming signals from the host at 606. If a signal arrives, it is received at 607 and decompressed at 608. At 609, the commands, parameters, and settings in the signal are applied to reconfigure the behavior of the DS.

[0111] Those skilled in the art will recognize that both processes in Figure 6A and Figure 6B can be executed simultaneously within the same system.

[0112] Possible reconfigurations include specifying which data elements will be included in future payloads (e.g., the final output but not the score). Another possible reconfiguration involves simply turning the transmission on or off, thus saving communication overhead cycles when the information is not needed.

[0113] Figure 7 is a sequence diagram illustrating an exemplary operation flow for controlling the operation of an embedded payload module (e.g., a data shipper module) from a host system according to one embodiment of the present disclosure. The illustrated flow assumes that the code has already been flushed to the MC and is running. In practice, the operation and interaction on the MC 20 may be handled by the data shipping module (e.g., DS 25 in Figure 3A), and the operation and interaction on the host system 10 may be handled by a monitoring application (e.g., monitor 11 in Figure 3A).

[0114] To initiate operation, user 1 instructs host system 10 to connect to MC 20 at 710. Host 10 then initiates a handshake with MC 20 at 711. If MC 20 is operational, it confirms the handshake at 712 and provides host 10 with configuration information regarding the format of the data payload to be transmitted. As described in relation to Figure 3A, these payloads may include analysis module outputs 231, internal data 232, and possibly raw data 221 from data collector buffer (DC) 22. The configuration information can set parameters for the format of this information that will be included in the payload, including, for example, frame length, the channel providing the data, data type, and other features. Monitor 11 on host 10 configures itself expecting such data.

[0115] The user instructs the host 10 to send a start signal at 713, generally through the activation of a "go" or "play" style button on the GUI, as will be further described below. The monitor 11 may include a control module (not shown) configured to format and generate this monitoring control signal and other monitoring control signals. In response to this instruction, the host 10 sends a start signal at 714 to the MC 20, and the MC 20 provides an acknowledgment at 715.

[0116] At 716, DS 25 begins transmitting payloads corresponding to each determination from the analysis module 23 on MC 20. At 717, information from these packets is buffered or stored on host 10, and the interface display (e.g., GUI 112) is updated in response. Operations 716 and 717 are repeated typically through the operation of the “Stop” or “Pause” style button on the GUI, as further described below, until the user instructs host 10 to send a stop signal at 718. In response to this command, host 10 sends a stop signal to MC 20 at 719, and MC 20 acknowledges at 720.

[0117] The monitoring control signals described with reference to Figure 7 are intended to control communication traffic from DS 25. In certain embodiments, these signals can stop and start some or all of the operation of the core application 28, but preferably not. The advantage of this configuration is that MC 20 can continue normal operation regardless of whether the host 10's monitor 11 is operational and connected. Thus, engineers can use the product as usual, placing it in various test situations and initiating data transmission only when necessary to more rigorously evaluate the operation of the analysis module 23.

[0118] Figure 7, as also shown in Figure 6B, illustrates a fully controlled interaction between host 10 and MC 20. Alternatively, DS 25 can be configured to continuously transmit data packets at startup, eliminating the requirement that MC 20 receive communications from host 10. Host 10 only needs to listen to incoming packets and parse the information within them, and may ignore packets when evaluation is not desired. In such an implementation, configuration data may be carried in the packet stream or may be available locally on host 10. User control may affect the information presented via the GUI, but not the actual ongoing communications from MC 20.

[0119] Here, some exemplary embodiments of GUI 112 are shown in Figures 9A to 9L.

[0120] The exemplary GUI shown in Figure 9A reflects the output of a classifier-type analysis module. In Figure 9A, a data connection button 901 is provided at the top of the GUI for accessing parameters of the communication settings with the MC and the circuit in which it is installed, and a module information button 902 is provided for loading information about the monitored analysis module. In some embodiments, the information retrieved by button 902 may be provided by an IDE that has merged the analysis module into the embedded firmware. In other embodiments, the information may be provided as part of a packet stream from the MC.

[0121] The retrieved information may include its name and its class map, as presented in 911 and 912, respectively. Class map 912 is prominent in the case of a classifier. In the illustrated example, there are three possible class outputs: “No Results,” “Normal,” and “Unbalanced.” One of these classes is output by the analysis module for each inference. Each class can be numerically labeled. Furthermore, each class may be color-coded or distinguished by differences in visual characteristics for easy reference on the main display below. Class map 912 can serve as a legend for these labels.

[0122] The exemplary GUI presents a data plot divided into three sections: raw output row 920, final output row 930, and class score chart 940. The data plot is also divided into columns, each representing one inference or decision case. Each column spans all three sections.

[0123] As shown in the diagram, the system has already begun collecting data to fill the data plot. During operation, the data plot may be filled from right to left in the form of a horizontal scroll chart record, with the most recent determinations placed in the rightmost column (e.g., raw prediction at 921 and final output at 9411) and the oldest visible determinations in the leftmost column.

[0124] In the illustrated embodiment, the visible information in each column is decomposed as follows:

[0125] The raw predictions 921 of the classifier appear in the upper column 920. Each raw prediction 921 is determined by a decision process in the classifier, which typically involves assigning a score to each possible class and selecting the class with the highest score as the correct class, as described above.

[0126] The class score chart 940 includes a series of line indicators 941 representing the relative class score of each of the three possible classes. Each class line indicator 941 has a different color or other visual characteristic, allowing the user to easily distinguish between classes.

[0127] The illustrative GUI positions each line indicator 941 on chart 940 vertically according to the represented score value. For most classifiers, the winning raw prediction presented in row 920 should reflect the highest score on chart 940 in the same column. This unique presentation allows engineers to determine how "confident" the AI / ML inference is by reviewing the comparison scores and how close they are vertically on chart 940. If the classifier makes a mistake in some cases, the engineer can see that the correct answer was nevertheless a close contender. The degree of error determined in this way will influence the engineer's corrective actions when tuning or improving the system.

[0128] In the exemplary GUI, the presented score values ​​are normalized on a scale from 0 to 1, as described above with reference to Figure 5B. Since the exemplary GUI is intended to provide engineers with quick and easy visual comparisons, preferred embodiments include such intuitive normalization, but other representations are also within the scope of this disclosure.

[0129] The lower column 930 displays the final output 931 of the classifier, which is sent to the application logic (e.g., output 231). This output 931 may differ from the raw prediction 921 due to smoothing or other post-analysis weighting and adjustments, as described above with reference to Figure 5C.

[0130] The smoothing buffer 923 contains the latest decision in the raw prediction row 920 and is visually connected to the output link box 933 in the final output row 930, indicating that the decision in the smoothing buffer 923 is influencing the current final output 9311 in the output link box 933 through smoothing.

[0131] Referring briefly to Figure 9C, the illustrated portion of the GUI includes a smoothing type indicator 9231, a smoothing window length indicator 9233, a buffer box 9235, and an output link box 933.

[0132] In the illustrated example, the indicated smoothing type (i.e., the algorithm used for smoothing), as indicated by indicator 9231, is "class score," and the class scores for all decisions in the smoothing buffer 923 are summed (or equivalently averaged) to select the final decision for display in the final output column 930. The number of decisions considered in the smoothing buffer 923 is the buffer length of 7 in the illustrated example, as indicated by indicator 9233.

[0133] Other smoothing algorithms include, for example, "voting smoothing" or "phenomenon smoothing," where the winner is selected by common occurrence in recent buffers, and "arithmetic smoothing," where the mean or median is selected. Arithmetic smoothing is most applicable when dealing with numerical output from regression analysis, rather than discrete classes established by a classifier. None of these alternative algorithms are illustrated, but their appropriate implementations should be evident from the context of those skilled in the art.

[0134] The smoothing type indicator 9231 also shows the formation of a buffer, which is a window in Figure 9C. In this form, a fixed-length smoothing buffer window (i.e., a fixed number of data points, indicated by 603) "slides" across the most recent raw predictions. As each new payload is received by the monitoring application, the new raw values ​​are added to the smoothing buffer, and the oldest values ​​are excluded from consideration.

[0135] Figure 9D shows another form of buffer, namely "grouped smoothing," in which the smoothing window length indicator 9233 indicates that the length is "variable," and in grouped smoothing, data is accumulated in the buffer until the application determines that it has completed according to a given criterion. Grouped smoothing is typically used when an ongoing process indicates when a new decision window begins and ends. For example, a user might collect repeated measurements of sound bouncing off a concrete column over a variable period, and the AI / ML makes a final decision about the quality of the column when the measurements are complete. In this example, smoothing is performed over all AI / ML inference events until the user is ready. Similarly, in an end-of-line inspection application, data is collected about products manufactured under the control of an assembly line, and a final decision is made when the product is removed. In other cases, an external event may trigger the end of the group.

[0136] The smoothing buffer box 9235, indicated by the dotted frame, visually represents the data contained in the smoothing buffer 923. Figures 9E-9G compare various instances of the smoothing buffer box 9235. In Figure 9E, as indicated by indicator 9233, the buffer length is 7, and therefore the buffer box 9235 contains 7 analysis results. In Figure 9F, the buffer length is reduced to 3, and correspondingly the buffer box 9235 becomes shorter. In Figure 9G, the buffer length is reduced to 1, which is equivalent to no smoothing at all.

[0137] As mentioned above, smoothing parameters can be adjusted and refined relatively easily on the microcontroller. In some embodiments, a GUI allows the user to adjust these parameters. The GUI can apply adjustments separately to the received data on the host 10 during review, while the monitored analysis module continues to draw its own inferences based on the previous parameters, or the GUI can adjust its internal parameters and send instructions to the microcontroller to refine the operation of the monitored analysis module. The user can decide to adjust and experiment with the parameters separately until the optimal parameter setting is found, and then transfer that parameter setting to the microcontroller.

[0138] Returning to Figure 9A, as mentioned earlier, the most recent data is stored in the rightmost column of the data plot in 9211 and 9311, with the data points to the left of that column being progressively older. The GUI may provide a zoom control 903a that allows the user to adjust the number of visible columns. While not always the case, typically manipulating the zoom control 903a adds or removes columns to the left of the data plot, while anchoring the rightmost column. The zoom control 903a adds or subtracts from the visible history, but preferably does not remove undisplayed information older than the displayed range from the application buffer (e.g., buffer 111 in Figure 3A).

[0139] The GUI can provide start and pause controls 903b. The user first uses the start control to begin browsing, which initiates the processing of new data and its display in the GUI. In certain embodiments, the start control can trigger a start signal to the MC (e.g., start signal 713 in Figure 7), while in other embodiments, it can simply instruct the monitoring application to begin processing data that has already been transmitted. Similarly, the stop / pause control instructs the monitoring application to stop the process and display the data, and in certain embodiments, it can trigger a stop signal to the MC (e.g., stop signal 718 in Figure 7). When the update is stopped, the GUI display is paused, and the user can zoom in / out and scroll to review the columns of collected data. In some embodiments, the user can save a record of the received data to a log file.

[0140] The GUI provides a scrollbar control 904 that allows the user to scroll back and forth, moving the data back into view on the display by "descending" the leftmost column. The data in the rightmost column is temporarily removed to provide presentation space. Preferably, the scrollbar control 904 is only operational when the data view is paused to avoid potential disruption from new data arriving "outside" the visible window.

[0141] The GUI may provide an inference time value indicator 913. In some embodiments, the inference time may be measured using a dedicated timer on the MC, starting the timer before each new inference and ending it thereafter. This is the most accurate approach. However, in other embodiments, the inference time may be measured using a host system to time the arrival of new inference data, and thus the inference time is approximated by the interval between outputs.

[0142] As described above, the GUI shown in Figure 9A is configured to present the class determination output from the classifier analysis module. The exemplary GUI shown in Figure 9B is similarly designed but reflects the output from the regression analysis module. The main feature changes are as follows: the raw prediction 921 is now numerical rather than an integer class, as is the smoothed final output value 931. The data format (e.g., number of decimal places) can be set using the value format control 905. The smoothing type indicator 9231 of the smoothing buffer 923 is "average score" in the illustrated example, meaning that the latest final output 9311 reflects the run-by average of the contents of the smoothing buffer 923. Finally, the class score chart 940 is replaced with a line graph 940′, and the raw prediction line 943a and the smoothed prediction line 943b are presented for easier comparison. In regression analysis, comparing these lines can be used to examine how well the smoothing and / or other adjustments from the raw predictions are performed, improving and stabilizing the final output.

[0143] The exemplary GUI shown in Figure 9H is a modification of the GUI shown in Figure 9A. This GUI provides a project selection control 906 that allows selection of a project to which an analysis module is associated, and an analysis module selection control 907 that allows selection of an analysis module from among multiple modules that may simultaneously be included in the selected project. Zoom, start, and pause controls are grouped together as an overall chart control 903. The GUI also provides a time axis 950 for tracking elapsed time in the chart, along with axis controls 908 for adjusting the axis scale and chart style. Settings for a forecast time indicator 914 describing the total time, last interval, and average interval are also provided.

[0144] The exemplary GUI shown in Figure 9I is a variation of the GUIs shown in Figures 9A and 9H, configured for a classifier where smoothing is not enabled. The raw prediction row 920, smoothing buffer 923, and output link box 933 are only significant for smoothing and are omitted for simplicity of display. Although not shown, similar adjustments to the exemplary GUI shown in Figure 9B can be made for cases where smoothing is not part of the regression analysis.

[0145] The exemplary GUI shown in Figure 9J is a variation of the GUIs shown in Figures 9A and 9H. Although it uses a different layout and style, this GUI provides the same features and functions, and their descriptions will not be repeated.

[0146] The exemplary GUI shown in Figure 9K is a variation of the GUI shown in Figures 9A and 9H, and implements an alternative class score chart 940″ that displays stacked bar indicators instead of the line indicators 941 displayed in the class score chart 940. Bands 941a″, 941b″, and 941c″ each represent different classes. Each band has a distinct color or other visual characteristic to allow the user to easily distinguish them. The relative ratio of the columns represents the relative score of the classes, with larger areas representing higher class scores. The total is always 100%.

[0147] In certain embodiments, a user can select between multiple GUIs using chart style selection controls, such as control 908 provided in the exemplary GUI of Figure 9H.

[0148] Figure 9L shows an exemplary GUI initially presented in Figure 9A, illustrating additional aspects of GUI program functionality.

[0149] As mentioned earlier, when the GUI is first initialized, the raw prediction row 920, the final output row 930, and the class score chart 940 are not filled. Rather, during initialization, the GUI begins to populate each section with values ​​as they arrive from the MC. Each new value fills the rightmost column (e.g., 9211, 9311, and 9411). The data scrolls to the left in the direction of arrow 90 each time a new data payload is received. Thus, the data plot scrolls like a chart record as new data is added. The first received raw prediction 9212 and the first received final output 9312 are seen in the leftmost column, but are not yet present in the leftmost column of the entire data plot.

[0150] The time axis 950 is adjustable according to the time scale control 908a. As shown, "ms" (milliseconds) is selected, and the axis 950 is marked in corresponding ms time units. The first mark 951 on the axis 950 indicates "0ms," corresponding to the start of the first raw forecast 9212 and the first final output 9312. The marking interval on the axis 950 may be automatically adjusted, preferably to prevent clutter. As shown, the second providing mark 952 indicates "500" (i.e., 500ms). As indicated by the indicator 913, the average inference time of 100ms corresponds to the interval between inference values ​​in the chart, and therefore the second providing mark 952 corresponds to the fifth received value. The total time indicator 914a shows the time elapsed since initialization (700ms), as shown.

[0151] Figure 10 is a flowchart illustrating an exemplary process for receiving and presenting analytical data in a monitoring application according to embodiments of the present disclosure. The following description assumes the use of the exemplary GUIs shown in Figures 9A–9L or similarly, but those skilled in the art can extrapolate to other GUI designs.

[0152] At startup, in 1001, the application configures itself and initializes its GUI. The initialized GUI can provide data for various fields if searchable or if default values ​​are provided and applicable, otherwise it can display placeholders or blank areas.

[0153] In some embodiments, if smoothing can change during live operation (not shown in Figure 10), the smoothing parameter indicators (e.g., indicators 9231 and 9233 in Figure 9C) may not be fixed at initialization but may be periodically updated in the GUI based on the payload or other signals via the communication channel.

[0154] After configuring and initializing the GUI, the system waits at 1002 to receive live data from the embedded side by looking at the incoming communication port on the host system.

[0155] When valid data for a new analysis event is received, the following actions are repeated:

[0156] Any previous data within the column area (e.g., raw forecast, class indicator, and final output) is scrolled one column to the left at 1003, and the rightmost column is temporarily left blank. (In the first iteration, the column area is blank, and this behavior can be omitted.) If all visible columns are full, the oldest (leftmost) column is scrolled off the screen and is no longer displayed, but it is desirable that its contents remain stored in memory.

[0157] At 1004, the new prediction data is added to the GUI. An exemplary execution of this operation has already been shown in Figure 9L, but again, any suitable GUI is within the scope of this system. In the context of Figure 9L, the new predictions in both the final output row 930 and the raw prediction row 920 (assuming the option of selecting both is available) are reported at positions 9211 and 9311 in the rightmost column, respectively, which were cleared at 1003 when the existing data was moved to the left.

[0158] In step 1005, the graph points are calculated according to the received scores, normalized to the GUI display range. In step 1006, the graph points are added to the graph area, for example, as line indicators. In the context of Figures 9A to 9L, the graph points are added to the class score chart 940 in the rightmost column.

[0159] The graph values ​​and display format vary depending on the settings. For classifiers, in some embodiments, the graph may include line indicators showing individual class scores, which are color-coded or otherwise visually distinguishable, as shown, for example, in Figure 9A. For regression determination, in some embodiments, the display may include a line graph with lines showing raw scores and smoothed scores, for example, as shown, for example, in Figure 9B. In certain embodiments, alternative graph formats, such as shown, for example, in Figure 9K, may be the default or may be selected by the user.

[0160] Following operation 1006, the most recent (rightmost) column of the data displays new data for both prediction and graphical score or value information.

[0161] In some embodiments, additional data can be updated in 1007. For example, as previously described, the inference interval (e.g., the inference time 913 in Figure 9L, or the interval time 914 in Figure 9H) may be calculated and displayed. Smoothing data may be updated if it changes during operation. Other embodiments may add additional updated data elements.

[0162] In certain embodiments, at 1008, the host system determines whether the user has put the monitoring application into a “paused” state by operating a control (e.g., control 903b in Figure 9A) which pauses display updates until resumed by the user. While in the “paused” state, certain embodiments may continue to accumulate and buffer new data at 1009 without displaying it, while other embodiments may discard such data, and yet another embodiment may instruct the DS on the MC to stop transmission, as described with reference to Figure 6B. During the “paused” state, the user can explore and inspect previously recorded data by using scroll controls (e.g., scroll bar 904 in Figure 9A) and / or adjust the number of visible columns by using zoom controls (e.g., zoom control 903a in Figure 9A). Inclusion is preferred, but certain embodiments may omit the “pause” function.

[0163] If the monitoring application is not paused (or has been resumed from paused mode), the host system determines at 1010 whether the process has terminated. If not, the process loops to approximately 1002 and enters another iteration. Typically, as illustrated with reference to Figure 9L, for example, the next iteration continues to accept new incoming data, filling the display from right to left. Once the number of available columns is filled, the next iteration of operation 1003 scrolls the old data from the left side of the display.

[0164] The process continues until the monitoring application is terminated at 1010.

[0165] The disclosed systems and methods are not limited to monitoring only one analysis module, but can easily monitor two or more modules in one or more embedded devices. For example, a monitoring application can open a separate GUI window for each module. As another example, controls within the GUI, such as buttons 906 and 907 in Figure 9H, may enable switching between monitoring the analysis module and the instrument.

[0166] Using a monitoring application, users can review the individual decisions of the embedded inference module in real time. Furthermore, the same data that the inference module is analyzing can be analyzed on a test module, as described above, to verify the accuracy of these decisions. If the decisions are inaccurate, users can take specific measures to improve the accuracy of the inference module. For example, a local copy of the inference module may be further trained on the received data, particularly the received data that the embedded inference module analyzed inaccurately, and once the refinement training of the local inference module is complete, it may replace the embedded inference module in the microcontroller. However, users may decide that certain types of errors can be corrected by adjusting the operating parameters of the existing inference module, including but not limited to the smoothing parameters described above. This form of improvement, if effective, is much faster and easier to implement than a complete retraining.

[0167] These and related processes, along with other necessary instructions, may be encoded as executable instructions on one or more non-temporary computer-readable media such as a hard disk drive or optical disc, and executed using one or more computer processors in conjunction with an operating system or other suitable means. Furthermore, one or more of the components described above may be stored on a computer-readable storage medium and implemented as instructions executable by at least one processor (and / or may include at least one processor).

[0168] These computer-readable program instructions can be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing device to generate a machine. The instructions are executed via the computer's processor or the processor of another programmable data processing device, thereby creating means for performing the functions / operations specified in the blocks of a flowchart and / or block diagram. Computer-readable program instructions can also be stored in a computer-readable storage medium that can instruct a computer, a programmable data processing device, and / or other device to function in a particular way. The resulting computer-readable storage medium then comprises a product containing instructions for implementing the modes of function / operations specified in the blocks or multiple blocks of a flowchart and / or block diagram.

[0169] Computer-readable program directives can also be loaded onto a computer, other programmable data processing device, or other device that executes a set of operational steps established within the directives to generate a computer implementation process. More specifically, the directives can be executed on a device to implement a function / operation specified in a block or multiple blocks of a flowchart and / or block diagram.

[0170] In software implementations, software may include a set of computer-executable instructions that are implemented on a computer system. Before being loaded onto a computer system, software may exist as encoded information on a suitable tangible, non-temporary computer-readable storage medium, such as magnetic, optical, or other appropriately encoded or recorded media. A computer-readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium may include, but is not limited to, any suitable combination of electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, and semiconductor storage devices. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, and mechanically encoded devices such as punched cards or raised structures in grooves on which instructions are recorded, as well as any suitable combination of the media described above. In certain embodiments, the computer-readable storage medium may take the form of existing data storage (such as "cloud storage") accessible via a network means (e.g., the Internet) to which it is operably coupled. When used herein, the computer-readable storage medium should not be construed as a transient signal in itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0171] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a computing device or processing device, or to an external computer or external storage device, via a network such as the Internet, a local area network, a wide area network, or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing device or processing device receives computer-readable program instructions from the network and transfers these instructions for storage on a computer-readable storage medium within the respective computing device or processing device.

[0172] Computer-readable program code or instructions for performing an operation may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages. Possible languages ​​include object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as Pascal and C. Computer-readable program instructions can run entirely on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer or server, or entirely on a remote computer or server. In scenarios involving a remote computer or server, the remote device may connect to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be established via an external computer (for example, via the Internet using an Internet service provider). In some embodiments, electronic circuits such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can execute computer-readable program instructions by utilizing state information of these instructions to personalize the electronic circuit and perform specific operations.

[0173] In some implementations, the system may include a dedicated processor, a processing segment of a system-on-a-chip (SoC), a segment of a field-programmable gate array (FPGA), or other suitable means, which execute processor instructions for performing the functions described herein or for emulating some of the structures defined herein. For example, suitable circuits using discrete logic gates, such as application-specific integrated circuits (ASICs), programmable logic arrays (PLAs), or field-programmable gate arrays (FPGAs), may also be developed in certain embodiments to perform these functions.

[0174] As an example, Figure 12 is a block diagram illustrating an exemplary computer system for program and / or hardware implementations of various aspects of the disclosed system and method. For example, in various embodiments, it functions as a host for hardware modules and / or runs software modules such as electronic design automation (EDA) tools, simulations, emulations, and firmware, according to different configurations of the disclosed system and method.

[0175] According to a particular embodiment, the computer system 1200 includes a processor 1202, main memory 1204, an interconnection bus 1206, a memory controller 1208 coupled to a memory device 12082, peripheral devices 1210, an input control device 1212, a portable storage medium drive 1214, a graphics subsystem 1216, and an output display 1218. Depending on the requirements of a particular embodiment and intended use, only all or some of the functionally shown system components may be required for actual implementation. In various embodiments, the processor 1202 includes a single microprocessor or multiple microprocessors to configure the computer system 1200 as a multiprocessor system. The main memory 1204 partially stores instructions and data executed by the processor 1202. The main memory 1204 preferably includes banks of dynamic random access memory (DRAM) and high-speed cache memory.

[0176] For simplicity, the components of the computer system 1200 are shown as being interconnected via an interconnection bus 1206. However, in alternative embodiments, the computer system 1200 is interconnected through one or more data transfer methods. For example, in a particular embodiment, the processor 1202 and the main memory 1204 are interconnected via a local microprocessor bus, while the memory controller 1208, peripheral devices 1210, portable storage media drives 1214, and graphics subsystem 1216 are interconnected via one or more input / output (I / O) buses. The memory device 12082 is preferably implemented as a non-volatile semiconductor memory for storing data and instructions used by the processor 1202. Furthermore, the memory device 12082 is preferably used to store the software necessary to load it into the main memory 1204, although in alternative embodiments, it may be represented in EDA tool simulations by an appropriate class (incorporating data structures and functions operable on those data structures) or similar methods known to those skilled in the art.

[0177] The portable storage medium drive 1214 operates to input and output data and code to and from the computer system 1200. In one configuration, software is stored on such a portable medium and input to the computer system 1200 via the portable storage medium drive 1214. In various embodiments, the peripheral device 1210 may include any type of computer support device, such as an input / output circuit (I / O) interface, to add additional functionality to the computer system 1200. For example, in certain embodiments, the peripheral device 1210 includes a network interface card for networking the computer system 1200. In certain embodiments, the peripheral device also includes a memory controller and non-volatile memory.

[0178] The input control device 1212 provides part of the user interface for the user of the computer system 1200. In various embodiments, the input control device 1212 may include an alphanumeric keypad for inputting alphanumeric and other key information, and a cursor control device such as a mouse, trackpad or stylus, or cursor directional keys.

[0179] To display text and graphic information, the computer system 1200 includes a graphics subsystem 1214 and an output display 1218. In various embodiments, the output display 1218 may include a cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma display, or an active-matrix organic light-emitting diode (AMOLED) display. The graphics subsystem 1216 receives text and graphic information and processes the information for output to the display 1218.

[0180] Using the disclosed system and method, engineers can visualize raw activity in otherwise closed embedded systems. This can be applied not only to the prototyping phase of evaluation boards but also to production hardware created by engineers. The disclosed system and method thereby facilitate a rapid transition to live testing of the complete device, as well as a deep review of the operation and internal state of analysis modules for analysis and debugging, representing a significant advance in the art.

[0181] In light of the above teachings, it will be apparent that numerous modifications and variations of the disclosed systems and methods are possible. Furthermore, it will be apparent that the disclosed systems and methods may be carried out in ways other than those expressly described herein. Moreover, any or all of the disclosed elements and operations may be selectively combined, and all particular locations in the sequence of elements or operations may be reversed or inserted without departing from the spirit or scope of the disclosed systems and methods as defined in the appended claims. Accordingly, the scope should be determined by reference to the scope of this disclosure and the appended claims, together with their equivalents of their entire scope.

Claims

1. A system for monitoring inference performed by a microcontroller (MC) in real time, the system includes: An MC that is communicatively connected to an electronic device, the MC includes: At least one MC processor, MC communication stack configured to send and receive signals, An MC memory in which MC program code executable by at least one of the MC processors is embedded, The aforementioned MC program code includes the following: An inference module configured to cause at least one MC processor to analyze a dataset and generate a corresponding decision value, wherein an electronic device selectively performs at least one operation based on the decision value. A payload module configured to generate a judgment data payload corresponding to each judgment value generated by the inference module, and to cause at least one of the MC processors to transmit the judgment data payload via the MC communication stack; Furthermore, a monitoring subsystem located outside the MC and connected to the MC in a communicative manner, the monitoring subsystem includes the following: At least one monitoring processor, A monitoring communication stack configured to send and receive signals, display, A monitoring memory in which monitoring program code executable by at least one of the monitoring processors is encoded, the monitoring program code includes: A monitoring module configured to cause at least one monitoring processor to process a plurality of judgment data payloads transmitted from the MC communication stack and received via the monitoring communication stack, An interface module configured to visually render a user interface on a display, and to cause at least one of the monitoring processors to execute such a process, where the user interface visually represents the determination value of at least one of a plurality of determination data payloads.

2. The system according to claim 1, wherein the determination value is generated based on at least one intermediate value generated during the analysis of the corresponding dataset, and the determination data payload further represents at least one intermediate value on which the determination value is based.

3. The system according to claim 1, wherein the decision data payload further represents the data subset corresponding to the decision value.

4. The monitoring program code further includes a test module configured to cause at least one of the monitoring processors to determine at least one of the plurality of determination data payloads, Based on the analysis of the resulting data subset, at least one validation value is generated, and the test results are generated based on a comparison between the resulting decision value and the generated validation value. The system according to claim 3.

5. The system according to claim 1, wherein the inference module is an artificial intelligence (AI) inference module.

6. The payload module is further configured to cause at least one of the MC processors to generate a data acquisition payload representing a dataset for each of the plurality of datasets, and to transmit the data acquisition payload through the MC communication stack, and the monitoring program code further comprises a data storage module configured to cause at least one of the MC processors to store the received data acquisition payload received through the monitoring communication stack in data storage. The system according to claim 5.

7. The judgment data payload further represents a subset of data corresponding to a judgment value, and the monitoring program code further comprises a data acquisition module configured to cause at least one of the at least one monitoring processors to process the subset of data represented in the judgment data payload received for AI training. The system according to claim 5.

8. The monitoring program code further comprises a control module configured to cause at least one of the monitoring processors to generate monitoring control signals for transmission via the monitoring communication stack. The payload module is configured to cause at least one of the MC processors to selectively transmit a decision data payload via the MC communication stack based on the reception of the monitoring control signal via the MC communication stack, and the inference module is configured to cause at least one of the MC processors to analyze the dataset and generate at least one corresponding decision value, regardless of the reception of the monitoring control signal via the MC communication stack. The system according to claim 1.

9. The system according to claim 1, wherein the MC program code further comprises a buffer module configured to cause at least one of the MC processors to organize data into a plurality of datasets configured to be analyzed by the inference module.

10. A method for monitoring inference performed by a microcontroller (MC) in real time, the method comprising: Construct an MC comprising at least one MC processor, an MC communication stack configured to send and receive signals, and an MC memory embedded with an inference module. A monitoring subsystem is constructed that is located outside the MC and is connected to the MC in a manner that enables communication with the MC, wherein the monitoring subsystem comprises at least one monitoring processor, a monitoring communication stack configured to send and receive signals, a monitoring memory, and a display. Multiple datasets are acquired by at least one of the MC processors; At least one of the MC processors analyzes each of the plurality of datasets and generates at least one corresponding decision value. A judgment data payload representing the aforementioned judgment value is generated, The aforementioned determination data payload is transmitted via the MC communication stack. At least one of the monitoring processors receives a plurality of determination data payloads transmitted from and received via the MC communication stack. At least one of the monitoring processors generates a user interface on the display that represents the judgment value of each processed judgment data payload of the plurality of judgment data payloads received.

11. A method according to claim 10, wherein the analysis of each of the plurality of datasets generates at least one intermediate value based on the dataset, generates at least one decision value based on the at least one intermediate value, and the decision data payload further represents at least one intermediate value on which the decision value is based.

12. A method according to claim 10, wherein each generated decision data payload further represents the data subset corresponding to the decision value, and the method further includes the data subset corresponding to the decision value. At least one monitoring processor, for at least one of multiple decision data payloads, Based on the analysis of the data subset represented by the aforementioned determination data payload, at least one validation value is generated. A method comprising generating a test result based on a comparison between the generated verification value and the determination value represented by the determination data payload.

13. The method according to claim 10, wherein the inference module is an artificial intelligence (AI) inference module, and each generated decision data payload further represents a subset of data corresponding to a decision value, A method further comprising processing a subset of data represented in a decision data payload received for AI training by at least one monitoring processor.

14. A method according to claim 10, further comprising generating a monitoring control signal for transmission via a monitoring communication stack by at least one monitoring processor, At least one of the MC processors selectively transmits a determination data payload based on having received a monitoring control signal via the MC communication stack. A method comprising at least one MC processor analyzing each of a plurality of datasets, regardless of whether or not it has received the monitoring control signal via the MC communication stack, and generating at least one corresponding determination value.

15. A method for training and updating the artificial intelligence (AI) module of a microcontroller (MC), An MC is established comprising at least one MC processor, an MC communication stack configured to send and receive signals, and an MC memory. Establish a trained AI module, The trained AI module is embedded in the MC memory. A test subsystem is established outside the MC and communicatively coupled to the MC, the test subsystem comprising at least one test processor, a test communication stack configured to send and receive signals, a test memory, and a display. Multiple test datasets are acquired by at least one of the MC processors, At least one MC processor analyzes the tests for each test dataset of multiple test datasets, generates at least one corresponding test decision value, generates a test data payload representing the test decision value, and transmits the generated test data payload through the MC communication stack. A method comprising: receiving a plurality of test data payloads originating from the MC communication stack via the test communication stack using at least one test processor; and generating a user interface on the display using at least one test processor that represents at least one test determination value for each processed test data payload of the plurality of received test data payloads.

16. The method according to claim 15, wherein each test data payload further represents a test dataset corresponding to a test determination value, A method comprising: using at least one of the test processors to analyze a test dataset represented by at least one of the multiple test data payloads; generating at least one corresponding test validation value; and generating a test result based on a comparison between the resulting judgment value and the generated validation value.

17. The method according to claim 16, wherein the MC processor analyzes the test dataset based on the value of at least one analysis parameter, At least one of the test processors generates an update instruction in response to the test result, At least one of the test processors transmits refinement instructions through the test communication stack. At least one of the MC processors receives the refinement instruction via the MC communication stack, A method for adjusting the value of at least one analysis parameter based on the update instruction using at least one MC processor.

18. The method according to claim 15, wherein the trained AI module is A training subsystem is established outside the MC and communicatively coupled to the MC, wherein the training subsystem comprises at least one training processor, a training communication stack configured to send and receive signals, and a training memory. Multiple training datasets are acquired by at least one of the MC processors. At least one of the MC processors generates a training data payload representing the training dataset for each of the plurality of training datasets, and transmits the generated training data payload via the MC communication stack. At least one of the training processors receives a plurality of training data payloads transmitted from the MC communication stack through the training communication stack. A method for generating a trained AI module by training the AI ​​module based on a plurality of received training data payloads using at least one of the training processors.

19. The method according to claim 18, wherein the test subsystem is the training subsystem.

20. The method according to claim 19, wherein each test data payload further represents a test dataset corresponding to a test decision value, At least one of the test processors analyzes the test dataset represented by at least one of the multiple test data payloads and generates at least one corresponding test validation value. A method comprising: retraining the AI ​​module based on at least one of the multiple test data payloads using at least one of the training processors to generate an improved AI module; and embedding the improved AI module in the MC memory.

Images