Method for customizing sensor functionality and smart device comprising sensor

Abstract

The invention relates to a method for customizing a sensor function based on a Wasm technology, and the method comprises the steps: providing a mathematical model for sensor training in an intelligent device and a post-processing program customized for the mathematical model, and the post-processing program is used for carrying out the post-processing of the output data of the mathematical model; deploying the mathematical model in the intelligent equipment; compiling the post-processing program into a Wasm program and deploying the Wasm program in the intelligent equipment; an interface defined based on the WASI specification is provided in the intelligent device, and the Wasm program can access and operate the sensor through the interface. Furthermore, the invention relates to a smart device comprising a sensor, the function of which is customized by means of the method according to the invention.

Description

Method for customizing sensor functions and smart device including sensor

[0001] Citation of Related Applications

[0002] This application claims the benefit of Chinese patent application No. 202310552597.X filed with the State Intellectual Property Office of the People's Republic of China on May 16, 2023, the entire contents of which are hereby incorporated by reference into this document in their entirety. Technical Field

[0003] The present invention relates to the field of computers, and in particular to a technology for customizing sensor functions based on Wasm technology. Background Art

[0004] In the field of intelligent technology, sensors can be used in autonomous driving, equipment detection, environmental detection, logistics detection, security detection, health detection, smart home appliances, retail industry, health care, agriculture, smart cities and other aspects.

[0005] Sensors in smart devices incorporate computational models or AI models developed, trained, or customized for specific purposes. These models are collectively referred to as mathematical models in this article. Sensors can use these mathematical models to provide computational or inference results based on sensor detection. The format of these computational or inference results may vary depending on the model.

[0006] For these different formats, smart devices need to post-process the results on-site to obtain the desired information. A typical post-processing method is to convert the calculation or reasoning results into a readable, visual, and structured metadata character stream.

[0007] As mentioned above, different models may output computation or inference results in different formats. This makes customizing post-processing functionality for models difficult. This functionality must be provided by the model developer. However, model developers typically lack understanding of the hardware and software architecture of smart devices and focus more on the post-processing logic and ensuring its compatibility with the model.

[0008] To customize post-processing functions, developers must know how to identify data in a specific model and how to communicate with sensors. However, developers often lack knowledge of the operating environment of smart devices, making it difficult to deploy compatible post-processing functions that can run on any smart device. Currently, various methods exist for customizing post-processing functions using Wasm technology, but they all present various challenges.

[0009] For example, the Camera Remote SDK, a software development kit provided by Sony, does not provide a general API for sensor control. Waft (WebAssembly Framework for Things), a new development framework proposed by Alibaba engineers, does not provide an API for accessing AI sensors or general sensors to read data. In US20220198071A1, Wasm technology is suitable for task sharing to reduce the burden on the main computing node, but does not provide a method for task simulation on child nodes. CN202210944973, CN202210096203, etc. lack sandbox control, have security issues, or do not comply with the wasi specification.

[0010] Furthermore, current model development relies heavily on several mainstream software frameworks, such as TensorFlow, PyTorch, and Onnx. While these frameworks do accelerate model development to some extent, mathematical models and possible post-processing programs often present issues during deployment and are difficult to debug, especially for low-level, cost-effective industrial equipment.

[0011] Summary of the Invention

[0012] The present invention aims to provide a method and module for customizing sensor functionality based on Wasm technology. These methods and modules do not suffer from the shortcomings of the prior art and offer additional advantages. This allows developers to focus solely on model post-processing logic without having to worry about the specific implementation and deployment of the model.

[0013] One aspect of the present invention relates to a method for customizing sensor functions based on Wasm technology, which includes: providing a mathematical model trained for a sensor in a smart device and a post-processing program customized for the mathematical model, wherein the post-processing program is used to perform post-processing on output data of the mathematical model; deploying the mathematical model in the smart device; compiling the post-processing program into a Wasm program and deploying it in the smart device; and providing an interface defined based on the WASI specification in the smart device, through which the Wasm program can access and operate the sensor.

[0014] Another aspect of the present invention relates to a smart device comprising a sensor, wherein the functionality of the sensor is customized according to the method of the present invention.

[0015] Another aspect of the present invention relates to a debugging tool for debugging a mathematical model, which includes a post-processing module and a compiler stack, wherein the post-processing module generates a post-processing template file for post-processing the mathematical model, provides the post-processing template file to the compiler stack, and receives output data of the mathematical model from a runtime running the mathematical model and provides it to a developer, wherein the compiler stack compiles the received post-processing template file into an executable file that can be executed by the runtime, and provides the executable file to the runtime, so that the runtime can use the mathematical model to generate the output data.

[0016] According to the present invention, Wasm technology is used to enable cross-platform and modular deployment of the post-processing function, thereby enabling customization of specific functions of the sensor in smart devices.

[0017] In addition, according to the present invention, developers can easily debug post-processing, and have the following advantages: no need to be forced to learn low-level languages ​​or hire special hardware engineers for debugging; guaranteed execution efficiency; easy to transplant. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 illustrates the creation and training of the mathematical model and the development of the post-processing procedure.

[0019] FIG2 shows the development process of a mathematical model according to the prior art.

[0020] FIG3 shows a debugging tool for debugging a mathematical model according to an embodiment of the present invention.

[0021] FIG4 shows a flow chart of a process of debugging a mathematical model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments given are not intended to limit the present invention. Unless there is a conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0023] In various smart devices of the Internet of Things, there are various sensors such as image sensors, sound sensors, light sensors, etc. Depending on the smart device, these sensors can be used to achieve different purposes.

[0024] To use sensors for specific purposes, developers can create and train corresponding mathematical models, for example, in a host computer. Developers can use model framework tools to create and train mathematical models. For example, development platforms such as TensorFlow, PyTorch, and OnNX can serve as model framework tools. Sensors can use mathematical models to perform calculations or inferences on sensor detection results to obtain calculation or inference results. The sensors then output these calculation or inference results in a specific format as output data.

[0025] For output data, developers also need to develop a post-processing program, for example, in a host computer. Developers can develop post-processing programs in their preferred programming language, such as Python, C / C++, Java, Go, or Rust. Using this post-processing program, the smart device can perform post-processing on the sensor's output data (e.g., format conversion) to provide a post-processed result containing the desired information.

[0026] As an example, Figure 1 illustrates the use of an IoT device containing a camera as a smart device. Here, a camera is a sensor within the meaning of this invention. Note that the smart device of this invention is not limited to IoT devices. Based on the technical implications of this invention, those skilled in the art can also apply this invention to fields such as autonomous driving, equipment testing, environmental testing, logistics testing, security testing, health testing, smart home appliances, the retail industry, healthcare, agriculture, and smart cities. These examples are not listed here.

[0027] The left side of Figure 1 shows the creation and training of a camera’s mathematical model and the development of a post-processing program. Here, the camera’s mathematical model can be an object detection model, such as a pedestrian detection model, a face recognition model, etc.

[0028] In existing technologies, the completed mathematical model and post-processing routines are typically deployed directly on the hardware of smart devices. For example, in the example shown in Figure 1, they are deployed in the camera's digital signal processor (DSP) and the processor of the IoT device, respectively. When deploying the mathematical model, factors such as the camera's power consumption, performance, and resources are taken into consideration. For example, the mathematical model can be lightweighted through methods such as depthwise separable convolution, grouped convolution, adjustable hyperparameters to reduce spatial resolution, knowledge distillation, low-rank pruning, and model compression.

[0029] The camera uses a mathematical model to calculate or infer the camera's detection results and outputs the calculation or inference results as output data. The IoT device containing the camera can then use a post-processing program to post-process the camera's output data to obtain the desired information. The mathematical model can be programmed in Python, C / C++, Java, Rust, Go, and other languages.

[0030] However, unlike existing technologies, the present invention proposes a technical solution for customizing sensor functions based on Wasm technology. Wasm, or WebAssembly, is a new bytecode format that is gaining widespread recognition due to its portability, compact size, and high security. Wasm allows users to write programs in a familiar language (currently supporting Python, C / C++, Rust, Java, or Go), and then run them on the web using a virtual machine engine. Wasm technology supports running Wasm programs in a sandbox environment.

[0031] As shown in Figure 1, in the present invention, the post-processing program is compiled into a Wasm program and deployed in an IoT device (e.g., a processor) to run in a virtual machine provided by the IoT device. Depending on the programming language of the post-processing program and the embedded environment of the Wasm program, various compilers can be used, such as the LLVM backend.

[0032] To enable Wasm programs to implement functionality consistent with post-processing programs, the present invention uses the WASI (WebAssembly System Interface) specification to define a unified sensor system interface for Wasm programs, referred to herein as WASI sensors. WASI is a new API system designed for WASM programs, a set of engine-independent, non-web system-oriented API standards. It defines how WASM programs interact with the host environment. For example, it includes standard system calls for the file system, network stack, time, and random number generator. These system calls are provided to Wasm programs through predefined API interfaces, allowing them to access sensor data resources in a sandboxed environment. With WASI support, Wasm programs can also run in non-web environments. As long as it supports the WASI standard, any runtime running Wasm programs can implement system calls in the host environment of IoT devices. The runtime library provides the program's execution environment, such as memory allocation, variable assignment, function calls, etc. It can also be used to manage threads or processes, execute interpreted code, dynamic link libraries, and shared libraries.

[0033] Through this sensor system interface, the Wasm program can correctly parse the output data generated by the sensor's mathematical model, and can access and operate the sensor (for example, turning it on, off, performing reasoning, retrieving results, etc.). As a result, the Wasm program can access and control the various functions provided by the sensor through the Wasi interface across the sandbox boundary, such as obtaining reasoning data, controlling the life cycle of the sensor, etc., which not only ensures the security and reliability of the program, but also protects the security of various types of information and equipment. Specifically, when a Wasm program needs to access sensor resources, it can pass the access request to the sensor through this interface. The sensor responds to this access request and returns the reasoning data to the Wasm program.

[0034] In the present invention, a smart device may include multiple sensors, and each sensor may be configured with at least one mathematical model to achieve a specific purpose. For example, in the example of Figure 1 , a camera may be provided with multiple mathematical models, including but not limited to a pedestrian detection model, a face recognition model, a text recognition model, and a license plate recognition model.

[0035] A post-processing program may be provided for each mathematical model to perform post-processing on the output data of the mathematical model. Of course, a post-processing program may also be provided for multiple mathematical models to perform post-processing on the output data of these mathematical models in combination.

[0036] Because the present invention customizes sensor functionality based on Wasm technology, developers can develop post-processing programs in any preferred language, regardless of the platform on which they run or how to adapt the program for different devices. Furthermore, Wasm technology restricts Wasm programs to a sandbox environment, providing high security.

[0037] Furthermore, this invention provides a sensor system interface based on the WASI specification. It runs in a sandbox environment to control the entire lifecycle, making it more secure. This invention also implements modular sensor control functionality, allowing sensor functionality to be expanded or modified by simply updating a single Wasm program, rather than the entire platform.

[0038] In summary, this paper proposes a cross-platform sensor control interface that allows developers to locally integrate mathematical model development and implement specific post-processing functions. Leveraging Wasm technology, this post-processing functionality is cross-platform and modularly deployable, enabling customized sensor functionality within smart devices.

[0039] Hereinafter, the present invention will explain in detail the deployment and debugging of Wasm programs.

[0040] FIG2 shows a process of developing a mathematical model using, for example, the Python programming language according to the prior art.

[0041] As shown in Figure 2, first, a mathematical model is created and trained. Model developers can use model framework tools to train the mathematical model. For example, development platforms such as TensorFlow, PyTorch, and Onnx can be used as model framework tools within a Python programming environment. Programming tools are not limited to Python; C / C++, Java, Rust, and Go are also possible.

[0042] Specifically, the model framework tool is first used to generate an initial model, and then the generated initial model is trained through machine learning. After the initial model training is completed, the trained initial model is exported.

[0043] Next, depending on the model type and operating environment, developers can use corresponding model deployment tools to deploy the completed mathematical model in a specific environment, such as an edge device such as a personal computer or smartphone. Here, the edge device can be the camera in Figure 1.

[0044] Finally, the deployed mathematical model can be run in the edge device to obtain calculation results or inference results.

[0045] However, currently, most developers are only familiar with programming languages ​​based on software framework requirements, such as Python, but are unfamiliar with hardware-level programming languages. This makes it difficult for algorithm developers to directly debug Wasm programs, severely limiting the richness of mathematical models.

[0046] Furthermore, even if mathematical models can be deployed using a hardware platform-specific software stack, they are usually implemented through a series of complex manual operations and are difficult to port to other devices.

[0047] Furthermore, developers primarily focus on model design creation without ensuring the efficiency requirements of industrial software.

[0048] To address the above issues, this paper proposes a debugging tool that can be built into a Docker image, providing developers with out-of-the-box functionality. As is well known, Docker images can be used to create development platforms for various programming languages, such as Python, Java, and Go. This article uses the Python programming language as an example.

[0049] In addition, for the operation of mathematical models, a runtime (e.g., WAMR Runtime) in edge devices (e.g., the smart devices mentioned above) is also necessary to apply the services to a wide range of hardware platforms and smoothly port the entire solution when necessary.

[0050] Specifically, FIG3 shows a debugging tool according to an embodiment of the present invention, which is used to debug a mathematical model to obtain higher task indicators, faster running speed, and smaller model volume.

[0051] According to the present invention, developers can, for example, create an entire debugging tool within a Docker image on their own host computer to provide a Docker container service from the Docker image. This allows developers to readily provide this service on nearly any PC or server operating system platform, such as on x86 or ARM hardware architectures. Docker containers are an open source application container engine that allows developers to package their applications and dependent packages into portable containers in a unified manner. Specifically, in order to create a debugging tool in a Docker image, for example, first write a Dockerfile file, which is used to define the environment and application in the Docker image; then, add the required debugging tools to the Dockerfile file, for example, you can choose to add tools such as gdb, strace, tcpdump as needed. In addition, depending on the type of debugging tool used and the requirements of the application, you may also need to set corresponding environment variables or expose the container's port in the Dockerfile file; then, build the Docker image, for example, execute the "docker build -t <image name>" command in the directory containing the Dockerfile file, which will start building the Docker image; next, start the Docker container, for example, run the following command to start the Docker container: "docker run -it –name <container name><image name>"; finally, test the debugging tool, run the required application in the started container, and then use the added debugging tool for debugging.

[0052] The debugging tool according to the present invention may include a post-processing module, a compiler stack, and an optional optimizer. The post-processing module may include a post-processing code module, a decoder, and a raw data viewer. The compiler stack may include a C++ converter, a WASM compiler, and a publisher. The optimizer may include a standard analyzer, a comparator, and a scheduler.

[0053] According to the present invention, in order to debug the mathematical model, a runtime (eg, WAMR runtime) may be utilized in the edge device to run the mathematical model.

[0054] According to the present invention, the Docker runtime environment of the debugging tool can be connected to the runtime of the edge device via a connector interface. The connector interface can be used to handle communication protocols, data format conversion, security authentication, and other issues to achieve communication and interaction. If necessary, data encryption and identity authentication are also considered to ensure communication security and data integrity. Here, the connection can be automatically established using various communication protocols, such as wired or wireless communication protocols.

[0055] In Figure 3, as shown by ①, developers can use the post-processing code module to fill in the template Python file with Python post-processing code for post-processing. The Python post-processing code can include instructions for post-processing the output data of the mathematical model. The post-processing code can be customized by the developer based on the desired post-processing functionality.

[0056] As shown in Mark ②, the C++ converter in the compiler stack converts the Python post-processing code from the post-processing code module into C++ code. Next, the WASM compiler in the compiler stack converts the C++ code into a Wasm executable file. Finally, as shown in Mark ③ and ④, the compiler stack uses the publisher to publish the converted Wasm executable file to the runtime, which runs the mathematical model, via the connector interface.

[0057] As shown in ⑤ and ⑥, the mathematical model outputs the calculation or reasoning results (i.e., output data) and outputs the calculation or reasoning results to the decoder in the post-processing module via the connector interface. The decoder decodes the calculation or reasoning results. As shown in ⑦, the raw data viewer in the post-processing module displays the decoded calculation or reasoning data to the developer in the form of images, sounds, text, etc. If the calculation or reasoning results are incorrect or unsatisfactory, the developer can adjust the mathematical model and then repeat the process shown in ① to ⑦ until the correct or ideal calculation or reasoning results are obtained.

[0058] Optionally, after obtaining correct or ideal calculation or inference results, the developer can also optimize the mathematical model.

[0059] As shown in mark ⑧, developers can submit performance standards, trial time cost limits and current results to the optimizer. In the optimizer, the standard analyzer parses the performance standard configuration to extract parameters and time costs, and the comparator compares the standard request and the current result. If the result meets the request, then as shown in mark As shown, the current result and time cost are directly returned to the developer. Otherwise, as shown in mark ⑨, the scheduler is started to test various optimization hyperparameters and scheduling strategies, and the publisher is controlled to run the optimized test code on the edge device in the process shown in marks ③ to ⑤. The result of each test case is sent to the comparator in the manner shown in mark ⑩ until a satisfactory value is reached or the best result within the set time is found. It is then returned to the developer along with the time cost.

[0060] FIG4 shows a flow chart of a process of debugging a mathematical model according to an embodiment of the present invention.

[0061] Determine whether to start debugging. If debugging is started, fill the post-processing code into the template file through the post-processing code module.

[0062] Next, the template file is converted into an execution file through the compiler stack.

[0063] Next, it checks whether the connector interface is ready for connection. If not, the device is restarted to connect until a successful connection is made. If a successful connection is made, the executable file is published to the runtime via the connector interface through the compiler stack.

[0064] Next, the mathematical model outputs the calculation or reasoning results (i.e., output data) to the post-processing module via the connector interface. The post-processing module then decodes the calculation or reasoning results and displays the decoded calculation or reasoning results to the developer.

[0065] If the inference result is incorrect or unsatisfactory, the developer can adjust the mathematical model and repeat the above process until the correct or ideal calculation or inference result is obtained.

[0066] After obtaining correct or ideal calculation or reasoning results, determine whether to further optimize the mathematical model.

[0067] If optimization is not required, the debugging process ends directly.

[0068] If optimization is required, the performance benchmark configuration, experimental time and cost constraints, and current results are submitted to the optimizer. The optimizer extracts the parameters and time constraints and compares the benchmark request with the initial results. If the results meet the requirements, the current results and time costs are directly returned to the developer. Otherwise, the scheduler is activated to test various optimization hyperparameters and scheduling strategies. The publisher is controlled to run the optimized test code on the edge device and send the results of each test case to the comparator until a satisfactory value is reached or the optimal result within the set time is found. This result, along with the time cost, is then returned to the developer. At this point, the debugging process ends.

[0069] According to the present invention, the following configuration can be adopted.

[0070] (1) A method for customizing sensor functions based on Wasm technology, comprising:

[0071] Providing a mathematical model trained for sensors in smart devices and a post-processing program customized for the mathematical model, wherein the post-processing program is used to perform post-processing on output data of the mathematical model;

[0072] deploying the mathematical model in the smart device;

[0073] Compiling the post-processing program into a Wasm program and deploying it in the smart device;

[0074] An interface defined based on the WASI specification is provided in the smart device, and the Wasm program can access and operate the sensor through the interface.

[0075] (2) The method according to (1) above, wherein the Wasm program and the post-processing program have the same post-processing function.

[0076] (3) The method according to (1) above, wherein the mathematical model is deployed in the sensor.

[0077] (4) The method according to (1) above, wherein the Wasm program runs in a virtual machine provided by the smart device.

[0078] (5) The method according to (1) above, wherein the post-processing is a format conversion of the output data.

[0079] (6) According to the method described in (1) above, the post-processing program and the mathematical model are developed using any one of the programming languages ​​​​Python, C / C++, Java, Rust and Go.

[0080] (7) The method according to (1) above, wherein the method further comprises the step of debugging the mathematical model in the step of deploying the mathematical model in the smart device, wherein the debugging comprises:

[0081] executing the mathematical model in a suitable runtime;

[0082] Creating a debugging tool in a Docker image and running the debugging tool in a Docker runtime environment;

[0083] The debugging tool fills the post-processing code for the post-processing into the template file;

[0084] The debugging tool compiles the template file into an execution file that can be executed by the runtime;

[0085] The runtime receives the execution file from the Docker runtime environment and executes the execution file to generate the output data using the mathematical model;

[0086] The debugging tool receives the output data from the runtime, decodes the output data, and provides the decoded output data to the developer;

[0087] In the case that the output data is incorrect, the developer adjusts the mathematical model;

[0088] Repeat the above steps until the correct output data is obtained.

[0089] (8) The method according to (7) above, wherein, when compiling the execution file,

[0090] The debugging tool converts the template file into C++ code, converts the C++ code into the execution file, and publishes the execution file to the runtime.

[0091] (9) According to the method described in (7) above, after obtaining the correct output data, the debugging tool analyzes the standard request set by the developer, compares the standard request and the output data, and when the result of the comparison is unsatisfactory, tests multiple scheduling strategies and finds the best scheduling strategy.

[0092] (10) The method according to (7) above, wherein the runtime is provided by the smart device.

[0093] (11) The method according to (7) above, wherein the Docker runtime environment is provided by the developer's host.

[0094] (12) The method according to (7) above, wherein the Docker image creates a development environment for any one of the programming languages ​​of Python, C / C++, Java, Rust, and Go, and the mathematical model, the debugging tool, and the post-processing program are developed or created using the same programming language.

[0095] (13) A smart device including a sensor,

[0096] The function of the sensor is customized according to any one of the methods described in (1) to (12) above.

[0097] (14) A debugging tool for debugging a mathematical model, comprising a post-processing module and a compiler stack,

[0098] The post-processing module generates a post-processing template file for post-processing the mathematical model, provides the post-processing template file to the compiler stack, and receives output data of the mathematical model from a runtime running the mathematical model and provides it to a developer.

[0099] The compiler stack compiles the received post-processing template file into an execution file that can be executed by the runtime, and provides the execution file to the runtime, so that the runtime can generate the output data using the mathematical model.

[0100] (15) The debugging tool according to (14) above, wherein the debugging tool and the mathematical model are developed or created using the same programming language.

[0101] (16) The debugging tool according to (14) above, wherein the post-processing module includes:

[0102] A post-processing code module, which is used to generate the post-processing template file and provide it to the compiler stack;

[0103] a decoder for receiving the output data of the mathematical model from the runtime and decoding the output data; and

[0104] A raw result viewer is used to provide the decoded output data to the developer.

[0105] (17) The debugging tool according to (14) above, wherein the compiler stack includes:

[0106] A C++ converter, which is used to convert the post-processing template file into C++ code;

[0107] A WASM compiler for converting the C++ code into the executable file; and

[0108] A publisher is used to publish the execution file to the runtime.

[0109] (18) The debugging tool according to (14) above further includes an optimizer, wherein the optimizer includes:

[0110] A standard analyzer for analyzing standard requests set by the developer;

[0111] a comparator for comparing the standard request with the output data of the mathematical model; and

[0112] The scheduler is configured to test multiple scheduling strategies and find the best scheduling strategy when the comparison result is unsatisfactory.

[0113] (19) The debugging tool according to (14) above, wherein:

[0114] The debugging tool is created in a Docker image.

[0115] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0116] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for customizing sensor functions based on Wasm technology, comprising: Providing a mathematical model trained for a sensor in an intelligent device and a post-processing program customized for the mathematical model, wherein the post-processing program is used to perform post-processing on output data of the mathematical model; Deploying the mathematical model in the smart device; Compiling the post-processing program into a Wasm program and deploying it in the smart device; An interface defined based on the WASI specification is provided in the smart device, and the Wasm program can access and operate the sensor through the interface.

2. The method according to claim 1, wherein: The Wasm program and the post-processing program have the same post-processing function.

3. The method according to claim 1, wherein: The mathematical model is deployed in the sensor.

4. The method according to claim 1, wherein: The Wasm program runs in a virtual machine provided by the smart device.

5. The method according to claim 1, wherein: The post-processing is the format conversion of the output data.

6. The method according to claim 1, wherein the post-processing program and the mathematical model are developed by any one of the programming languages ​​of Python, C / C++, Java, Rust and Go.

7. The method according to claim 1, wherein: The method further comprises the step of debugging the mathematical model in the step of deploying the mathematical model in the smart device, wherein the debugging comprises: executing the mathematical model in a suitable runtime; Creating a debugging tool in a Docker image and running the debugging tool in a Docker runtime environment; The debugging tool fills the post-processing code for the post-processing into the template file; The debugging tool compiles the template file into an execution file that can be executed by the runtime; The runtime receives the execution file from the Docker runtime environment and executes the execution file to generate the output data using the mathematical model; The debugging tool receives the output data from the runtime, decodes the output data, and provides the decoded output data to the developer; In case the output data is incorrect, the developer adjusts the mathematical model; Repeat the above steps until the correct output data is obtained.

8. The method according to claim 7, wherein: When compiling the executable file, The debugging tool converts the template file into C++ code, converts the C++ code into the execution file, and publishes the execution file to the runtime.

9. The method according to claim 7, wherein: After obtaining the correct output data, the debugging tool analyzes the standard request set by the developer, compares the standard request and the output data, and when the result of the comparison is not satisfactory, tests multiple scheduling strategies and finds the best scheduling strategy therefrom.

10. The method according to claim 7, wherein: The runtime is provided by the smart device.

11. The method according to claim 7, wherein: The Docker operating environment is provided by the developer's host.

12. The method according to claim 7, wherein: The Docker image creates a development environment for any one programming language among Python, C / C++, Java, Rust and Go, and the mathematical model, the debugging tool and the post-processing program are developed or created using the same programming language.

13. A smart device comprising a sensor, in, The functionality of the sensor is customized using the method according to any one of claims 1 to 12.

14. A debugging tool for debugging a mathematical model, comprising a post-processing module and a compiler stack, in, The post-processing module generates a post-processing template file for post-processing of the mathematical model, provides the post-processing template file to the compiler stack, and receives output data of the mathematical model from a runtime running the mathematical model to provide to a developer, The compiler stack compiles the received post-processing template file into an execution file executable by the runtime, and provides the execution file to the runtime, so that the runtime can generate the output data using the mathematical model.

15. The debugging tool according to claim 14, wherein: The debugging tool and the mathematical model are developed or created using the same programming language.

16. The debugging tool according to claim 14, wherein: The post-processing module comprises: A post-processing code module, which is used to generate the post-processing template file and provide it to the compiler stack; a decoder for receiving the output data of the mathematical model from the runtime and decoding the output data; and A raw result viewer is used to provide the decoded output data to the developer.

17. The debugging tool according to claim 14, wherein: The compiler stack includes: A C++ converter, which is used to convert the post-processing template file into C++ code; A WASM compiler, which is used to convert the C++ code into the execution file; and A publisher is used to publish the execution file to the runtime.

18. The debugging tool according to claim 14, further comprising an optimizer, wherein the optimizer comprises: A standard analyzer for analyzing standard requests set by the developer; a comparator for comparing the standard request with the output data of the mathematical model; as well as The scheduler is used for testing a plurality of scheduling strategies and finding the best scheduling strategy therefrom when the result of the comparison is not satisfactory.

19. The debugging tool according to claim 14, wherein: The debugging tool is created in a Docker image.

Images