AI service deployment system of embedded platform, AI service loading method, storage medium, electronic device and embedded device

Abstract

The application discloses an AI service deployment system of an embedded platform, an AI service loading method, a storage medium, an electronic device and an embedded device. The system comprises a cloud deployment device and an AI service loading device. The AI service loading device comprises an interface unit and a scheduling component. The cloud deployment device provides the AI service loading device with adaptive service components based on information of the embedded device and information of a target AI service to be deployed provided by the AI service loading device. The interface unit realizes the interaction between the AI service loading device and the user and the cloud deployment device. The scheduling component obtains the service components from the cloud deployment device based on a deployment instruction of the target AI service issued by the user, and loads and triggers the running of the service components. The application can effectively improve the deployment efficiency of the AI service.

Description

Technical Field

[0001] This application relates to the fields of embedded systems and artificial intelligence technology, and in particular to AI service deployment systems for embedded platforms, AI service loading methods, storage media, electronic devices, and embedded devices. Background Technology

[0002] In recent years, artificial intelligence (AI) technology, especially deep learning, has made groundbreaking progress in fields such as computer vision, speech recognition, and natural language processing. This has led to a demand for widely applying trained AI models to various devices and environments, with deployment scenarios gradually expanding from resource-rich data centers and cloud servers to mobile terminals and resource-constrained edge and embedded devices.

[0003] Given this trend, the efficient and flexible deployment and management of AI models has become a key issue. Current mainstream deployment methods typically involve tightly coupling and encapsulating the complete algorithmic logic required to implement a specific function or service with the corresponding AI model, deploying it as a separate, monolithic software unit. During inference, this unit performs computations by calling acceleration libraries for specific hardware (such as GPUs and NPUs). This "monolithic" deployment architecture is still applicable in environments such as servers.

[0004] However, with the deepening of intelligentization, the need to deploy multiple AI functions (such as simultaneous facial recognition, behavior analysis, and anomaly detection) on a single embedded device (such as a smart camera, drone, or robot) is becoming increasingly urgent. Embedded platforms inherently have significant constraints, including limited computing power, small memory, limited storage space, and strict power consumption budgets. If the aforementioned "monolithic" deployment method continues, each independent functional unit will carry and exclusively occupy all the models and operating environments it requires, resulting in severe fragmentation and redundant use of device resources. When attempting to deploy multiple AI models or services, the device is highly likely to be unable to handle the task due to insufficient memory, computing power, or storage resources, greatly limiting the functional flexibility and scalability of embedded devices. Summary of the Invention

[0005] This application provides an AI service deployment system, AI service loading method, apparatus, storage medium, electronic device, and embedded device for embedded platforms, which can effectively improve the deployment efficiency of AI services.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] An AI service deployment system for an embedded platform, characterized in that it includes: a cloud deployment device and an embedded device, wherein the embedded device includes an interface unit and a scheduling component;

[0008] A cloud deployment device is used to provide the embedded device with business components adapted to the embedded device and the target AI service based on the information of the embedded device and the information of the target AI service to be deployed.

[0009] The interface unit is used to enable the interaction between the embedded device and the user and the cloud deployment device;

[0010] The scheduling component is used to provide the embedded device information and the target AI service information to the cloud deployment device through the interface unit based on the deployment instruction of the target AI service issued by the user received through the interface unit; it is also used to obtain the business component from the cloud deployment device through the interface unit, and load and trigger the operation of the business component.

[0011] The business component is used to call one or more atomic components according to the business logic of the target AI service to execute the target AI service on the embedded device; the atomic component is used to call a hardware acceleration component bound to the hardware of the embedded device based on the model data of the neural network model to execute the inference of the neural network model; the atomic component can be called by different business components.

[0012] Preferably, the cloud deployment device includes an e-commerce interaction unit and a service warehouse unit;

[0013] The e-commerce interaction unit is used to display AI services adapted to various embedded devices, and also to interact with the interface unit. Based on the information of the embedded device and the target AI service provided by the interface unit, it determines the business components adapted to the embedded device and the target AI service, as well as the first atomic component and the first hardware acceleration component related to the business components and not saved by the embedded device. It is also used to obtain the determined business components, the first atomic component, and the first hardware acceleration component from the service repository unit, and send them to the scheduling component through the interface unit.

[0014] The service repository unit is used to store various business components, atomic components, and hardware acceleration components for different embedded platforms.

[0015] Preferably, the cloud deployment device is further configured to receive sensor data of the first neural network model corresponding to the second atomic component through the interface unit; and to perform model optimization training on the first neural network model using the sensor data, and update the second atomic component based on the training results.

[0016] Preferably, the atomic component includes a model file that carries the model data of the neural network model;

[0017] In the cloud deployment device, the model file in the second atomic component is updated based on the training results to achieve the update of the second atomic component.

[0018] Preferably, the scheduling component is further used to statistically analyze the usage of various business components and atomic components by the embedded device, and report it to the e-commerce interaction unit through the interface unit as the basis for displaying various AI services.

[0019] Preferably, the embedded device further includes a custom service editing unit for providing a service editing interface to the user, allowing the user to create trigger conditions and response operations based on the output of a specified AI service and the device functions of the embedded device through predefined logical symbols;

[0020] The scheduling component is used to verify the triggering conditions and response operations, and after passing the verification, to start the business component corresponding to the specified AI service, and to execute the response operation when it is determined that the triggering conditions are met.

[0021] Preferably, the scheduling component is further configured to check whether the atomic component called by the business component has been loaded when loading the business component; if so, reuse the corresponding atomic component and its output result; otherwise, load the corresponding atomic component.

[0022] And / or,

[0023] The scheduling component is further configured to check, according to the needs of the business component, whether a data channel and buffer have been created for the required sensor; if so, the data channel and buffer of the sensor are reused; otherwise, a data channel and buffer are created for the sensor.

[0024] And / or,

[0025] The scheduling component is used to further obtain atomic components and hardware acceleration components related to the business component that are not locally stored when obtaining the business component from the cloud deployment device.

[0026] And / or,

[0027] The scheduling component is used to delete the business component that has been closed the longest when acquiring business components, atomic components and / or hardware acceleration components if local resources are insufficient.

[0028] And / or,

[0029] The scheduling component is used to determine the business component with the least impact that needs to be closed when loading the first business component if local resources are insufficient, close the determined business component, and then load the first business component.

[0030] And / or,

[0031] The scheduling component is used to further delete atomic components called by a business component that are not currently being called by other loaded or pending business components when a business component is deleted.

[0032] Preferably, the business component includes a business logic code library and a business component configuration file. The business component configuration file records the atomic components and sensor resources that need to be called, and the business logic code library is used to store the business logic for implementing the target AI service.

[0033] Preferably, the hardware acceleration component is used to encapsulate the underlying hardware call interface provided by the embedded device into a standard call interface, and execute the inference of the neural network model by calling the underlying hardware.

[0034] Preferably, the atomic component includes a model file, an atomic component configuration file, and an algorithm library;

[0035] The model file is used to record the model data;

[0036] The atomic component configuration file is used to record the input data format, data length, and model reporting parameters of the corresponding neural network model;

[0037] The algorithm library is used to preprocess the data input to the atomic components to adapt to the neural network model; it is also used to call the hardware acceleration component through the standard calling interface to trigger the inference execution of the neural network model; and it is also used to postprocess the inference results returned by the hardware acceleration component to generate standardized atomic component output results.

[0038] Preferably, the business component, the atomic component, and the hardware acceleration component each provide their own component information acquisition interface, initialization interface, inference interface, and resource release interface;

[0039] The scheduling component is used to obtain the device resource information required by the business component and its dependent lower-level components by calling the component information acquisition interface of the business component; it is also used to allocate device resources to the business component based on the device resource information of each level of components.

[0040] Specifically, the business component obtains the device resource information required by the atomic component and the hardware acceleration component it calls by calling the component information acquisition interface of the atomic component it depends on; the atomic component obtains the device resource information required by the acceleration component by calling the component information acquisition interface of the hardware acceleration component.

[0041] The scheduling component is further configured to complete the initialization of the service component by calling the initialization interface of the service component and passing the allocated device resources; the service component is further configured to complete the initialization of the atomic component by calling the initialization interface of the atomic component it depends on during its initialization process; the atomic component is further configured to complete the initialization of the hardware acceleration component by calling the initialization interface of the hardware acceleration component during its initialization process.

[0042] The scheduling component is further configured to trigger the execution of the target service by calling the inference interface of the service component; the service component is further configured to trigger the corresponding atomic component to execute the inference of the corresponding neural network model by calling the inference interface of the atomic component it depends on; the atomic component is further configured to trigger the underlying hardware associated with the hardware acceleration component to execute the inference by calling the inference interface of the hardware acceleration component.

[0043] The scheduling component is further configured to release the device resources occupied by the business component by calling the resource release interface of the business component; the business component is further configured to release the device resources occupied by the atomic component by calling the resource release interface of the atomic component it depends on during its resource release process; the atomic component is further configured to release the device resources occupied by the acceleration component by calling the resource release interface of the acceleration component during its resource release process.

[0044] Preferably, the embedded device includes an AI service loading device, and the scheduling component is located in the AI ​​service loading device.

[0045] An AI service loading method for an embedded platform, applied in an embedded device, includes:

[0046] In response to a user's deployment command for a target AI service, the system provides information about the embedded device and the target AI service to the cloud, and retrieves business components adapted to the embedded device and the target AI service from the cloud.

[0047] Load and trigger the execution of the business component so that the business component calls one or more atomic components to execute the target AI service according to the business logic of the target AI service;

[0048] The step of calling one or more atomic components to execute the target AI service includes: calling the atomic component through an interface provided by the atomic component, wherein the atomic component, based on the model data of the neural network model, calls a hardware acceleration component bound to the embedded device hardware to execute the inference of the neural network model.

[0049] Preferably, when acquiring the business component, the method further includes: acquiring atomic components and hardware acceleration components that the business component depends on and that are not saved by the embedded device from the cloud.

[0050] Preferably, the method further includes:

[0051] In response to a user's model optimization instruction for the first neural network model corresponding to the first atomic component, the sensor data corresponding to the first neural network model and the model optimization instruction are uploaded to the cloud, so that the cloud can use the sensor data to perform model optimization training on the first neural network model and update the first atomic component.

[0052] Preferably, the atomic component includes a model file for carrying model data of the neural network model;

[0053] Updating the first atomic component includes updating the model data in the model file based on the training results of the model optimization training.

[0054] Preferably, the method further includes: statistically analyzing the usage of the embedded device for each business component and atomic component, and reporting it to the cloud, so that the cloud updates the display order of various AI services based on the usage.

[0055] Preferably, the method further includes:

[0056] Based on user input, determine the trigger conditions and response operations of the custom AI service created by the user by combining the output of the specified AI service with the device functions of the embedded device using predefined logical symbols;

[0057] Verify the triggering condition and the response operation, and start the business component corresponding to the specified AI service after passing the verification. When it is determined that the triggering condition is met, execute the response operation.

[0058] Preferably, the method further includes:

[0059] When loading the business component, check whether the atomic component called by the business component has been loaded. If so, reuse the corresponding atomic component and its output result; otherwise, load the corresponding atomic component.

[0060] And / or,

[0061] When loading the business component, check whether a data channel and buffer have been created for the required sensor according to the needs of the business component. If so, reuse the data channel and buffer of the sensor; otherwise, create a data channel and buffer for the sensor.

[0062] And / or,

[0063] When acquiring business components, if local resources are insufficient, delete the business component that has been closed for the longest time;

[0064] And / or,

[0065] When loading the first business component, if local resources are insufficient, determine the business component with the least impact that needs to be closed before loading the first business component, close the determined business component, and then load the first business component.

[0066] And / or,

[0067] When deleting a business component, further delete the atomic components that are called by that business component and are not currently being called by other loaded or pending business components.

[0068] An embedded device, the means being located within the embedded device, includes an interface unit and a scheduling component;

[0069] The interface unit is used to enable the device to interact with the user and the cloud;

[0070] The scheduling component is used to provide the embedded device information and the target AI service information to the cloud through the interface unit based on the deployment instruction of the target AI service issued by the user received through the interface unit; it is also used to obtain the business component from the cloud deployment device through the interface unit, and load and trigger the operation of the business component.

[0071] The business component is used to call one or more atomic components according to the business logic of the target AI service to execute the target business on the embedded device; the atomic component is used to call a hardware acceleration component bound to the hardware of the embedded device based on the model data of the neural network model to execute the inference of the neural network model.

[0072] A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, can implement the AI ​​service loading method described above.

[0073] An electronic device, comprising at least a computer-readable storage medium and a processor;

[0074] The processor is configured to read the executable instructions from the computer-readable storage medium and execute the instructions to implement the AI ​​service loading method described above.

[0075] As can be seen from the above technical solution, the AI ​​service deployment system in this application includes a cloud deployment device and an embedded device. The embedded device includes an interface unit and a scheduling component. The scheduling component receives a deployment instruction for the target AI service from a user through the interface unit. In response to the instruction, it provides the cloud deployment device with information about the embedded device and the target AI service through the interface unit. Based on the information of the embedded device and the target AI service, the cloud deployment device sends an adapted business component to the scheduling component through the interface unit. The scheduling component loads and triggers the execution of the corresponding business component. The business component, according to the business logic of the target AI service, calls one or more atomic components to execute the target business on the embedded device. The atomic component is used to call hardware acceleration components bound to the embedded device hardware based on model data of the neural network model to execute the inference of the neural network model. The atomic component can be called by different business components. In the above deployment system, through component layering and dynamic scheduling, different AI services can reuse the same atomic components, thereby achieving the reuse of basic neural network models and resources, breaking through the resource bottleneck when deploying multiple AI services on embedded devices, and effectively improving the deployment efficiency of AI services. Attached Figure Description

[0076] Figure 1 This is a schematic diagram of the basic structure of the AI ​​service deployment system of the embedded platform in this application;

[0077] Figure 2 This is a schematic diagram of the specific structure of the AI ​​service deployment system in a specific embodiment of this application;

[0078] Figure 3 This is a schematic diagram of the software architecture of the AI ​​service loading device, which consists of four components, according to a specific embodiment of this application.

[0079] Figure 4 This is a schematic diagram illustrating the interface call order and service lifecycle of each component in the AI ​​service loading device according to a specific embodiment of this application;

[0080] Figure 5a This is a schematic diagram of the image data enhancement method given in a specific embodiment of this application;

[0081] Figure 5b This is a schematic diagram illustrating the methods for enhancing audio and text data as given in a specific embodiment of this application;

[0082] Figure 6This is a basic flowchart illustrating the AI ​​service loading method applied to embedded devices in this application;

[0083] Figure 7 This is a schematic diagram of the basic structure of the electronic device provided in this application. Detailed Implementation

[0084] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0085] This application provides an AI service deployment system and an embedded device to address the specific circumstances and needs of embedded devices. Figure 1 This is a schematic diagram of the basic structure of the AI ​​service deployment system for an embedded platform in this application. Figure 1 As shown, the system includes a cloud deployment device and an embedded device, wherein the embedded device includes an interface unit and a scheduling component.

[0086] A cloud-based deployment device is used to provide embedded devices with business components adapted to both the embedded devices and the target AI services to be deployed, based on information about the embedded devices and the target AI services to be deployed.

[0087] An interface unit is used to enable interaction between embedded devices and users and cloud-deployed devices;

[0088] The scheduling component is used to provide information about the embedded device and the target AI service to the cloud deployment device through the interface unit based on the deployment instructions for the target AI service received from the user through the interface unit; it is also used to obtain business components from the cloud deployment device through the interface unit, and load and trigger the operation of the business components.

[0089] Among them, the business component is used to call one or more atomic components to execute the target AI service on the embedded device according to the business logic of the target AI service; the atomic component is used to call the hardware acceleration component bound to the embedded device hardware based on the model data of the neural network model to execute the inference of the neural network model; the atomic component can be called by different business components.

[0090] In a specific implementation, the aforementioned embedded device may directly include the aforementioned interface unit and scheduling component; alternatively, the embedded device may also include an AI service loading device, in which the scheduling component may be located, and the interface unit may be located inside or outside the AI ​​service loading device; AI service loading is achieved through the AI ​​service loading device.

[0091] In the aforementioned AI service deployment system and embedded devices, the scheduling component serves as a resident management framework on the device side, uniformly managing the acquisition, loading, and execution triggering of business components. Simultaneously, the business components used to implement the target AI service do not require pre-installation; they are loaded instantly via user commands, overcoming the static deployment limitations of traditional firmware burning and effectively adapting to resource-constrained scenarios in embedded devices. In the implementation of the target AI service, business components and atomic components are layered, decoupling business logic from model inference capabilities. Business components only declare dependencies, while atomic components exist independently, enabling cross-business reuse of neural network models. Furthermore, multiple business components can call the same atomic component instance, effectively reducing memory and computing power consumption, particularly suitable for resource- and computing-intensive embedded devices. The separation of atomic components from hardware acceleration components decouples model inference from hardware calls. When migrating AI services to a new embedded platform, only the hardware acceleration component needs replacement; atomic and business components remain unchanged, effectively reducing development costs. The interface unit and scheduling component collaborate to report device information to the cloud, which then adapts and distributes resources on demand, achieving precise resource adaptation. As can be seen from the above, the AI ​​service deployment system and embedded device provided in this application can effectively improve the deployment efficiency of AI services and are suitable for the deployment needs of embedded devices.

[0092] The AI ​​deployment system and embedded device provided in this application will be described in detail below through specific embodiments. Figure 2 This is a schematic diagram illustrating the specific structure of the AI ​​service deployment system in a specific embodiment of this application. In this embodiment, it is assumed that the embedded device includes an AI service loading unit to implement AI service loading, and that the underlying hardware for implementing model inference computation is an NPU. The specific implementation of the AI ​​service deployment system and the embedded device in this embodiment will be described below from several aspects.

[0093] I. System Architecture

[0094] like Figure 2 As shown, the deployment system is mainly divided into two parts: cloud deployment device and embedded device, the embedded device including AI service loading device.

[0095] The cloud deployment device is deployed on a cloud server and mainly includes:

[0096] Service Repository Unit: Used to store all developed business components, atomic components, and hardware acceleration components for different hardware platforms and versions.

[0097] The e-commerce interaction unit is used to display AI services adapted to various embedded devices. It also interacts with the interface unit to determine the business components adapted to the embedded devices and target AI services based on the embedded device information and target AI service information provided by the interface unit, as well as the atomic components (referred to as the first atomic components) and hardware acceleration components (referred to as the first hardware acceleration components) related to the business components but not saved by the embedded devices. It is also used to obtain the determined business components, the first atomic components, and the first hardware acceleration components from the service repository unit and distribute them to the scheduling component through the interface unit. More specifically, it can serve as an entry point for users (e.g., through a mobile APP or Web terminal) to display various AI services (such as "face recognition", "fall detection", "pet care", etc.) and can intelligently recommend adapted business components based on the user's registered device information (such as chip model and memory size).

[0098] Model training unit: Used to receive sensor data of the neural network model (referred to as the first neural network model) corresponding to a certain atomic component (referred to as the second atomic component) through the interface unit; also used to optimize and train the first neural network model using the corresponding sensor data, and update the second atomic component based on the training results; more specifically, when a user reports that a certain neural network model is not performing well, the user can be guided to upload a small amount of relevant sensor data (such as images and audio), and after the uploaded sensor data is enhanced, the original model is fine-tuned and trained to generate an optimized first neural network model, update the second atomic component based on the training results, and store the updated second atomic component in the service repository unit.

[0099] The AI ​​service loading device, deployed on embedded devices (such as smart cameras), is the core of the system's operation and mainly includes:

[0100] Interface Unit: Serving as the device's unified communication interface, it is responsible for bidirectional communication with the user's app and cloud-deployed devices. For example, it receives AI service deployment commands from the user, reports device information to the cloud, and downloads business components from the cloud.

[0101] Scheduling Component: This component runs residently as the management framework on the device side. It is the central hub for resource management and task scheduling, responsible for receiving instructions from interface units, retrieving business components from the cloud, managing the lifecycle of components (loading, initialization, running, unloading), and allocating and reclaiming device resources (CPU / memory / sensor channels).

[0102] The overall system architecture provided in this application allows for the download of adapted business components and atomic components from the cloud based on user needs, effectively enabling dynamic loading of AI services; and it can promptly detect model limitations and utilize user-provided data for customized model optimization.

[0103] II. Core Component Structure and Interaction

[0104] exist Figure 2 In the system shown, the AI ​​service loading device involves four core functional components when loading and running AI services: a scheduling component, a business component, an atomic component, and a hardware acceleration component. These components have a hierarchical calling relationship. The scheduling component is a resident component, while the business component, atomic component, and hardware acceleration component are dynamically loaded during the loading and running of the AI ​​service. The software architecture consisting of these four components is as follows: Figure 3 As shown in the image. Since the interface unit is mainly responsible for communicating with the outside world, it will not be described in detail here.

[0105] 1. Scheduling component

[0106] Function: The scheduling component is the core management unit of the AI ​​service loading device. As the operating framework of the resident embedded device system, it performs business lifecycle management, resource coordination and allocation, device information reporting and status synchronization, and provides custom AI service editing functions.

[0107] In more detail, the scheduling component allocates device resources to business components, enables and disables business components, interacts with the cloud, and manages various business components, atomic components, and hardware acceleration components. The scheduling component can also collect all AI service results from the embedded device, allowing users to use simple logic in the app to chain together the AI ​​service functions provided by the device to create custom AI services (described in a separate section later). With user consent, the scheduling component can collect and upload information such as usage data, fault conditions, user preferences, and embedded device resource status to the cloud, assisting product managers in understanding market conditions.

[0108] 2. Business Components

[0109] Functionality: This corresponds to a complete, user-perceptible AI service (such as "home security"), internally encapsulating the business logic for implementing that AI service. For example, the "home security" AI service might first call the "human detection" atomic component, and after detecting a human, call the "face recognition" atomic component to determine identity, and finally decide whether to issue an alarm based on the recognition result;

[0110] Composition: It includes a business logic code library and business component configuration files. The business component algorithm library contains the business logic required for the implementation of this business, and the business component configuration file records the atomic components (such as "human detection v2.1" and "face recognition v1.5") and sensor resources required by this business.

[0111] Typically, a single model is insufficient to implement complex business functions. It's necessary to design logic to connect multiple atomic components. For example, a robotic vacuum cleaner needs a SLAM model to create a map, while a vision-based garbage detection model identifies the garbage that needs cleaning. Therefore, the business component for cleaning a robotic vacuum cleaner would include atomic components of both the SLAM model and the garbage detection model. Some business functions only require a single atomic component, but still need some logical judgment to encapsulate the service. For instance, a door lock might include a package detection function, reporting to the owner when a package arrives and when it's taken away by the courier. This function only needs a package detection model, but whether to report immediately upon arrival or wait for a period of time is all reflected in the business component.

[0112] Different business components can also reuse the same atomic component, saving resources and computing power within the device.

[0113] 3. Atomic Components

[0114] Function: Strongly bound to a specific neural network model architecture (such as YOLOv5, MobileNet), it serves as the execution unit for model inference. Its core responsibilities are: performing model-specific preprocessing on input data (such as noise reduction and speech enhancement filtering commonly used in audio algorithms), calling hardware acceleration components to execute model inference, post-processing the raw data output by inference (such as non-maximum suppression), and finally outputting standardized results (such as bounding boxes, classification labels, and confidence scores) that can be understood by upper-layer business components.

[0115] Composition: Includes an algorithm library (implementing pre- and post-processing logic), model files (in .pt or .onnx format, storing model structure and parameters), and atomic component configuration files (recording model input / output dimensions, version, etc.). Model files are independently replaceable. When optimizing the model, only the model files need to be updated; the algorithm library and upper-layer business components remain unchanged. In other words, updating the atomic components is achieved through updating the model files.

[0116] 4. Hardware acceleration components

[0117] Function: It is bound to a specific hardware acceleration unit of the device (specifically, a certain NPU chip in this embodiment). Its core function is to encapsulate the various heterogeneous computing interfaces provided by the underlying chip manufacturer and provide a unified, standard model inference interface to the upper-layer atomic components. It receives model data and input data from the atomic components, converts them into the format required by the hardware driver, calls the NPU chip to complete the calculation, and returns the result;

[0118] Composition: It is usually a dynamic library, which is pre-installed with the device system firmware or provided by the chip manufacturer.

[0119] Because different products have different positioning, the chip platforms selected during product design vary widely. The usage processes and interfaces of chips provided by different manufacturers are also completely different. The hardware acceleration component encapsulates all the interfaces required for deep learning inference provided by the manufacturers, as specified in the AI ​​service deployment system application, to accelerate the inference of neural network models. The specific content of the interfaces is introduced in Part Three. In different products, as long as the same chip platform is used, the same hardware acceleration component can be reused.

[0120] III. Layer-by-layer initialization and interface call process

[0121] In this application, components communicate through standard interfaces via a mechanism of "layer-by-layer device information acquisition," "layer-by-layer initialization," "layer-by-layer invocation," and "layer-by-layer release." Specifically, business components, atomic components, and hardware acceleration components each include interfaces for acquiring component information, initialization, inference, and releasing resources for the upper layers. For example... Figure 4 As shown, taking the running of a certain AI service as an example, the order of interface calls and the service lifecycle are as follows:

[0122] Step 1: Resource Information Acquisition;

[0123] After receiving the AI ​​service activation command (i.e., deployment command), the scheduling component first calls the component information retrieval interface of the business component corresponding to the target AI service. The business component then calls the same interface of each atomic component declared in its business component configuration file, and the atomic component then calls the same interface of its corresponding hardware acceleration component. Resource requirement information (such as memory size and required sensor type) is thus aggregated layer by layer from bottom to top to the scheduling component.

[0124] Step 2: Resource allocation and initialization;

[0125] The scheduling component determines whether device resources are sufficient based on the aggregated information. If sufficient, it allocates resources (such as memory blocks and camera data channels) to the service, then calls the service component's initialization interface and passes in the allocated resources. Next, during the service component's initialization process, it calls the initialization interfaces of its declared atomic components. Finally, during the atomic component initialization process, it calls the hardware acceleration component's initialization interface, completing the initialization of the entire execution chain.

[0126] Step 3: Business reasoning execution;

[0127] Once sensor data is ready, the scheduling component calls the inference interface of the business component to run the business logic. The business component executes its business logic, calling the inference interfaces of its atomic components in sequence. After preprocessing, the atomic components call the inference interface of the acceleration component to complete hardware-accelerated inference, perform post-processing, and return the results to the business component. The business component integrates all results to generate the final business output (such as generating an alarm event). Step 3 is executed repeatedly until the AI ​​service is shut down.

[0128] Step 4: Resource release;

[0129] When the AI ​​service stops, the scheduling component calls the resource release interface of the business component. This call is also passed down layer by layer until all occupied computing resources and data channels are released.

[0130] IV. Resource Reuse and Management Mechanism

[0131] To maximize resource utilization, when a business component requests the initialization of an atomic component, the scheduling component checks if an instance of the same version of the atomic component, processing the same data source, is already running on the device. If it exists, the new business component is guided to reuse the existing atomic component instance and its output results, rather than creating a new instance. This avoids the same neural network model being repeatedly loaded and computed in memory, which is especially suitable for multi-business concurrent scenarios (such as running "regional intrusion detection" and "personnel statistics" services simultaneously, both of which may depend on the same "human detection" atomic component).

[0132] In addition, the process of running multiple AI services in an embedded device can be as follows:

[0133] 1. When the AI ​​service loading device initializes, it reads information such as the sensor status, device resources, and deployed AI services of the embedded device.

[0134] 2. When a user instructs the deployment of an AI service, the scheduling component first checks whether the atomic components that the business component depends on have already been enabled in other services. If so, the already enabled atomic components are reused; otherwise, new atomic components are created.

[0135] 3. The scheduling component creates sensor data channels and buffers according to the needs of the business components. If the data channels can be reused, the created data channels and buffers are reused.

[0136] 4. When the scheduling component retrieves business components, atomic components, and hardware acceleration components from the cloud, if device resources are insufficient, the scheduling component can delete the business component that has been shut down for the longest time to free up resources.

[0137] 5. When the scheduling component loads a service component A, if device resources are insufficient, the scheduling component can calculate, based on the coupling between the service component and atomic components, which minimizes the impact of shutting down other service components while still allowing the service component to be loaded, and then shut down the service component. Subsequently, it requests device resources, creates a service result buffer, and loads service component A.

[0138] 6. When a user instructs the deletion of an AI service, the scheduling component deletes the corresponding business component and atomic components that are not depended upon by other business components.

[0139] V. Editing Custom AI Services

[0140] The AI ​​service loading device can further include a custom service editing unit, which provides a service editing interface to users, allowing them to create trigger conditions and response operations based on the output of a specified AI service and the device functions of the embedded device using predefined logical symbols; and a scheduling component, which verifies the trigger conditions and response operations, and starts the business component corresponding to the specified AI service after verification, and executes the response operation when the trigger conditions are met. Thus, the AI ​​service loading device can achieve a high degree of functional scalability.

[0141] More specifically, the custom service editing unit of the AI ​​service loading device provides a graphical business logic editing interface on the user's APP. Without programming, users can drag and drop logic symbols such as "IF", "AND", "OR", and "DURATION" to connect multiple existing basic AI services (essentially the outputs of business components) of the device with device actions (such as recording, making sounds, and sending messages) to create triggering conditions and one or more corresponding response operations.

[0142] For example, a user can create a trigger condition where the "human detection" algorithm continuously detects a target on the "living room camera," and the "face recognition" algorithm identifies the target as a "stranger," and the time is "nighttime." The user can then create a corresponding response operation for this trigger condition: execute a "loud alarm" and "send a push notification to the phone."

[0143] After saving the above triggering conditions and corresponding response operations, users can combine the triggering conditions and their corresponding response operations to form a custom AI service.

[0144] The scheduling component verifies the user-defined AI service, checking whether the logic of the custom AI service can meet the triggering conditions, and whether the basic AI services required by the custom AI service can run simultaneously on the embedded device (i.e., whether the resources are sufficient). After passing the verification, the scheduling component officially generates a custom AI service.

[0145] The loading process for the scheduling component for this custom AI service is as follows:

[0146] 1. When a custom AI service is deployed, the business components must start all the basic AI services required by the custom AI service, including the corresponding business components and atomic components.

[0147] 2. Create corresponding triggering conditions based on the implementation logic of the custom AI service;

[0148] 3. When the custom AI service is running, the business components corresponding to the basic AI service will send the results to the service result buffer. When the triggering conditions are met, the response operation of the custom AI service will be executed.

[0149] According to this description, the execution of relevant business components (human detection, face recognition) is dynamically coordinated at runtime, and response operations are triggered when logical conditions are met.

[0150] The cloud-deployed device can also track the usage of all customer-customized AI services, analyze the usage of each customer's atomic and business components, and develop new AI services in response to market trends.

[0151] VI. Cloud Adaptation and Optimization Closed Loop

[0152] Cloud-based deployment enables precise adaptation and continuous optimization of business operations:

[0153] 1. Intelligent Recommendation and Deployment: When a user selects an AI service from the interface provided by the e-commerce interaction unit, the e-commerce interaction unit will select the atomic component version with the best performance and resource consumption from the service repository based on information such as the chip model and memory capacity of the user's device, and deploy it together with the business component to achieve personalized deployment.

[0154] 2. Data Feedback and Model Optimization: Users can provide feedback on unsatisfactory business performance (e.g., high false alarm rates). After authorization, relevant sensor data (e.g., images) can be fed back to the model training unit of the cloud-deployed device. This unit can utilize data augmentation techniques (e.g., image rotation, cropping, noise addition) to augment small sample data, then fine-tune the model in the atomic component, generating an optimized new model file, which is then packaged as a new version of the atomic component. User devices can automatically or manually download and update this atomic component during idle periods, thereby improving business performance.

[0155] The model training unit can perform data augmentation on the sensor data uploaded by the user using various existing methods. Specifically, image data augmentation methods can be divided into three main categories: geometric transformation, pixel transformation, and complex transformation, such as... Figure 5a As shown. The audio and text data also include corresponding enhancement methods, such as... Figure 5b As shown.

[0156] The above describes the implementation of the AI ​​service deployment system and AI service loading device in this application. As can be seen from the above, the AI ​​service loading device of this application has the following advantages compared to existing loading methods on embedded devices:

[0157] 1. Different AI services can reuse the same atomic components, and different atomic components can reuse sensor data in the same format, which greatly saves the computing resources of embedded devices;

[0158] 2. The scheduling component supports customers in designing their own AI services, increasing service scalability and product competitiveness.

[0159] 3. When migrating the entire loading device to the new chip platform, only the NPU / GPU acceleration components need to be modified; the other components remain unchanged. When developing new business components, old atomic components (models) can also be reused, reducing development workload. When atomic components (models) are optimized, they can be directly replaced to improve service performance.

[0160] 4. For the same AI service, users can choose atomic components (models) with different performance levels based on the resources of their embedded devices to obtain the service effect they expect.

[0161] 5. This device can be used not only for AI services based on deep learning of images, but also for AI services based on neural network models of other sensors, and can fuse service results, including but not limited to audio, point cloud and other data.

[0162] 6. The scheduling component can collect usage information for each component, especially atomic and business components that are relied upon in customer-defined AI services, and assist developers in further development based on popularity.

[0163] Compared to existing loading methods on embedded devices, the AI ​​service deployment system proposed in this application has the following advantages:

[0164] 1. Based on the embedded device resources available to the user, select suitable business components and atomic components to provide customized AI services, rather than using the same AI service model for all devices.

[0165] 2. It can collect user preferences for AI services, assisting R&D personnel in further development based on market conditions;

[0166] 3. Based on user feedback, we can customize and train deep learning AI models for C-end users who can only provide a small amount of data feedback, thereby optimizing the AI ​​service effect.

[0167] This application also provides an AI service loading method for embedded devices, which can be implemented in the embedded device described in this application and the aforementioned... Figure 2 The AI ​​service is implemented in the loading device. Figure 6 A basic flowchart illustrating the method for loading this AI service is shown below. Figure 6 As shown, the method includes:

[0168] Step 601: In response to the deployment command issued by the user for the target AI service, provide the cloud with information about the embedded device and the target AI service, and obtain the business components adapted to the embedded device and the target AI service from the cloud;

[0169] Step 602: Load and trigger the execution of business components so that the business components can call one or more atomic components to execute the target AI service according to the business logic of the target AI service;

[0170] Specifically, calling one or more atomic components to execute the target AI service may include: calling the atomic component through the interface provided by the atomic component, and the atomic component calling the hardware acceleration component bound to the embedded device hardware to execute the inference of the neural network model based on the model data of the neural network model.

[0171] When acquiring the business component, the method further includes: acquiring atomic components and hardware acceleration components that the business component depends on and that are not saved by the embedded device from the cloud.

[0172] In the above AI service loading method, to achieve customized training optimization of the model, the method may further include:

[0173] In response to the user's model optimization instruction for the first neural network model corresponding to the first atomic component, the sensor data and model optimization instruction corresponding to the first neural network model are uploaded to the cloud, so that the cloud can use the sensor data to perform model optimization training on the first neural network model and update the first atomic component.

[0174] Preferably, the usage of various business components and atomic components by embedded devices can also be statistically analyzed and reported to the cloud, so that the cloud can update the recommended display order of various AI services based on the usage.

[0175] To implement custom AI services, the above AI service loading method can further include:

[0176] Based on user input, determine the trigger conditions and response operations of the custom AI service created by the user by combining the output of the specified AI service with the device functions of the embedded device using predefined logical symbols.

[0177] Verify the triggering conditions and response operations, and start the business component corresponding to the specified AI service after passing the verification. When it is determined that the triggering conditions are met, execute the response operation.

[0178] To achieve resource and computing power reuse, the above AI service loading method can check whether the atomic components called by the business component have been loaded when loading the business component. If so, the corresponding atomic components and their output results are reused; otherwise, the corresponding atomic components are loaded.

[0179] When loading a business component, you can check whether a data channel and buffer have been created for the required sensor according to the needs of the business component. If so, reuse the data channel and buffer of the corresponding sensor; otherwise, create a data channel and buffer for the corresponding sensor.

[0180] When acquiring business components, if local resources are insufficient, the business component that has been closed for the longest time can be deleted;

[0181] When loading the first business component, if local resources are insufficient, the business component with the least impact that needs to be shut down can be determined, and the determined business component can be shut down before loading the first business component.

[0182] When deleting a business component, you can further delete atomic components that are called by that business component and are not currently being called by other loaded or pending business components.

[0183] This application also provides a computer-readable storage medium that stores instructions, which, when executed by a processor, can perform the steps in the AI ​​service loading method described above. In practical applications, the computer-readable medium may be included in the devices / apparatus / systems of the above embodiments, or it may exist independently and not assembled into the device / apparatus / system. The instructions stored in the computer-readable storage medium, when executed by a processor, can perform the steps in the AI ​​service loading method described above.

[0184] According to the embodiments disclosed in this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof, but not intended to limit the scope of protection of this application. In the embodiments disclosed in this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0185] Figure 7 An electronic device is also provided for this application. For example... Figure 7 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0186] The electronic device may include a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 702 is executed, an AI service loading method can be implemented.

[0187] Specifically, in practical applications, this electronic device may also include components such as a power supply 703 and an input / output unit 704. Those skilled in the art will understand that... Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0188] The processor 701 is the control center of the electronic device. It connects various parts of the electronic device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702, and calling data stored in the memory 702, it performs various functions of the server and processes data, thereby monitoring the electronic device as a whole.

[0189] The memory 702 can be used to store software programs and modules, i.e., the aforementioned computer-readable storage medium. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on server usage, etc. Furthermore, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0190] The electronic device also includes a power supply 703 that supplies power to the various components. This power supply can be logically connected to the processor 701 via a power management system, enabling functions such as charging, discharging, and power consumption management. The power supply 703 may also include one or more DC or AC power supplies, a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator, or any other components.

[0191] The electronic device may also include an input / output unit 704, which can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, and optical signal inputs related to user settings and function control. The input / output unit 704 can also be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof.

[0192] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An AI service deployment system for an embedded platform, characterized in that, include: A cloud-based deployment device and an embedded device, wherein the embedded device includes an interface unit and a scheduling component; A cloud deployment device is used to provide the embedded device with business components adapted to the embedded device and the target AI service based on the information of the embedded device and the information of the target AI service to be deployed. The interface unit is used to enable the interaction between the embedded device and the user and the cloud deployment device; The scheduling component is used to provide the embedded device information and the target AI service information to the cloud deployment device through the interface unit based on the deployment instruction of the target AI service issued by the user received through the interface unit; it is also used to obtain the business component from the cloud deployment device through the interface unit, and load and trigger the operation of the business component. The business component is used to call one or more atomic components to execute the target AI service on the embedded device according to the business logic of the target AI service; the atomic component is used to call a hardware acceleration component bound to the embedded device based on the model data of the neural network model to execute the inference of the neural network model; the atomic component can be called by different business components.

2. The system according to claim 1, characterized in that, The cloud deployment device includes an e-commerce interaction unit and a service warehouse unit; The e-commerce interaction unit is used to display AI services adapted to various embedded devices, and also to interact with the interface unit. Based on the information of the embedded device and the target AI service provided by the interface unit, it determines the business components adapted to the embedded device and the target AI service, as well as the first atomic component and the first hardware acceleration component related to the business components and not saved by the embedded device. It is also used to obtain the determined business components, the first atomic component, and the first hardware acceleration component from the service repository unit, and send them to the scheduling component through the interface unit. The service repository unit is used to store various business components, atomic components, and hardware acceleration components for different embedded platforms.

3. The system according to claim 1, characterized in that, The cloud deployment device is further configured to receive sensor data of the first neural network model corresponding to the second atomic component through the interface unit; and to perform model optimization training on the first neural network model using the sensor data, and update the second atomic component based on the training results.

4. The system according to claim 3, characterized in that, The atomic component includes a model file that carries the model data of the neural network model; In the cloud deployment device, the model file in the second atomic component is updated based on the training results to achieve the update of the second atomic component.

5. The system according to claim 2, characterized in that, The scheduling component is further used to statistically analyze the usage of various business components and atomic components by the embedded device, and report it to the e-commerce interaction unit through the interface unit, serving as the basis for displaying various AI services.

6. The system according to claim 1, characterized in that, The embedded device further includes a custom service editing unit, which provides a service editing interface to the user, allowing the user to create trigger conditions and response operations based on the output of a specified AI service and the device functions of the embedded device through predefined logical symbols. The scheduling component is used to verify the triggering conditions and response operations, and after passing the verification, to start the business component corresponding to the specified AI service, and to execute the response operation when it is determined that the triggering conditions are met.

7. The system according to claim 1, characterized in that, The scheduling component is further configured to check whether the atomic component called by the business component has been loaded when loading the business component; if so, reuse the corresponding atomic component and its output result; otherwise, load the corresponding atomic component. And / or, The scheduling component is further configured to check, according to the needs of the business component, whether a data channel and buffer have been created for the required sensor; if so, the data channel and buffer of the sensor are reused; otherwise, a data channel and buffer are created for the sensor. And / or, The scheduling component is used to further obtain atomic components and hardware acceleration components related to the business component that are not locally stored when obtaining the business component from the cloud deployment device. And / or, The scheduling component is used to delete the business component that has been closed the longest when acquiring business components, atomic components and / or hardware acceleration components if local resources are insufficient. And / or, The scheduling component is used to determine the business component with the least impact that needs to be closed when loading the first business component if local resources are insufficient, close the determined business component, and then load the first business component. And / or, The scheduling component is used to further delete atomic components called by a business component that are not currently being called by other loaded or pending business components when a business component is deleted.

8. The system according to claim 1, characterized in that, The business component includes a business logic code library and a business component configuration file. The business component configuration file records the atomic components and sensor resources that need to be called, and the business logic code library is used to store the business logic for implementing the target AI service.

9. The system according to claim 1, characterized in that, The hardware acceleration component is used to encapsulate the underlying hardware call interface provided by the embedded device into a standard call interface, and execute the inference of the neural network model by calling the underlying hardware.

10. The system according to claim 9, characterized in that, The atomic components include model files, atomic component configuration files, and algorithm libraries; The model file is used to record the model data; The atomic component configuration file is used to record the input data format, data length, and model reporting parameters of the corresponding neural network model; The algorithm library is used to preprocess the data input to the atomic components to adapt to the neural network model; it is also used to call the hardware acceleration component through the standard calling interface to trigger the inference execution of the neural network model; and it is also used to postprocess the inference results returned by the hardware acceleration component to generate standardized atomic component output results.

11. The system according to claim 1, characterized in that, The business component, the atomic component, and the hardware acceleration component each provide their own component information acquisition interface, initialization interface, inference interface, and resource release interface; The scheduling component is used to obtain the device resource information required by the business component and its dependent lower-level components by calling the component information acquisition interface of the business component; it is also used to allocate device resources to the business component based on the device resource information of each level of components. Specifically, the business component obtains the device resource information required by the atomic component and the hardware acceleration component it calls by calling the component information acquisition interface of the atomic component it depends on; the atomic component obtains the device resource information required by the acceleration component by calling the component information acquisition interface of the hardware acceleration component. The scheduling component is further configured to complete the initialization of the service component by calling the initialization interface of the service component and passing the allocated device resources; the service component is further configured to complete the initialization of the atomic component by calling the initialization interface of the atomic component it depends on during its initialization process; the atomic component is further configured to complete the initialization of the hardware acceleration component by calling the initialization interface of the hardware acceleration component during its initialization process. The scheduling component is further configured to trigger the execution of the target AI service by calling the inference interface of the business component; the business component is further configured to trigger the corresponding atomic component to execute the inference of the corresponding neural network model by calling the inference interface of the atomic component it depends on; the atomic component is further configured to trigger the underlying hardware associated with the hardware acceleration component to execute the inference by calling the inference interface of the hardware acceleration component. The scheduling component is further configured to release the device resources occupied by the business component by calling the resource release interface of the business component; the business component is further configured to release the device resources occupied by the atomic component by calling the resource release interface of the atomic component it depends on during its resource release process; the atomic component is further configured to release the device resources occupied by the acceleration component by calling the resource release interface of the acceleration component during its resource release process.

12. The system according to any one of claims 1 to 11, characterized in that, The embedded device includes an AI service loading device, and the scheduling component is located in the AI ​​service loading device.

13. A method for loading AI services on an embedded platform, characterized in that, Used in embedded devices, including: In response to a user's deployment command for a target AI service, the system provides information about the embedded device and the target AI service to the cloud, and retrieves business components adapted to the embedded device and the target AI service from the cloud. Load and trigger the execution of the business component so that the business component calls one or more atomic components to execute the target AI service according to the business logic of the target AI service; The step of calling one or more atomic components to execute the target AI service includes: calling the atomic component through an interface provided by the atomic component, wherein the atomic component, based on model data of the neural network model, calls a hardware acceleration component bound to the embedded device to execute the inference of the neural network model.

14. The method according to claim 13, characterized in that, When acquiring the business component, the method further includes: acquiring atomic components and hardware acceleration components that the business component depends on and that are not saved by the embedded device from the cloud.

15. The method according to claim 13, characterized in that, The method further includes: In response to a user's model optimization instruction for the first neural network model corresponding to the first atomic component, the sensor data corresponding to the first neural network model and the model optimization instruction are uploaded to the cloud, so that the cloud can use the sensor data to perform model optimization training on the first neural network model and update the first atomic component.

16. The method according to claim 15, characterized in that, The atomic component includes a model file, which carries the model data of the neural network model; Updating the first atomic component includes updating the model data in the model file based on the training results of the model optimization training.

17. The method according to claim 13, characterized in that, The method further includes: statistically analyzing the usage of the embedded device for each business component and atomic component, and reporting it to the cloud, so that the cloud updates the display order of various AI services based on the usage.

18. The method according to claim 13, characterized in that, The method further includes: Based on user input, determine the trigger conditions and response operations of the custom AI service created by the user by combining the output of the specified AI service with the device functions of the embedded device using predefined logical symbols; Verify the triggering condition and the response operation, and start the business component corresponding to the specified AI service after passing the verification. When it is determined that the triggering condition is met, execute the response operation.

19. The method according to claim 13, characterized in that, The method further includes: When loading the business component, check whether the atomic component called by the business component has been loaded. If so, reuse the corresponding atomic component and its output result; otherwise, load the corresponding atomic component. And / or, When loading the business component, check whether a data channel and buffer have been created for the required sensor according to the needs of the business component. If so, reuse the data channel and buffer of the sensor; otherwise, create a data channel and buffer for the sensor. And / or, When acquiring business components, if local resources are insufficient, delete the business component that has been closed for the longest time; And / or, When loading the first business component, if local resources are insufficient, determine the business component with the least impact that needs to be closed before loading the first business component, close the determined business component, and then load the first business component. And / or, When deleting a business component, further delete the atomic components that are called by that business component and are not currently being called by other loaded or pending business components.

20. An embedded device, characterized in that, Includes interface units and scheduling components; The interface unit is used to enable the interaction between the embedded device and the user and the cloud deployment device; The scheduling component is used to provide the embedded device information and the target AI service information to the cloud deployment device through the interface unit based on the deployment instruction of the target AI service issued by the user received through the interface unit; it is also used to obtain the business component from the cloud deployment device through the interface unit, and load and trigger the operation of the business component. The business component is used to call one or more atomic components according to the business logic of the target AI service to execute the target AI service on the embedded device; the atomic component is used to call a hardware acceleration component bound to the embedded device based on the model data of the neural network model to execute the inference of the neural network model.

21. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instruction is executed by the processor, it can implement the AI ​​service loading method according to any one of claims 13 to 19.

22. An electronic device, characterized in that, The electronic device includes at least a computer-readable storage medium and a processor; The processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the AI ​​service loading method according to any one of claims 13 to 19.

Images