Artificial intelligence model adaptive deployment method and device, equipment and medium

Abstract

The invention relates to the technical field of electric power communication, in particular to an artificial intelligence model adaptive deployment method and device, equipment and a medium, and the method comprises the steps: generating an equipment fingerprint vector based on hardware parameters of edge equipment, and determining a model customization strategy matched with the equipment fingerprint vector according to a preset capability matrix mapping rule; performing metadata analysis on the to-be-transmitted artificial intelligence model, and determining a type label of the artificial intelligence model and a corresponding lossless compression strategy according to a preset model feature library; and converting a to-be-transmitted artificial intelligence model into a target format according to the model customization strategy and the lossless compression strategy, transmitting the artificial intelligence model converted into the target format to edge equipment in a blocking manner, and deploying the artificial intelligence model. Therefore, the problems of unreasonable resource utilization, poor network adaptability and the like of the artificial intelligence model in edge device deployment in the prior art are solved, and efficient transmission and deployment of the artificial intelligence model are realized.

Description

Technical Field

[0001] The present application relates to the field of power communication technology, and in particular to a method, device, equipment and medium for adaptive deployment of an artificial intelligence model. Background Art

[0002] Currently, the large-scale application of artificial intelligence models in power transmission, transformation and distribution and other professional scenarios faces significant bottlenecks: on the one hand, the Internet of Things management platform has strict size restrictions on model import, and high-precision models need to be updated repeatedly due to frequent iterations, version management is chaotic, and the operation and maintenance costs of manual on-site upgrades are high; on the other hand, the frequent upgrades of uncompressed models have a high network bandwidth occupancy rate, which causes traffic congestion when multiple professional upgrades are carried out in parallel. In addition, the heterogeneity of edge device resources is significant, which can easily cause waste of high-end device resources or memory overflow of low-end devices. At the same time, there is a lack of full life cycle management mechanisms such as model version control and exception rollback. Manual intervention is required when the upgrade fails, which restricts the efficient implementation of the model in complex power scenarios. Summary of the Invention

[0003] In order to overcome the deficiencies in the prior art, the present application provides an artificial intelligence model adaptive deployment method, device, equipment and medium, which can realize the efficient transmission and deployment of artificial intelligence models.

[0004] In a first aspect, the present application provides a method for adaptively deploying an artificial intelligence model, the method comprising the following steps:

[0005] Generate a device fingerprint vector based on the hardware parameters of the edge device, and determine a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule;

[0006] Perform metadata analysis on the AI model to be transmitted, and determine the type label of the AI model and the corresponding lossless compression strategy based on a preset model feature library;

[0007] The artificial intelligence model to be transmitted is converted into a target format according to the model customization strategy and the lossless compression strategy, and the artificial intelligence model converted into the target format is transmitted in blocks to the edge device and deployed.

[0008] In one possible implementation, generating a device fingerprint vector based on hardware parameters of the edge device includes the following steps:

[0009] Obtaining hardware parameters of the edge device; the hardware parameters include one or more of the computing power, storage capacity, memory capacity, and processor type of the edge device;

[0010] The hardware parameters are quantized to generate a device fingerprint vector.

[0011] In a possible implementation, the capability matrix mapping rule is preset in the following manner:

[0012] Classifying edge devices into different levels based on the device fingerprint vector; wherein hardware parameters of edge devices of different levels correspond to different ranges;

[0013] Corresponding model customization strategies are set for edge devices of different levels as capability matrix mapping rules; different model customization strategies correspond to different model compression parameters; the model compression parameters include one or more of pruning rate, quantization type, and entropy coding.

[0014] In one possible implementation, performing metadata analysis on the artificial intelligence model to be transmitted and determining the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library includes the following steps:

[0015] Obtaining an artificial intelligence model to be transmitted, and performing metadata analysis on the artificial intelligence model to extract its structural features; the structural features include a distribution ratio of a network structure;

[0016] The extracted structural features are matched with the preset model feature library to obtain the type label of the artificial intelligence model to be transmitted and the corresponding lossless compression strategy; wherein, the model feature library includes the structural features, type labels and lossless compression strategies corresponding to the artificial intelligence model; the type labels include one or more of YOLO, ResNet, and Transformer, and the lossless compression strategies include dictionary compression and sparse coding.

[0017] In one possible implementation, transferring the AI model converted into the target format to the edge device in blocks and deploying it includes the following steps:

[0018] The AI model converted to the target format is split into segments according to the set size to generate segment metadata;

[0019] The block metadata is sent to the edge device in sequence, and a double verification mechanism is adopted on the edge device to perform block verification on each received block metadata, and to perform overall verification on all received block metadata combined into complete model metadata;

[0020] Environmental adaptation, resource-aware loading, and abnormality self-healing are performed based on the model metadata.

[0021] In one possible implementation, abnormal self-healing is performed in the following manner:

[0022] During the operation of the artificial intelligence model, the inference latency and accuracy are collected, and a sliding window is set;

[0023] Calculate the average latency within the sliding window and the changing trend of the average accuracy of adjacent sliding windows;

[0024] If the average delay or the change trend exceeds the set threshold, an alarm is triggered and the previous version of the artificial intelligence model is extracted from the constructed version management database to replace the current version.

[0025] In a possible implementation, the method further includes the following steps:

[0026] Deploy AI models globally with a gradual scale-out strategy.

[0027] In a second aspect, the present application provides an artificial intelligence model adaptive deployment device, the device comprising:

[0028] A first determination module is configured to generate a device fingerprint vector based on hardware parameters of the edge device, and determine a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule;

[0029] A second determination module is used to perform metadata analysis on the artificial intelligence model to be transmitted, and determine the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library;

[0030] A transmission and deployment module is used to convert the artificial intelligence model to be transmitted into a target format according to the model customization strategy and the lossless compression strategy, and transmit the artificial intelligence model converted into the target format in blocks to the edge device for deployment.

[0031] In a third aspect, the present application provides an electronic device comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the artificial intelligence model adaptive deployment method as described in any one of the first aspects are performed.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the artificial intelligence model adaptive deployment method as described in any one of the first aspects.

[0033] This embodiment provides an artificial intelligence model adaptive deployment method, apparatus, device, and medium. The method generates a device fingerprint vector based on the hardware parameters of the edge device, and determines a model customization strategy that matches the device fingerprint vector based on a preset capability matrix mapping rule. The method also performs metadata analysis on the artificial intelligence model to be transmitted, and determines the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library. The artificial intelligence model to be transmitted is converted into a target format based on the model customization strategy and the lossless compression strategy, and the artificial intelligence model converted into the target format is transmitted in blocks to the edge device for deployment. This method solves the problems of irrational resource utilization and poor network adaptability in the deployment of artificial intelligence models on edge devices in the prior art, and enables efficient transmission and deployment of artificial intelligence models. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A flowchart of the method for adaptive deployment of an artificial intelligence model according to an embodiment of the present application is shown;

[0036] Figure 2 A flowchart of performing metadata analysis on an artificial intelligence model to be transmitted and determining the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library according to an embodiment of the present application is shown;

[0037] Figure 3 A flowchart of an embodiment of the present application is shown for transmitting the artificial intelligence model converted into a target format to an edge device in blocks and deploying the same;

[0038] Figure 4 A schematic diagram of the structure of the artificial intelligence model adaptive deployment device according to an embodiment of the present application is shown;

[0039] Figure 5 A structural block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0041] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0042] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0043] In view of the technical problems raised by the background technology, the present application provides an artificial intelligence model adaptive deployment method, device, equipment and medium, which can realize the efficient transmission and deployment of artificial intelligence models.

[0044] In one embodiment, see the attached Figure 1 , this application provides an artificial intelligence model adaptive deployment method, the method comprising the following steps:

[0045] S1. Generate a device fingerprint vector based on the hardware parameters of the edge device, and determine a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule;

[0046] S2. Perform metadata analysis on the artificial intelligence model to be transmitted, and determine the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library;

[0047] S3. Convert the artificial intelligence model to be transmitted into the target format according to the model customization strategy and the lossless compression strategy, and transmit the artificial intelligence model converted into the target format in blocks to the edge device for deployment.

[0048] In step S1, the main task is to construct a device fingerprint vector by quantifying and defining the computing power, storage, memory, and processor type characteristics of edge devices. Based on this, a capability matrix mapping rule is established to match the optimal model customization strategy. First, the hardware capabilities of edge devices are standardized and graded to form a device fingerprint vector. This allows for precise matching from "device profiling" to "customization decision-making," ensuring that the AI model adapts to the resource constraints of edge devices while maximizing the accuracy and efficiency of the AI model.

[0049] In one embodiment, the constructed device fingerprint vector is C device =(S compute ,M storage ,M memory ,P type ), S compute ,M storage ,M memory ,P type They represent the normalized computing power value, storage capacity, memory capacity, and processor type code respectively.

[0050] The computing power value is normalized based on the actual TOPS value (tera operations per second) of the edge device:

[0051]

[0052] The storage capacity is divided into levels according to the available storage capacity and normalized:

[0053]

[0054] The memory capacity is divided into levels according to the available memory capacity and normalized:

[0055]

[0056] For processor type coding, the level setting can be based on the processor type:

[0057]

[0058] For example, the hardware parameters of the first power edge device are TOPS=5, storage capacity 8GB, memory capacity 4GB, and CPU type. Storage capacity normalization Memory capacity normalization Processor type code P type =0.3; device fingerprint vector C device =(0.227,0.29,0.125,0.3).

[0059] The hardware parameters of the second power edge device are TOPS=25, storage capacity 128GB, memory capacity 64GB, and NPU type. Storage Normalization Memory normalization Processor code P type =1; device fingerprint vector C device =(1.136,0.6,2,1).

[0060] In the present application, edge devices are divided into different levels based on the device fingerprint vector; wherein, the hardware parameters of edge devices of different levels correspond to different ranges; and corresponding model customization strategies are set for edge devices of different levels as capability matrix mapping rules; wherein, different model customization strategies correspond to different model compression parameters; the model compression parameters include one or more of pruning rate, quantization type, and entropy coding.

[0061] In one embodiment, devices are divided into low-end, mid-end, and high-end levels based on their computing power, storage capacity, memory capacity, and processor type code, with ranges set for each indicator for each level. For example, low-end devices have a computing power value of <0.4, storage capacity <0.3, memory capacity <0.25, and a processor type code ≤0.3 (CPU); mid-end devices have a computing power value between 0.4-0.7, storage capacity between 0.3-0.6, memory capacity between 0.25-0.5, and a processor type code between 0.3-0.6 (GPU); and high-end devices have a computing power value ≥0.7, storage capacity ≥0.6, memory capacity ≥0.5, and a processor type code ≥0.6 (NPU). The model compression parameters set for low-end devices are: high pruning rate (70%), INT8 full quantization and high entropy coding (LZMA compression); the model compression parameters set for mid-end devices are: moderate pruning rate (30%), combination of FP16 core layer and INT8 non-core layer and medium entropy coding (Zstandard compression); the model compression parameters set for high-end devices are: low pruning rate (10%), retaining FP16 precision and not using entropy coding.

[0062] According to the above-mentioned capability matrix mapping rules, it can be determined that the above-mentioned first power edge device belongs to a low-end device, and the model customization parameters of pruning rate 70%, INT8 full quantization, and LZMA compression are selected; the above-mentioned second power edge device belongs to a high-end device, and the model customization parameters of pruning rate 10%, FP16 retention, and no entropy coding are selected.

[0063] See the instructions attached Figure 2In step S2, metadata analysis is performed on the artificial intelligence model to be transmitted, and the type label of the artificial intelligence model and the corresponding lossless compression strategy are determined according to a preset model feature library, including the following steps:

[0064] S201. Acquire an artificial intelligence model to be transmitted, and perform metadata analysis on the artificial intelligence model to extract its structural features; the structural features include a distribution ratio of a network structure;

[0065] S202. Match the extracted structural features with a preset model feature library to obtain the type label of the artificial intelligence model to be transmitted and the corresponding lossless compression strategy; wherein the model feature library includes the structural features, type labels and lossless compression strategies corresponding to the artificial intelligence model; the type labels include one or more of YOLO, ResNet, and Transformer, and the lossless compression strategies include dictionary compression and sparse coding.

[0066] That is, in step S2, a model-aware lossless compression mechanism is introduced mainly targeting the structural characteristics of different artificial intelligence models. It can automatically match the best lossless compression algorithm according to the model architecture (such as YOLO, ResNet, Transformer), and realize model compression based on the customized conclusion of the device fingerprint recognition model, and finally convert it into a target format suitable for edge device resources, such as the RKNN format, with zero loss of model accuracy after compression.

[0067] In one embodiment, the preset model feature library is shown in Table 1.

[0068]

[0069]

[0070] Table 1

[0071] For example, the YOLO series of models, such as YOLOv8, have distinct model structures. When predefining the model feature library, a key feature is set: a combination of Conv (convolutional layer) + BN (batch normalization layer) + SiLU (activation function) with a ratio greater than 80%. When performing metadata analysis on the AI model to be transmitted, the proportion of each layer in the model is accurately counted. If the proportion of the Conv+BN+SiLU combination meets the above conditions, the model is identified as a YOLO series model and the corresponding "YOLO" type label is output.

[0072] The core feature of the ResNet series of models lies in their residual structure. There are a large number of residual connection structures in the model, such as y = F(x) + x. As the network depth increases, the resolution of the feature map gradually decreases and the number of channels gradually increases. When predefining the model feature library, the residual connection structure is set to account for > 60%, and the downsampling method and the number of channels change in accordance with the ResNet law. When performing metadata analysis on the artificial intelligence model to be transmitted, it is found that it has continuous residual blocks. Each residual block contains multiple convolutional layers, and downsampling is achieved through strided convolution at different stages. By adjusting the feature map size and the number of channels, the "ResNet" type label can be output.

[0073] The Transformer model's distinguishing feature is its self-attention mechanism. When the model includes positional encoding and a multi-head attention module, the number of multi-head attention layers should account for >30% of the total number of layers in the predefined model feature library, and the model should have a complete positional encoding and feedforward neural network structure. When performing metadata analysis on the AI model to be transmitted, if this condition is met, the "Transformer" type label will be output.

[0074] Among them, the compression algorithm of dictionary compression (LZ77 variant) is used for the YOLO model; the compression algorithm of Huffman coding combined with parameter sharing is used for the ResNet / CNN model; the compression algorithm of sparse matrix storage combined with run-length coding is used for the Transformer model; and the compression algorithm of differential coding combined with arithmetic coding is used for the LSTM / RNN model. The principles of various compression algorithms should be technical means well known to those skilled in the art and will not be described in detail here. For example, when compressing the YOLO model: (1) Dictionary construction: extract all convolution kernel weights, count the occurrence frequencies, sort by frequency, and construct a dictionary with the top 10% high-frequency values (such as the value 0.25 appears 100,000 times). High-frequency values are assigned 3-bit short codes (such as 001), and low-frequency values retain the original precision. (2) Weight replacement: traverse the model weights, replace the original values with dictionary coding, and use the LZ77 algorithm to compress the coding sequence to eliminate local repetitions. When compressing the Transformer model: (1) Sparseness extraction: Count the proportion of non-zero elements in the attention matrix (usually <10%), and record the non-zero value coordinates (row and column indexes) and values. (2) Run-length encoding: Scan the coordinates by row, record the length of the continuous zero value interval as the run, and store the non-zero values directly (value + coordinate offset). Thus, through this static rule matching, resource-adaptive compression is achieved, making the model 100% adaptable to the limitations of the IoT platform, ensuring that the compressed model is adaptable to heterogeneous devices and ensuring the fast and stable transmission of subsequent models.

[0075] In step S3, the artificial intelligence model to be transmitted is compressed according to the model customization strategy obtained in step S1 and the lossless compression strategy obtained in step S2, and converted into an RKNN target format suitable for edge device resources.

[0076] For further information, see the appendix of the manual. Figure 3 , the artificial intelligence model converted into the target format is transferred to the edge device in blocks and deployed, including the following steps:

[0077] S301, splitting the artificial intelligence model converted into the target format according to the set size to generate block metadata;

[0078] S302: Send the block metadata to the edge device in sequence, and use a double verification mechanism on the edge device to perform block verification on each received block metadata, and to perform overall verification on all received block metadata combined into complete model metadata;

[0079] S303: Perform environment adaptation, resource-aware loading, and abnormality self-healing based on the model metadata.

[0080] In steps S301 and S302, efficient and reliable block transmission and double verification mechanisms are used to ensure lossless delivery of the RKNN model from the cloud to the edge device.

[0081] In one embodiment, the appropriate block size is determined based on factors such as network bandwidth, device storage capacity, and model size. For example, the block size is divided into fixed sizes (such as 10MB) to generate block metadata.

[0082] Each block metadata includes header metadata and payload data, and the structure is shown in Table 2 below.

[0083] Field illustrate Chunk identifier (4 bytes) Fixed value 0x4D4F444C ("MODL"), marking the start of the block Block sequence number and total number of blocks 2-byte sequence number (0 to 65535) + 2-byte total number of blocks Hash type (1 byte) Algorithm identifier (0=SHA256, 1=CRC32) Block hash value (32 bytes) Hash value of the payload data Payload data (variable length) RKNN model blocks (e.g. 10MB)

[0084] Table 2

[0085] Furthermore, a reliable network connection is established between the cloud (e.g., IoT management platform) and the edge device based on a transmission protocol (e.g., HTTP, TCP, MQTT, etc.). The metadata blocks are sent sequentially, and MQTT resumable transmission is supported. This means that during transmission, the transmission status of each metadata block is recorded. If the transmission is interrupted, the edge device can request the cloud to resume transmission of the unfinished metadata blocks from the point of interruption based on the recorded status.

[0086] After receiving each block, the edge device immediately calculates the CRC checksum of the block metadata and compares it with the CRC code recorded in the block metadata. If the checksum fails, it means that the block metadata may have been damaged during transmission and needs to be re-requested. In addition, after the edge device receives all the block metadata, it merges all the block metadata into a complete model file in sequence and calculates the HMAC-SHA256 hash value of the file. The calculated hash value is compared with the HMAC-SHA256 hash value of the original model sent by the cloud to ensure that the model file has not been tampered with during transmission.

[0087] In other embodiments, network quality monitoring is also performed during data transmission. For example, the health score model used is as follows:

[0088]

[0089] If S helth ≥0.8, normal transmission; if 0.6 helth <0.8, limit the speed to 90% of the bandwidth and suspend low-priority tasks; if S helth ≤0.8, only transmits control instructions and triggers manual intervention.

[0090] In step S303, the model is mainly adapted to the environment of the edge device, resource-aware loading and abnormal self-healing are achieved mainly through the edge self-deployment mechanism.

[0091] In one embodiment, NPU driver detection is used for environment adaptation. Before deploying the model, the edge device checks whether the NPU driver required by the RKNN Toolkit is installed. The driver installation status can be detected by querying system files or executing specific commands. If the driver is detected to be missing, the minimum dependency package is pulled from the local image repository for installation. The dependency package can be installed using system commands (such as apt-get, yum, etc.) or custom scripts.

[0092] For resource-aware loading, the edge device applies to the system for the required memory resources in advance based on the memory requirement information recorded in the model metadata. Specifically, the loading interface provided by the RKNN Toolkit (such as rknn.load_rknn) is used to load the merged complete model into the memory of the edge device. For example, pre-apply for memory (such as 1.5GB) based on the model metadata (input size 640×640) to avoid runtime overflow. Among them, when the edge device needs to run multiple models at the same time, computing resources are dynamically allocated based on the computing power requirements of each model and the total computing power of the device. For example, a dual-core NPU allocates 70% of computing power for inference and 30% for data preprocessing.

[0093] ​For abnormal self-healing, a sliding window algorithm is used to calculate the average inference latency and accuracy of the model in real time during operation. The inference latency and prediction results can be recorded at regular intervals (such as 1 second), and then the average value is calculated based on the size of the sliding window (such as 10 samples). When the average inference latency exceeds the preset threshold (such as 200ms) or the accuracy drops by more than a preset ratio (such as 10%), the alarm mechanism is triggered. The administrator can be notified through logging, email or text messages. Once the abnormal alarm is triggered, the edge device immediately loads the previous version of the model from the backup directory for hot replacement.

[0094] Among them, it should be noted that in this application, the transmitted artificial intelligence model is version controlled. For example, version naming specifications are implemented, and a version management database or storage system is established to record detailed information for each version. In this way, the corresponding model version file and its related information can be quickly retrieved by version number, and the rollback function of the model version can be realized. When serious problems are found in the newly deployed version during model deployment and operation, it can be quickly rolled back to the last stable version. In one embodiment, the naming format is major version.minor version.revision number-device type (such as v2.1.3-rk3588), and supports fast retrieval and rollback by version number, adopts incremental update transmission method, and only transmits the difference part of the model (such as fully connected layer parameter changes), reducing the volume by 70%.

[0095] Furthermore, the present application implements a gradual expansion strategy when globally deploying the artificial intelligence model. In one embodiment, the gradual expansion strategy is shown in Table 3.

[0096]

[0097] Table 3

[0098] Specifically, this application constructs a gradual expansion strategy based on three dimensions: device ratio, monitoring metrics, and circuit breaker conditions. During the internal testing phase, 5% of devices are selected for testing, focusing on monitoring model latency, accuracy, and NPU utilization. These metrics provide a direct reflection of the model's fundamental performance on the device. A circuit breaker condition is set when a single device experiences three consecutive failures. Once triggered, this indicates a serious model issue in the small-scale testing phase, necessitating immediate suspension of internal testing to troubleshoot and fix the issue and prevent potential defects from spreading. During the public beta phase, the device ratio is expanded to 20%, with monitoring metrics shifting to business success rate and device stability, focusing more on the model's ability to support services across a wide range of application scenarios and its impact on the overall operational status of the device. When the overall failure rate exceeds 1%, a circuit breaker is triggered, prompting the immediate halt of public testing to prevent further deterioration and ensure normal business operations. During the full release phase, the model is released to 100% of devices, and model performance is evaluated by comparing it against a global performance baseline. There are no clear circuit breaker conditions at this stage, indicating that the reliability of the model has been fully verified through internal and public testing before entering the full-scale stage. The full-scale release is based on the results of rigorous early testing and aims to achieve comprehensive application of the model.

[0099] It can be seen that the adaptive deployment method of artificial intelligence models provided by this application realizes the transmission and unified management of artificial intelligence models based on the Internet of Things management platform through a full-link collaborative mechanism of dynamic compression-sharded transmission-edge self-maintenance. First, a capability matrix is constructed based on device fingerprints, and the pruning rate, quantization strategy and compression algorithm are dynamically matched to compress the model into an RKNN format suitable for edge device resources, while ensuring that the loss of model accuracy is controllable. Secondly, the sharding adaptive protocol is combined with traffic control technology to reduce the parallel upgrade bandwidth occupancy rate to below 25%, significantly alleviating network congestion problems. In addition, the edge end uses a lightweight decompression engine, a resource-aware loading module and an abnormal self-healing mechanism to achieve rapid model deployment and operational stability. This will comprehensively improve the agile deployment and intelligent operation and maintenance capabilities of artificial intelligence models in Internet of Things scenarios.

[0100] Based on the same inventive concept, an artificial intelligence model adaptive deployment device is also provided in an embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned artificial intelligence model adaptive deployment method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0101] As the instruction manual Figure 4 As shown, an embodiment of the present application provides an artificial intelligence model adaptive deployment device, the device comprising:

[0102] A first determination module 401 is configured to generate a device fingerprint vector based on hardware parameters of the edge device, and determine a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule;

[0103] A second determination module 402 is configured to perform metadata analysis on the artificial intelligence model to be transmitted, and determine a type label of the artificial intelligence model and a corresponding lossless compression strategy based on a preset model feature library;

[0104] The transmission and deployment module 403 is used to convert the artificial intelligence model to be transmitted into the target format according to the model customization strategy and the lossless compression strategy, and transmit the artificial intelligence model converted into the target format in blocks to the edge device for deployment.

[0105] In one embodiment, the first determination module 401 generates a device fingerprint vector based on the hardware parameters of the edge device, including: obtaining the hardware parameters of the edge device; the hardware parameters include one or more of the computing power value, storage capacity, memory capacity, and processor type of the edge device; and quantifying the hardware parameters to generate a device fingerprint vector.

[0106] In one embodiment, the first determination module 401 presets a capability matrix mapping rule, including: dividing edge devices into different levels based on the device fingerprint vector; wherein the hardware parameters of edge devices of different levels correspond to different ranges; setting corresponding model customization strategies for edge devices of different levels as capability matrix mapping rules; wherein different model customization strategies correspond to different model compression parameters; the model compression parameters include one or more of pruning rate, quantization type, entropy coding, and compression method.

[0107] In one embodiment, the second determination module 402 performs metadata analysis on the artificial intelligence model to be transmitted, and determines the type label of the artificial intelligence model and the corresponding lossless compression strategy based on the preset model feature library, including: obtaining the artificial intelligence model to be transmitted, and performing metadata analysis on the artificial intelligence model to extract its structural features; the structural features include the distribution ratio of the network structure; matching the extracted structural features with the preset model feature library to obtain the type label of the artificial intelligence model to be transmitted and the corresponding lossless compression strategy; wherein, the model feature library includes the structural features, type labels and lossless compression strategies corresponding to the artificial intelligence model; the type labels include one or more of YOLO, ResNet, and Transformer, and the lossless compression strategies include dictionary compression and sparse coding.

[0108] In one embodiment, the transmission and deployment module 403 transmits the artificial intelligence model blocks converted into the target format to the edge device and deploys them, including: dividing the artificial intelligence model converted into the target format into blocks according to the set size to generate block metadata; sending the block metadata to the edge device in sequence, and using a double verification mechanism on the edge device side to perform block verification on each of the received block metadata, and merging all the received block metadata into complete model metadata for overall verification; performing environmental adaptation, resource-aware loading and abnormal self-healing based on the model metadata. Among them, abnormal self-healing is performed in the following manner: during the operation of the artificial intelligence model, the inference delay and accuracy are collected, and a sliding window is set; the average delay in the sliding window and the changing trend of the average accuracy of adjacent sliding windows are calculated; if the average delay or the changing trend exceeds the set threshold, an alarm is triggered and the previous version of the artificial intelligence model is extracted from the constructed version management database to replace the current version.

[0109] In one embodiment, the apparatus further comprises:

[0110] The release module is used to deploy AI models globally according to a gradual scale-out strategy.

[0111] The present application provides an adaptive deployment device for an artificial intelligence model. A first determination module generates a device fingerprint vector based on the hardware parameters of an edge device, and determines a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule. A second determination module performs metadata analysis on the artificial intelligence model to be transmitted, and determines the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library. A transmission deployment module converts the artificial intelligence model to be transmitted into a target format according to the model customization strategy and the lossless compression strategy, and transmits the artificial intelligence model converted to the target format in blocks to the edge device for deployment. This solves the problems of irrational resource utilization and poor network adaptability in the deployment of artificial intelligence models on edge devices in the prior art, and achieves efficient transmission and deployment of artificial intelligence models.

[0112] Based on the same concept of the present invention, the specification Figure 5As shown, an embodiment of the present application provides a structure of an electronic device 500, which includes: at least one processor 501, at least one network interface 504 or other user interface 503, a memory 505, and at least one communication bus 502. The communication bus 502 is used to achieve connection and communication between these components. The electronic device 500 optionally includes a user interface 503, including a display (for example, a touch screen, LCD, CRT, holographic imaging (Holographic) or projection (Projector), etc.), a keyboard or a pointing device (for example, a mouse, trackball (trackball), touchpad or touch screen, etc.).

[0113] The memory 505 may include a read-only memory and a random access memory, and provides instructions and data to the processor 501. A portion of the memory 505 may also include a non-volatile random access memory (NVRAM).

[0114] In some embodiments, the memory 505 stores the following elements, executable modules, or data structures, or a subset or extended set thereof:

[0115] Operating system 5051, including various system programs for implementing various basic services and processing hardware-based tasks;

[0116] The application module 5052 includes various application programs, such as a launcher, a media player, a browser, etc., which are used to implement various application services.

[0117] In an embodiment of the present application, by calling the program or instructions stored in the memory 505, the processor 501 is used to execute steps in a method for adaptive deployment of an artificial intelligence model, thereby enabling efficient transmission and deployment of the artificial intelligence model.

[0118] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the artificial intelligence model adaptive deployment method are executed.

[0119] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned artificial intelligence model adaptive deployment method can be executed.

[0120] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0121] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0122] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0124] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed in the present application, or replace some of the technical features therein with equivalents. However, these modifications, changes, or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for adaptive deployment of artificial intelligence models, characterized in that: The method comprises the following steps: Generate a device fingerprint vector based on the hardware parameters of the edge device, and determine a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule; Perform metadata analysis on the AI model to be transmitted, and determine the type label of the AI model and the corresponding lossless compression strategy based on a preset model feature library; The artificial intelligence model to be transmitted is converted into a target format according to the model customization strategy and the lossless compression strategy, and the artificial intelligence model converted into the target format is transmitted in blocks to the edge device and deployed.

2. The method for adaptive deployment of an artificial intelligence model according to claim 1, characterized in that: Generating a device fingerprint vector based on the hardware parameters of the edge device includes the following steps: Obtaining hardware parameters of the edge device; the hardware parameters include one or more of the computing power, storage capacity, memory capacity, and processor type of the edge device; The hardware parameters are quantized to generate a device fingerprint vector.

3. The method for adaptive deployment of an artificial intelligence model according to claim 2, characterized in that: Preset capability matrix mapping rules as follows: Classifying edge devices into different levels based on the device fingerprint vector; wherein hardware parameters of edge devices of different levels correspond to different ranges; Corresponding model customization strategies are set for edge devices of different levels as capability matrix mapping rules; different model customization strategies correspond to different model compression parameters; the model compression parameters include one or more of pruning rate, quantization type, and entropy coding.

4. The method for adaptive deployment of an artificial intelligence model according to claim 1, characterized in that: The metadata analysis of the artificial intelligence model to be transmitted and the determination of the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library include the following steps: Obtaining an artificial intelligence model to be transmitted, and performing metadata analysis on the artificial intelligence model to extract its structural features; the structural features include a distribution ratio of a network structure; The extracted structural features are matched with the preset model feature library to obtain the type label of the artificial intelligence model to be transmitted and the corresponding lossless compression strategy; wherein, the model feature library includes the structural features, type labels and lossless compression strategies corresponding to the artificial intelligence model; the type labels include one or more of YOLO, ResNet, and Transformer, and the lossless compression strategies include dictionary compression and sparse coding.

5. The method for adaptive deployment of an artificial intelligence model according to claim 1, characterized in that: The artificial intelligence model converted into the target format is transferred to the edge device in blocks and deployed, including the following steps: The AI model converted to the target format is split into segments according to the set size to generate segment metadata; The block metadata is sent to the edge device in sequence, and a double verification mechanism is adopted on the edge device to perform block verification on each received block metadata, and to perform overall verification on all received block metadata combined into complete model metadata; Environmental adaptation, resource-aware loading, and abnormality self-healing are performed based on the model metadata.

6. The method for adaptive deployment of an artificial intelligence model according to claim 5, characterized in that: Abnormal self-healing is performed in the following ways: During the operation of the artificial intelligence model, the inference latency and accuracy are collected, and a sliding window is set; Calculate the average latency within the sliding window and the changing trend of the average accuracy of adjacent sliding windows; If the average delay or the change trend exceeds the set threshold, an alarm is triggered and the previous version of the artificial intelligence model is extracted from the constructed version management database to replace the current version.

7. The method for adaptive deployment of an artificial intelligence model according to claim 1, characterized in that: The method further comprises the following steps: Deploy AI models globally with a gradual scale-out strategy.

8. An artificial intelligence model adaptive deployment device, characterized in that: The device comprises: A first determination module is configured to generate a device fingerprint vector based on hardware parameters of the edge device, and determine a model customization strategy that matches the device fingerprint vector according to a preset capability matrix mapping rule; A second determination module is used to perform metadata analysis on the artificial intelligence model to be transmitted, and determine the type label of the artificial intelligence model and the corresponding lossless compression strategy based on a preset model feature library; A transmission and deployment module is used to convert the artificial intelligence model to be transmitted into a target format according to the model customization strategy and the lossless compression strategy, and transmit the artificial intelligence model converted into the target format in blocks to the edge device for deployment.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. When the machine-readable instructions are executed by the processor, the steps of the artificial intelligence model adaptive deployment method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the artificial intelligence model adaptive deployment method as described in any one of claims 1 to 7.

Images