Internet of Things AI model reasoning method and device and storage medium

Abstract

The invention provides an Internet of Things AI model reasoning method and device and a storage medium, and the method comprises the steps: collecting data through an Internet of Things terminal, and determining whether the Internet of Things terminal meets a first reasoning condition or not; if the Internet of Things terminal meets the first reasoning condition, generating a reasoning request; the edge server performs reasoning through a deployed first AI model based on the reasoning request to generate a first reasoning result, and determines whether the first reasoning result meets a second reasoning condition; if the first reasoning result meets a second reasoning condition, the cloud server generates a second reasoning result through a deployed second AI model based on the reasoning request; and the edge server generates a target reasoning result based on the reasoning request, the first reasoning result and the second reasoning result. Therefore, according to the method and the device, a stable, reliable, low-delay and high-precision reasoning result is obtained through cloud side-end collaborative reasoning, and the situation of reasoning failure is avoided, so that services are provided for the user in time, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of model reasoning technology, and in particular to an Internet of Things AI model reasoning method, device and storage medium. Background Art

[0002] With the rapid development of the Internet of Things, the Internet of Things and artificial intelligence (AI) models are deeply integrated, bringing more powerful perception and cognition capabilities, decision-making execution capabilities, and autonomous learning and optimization capabilities to the Internet of Things, thereby realizing the intelligence and automation of complex systems. Among them, the Internet of Things can deploy AI models through cloud deployment, edge deployment, and terminal deployment. However, any single-mode IoT AI model deployment in cloud deployment, edge deployment, and terminal deployment can only use a single-mode IoT AI model for reasoning, making it impossible for the AI ​​models on the cloud, edge, and terminal to perform collaborative reasoning, which may make the reasoning results inaccurate. At the same time, when the model performance deteriorates, reasoning failures may occur, and services cannot be provided to users in a timely manner, resulting in poor user experience. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides an IoT AI model reasoning method, device and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an IoT AI model reasoning method, which is applied to a reasoning framework, wherein the reasoning framework includes a cloud server, an edge server, and an IoT terminal, wherein the cloud server is connected to at least one edge server, and the edge server is connected to at least one IoT terminal, and the cloud server and the edge server are both deployed with corresponding AI models; the method includes:

[0005] The Internet of Things terminal collects data and determines whether the Internet of Things terminal meets the first reasoning condition;

[0006] If the IoT terminal meets the first reasoning condition, generating a reasoning request based on the type of the IoT terminal and the collected data;

[0007] The edge server generates a first reasoning result by reasoning based on the reasoning request through the deployed first AI model, and determines whether the first reasoning result meets a second reasoning condition;

[0008] If the first reasoning result satisfies the second reasoning condition, the cloud server generates a second reasoning result based on the reasoning request by using the deployed second AI model;

[0009] The edge server generates a target inference result based on the inference request, the first inference result, and the second inference result.

[0010] Optionally, in the embodiment of the present disclosure, determining whether the IoT terminal satisfies the first reasoning condition includes:

[0011] Determining whether the IoT terminal belongs to a preset type;

[0012] If the Internet of Things terminal belongs to a preset type, the Internet of Things terminal infers the collected data to obtain a third inference result, and determines whether the third inference result meets a threshold condition;

[0013] If the third reasoning result does not meet the threshold condition, determining that the Internet of Things terminal meets the first reasoning condition;

[0014] If the IoT terminal does not belong to the preset type, determining whether the IoT terminal meets the access condition;

[0015] If the Internet of Things terminal meets the access condition, it is determined that the Internet of Things terminal meets the first reasoning condition.

[0016] Optionally, in an embodiment of the present disclosure, the edge server generates a target reasoning result based on the reasoning request, the first reasoning result, and the second reasoning result, including:

[0017] The edge server determines at least one fused inference result for inference result fusion based on the inference request, the first inference result, and the second inference result;

[0018] Determining the weight coefficient of each fusion reasoning result;

[0019] A target reasoning result is generated according to each fused reasoning result and a weight coefficient of each fused reasoning result.

[0020] Optionally, in the embodiment of the present disclosure, the method further includes:

[0021] The edge server obtains a first data set from the IoT terminal, wherein the first data set includes inference data;

[0022] The edge server obtains a second data set, wherein the second data set includes training data;

[0023] The edge server determines whether there is a difference between the data distribution of the first data set and the data distribution of the second data set;

[0024] If it is determined that there is a difference, annotating the first data set, and sending the annotated data and retraining instruction information to the connected cloud server;

[0025] After receiving the retraining instruction information, the cloud server determines a retraining mode and performs retraining based on the retraining mode.

[0026] Optionally, in the embodiment of the present disclosure, the edge server determines whether there is a difference between the data distribution of the first data set and the data distribution of the second data set, including:

[0027] Adding corresponding labels to the first data set and the second data set respectively, and merging the first data set with the added labels with the second data set to obtain a third data set;

[0028] Based on the third data set, classify the third data set by using a preset binary classification algorithm set to obtain a classification effect of each algorithm in the preset binary classification algorithm set;

[0029] Based on the classification effects of the algorithms, determining the accuracy of the preset binary classification algorithm set;

[0030] Based on the accuracy, it is determined whether there is a difference between the data distribution of the first data set and the data distribution of the second data set.

[0031] Optionally, in the embodiment of the present disclosure, determining the accuracy of the preset binary classification algorithm set based on the classification effect of each algorithm includes:

[0032] Based on the classification effect of each algorithm, a result matrix is ​​determined by each evaluation indicator in the evaluation indicator set, wherein each row of the result matrix represents each binary classification algorithm, and each column of the result matrix represents the indicator value of each evaluation indicator corresponding to each binary classification algorithm;

[0033] Determining a target evaluation indicator from the evaluation indicator set;

[0034] Based on the result matrix, determining the accuracy of the target evaluation indicator;

[0035] The accuracy of the target evaluation indicator is determined as the accuracy of the preset binary classification algorithm set.

[0036] Optionally, in an embodiment of the present disclosure, after receiving the retraining instruction information, the cloud server determines a retraining mode and performs retraining based on the retraining mode, including:

[0037] After receiving the retraining instruction information, the cloud server determines a target number of edge servers that send the retraining instruction information;

[0038] If the target quantity is greater than the quantity threshold, determining that the retraining mode is the first retraining mode, and performing retraining based on the first retraining mode;

[0039] If the target number is less than or equal to the number threshold, the retraining mode is determined to be the second retraining mode, and retraining is performed based on the second retraining mode.

[0040] Optionally, in the embodiment of the present disclosure, the retraining based on the first retraining mode includes:

[0041] Determining first training data based on the received labeled data;

[0042] The cloud server retrains the second AI model based on the first retraining data, generates a target model file of the latest version of the second AI model based on the retrained second AI model, and pushes the target model file to the connected edge server;

[0043] The edge server connected to the cloud server updates the first AI model based on the target model file.

[0044] Optionally, in the embodiment of the present disclosure, the retraining based on the second retraining mode includes:

[0045] Determine second training data based on the labeled data;

[0046] The edge server that sends the retraining indication information retrains its respective first AI model based on the second retraining data, and performs model inference based on the retrained first AI model.

[0047] According to a second aspect of an embodiment of the present disclosure, there is provided an IoT AI model reasoning device, the device comprising:

[0048] A determination module, used for the Internet of Things terminal to collect data and determine whether the Internet of Things terminal meets the first reasoning condition;

[0049] A first generating module, configured to generate an inference request based on the type of the Internet of Things terminal and the collected data if the Internet of Things terminal meets the first inference condition;

[0050] A first reasoning module, configured for the edge server to generate a first reasoning result by reasoning through a deployed first AI model based on the reasoning request, and to determine whether the first reasoning result meets a second reasoning condition;

[0051] A second reasoning module, configured to generate a second reasoning result by the cloud server through a deployed second AI model based on the reasoning request if the first reasoning result satisfies the second reasoning condition;

[0052] The second generating module is used for the edge server to generate a target reasoning result based on the reasoning request, the first reasoning result and the second reasoning result.

[0053] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0054] processor;

[0055] a memory for storing processor-executable instructions;

[0056] The processor is configured to implement the method described in the first aspect above.

[0057] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, wherein the computer storage medium stores computer-executable instructions; after the computer-executable instructions are executed by a processor, the method described in the first aspect above can be implemented.

[0058] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0059] The present disclosure proposes an IoT AI model reasoning method, device and storage medium, the method includes collecting data from an IoT terminal and determining whether the IoT terminal meets a first reasoning condition; if the IoT terminal meets the first reasoning condition, generating a reasoning request based on the type of the IoT terminal and the collected data; the edge server generates a first reasoning result based on the reasoning request by reasoning through the deployed first AI model, and determines whether the first reasoning result meets a second reasoning condition; if the first reasoning result meets the second reasoning condition, the cloud server generates a second reasoning result based on the reasoning request through the deployed second AI model; the edge server generates a target reasoning result based on the reasoning request, the first reasoning result and the second reasoning result. Therefore, the present disclosure can obtain a stable, reliable, low-latency and highly accurate reasoning result through cloud-edge collaborative reasoning, and when the model performance on one side decreases, the reasoning result can be obtained by performing model reasoning on other sides to avoid reasoning failures, thereby providing services to users in a timely manner and improving user experience.

[0060] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0062] Figure 1is a schematic diagram of cloud deployment of an IoT AI model according to some embodiments of the present disclosure;

[0063] Figure 2 is a schematic diagram of edge deployment of an IoT AI model according to some embodiments of the present disclosure;

[0064] Figure 3 is a schematic diagram of a terminal-side deployment of an IoT AI model according to some embodiments of the present disclosure;

[0065] Figure 4 It is a flowchart of an IoT AI model reasoning method according to some embodiments of the present disclosure;

[0066] Figure 5 is a structural schematic diagram of a reasoning framework according to some embodiments of the present disclosure;

[0067] Figure 6 It is a flowchart of an IoT AI model reasoning method according to some embodiments of the present disclosure;

[0068] Figure 7 is a flowchart of a method for determining retraining according to some embodiments of the present disclosure;

[0069] Figure 8 is a flowchart of a retraining method according to some embodiments of the present disclosure;

[0070] Fig. 9 It is a structural schematic diagram of an IoT AI model reasoning device according to some embodiments of the present disclosure;

[0071] Fig.10 It is a block diagram of an electronic device suitable for executing an IoT AI model reasoning method according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0072] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.

[0073] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0074] In the disclosed embodiment, the cloud deployment of the IoT AI model is to perform AI model training and reasoning on a cloud server. Figure 1 A schematic diagram of cloud deployment of an IoT AI model provided by an embodiment of the present disclosure. Figure 1 As shown in the figure, the cloud server can use the communication network to obtain the data sent by the IoT terminal, and infer the data based on the deployed AI model, and send the inference results to the IoT terminal. Among them, the cloud server collects the full amount of IoT device data, performs data preprocessing such as cleaning, labeling, and feature engineering, and uses the preprocessed data to train the AI ​​model offline, and then deploys the trained AI model on the cloud server.

[0075] In the disclosed embodiment, the above-mentioned IoT AI model cloud deployment solution realizes the centralized deployment of AI capabilities, shared computing power and unified update. The powerful computing power and large storage space can process a large amount of data and complex models, quickly adjust resources according to business needs, and support simultaneous access by massive IoT terminals. However, the massive data perceived by the IoT terminals needs to be transmitted to the cloud server through the network, and network delays may cause real-time performance to degrade.

[0076] Furthermore, in the disclosed embodiment, the edge deployment of the IoT AI model is to deploy the AI ​​model through an edge server close to the data source to serve nearby IoT terminals. Figure 2 This is a schematic diagram of edge deployment of an IoT AI model proposed in an embodiment of the present disclosure. Figure 2 As shown, the cloud server uses the full data training to obtain a general AI model, and compresses the AI ​​model to perform lightweight optimization to obtain an AI model that can be deployed on the edge. The compressed AI model is distributed to the edge server, and the IoT terminal sends the perceived data to the edge server. The edge server uses the deployed AI model for reasoning without having to transmit it back to the cloud, and transmits the reasoning results to the physical terminal.

[0077] In the disclosed embodiment, the edge deployment of the IoT AI model can realize real-time or quasi-real-time reasoning of data locally without transmitting data, thus reducing the data transmission cost. However, the computing and storage capabilities of the above solution are relatively weak and are not suitable for processing large amounts of data or complex models.

[0078] Furthermore, in the embodiments of the present disclosure, Figure 3The following is a schematic diagram of the terminal deployment of an IoT AI model proposed in the embodiment of the present disclosure. Figure 3 As shown in the figure, the deployment of IoT AI models on the terminal side is based on the hardware performance limitations of IoT terminals. The full model trained on the cloud is compressed using methods such as model compression, quantization, and distillation to obtain a lightweight model. The compressed model and the minimum operating environment required for model execution are integrated into the chip system of the IoT terminal. Based on this, after collecting data, the IoT terminal can directly use the AI ​​model to infer the perception data on the terminal side and obtain the inference result.

[0079] In the disclosed embodiment, the terminal deployment of the IoT AI model can be completely offline and does not require a network connection, thereby providing users with instant feedback. However, the computing and storage capabilities of the above terminal deployment are limited and may not be suitable for complex models.

[0080] Among them, in the embodiments of the present disclosure, the above-mentioned single-mode IoT AI deployment cannot comprehensively use multiple different models for reasoning, making it difficult for centralized cloud deployment to effectively merge real-time data from massive distributed edge nodes, and it is also difficult for AI models distributed at various edge terminals to achieve fusion between models. The data deployed on the terminal side cannot be shared and uploaded to the cloud, and the model on the terminal side cannot collaborate with the global model.

[0081] Furthermore, in the disclosed embodiments, the single-mode IoT AI deployment cannot meet the needs of various scenarios. Specifically, cloud deployment requires access to the AI ​​model of the cloud server through the network, which will result in a certain network transmission delay and is difficult to meet the requirements of the real-time requirements in engineering control; edge deployment requires deploying AI models one by one on the edge server, which is complex to manage and maintain; and end-side deployment is limited by the computing performance and storage capacity of the terminal node, and the deployment process needs to be customized for different hardware, which has poor portability.

[0082] Figure 4 is a flowchart of an IoT AI model reasoning method according to some embodiments of the present disclosure, such as Figure 4 As shown, the method is applied to the reasoning framework and can include the following steps:

[0083] Step 401: The IoT terminal collects data and determines whether the IoT terminal meets a first reasoning condition.

[0084] It should be noted that the executor of the above-mentioned IoT AI model reasoning method is an IoT AI model reasoning device, which can be implemented by software and / or hardware. The IoT AI model reasoning device in this embodiment can be configured in an electronic device.

[0085] In this example embodiment, the electronic device may include devices such as terminal devices and servers, and this embodiment does not limit the electronic device.

[0086] In the disclosed embodiment, Figure 5 This is a schematic diagram of a reasoning framework proposed in an embodiment of the present disclosure. Figure 5 As shown, the above-mentioned reasoning framework includes a cloud server, an edge server and an IoT terminal. The cloud server is connected to at least one edge server, and the edge server is connected to at least one IoT terminal. The cloud server and the edge server are both deployed with corresponding AI models.

[0087] Among them, in the embodiment of the present disclosure, the above-mentioned cloud server includes a cloud training module, a cloud reasoning module, a cloud model warehouse and an IoT terminal management module. Specifically, in the embodiment of the present disclosure, the above-mentioned cloud training module is built on a distributed deep learning training framework (such as TensorFlow On Spark), and can use a variety of optimization algorithms (such as Adam, Adagrad, etc.) to efficiently train various neural network models (such as CNN, RNN, GAN, etc.) in parallel, support data parallelism and model parallelism, and realize the parallel extraction, preprocessing, and normalization of massive heterogeneous data. Through multi-GPU acceleration of deep learning model training, higher training throughput is obtained. At the same time, the trained model is published to the cloud model warehouse, and the model size is compressed by model compression, quantization, etc., and distributed to the AI ​​model warehouse of the edge server to form a trained AI model; the above-mentioned cloud model warehouse is based on a model version control system (such as MLflow), and stores trained AI model files of different functions and different versions in the form of container mirroring or model file packages, and records model metadata, including model creation information, model parameters, training indicators, etc., supports model comparison, query and rapid distribution, and can provide adapted models for heterogeneous edge terminals to achieve dynamic upgrades and deployments. Figure 5 As shown, model file A, model file B, and model file C in the cloud model warehouse represent models with different functions, respectively, and version number V1 and the like represent model files of different versions; the cloud reasoning module obtains model files from the cloud model warehouse, deserializes and deploys the model, and accesses various heterogeneous edge device requests, using a high-performance model parallel reasoning engine to perform efficient reasoning on the input. At the same time, the computing device can be configured with GPU / CPU / FPGA, supports customized reasoning pipelines, achieves high-throughput and low-latency services, and provides centralized cloud AI inference capabilities for IoT applications; the IoT terminal management module monitors the operating status of the IoT terminal through heartbeat packets, and performs status control, authority management, and software and hardware control.

[0088] Furthermore, in the disclosed embodiment, the edge server and the IoT terminal are placed in the same local area network, providing the IoT AI model with model retraining and reasoning capabilities on the edge side, sniffing IoT terminal data in real time, and initiating model online retraining, online updating, online deployment and model fusion on demand. The edge server includes an edge training module, an edge reasoning module, an edge model warehouse and a data sniffing module.

[0089] Specifically, in the disclosed embodiment, the edge training module uses actual IoT terminal data to locally adjust the model parameters of the edge model warehouse to make the model more suitable for the IoT deployment environment; the edge model warehouse provides for receiving cloud warehouse model files and managing local model files, wherein the model files of the cloud model warehouse can be directly received, the model can be stored in the form of files, and based on the model version management platform, the model of the edge server can be stored, version controlled and retrained. Figure 5 In the figure, model file A, model file B, and model file C in the edge model warehouse represent models with different functions, respectively, and version numbers V1 and the like represent model files of different versions; the edge reasoning module obtains the model file from the edge model warehouse, deserializes the deployment model, and builds an inference pipeline to perform real-time low-latency reasoning using edge data. At the same time, the computing device can be configured with GPU / CPU / FPGA to support customized inference pipelines, achieve high-throughput and low-latency services, and provide edge-side AI inference capabilities for IoT applications; the data sniffing module includes model fusion management, model accuracy management, and data distribution management, wherein model fusion management is to call the AI ​​inference services of the cloud server and edge server to achieve fusion inference results and AI model fusion; model accuracy management is to identify whether the accuracy of the model inference data meets the standards; data distribution management is to analyze whether the deployed AI model needs to be retrained. If it is determined that the AI ​​model needs to be retrained, the AI ​​model retraining process of the edge server or cloud server is initiated.

[0090] Furthermore, in the disclosed embodiment, the IoT terminal includes two types: IoT fat terminal and IoT thin terminal. The IoT fat terminal includes sensors and a terminal-side reasoning module. The terminal-side reasoning module can perform real-time reasoning processing on the collected data through an integrated lightweight AI model, and the reasoning result can trigger the response action of the IoT terminal. The IoT thin terminal includes sensors that can detect the data of a specific object and send the AI ​​reasoning request to the edge server. At the same time, the IoT fat terminal and the thin terminal will also report the model's usage performance data for the edge server to monitor the model effect.

[0091] Among them, in the embodiment of the present disclosure, the above-mentioned end-side reasoning module receives the model file from the edge model warehouse, uses the extremely lightweight AI reasoning framework, is embedded in the IoT fat terminal, and quickly performs online reasoning on the collected data; the sensor supports multiple types of IoT sensors with standard communication protocols, collects environmental data and transmits it to the end-side reasoning module or edge server.

[0092] In the disclosed embodiment, the IoT terminal may collect data through sensors, and after collecting the data, may determine whether the IoT terminal satisfies a first reasoning condition to determine whether to send a reasoning request to a connected edge server.

[0093] Specifically, in the embodiment of the present disclosure, the method for determining whether the IoT terminal satisfies the first reasoning condition may include the following steps:

[0094] Step 4011, determining whether the IoT terminal belongs to a preset type;

[0095] In the embodiment of the present disclosure, the above-mentioned preset type may be an Internet of Things fat terminal.

[0096] Step 4012: If the IoT terminal belongs to the preset type, the IoT terminal infers the collected data to obtain a third inference result, and determines whether the third inference result meets the threshold condition;

[0097] In the disclosed embodiment, if it is determined that the IoT terminal is an IoT fat terminal, the collected data can be inferred by a third AI model deployed on the IoT fat terminal to obtain a third inference result.

[0098] And, in the embodiment of the present disclosure, after the third reasoning result is obtained through the above steps, the third reasoning result can be sent to the edge server, and the edge server can receive the third reasoning result through the data sniffing module, and determine whether the third reasoning result meets the threshold condition. Specifically, in the embodiment of the present disclosure, the method for determining whether the third reasoning result meets the threshold condition can include: determining the first model accuracy of the third reasoning result, if the first model accuracy of the third reasoning result exceeds the first preset threshold, then determining that the third reasoning result meets the threshold condition; if the first model accuracy of the third reasoning result does not exceed the first preset threshold, then determining that the third reasoning result does not meet the threshold condition. Among them, the first preset threshold can be set as needed.

[0099] And, in the embodiment of the present disclosure, the first model accuracy of the third reasoning result can be determined based on the type of the third reasoning result. In the embodiment of the present disclosure, the third reasoning result may include two types, namely, classified data and predicted data. Among them, if the above-mentioned third reasoning result is classified data, for example, the Internet of Things identifies whether there is an abnormality in the product and outputs a category of "yes" or "no", the accuracy of the third reasoning result can be determined by the first evaluation indicator; if the above-mentioned third reasoning result is predicted data, for example, the Internet of Things determines the number of products and outputs a continuous number, the accuracy of the third reasoning result can be determined by the second evaluation indicator. Among them, the above-mentioned first evaluation indicator may include any one of the precision rate, the recall rate, and the F1 value, and the above-mentioned second evaluation indicator may include any one of the MSE and the MAE.

[0100] Step 4013: If the third reasoning result does not meet the threshold condition, determine that the Internet of Things terminal meets the first reasoning condition;

[0101] In the disclosed embodiment, if the third reasoning result does not meet the threshold condition, it means that the model accuracy of the third reasoning result does not meet the requirement and the edge server needs to perform reasoning again, then it is determined that the Internet of Things terminal meets the first reasoning condition.

[0102] Furthermore, in the embodiment of the present disclosure, if the third reasoning result meets the threshold condition, it means that the model accuracy of the third reasoning result meets the requirement, and there is no need for the edge server to perform re-reasoning, then the third reasoning result is directly sent to the edge server for model fusion.

[0103] Step 4014: If the IoT terminal does not belong to the preset type, determine whether the IoT terminal meets the access condition;

[0104] In the disclosed embodiment, if the IoT terminal does not belong to the preset type, it means that the IoT terminal is an IoT thin terminal. At this time, the third AI model is not deployed in the IoT thin terminal, and the collected data cannot be modeled. Based on this, it is necessary to determine whether the IoT terminal meets the access conditions so that the collected data can be transmitted to the connected edge server and model inference can be performed through the edge server.

[0105] Among them, in an embodiment of the present disclosure, the above-mentioned method for determining whether the Internet of Things terminal meets the access conditions may include: determining whether the Internet of Things terminal can access the edge server, if it is determined that the Internet of Things terminal can access the edge server, then determining that the Internet of Things terminal meets the access conditions; if it is determined that the Internet of Things terminal cannot access the edge server, then determining that the Internet of Things terminal does not meet the access conditions.

[0106] Step 4015: If the Internet of Things terminal meets the access condition, it is determined that the Internet of Things terminal meets the first reasoning condition.

[0107] In the disclosed embodiment, if the Internet of Things terminal meets the access condition, it means that the Internet of Things terminal can access the edge server, that is, the collected data can be transmitted to the edge server, and the collected data can be inferred through the edge server, then it is determined that the Internet of Things terminal meets the first inference condition.

[0108] In the disclosed embodiment, if the IoT terminal does not meet the access condition, it means that the IoT terminal cannot access the edge server, that is, it cannot transmit the collected data to the edge server and perform reasoning on the collected data through the edge server. Based on this, the IoT terminal cannot perform data reasoning, triggers an abnormal prompt, and the process ends.

[0109] Step 402: If the IoT terminal meets the first reasoning condition, a reasoning request is generated based on the type of the IoT terminal and the collected data.

[0110] In the disclosed embodiment, if the IoT terminal meets the first reasoning condition, it means that the IoT terminal requires an edge server to perform model reasoning, then a reasoning request is generated based on the type of the IoT terminal and the collected data, and the reasoning request is sent to the edge server.

[0111] Among them, in the disclosed embodiment, the data included in the corresponding reasoning request is different depending on the type of the IoT terminal. Specifically, in the disclosed embodiment, if the IoT terminal belongs to the preset type (IoT fat terminal), the reasoning request is generated based on the third reasoning result and the collected data; if the IoT terminal does not belong to the preset type (IoT thin terminal), the reasoning request is generated based on the collected data.

[0112] Step 403: The edge server generates a first reasoning result by reasoning based on the reasoning request through the deployed first AI model, and determines whether the first reasoning result meets the second reasoning condition.

[0113] In the disclosed embodiment, after the edge server receives the inference request sent by the IoT terminal, it can collect data in the inference request, generate a first inference result through inference using the deployed first AI model, and determine whether the first inference result meets the second inference condition.

[0114] Specifically, in the embodiment of the present disclosure, the method for determining whether the first reasoning result satisfies the second reasoning condition may include: determining the second model accuracy of the first reasoning result, and if the second model accuracy of the first reasoning result exceeds the second preset threshold, determining that the first reasoning result does not satisfy the second reasoning condition; if the second model accuracy of the first reasoning result does not exceed the second preset threshold, determining that the first reasoning result satisfies the second reasoning condition. The second preset threshold can be set as needed.

[0115] Furthermore, in the embodiment of the present disclosure, the method for determining the second model accuracy of the first reasoning result is the same as the method for determining the first model accuracy of the third reasoning result. For details, please refer to the detailed introduction in the above embodiment, and the embodiment of the present disclosure will not be repeated again.

[0116] Step 404: If the first reasoning result meets the second reasoning condition, the cloud server generates a second reasoning result based on the reasoning request by deploying the second AI model.

[0117] In the embodiment of the present disclosure, after determining through the above steps that the first reasoning result satisfies the second reasoning condition, the reasoning request can be sent to the cloud server, so that the cloud server can generate a second reasoning result based on the reasoning request through the deployed second AI model.

[0118] Furthermore, in the disclosed embodiment, if the first reasoning result does not satisfy the second reasoning condition, there is no need to send the reasoning request to the cloud server, and the second reasoning result can be directly sent to the edge server for model fusion.

[0119] Step 405: The edge server generates a target reasoning result based on the reasoning request, the first reasoning result, and the second reasoning result.

[0120] In the embodiment of the present disclosure, the first reasoning result and the second reasoning result are obtained through the above steps, and the target reasoning result can be generated based on the reasoning request, the first reasoning result and the second reasoning result through the model fusion management in the edge server.

[0121] Specifically, in an embodiment of the present disclosure, the method for the edge server to generate a target reasoning result based on the reasoning request, the first reasoning result, and the second reasoning result may include the following steps:

[0122] Step 4051: The edge server determines at least one fused reasoning result for reasoning result fusion based on the reasoning request, the first reasoning result, and the second reasoning result;

[0123] Step 4052: Determine the weight coefficient of each fusion reasoning result;

[0124] Step 4053: Generate a target reasoning result according to each fused reasoning result and the weight coefficient of each fused reasoning result.

[0125] In an embodiment of the present disclosure, if the reasoning request includes the third reasoning result, it is determined that at least one fused reasoning result for reasoning result fusion includes the first reasoning result, the second reasoning result and the third reasoning result; if the reasoning request does not include the third reasoning result, it is determined that at least one fused reasoning result for reasoning result fusion includes the first reasoning result and the second reasoning result.

[0126] Furthermore, in the embodiment of the present disclosure, after the fusion reasoning results are determined through the above steps, the weight coefficients of each fusion reasoning result can be determined, and the target reasoning result can be generated according to each fusion reasoning result and the weight coefficients of each fusion reasoning result. The weight coefficients of each fusion reasoning result can be determined according to expert experience, and the sum of the weight coefficients of each fusion reasoning result is 1.

[0127] For example, in the disclosed embodiment, assuming that the third reasoning result of the IoT fat terminal is x, the first reasoning result of the edge server is y, and the second reasoning result of the cloud server is z, and the corresponding weight coefficients are a, b, c, and a≥0, b≥0, c≥0, a+b+c=1, then the target reasoning result is ax+by+cz. And, in the disclosed embodiment, if the model is fused based on one or two model reasoning results, the corresponding weight coefficient needs to be set to 0. Table 1 is a comparative relationship between the fusion reasoning result and the target reasoning result proposed in the disclosed embodiment.

[0128] Table 1

[0129]

[0130] In the disclosed embodiments, through the above-mentioned reasoning framework and method, it is possible to start terminal-side reasoning, edge reasoning and cloud-side reasoning respectively to perform efficient model reasoning according to different scenarios and business models in an IoT environment.

[0131] Specifically, in the disclosed embodiment, if the type of the IoT terminal belongs to the preset type (IoT fat terminal), a very simple lightweight third AI model is deployed on the IoT fat terminal device to perform real-time low-power model reasoning on the data collected by the sensor to output the third reasoning result, so as to achieve the effect of instant response; if the third reasoning result performs poorly or requires more accurate prediction, the edge server is requested to provide reasoning service through program scheduling, and the edge server obtains the first reasoning result by dynamically scheduling resources to achieve efficient heterogeneous parallel reasoning; if the first reasoning result cannot meet the accuracy requirements, the cloud server will continue to be requested to provide reasoning service, and the cloud server can dynamically adjust resources through load balancing and elastic expansion and contraction mechanisms, and obtain the second reasoning result by distributed large-scale model prediction calculation based on technologies such as containers. Based on this, stable, reliable, low-latency and highly accurate reasoning results can be obtained through cloud-edge collaborative reasoning, and when the model performance on one side decreases, the reasoning result can be obtained through model reasoning on other sides to avoid reasoning failures, so that services can be provided to users in a timely manner, improving user experience.

[0132] Furthermore, in the embodiment of the present disclosure, after obtaining the target inference result through the above method, it is also possible to determine whether there is a distribution difference between the IoT end-side inference data and the edge server's model training data based on the model sniffing function of the edge server, so as to determine whether the first AI model deployed by the edge server is adapted to the end-side data distribution. If it is determined that the data distribution has changed, a retraining request can be initiated through the model sniffer of the edge server to retrain the model. Based on this, in the embodiment of the present disclosure, the above method can also include the following steps:

[0133] Step 406: The edge server obtains a first data set from the IoT terminal, wherein the first data set includes inference data;

[0134] Step 407: The edge server obtains a second data set, where the second data set includes training data.

[0135] Step 408: The edge server determines whether there is a difference between the data distribution of the first data set and the data distribution of the second data set;

[0136] Step 409: If it is determined that there is a difference, the first data set is labeled, and the labeled data and retraining instruction information are sent to the connected cloud server;

[0137] Step 410: After receiving the retraining instruction information, the cloud server determines a retraining mode and performs retraining based on the retraining mode.

[0138] In the disclosed embodiment, the first data set can be periodically obtained from the IoT terminal as needed and stored in the edge server. If the data in the first data set is structured data of indicators and / or text, the first data set is stored in a structured database or file storage; if the data in the first data set is unstructured data of images and / or videos and / or logs, the first data set is stored in the edge server through a file object.

[0139] Furthermore, in an embodiment of the present disclosure, after obtaining the first data set, it can be determined whether there is a difference between the data distribution of the first data set and the data distribution of the second data set, so as to determine whether the deployed AI model needs to be retrained.

[0140] Specifically, in the embodiment of the present disclosure, the method for determining whether there is a difference between the data distribution of the first data set and the data distribution of the second data set may include the following steps:

[0141] Step 1: Add corresponding labels to the first data set and the second data set respectively, and merge the first data set with the added labels with the second data set to obtain a third data set;

[0142] In the embodiment of the present disclosure, corresponding tags may be added to the first data set and the second data set respectively according to rules or needs to distinguish the first data set from the second data set.

[0143] For example, in an embodiment of the present disclosure, the label "1" can be added to the training data in the second data set, the label "2" can be added to the inference data in the first data set, and the labeled first data set and the second data set can be merged, and then shuffled and randomly disrupted in order to obtain a third data set.

[0144] Step 2: Based on the third data set, classify the third data set by using a preset binary classification algorithm set to obtain a classification effect of each algorithm in the preset binary classification algorithm set;

[0145] In the embodiment of the present disclosure, the above preset binary classification algorithm set {F i , i=1, 2, ..., n} may include an indicator data class binary classification algorithm and an image and text binary classification algorithm. The indicator data class binary classification algorithm may include one or more of molbileNet, InceptionNet, and BERT; the image and text binary classification algorithm may include one or more of decision tree, support vector machine, and xgboost.

[0146] And, in the embodiment of the present disclosure, the third data set can be classified by the above-mentioned preset binary classification algorithm set to obtain the classification effect of each algorithm in the preset binary classification algorithm set. Specifically, in the embodiment of the present disclosure, the above-mentioned method of classifying the third data set by the preset binary classification algorithm set to obtain the classification effect of each algorithm in the preset binary classification algorithm set may include: dividing the third data set into training samples and verification samples, training each algorithm of the preset binary classification algorithm set using the training samples, and calculating the classification effect of each trained algorithm through the verification samples.

[0147] In the embodiment of the present disclosure, the classification effect of each trained algorithm can be calculated by verifying samples based on at least one evaluation index. In the embodiment of the present disclosure, the evaluation index can include at least one of precision, recall, F1, AUC, and ROC.

[0148] Furthermore, in the embodiments of the present disclosure, evaluation indicators may be selected according to specific practical problems.

[0149] Step 3: Based on the classification effect of each algorithm, determine the accuracy of the preset binary classification algorithm set;

[0150] In the embodiment of the present disclosure, after the classification effect of each algorithm is obtained through the above steps, the accuracy of the preset binary classification algorithm set can be determined based on the classification effect of each algorithm.

[0151] Specifically, in the embodiment of the present disclosure, the method for determining the accuracy of the preset binary classification algorithm set based on the classification effect of each algorithm may include the following steps:

[0152] Step a: Based on the classification effect of each algorithm, a result matrix is ​​determined by each evaluation indicator in the evaluation indicator set;

[0153] In the embodiment of the present disclosure, each row of the above result matrix represents each binary classification algorithm, and each column of the result matrix represents the index value of each evaluation index corresponding to each binary classification algorithm. For example, the result matrix {E ij , i = 1, 2, ..., n, j = 1, 2, ..., m}, where n is the number of algorithms in the preset binary classification algorithm set, and m is the number of evaluation indicators. Table 2 is a corresponding relationship table of each binary classification algorithm and the indicator value of each evaluation indicator proposed in the embodiment of the present disclosure.

[0154] Table 2

[0155] algorithm Evaluation index 1 Evaluation index 2 …… Evaluation index m F1 E11 E12 …… E1m F2 E21 E22 …… E2m …… …… …… …… …… Fn En1 En2 …… Enm

[0156] Step b, determining the target evaluation index from the evaluation index set;

[0157] In the disclosed embodiment, an evaluation indicator may be randomly selected from the evaluation indicator set, and the evaluation indicator may be determined as a target evaluation indicator.

[0158] Step c: Based on the result matrix, determine the accuracy of the target evaluation indicator;

[0159] In the disclosed embodiment, after the result matrix is ​​obtained through the above steps, the accuracy of the target evaluation indicator can be determined based on the result matrix.

[0160] Specifically, in the embodiments of the present disclosure, the method for determining the accuracy of the target evaluation indicator based on the result matrix may include: determining the target column of the target evaluation indicator in the result matrix, comparing the target column with a second preset threshold value to obtain a comparison result vector, and inputting the comparison result vector into a quantity function to obtain the number of accurate results, and then dividing the number of accurate results by the number of algorithms in a preset binary classification algorithm set to obtain the accuracy of the target evaluation indicator.

[0161] In the embodiment of the present disclosure, the comparison result vector F is a vector consisting of a set of Boolean values ​​TRUE and FALSE to represent the binary classification algorithm F in the preset binary classification algorithm set. iWhether the first data set and the second data set can be distinguished. And, in the embodiment of the present disclosure, if the column value in the target column is greater than the second preset threshold, it means that the binary classification algorithm corresponding to the column value can distinguish the first data set and the second data set, and the comparison result corresponding to the column value is TRUE; if the column value in the target column is less than or equal to the second preset threshold, it means that the binary classification algorithm corresponding to the column value cannot distinguish the first data set and the second data set, and the comparison result corresponding to the column value is FALSE. Among them, the above-mentioned second preset threshold can be set as needed.

[0162] And, in the embodiment of the present disclosure, after the comparison result vector is obtained by the above method, the comparison result vector can be input into the quantity function to obtain the number of accurate results, wherein the number of accurate results is the number of TRUE elements in the comparison vector result. Table 3 is an explanation of the quantity function proposed in the embodiment of the present disclosure.

[0163] Table 3

[0164]

[0165] Step d: Determine the accuracy of the target evaluation indicator as the accuracy of the preset binary classification algorithm set.

[0166] Step 4: Based on the accuracy, determine whether there is a difference between the data distribution of the first data set and the data distribution of the second data set.

[0167] Among them, in the embodiment of the present disclosure, after the accuracy rate is obtained through the above steps, it can be determined based on the accuracy rate whether there is a difference between the data distribution of the first data set and the data distribution of the second data set.

[0168] Specifically, in the embodiment of the present disclosure, the method for determining whether there is a difference between the data distribution of the first data set and the data distribution of the second data set based on the accuracy rate may include: if the accuracy rate exceeds the third preset threshold, it means that the binary classification algorithm in the above-mentioned preset binary classification algorithm set that exceeds the third preset threshold can distinguish the first data set from the second data set, and then it is determined that there is a difference between the data distribution of the first data set and the data distribution of the second data set; if the accuracy rate does not exceed the third preset threshold, it means that the binary classification algorithm in the above-mentioned preset binary classification algorithm set cannot distinguish the first data set from the second data set, and then it is determined that there is no difference between the data distribution of the first data set and the data distribution of the second data set. Among them, the third preset threshold can be set as needed, and for example, the third preset threshold can be 50%.

[0169] Further, in the embodiment of the present disclosure, if the existence of a difference is determined through the above steps, the first data set can be manually labeled, and the labeled data and the data in the second data set are merged and the retraining instruction information is sent to the connected cloud server.

[0170] And, in the embodiment of the present disclosure, after the cloud server receives the retraining instruction information, it can determine the retraining mode and perform retraining based on the retraining mode. Specifically, in the embodiment of the present disclosure, after the cloud server receives the retraining instruction information, the method for determining the retraining mode and performing retraining based on the retraining mode may include the following steps:

[0171] Step 4101: After receiving the retraining instruction information, the cloud server determines the target number of edge servers that send the retraining instruction information;

[0172] In the embodiment of the present disclosure, after receiving the retraining instruction information, the cloud server may determine the number of edge servers that send the retraining instruction information, and determine the number as the target number.

[0173] Step 4102: If the target quantity is greater than the quantity threshold, determine that the retraining mode is the first retraining mode, and perform retraining based on the first retraining mode;

[0174] In the embodiment of the present disclosure, if the target quantity is greater than the quantity threshold, it means that the data distribution corresponding to the edge server that is currently greater than the quantity threshold has changed, and the retraining mode is determined to be the first retraining mode, and retraining is performed based on the first retraining mode. The quantity threshold can be set as needed, for example, the quantity threshold is 4.

[0175] Among them, in the embodiment of the present disclosure, the method for retraining based on the first retraining mode may include the following steps:

[0176] Step 41021: Determine the first training data based on the received labeled data;

[0177] Step 41022: The cloud server retrains the second AI model based on the first retraining data, generates a target model file of the latest version of the second AI model based on the retrained second AI model, and pushes the target model file to the connected edge server;

[0178] Step 41023: The edge server connected to the cloud server updates the first AI model based on the target model file.

[0179] In the disclosed embodiment, the above method can be used to retrain the second AI model of the cloud server and the first AI model of the edge server, thereby realizing an integrated mechanism of AI model retraining and reasoning deployed on the cloud server and the edge server.

[0180] Furthermore, in the embodiment of the present disclosure, after retraining through the above steps, model reasoning can be performed through the second AI model retrained by the cloud server and the first AI model retrained by the edge server.

[0181] Step 4103: If the target quantity is less than or equal to the quantity threshold, determine that the retraining mode is the second retraining mode, and perform retraining based on the second retraining mode.

[0182] In an embodiment of the present disclosure, if the target number is less than or equal to the number threshold, indicating that the data distribution corresponding to the edge server that is currently less than or equal to the number threshold has changed, the retraining mode is determined to be the second retraining mode, and retraining is performed based on the second retraining mode.

[0183] Specifically, in the embodiment of the present disclosure, the method for retraining based on the second retraining mode may include the following steps:

[0184] Step 41031: Determine the second training data based on the labeled data;

[0185] Step 41032: The edge servers that send the retraining indication information retrain their respective first AI models based on the second retraining data, and perform model inference based on the retrained first AI model.

[0186] In the disclosed embodiment, in the second retraining mode, the edge server that sends the retraining instruction information needs to be retrained, and the cloud server does not need to be retrained. Based on this, the edge server that sends the retraining instruction information retrains its respective first AI model based on the second retraining data, and performs model inference based on the retrained first AI model.

[0187] Furthermore, in the embodiment of the present disclosure, after retraining through the above steps, model reasoning can be performed through the first AI model retrained by the edge server.

[0188] The present disclosure proposes an IoT AI model reasoning method, which includes collecting data from an IoT terminal and determining whether the IoT terminal meets a first reasoning condition; if the IoT terminal meets the first reasoning condition, generating a reasoning request based on the type of the IoT terminal and the collected data; the edge server generates a first reasoning result based on the reasoning request by reasoning through the deployed first AI model, and determines whether the first reasoning result meets a second reasoning condition; if the first reasoning result meets the second reasoning condition, the cloud server generates a second reasoning result based on the reasoning request through the deployed second AI model; the edge server generates a target reasoning result based on the reasoning request, the first reasoning result, and the second reasoning result. Thus, the present disclosure can obtain a stable, reliable, low-latency, and highly accurate reasoning result through cloud-edge collaborative reasoning, and when the model performance on one side decreases, the reasoning result can be obtained by performing model reasoning on other sides to avoid reasoning failures, thereby providing services to users in a timely manner and improving user experience.

[0189] Based on the above description, Figure 6 The following is a flow chart of an IoT AI model reasoning method proposed in an embodiment of the present disclosure. The method may include the following steps:

[0190] Step 601: The IoT terminal collects sensor data;

[0191] Step 602: determine whether the IoT terminal is a fat terminal. If the IoT terminal is a fat terminal, proceed to step 604; if the IoT terminal is not a fat terminal, proceed to step 603.

[0192] Step 603: The IoT terminal checks the network connection status to determine whether it can access the edge server. If it can access the edge server, the collected data is transmitted to the edge server, and step 606 is continued. If it cannot access the edge server, it means that the IoT terminal device cannot perform data reasoning, an abnormal prompt is triggered, and the process ends.

[0193] Step 604: The IoT terminal performs model inference service and sends the inference result (the third inference result) to the edge server;

[0194] Step 605: The data sniffing module of the edge server receives the inference result (the third inference result) sent by the IoT terminal, and determines whether the inference accuracy of the inference result exceeds the threshold (the first preset threshold). If it is determined that it exceeds the threshold, continue to step 609; otherwise, continue to step 606;

[0195] Step 606: The edge server performs model reasoning to obtain a reasoning result of the edge server (the first reasoning result described above);

[0196] Step 607: determine whether the inference result exceeds a threshold (the second preset threshold mentioned above); if it is determined to exceed the threshold, continue to step 609; otherwise, continue to step 608;

[0197] Step 608: The cloud server executes the model reasoning service through the cloud reasoning module, and sends the obtained model reasoning result (the second reasoning result) to the edge server;

[0198] Step 609: The edge server integrates the inference results of the IoT fat terminal, the edge server, and the cloud server into a model result to obtain a target inference result.

[0199] In the embodiment of the present disclosure, for other contents of the above steps 601 to 609, reference may be made to the detailed description in the above embodiment, and the embodiment of the present disclosure will not be elaborated here.

[0200] Based on the above description, Figure 7 A flowchart of a retraining determination method proposed in an embodiment of the present disclosure is provided. The method may include the following steps:

[0201] Step 701: The edge server obtains and stores a training data sample segment A (the second data set mentioned above);

[0202] Among them, if the training data sample segment A is structured data such as indicators and / or text, it is stored in a structured database or file storage; if the training data sample segment A is unstructured data such as images and / or videos and / or logs, it is stored in the edge server through file objects.

[0203] Step 702: The edge server presets a binary classification machine learning algorithm set {F i , i=1,2,…,n};

[0204] Among them, the preset binary classification machine learning algorithm set includes indicator data class binary classification algorithms, such as decision tree, support vector machine, xgboost. In addition, the preset binary classification machine learning algorithm set also includes image and text binary classification algorithms, such as molbileNet, InceptionNet, BERT, etc.

[0205] Step 703: The edge server obtains the inference data sample segment B from the IoT terminal and stores it in the edge server;

[0206] Among them, if the inference data sample segment B is structured data such as indicators and texts, it is stored in a structured database or file storage; if the inference data sample segment B is unstructured data such as images, videos, logs, etc., it is stored in the edge server through file objects.

[0207] Step 704: add a label "1" to the training data sample segment A, add a label "2" to the inference data sample segment B, and merge the labeled training data sample segment A and the inference data sample segment B to obtain data set C;

[0208] Among them, the labeled training data sample fragment A and the inference data sample fragment B are shuffled, the sample order is randomly disrupted, and then merged to form data set C.

[0209] Step 705: classify the data set C using a preset binary classification algorithm set, and obtain an accuracy result matrix (the above result matrix) based on the classification effect of each algorithm;

[0210] Step 706: Calculate the overall accuracy of the binary classification algorithm set according to the accuracy result matrix (the above result matrix). If the overall accuracy exceeds the third preset threshold, continue to step 707; if the overall accuracy does not exceed the third preset threshold, continue to step 703;

[0211] Step 707: annotate the inference data sample segment B, merge the annotated data into the training data sample segment A of the edge server, and send the merged data and retraining instruction information to the cloud server.

[0212] In the embodiment of the present disclosure, for other contents of the above steps 701 to 707, reference may be made to the detailed description in the above embodiment, and the embodiment of the present disclosure will not be elaborated here.

[0213] Based on the above description, Figure 8 A flowchart of a retraining method proposed in an embodiment of the present disclosure is provided. The method may include the following steps:

[0214] Step 801, determining whether the data of the edge server that sends the retraining instruction information exceeds a threshold (the above-mentioned quantity threshold), if the data exceeds the threshold, continuing to step 802; otherwise, continuing to step 806;

[0215] Step 802: The edge server that sends the retraining instruction information obtains training data and performs a retraining process;

[0216] Step 803: The edge server performs model training on the deployed AI model based on the training data to generate a model file;

[0217] Step 804: The edge server pushes the model file to the image repository of the edge server;

[0218] Step 805: The edge server deserializes the model file to form a running model and provides model reasoning services;

[0219] Step 806: The cloud server obtains training data and performs a retraining process;

[0220] Step 807: The cloud server performs model training on the deployed AI model based on the training data to generate a model file;

[0221] Step 808: The cloud server pushes the model file to the image repository of the edge server and the image repository of the edge server;

[0222] Step 809: The cloud server deserializes the model file to form a running model and provides model reasoning services.

[0223] In the embodiment of the present disclosure, for other contents of the above steps 801 to 809, reference may be made to the detailed description in the above embodiment, and the embodiment of the present disclosure will not be elaborated here.

[0224] Fig. 9 According to some embodiments of the present disclosure, an IoT AI model inference device is shown, which is applied to an inference framework. The inference framework includes a cloud server, an edge server and an IoT terminal. The cloud server is connected to at least one edge server, and the edge server is connected to at least one IoT terminal. The cloud server and the edge server are both deployed with corresponding AI models, such as Fig. 9 As shown, the device may include:

[0225] The determination module 901 is used for the IoT terminal to collect data and determine whether the IoT terminal meets the first reasoning condition;

[0226] A first generating module 902, configured to generate an inference request based on the type of the IoT terminal and the collected data if the IoT terminal meets the first inference condition;

[0227] The first reasoning module 903 is used for the edge server to generate a first reasoning result by reasoning through the deployed first AI model based on the reasoning request, and determine whether the first reasoning result meets the second reasoning condition;

[0228] A second reasoning module 904 is configured to generate a second reasoning result by the cloud server through a deployed second AI model based on the reasoning request if the first reasoning result satisfies a second reasoning condition;

[0229] The second generating module 905 is used for the edge server to generate a target reasoning result based on the reasoning request, the first reasoning result and the second reasoning result.

[0230] In the embodiment of the present disclosure, the determination module 901 is specifically used to:

[0231] Determine whether the IoT terminal belongs to the preset type;

[0232] If the IoT terminal belongs to the preset type, the IoT terminal infers the collected data to obtain a third inference result, and determines whether the third inference result meets the threshold condition;

[0233] If the third reasoning result does not meet the threshold condition, it is determined that the Internet of Things terminal meets the first reasoning condition;

[0234] If the IoT terminal does not belong to the preset type, determining whether the IoT terminal meets the access condition;

[0235] If the Internet of Things terminal meets the access condition, it is determined that the Internet of Things terminal meets the first reasoning condition.

[0236] Optionally, in the embodiment of the present disclosure, the second generating module 905 is specifically used to:

[0237] The edge server determines at least one fused reasoning result for reasoning result fusion based on the reasoning request, the first reasoning result, and the second reasoning result;

[0238] Determine the weight coefficient of each fusion reasoning result;

[0239] The target reasoning result is generated according to each fusion reasoning result and the weight coefficient of each fusion reasoning result.

[0240] Optionally, in the embodiment of the present disclosure, the above device is also used for:

[0241] The edge server obtains a first data set from the IoT terminal, wherein the first data set includes inference data;

[0242] The edge server obtains a second data set, wherein the second data set includes training data;

[0243] The edge server determines whether there is a difference between the data distribution of the first data set and the data distribution of the second data set;

[0244] If it is determined that there is a difference, the first data set is labeled, and the labeled data and retraining instruction information are sent to a connected cloud server;

[0245] After receiving the retraining instruction information, the cloud server determines a retraining mode and performs retraining based on the retraining mode.

[0246] Optionally, in the embodiments of the present disclosure, the above device is specifically used for:

[0247] Add corresponding labels to the first data set and the second data set respectively, and merge the first data set with the added labels and the second data set to obtain a third data set;

[0248] Based on the third data set, classify the third data set by using a preset binary classification algorithm set to obtain a classification effect of each algorithm in the preset binary classification algorithm set;

[0249] Based on the classification effect of each algorithm, determine the accuracy of the preset binary classification algorithm set;

[0250] Based on the accuracy, it is determined whether there is a difference between the data distribution of the first data set and the data distribution of the second data set.

[0251] Optionally, in the embodiments of the present disclosure, the above device is specifically used for:

[0252] Based on the classification effect of each algorithm, a result matrix is ​​determined by each evaluation indicator in the evaluation indicator set, wherein each row of the result matrix represents each binary classification algorithm, and each column of the result matrix represents the indicator value of each evaluation indicator corresponding to each binary classification algorithm;

[0253] Determine the target evaluation indicator from the evaluation indicator set;

[0254] Based on the result matrix, determine the accuracy of the target evaluation indicators;

[0255] The accuracy of the target evaluation indicator is determined as the accuracy of the preset binary classification algorithm set.

[0256] Optionally, in the embodiments of the present disclosure, the above device is specifically used for:

[0257] After receiving the retraining instruction information, the cloud server determines a target number of edge servers that send the retraining instruction information;

[0258] If the target quantity is greater than the quantity threshold, determining the retraining mode to be the first retraining mode, and performing retraining based on the first retraining mode;

[0259] If the target quantity is less than or equal to the quantity threshold, the retraining mode is determined to be the second retraining mode, and retraining is performed based on the second retraining mode.

[0260] Optionally, in the embodiments of the present disclosure, the above device is specifically used for:

[0261] Determine the first training data based on the received labeled data;

[0262] The cloud server retrains the second AI model based on the first retraining data, generates a target model file of the latest version of the second AI model based on the retrained second AI model, and pushes the target model file to the connected edge server;

[0263] The edge server connected to the cloud server updates the first AI model based on the target model file.

[0264] Optionally, in the embodiments of the present disclosure, the above device is specifically used for:

[0265] Determine the second training data based on the labeled data;

[0266] The edge server that sends the retraining indication information retrains its respective first AI model based on the second retraining data, and performs model inference based on the retrained first AI model.

[0267] In one or more embodiments of the present disclosure, the IoT terminal collects data and determines whether the IoT terminal meets the first reasoning condition; if the IoT terminal meets the first reasoning condition, an inference request is generated based on the type of the IoT terminal and the collected data; the edge server generates a first reasoning result based on the reasoning request through the deployed first AI model, and determines whether the first reasoning result meets the second reasoning condition; if the first reasoning result meets the second reasoning condition, the cloud server generates a second reasoning result based on the reasoning request through the deployed second AI model; the edge server generates a target reasoning result based on the reasoning request, the first reasoning result and the second reasoning result. Therefore, the present disclosure can obtain stable, reliable, low-latency and high-precision reasoning results through cloud-edge collaborative reasoning, and when the model performance on one side decreases, the reasoning result can be obtained through model reasoning on other sides to avoid reasoning failures, thereby providing services to users in a timely manner and improving user experience.

[0268] like Fig.10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0269] A number of components in the electronic device 1000 are connected to the I / O interface 1005, including: an input unit 1008, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the electronic device 1000 to exchange information / data with other electronic devices through a computer network such as the Internet and / or various telecommunication networks.

[0270] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the IoT AI model reasoning method. For example, in some embodiments, the IoT AI model reasoning method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the IoT AI model reasoning method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the IoT AI model reasoning method in any other appropriate manner (e.g., by means of firmware).

[0271] Various embodiments of the devices and techniques described above herein can be implemented in digital electronic circuit devices, integrated circuit devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), devices on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable device including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage device, at least one input device, and at least one output device, and transmit data and instructions to the storage device, the at least one input device, and the at least one output device.

[0272] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0273] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution device, device, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor devices, devices, or electronic devices, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage electronic device, a magnetic storage electronic device, or any suitable combination of the foregoing.

[0274] To provide interaction with a user, the devices and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0275] The apparatus and techniques described herein may be implemented in a computing device including backend components (e.g., as a data server), or a computing device including middleware components (e.g., an application server), or a computing device including frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the apparatus and techniques described herein), or a computing device including any combination of such backend components, middleware components, or frontend components. The components of the apparatus may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0276] A computer device may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed device, or a server combined with a blockchain.

[0277] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0278] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An IoT AI model reasoning method, characterized in that: Applied to an inference framework, the inference framework includes a cloud server, an edge server and an IoT terminal, the cloud server is connected to at least one edge server, the edge server is connected to at least one IoT terminal, and the cloud server and the edge server are both deployed with corresponding AI models; the method includes: The Internet of Things terminal collects data and determines whether the Internet of Things terminal meets the first reasoning condition; If the IoT terminal meets the first reasoning condition, generating a reasoning request based on the type of the IoT terminal and the collected data; The edge server generates a first reasoning result by reasoning based on the reasoning request through the deployed first AI model, and determines whether the first reasoning result meets a second reasoning condition; If the first reasoning result satisfies the second reasoning condition, the cloud server generates a second reasoning result based on the reasoning request by using the deployed second AI model; The edge server generates a target inference result based on the inference request, the first inference result, and the second inference result.

2. The method according to claim 1, characterized in that The determining whether the Internet of Things terminal satisfies the first reasoning condition includes: Determining whether the IoT terminal belongs to a preset type; If the Internet of Things terminal belongs to a preset type, the Internet of Things terminal infers the collected data to obtain a third inference result, and determines whether the third inference result meets a threshold condition; If the third reasoning result does not meet the threshold condition, determining that the Internet of Things terminal meets the first reasoning condition; If the IoT terminal does not belong to the preset type, determining whether the IoT terminal meets the access condition; If the Internet of Things terminal meets the access condition, it is determined that the Internet of Things terminal meets the first reasoning condition.

3. The method according to claim 1, characterized in that The edge server generates a target reasoning result based on the reasoning request, the first reasoning result, and the second reasoning result, including: The edge server determines at least one fused inference result for inference result fusion based on the inference request, the first inference result, and the second inference result; Determining the weight coefficient of each fusion reasoning result; A target reasoning result is generated according to each fused reasoning result and a weight coefficient of each fused reasoning result.

4. The method according to claim 1, characterized in that The method further comprises: The edge server obtains a first data set from the IoT terminal, wherein the first data set includes inference data; The edge server obtains a second data set, wherein the second data set includes training data; The edge server determines whether there is a difference between the data distribution of the first data set and the data distribution of the second data set; If it is determined that there is a difference, annotating the first data set, and sending the annotated data and retraining instruction information to the connected cloud server; After receiving the retraining instruction information, the cloud server determines a retraining mode and performs retraining based on the retraining mode.

5. The method according to claim 4, characterized in that The edge server determines whether there is a difference between the data distribution of the first data set and the data distribution of the second data set, including: Adding corresponding labels to the first data set and the second data set respectively, and merging the first data set with the added labels with the second data set to obtain a third data set; Based on the third data set, classify the third data set by using a preset binary classification algorithm set to obtain a classification effect of each algorithm in the preset binary classification algorithm set; Based on the classification effects of the algorithms, determining the accuracy of the preset binary classification algorithm set; Based on the accuracy, it is determined whether there is a difference between the data distribution of the first data set and the data distribution of the second data set.

6. The method according to claim 5, characterized in that The step of determining the accuracy of the preset binary classification algorithm set based on the classification effect of each algorithm includes: Based on the classification effect of each algorithm, a result matrix is ​​determined by each evaluation indicator in the evaluation indicator set, wherein each row of the result matrix represents each binary classification algorithm, and each column of the result matrix represents the indicator value of each evaluation indicator corresponding to each binary classification algorithm; Determining a target evaluation indicator from the evaluation indicator set; Based on the result matrix, determining the accuracy of the target evaluation indicator; The accuracy of the target evaluation indicator is determined as the accuracy of the preset binary classification algorithm set.

7. The method according to claim 4, characterized in that After receiving the retraining instruction information, the cloud server determines a retraining mode and performs retraining based on the retraining mode, including: After receiving the retraining instruction information, the cloud server determines a target number of edge servers that send the retraining instruction information; If the target quantity is greater than the quantity threshold, determining that the retraining mode is the first retraining mode, and performing retraining based on the first retraining mode; If the target number is less than or equal to the number threshold, the retraining mode is determined to be the second retraining mode, and retraining is performed based on the second retraining mode.

8. The method according to claim 7, characterized in that The retraining based on the first retraining mode includes: Determining first training data based on the received labeled data; The cloud server retrains the second AI model based on the first retraining data, generates a target model file of the latest version of the second AI model based on the retrained second AI model, and pushes the target model file to the connected edge server; The edge server connected to the cloud server updates the first AI model based on the target model file.

9. The method according to claim 7, characterized in that The retraining based on the second retraining mode includes: Determine second training data based on the labeled data; The edge server that sends the retraining indication information retrains its respective first AI model based on the second retraining data, and performs model inference based on the retrained first AI model.

10. An IoT AI model reasoning device, characterized in that: Applied to an inference framework, the inference framework includes a cloud server, an edge server and an IoT terminal, the cloud server is connected to at least one edge server, the edge server is connected to at least one IoT terminal, and the cloud server and the edge server are both deployed with corresponding AI models; the device includes: A determination module, used for the Internet of Things terminal to collect data and determine whether the Internet of Things terminal meets the first reasoning condition; A first generating module, configured to generate an inference request based on the type of the Internet of Things terminal and the collected data if the Internet of Things terminal meets the first inference condition; A first reasoning module, configured for the edge server to generate a first reasoning result by reasoning through a deployed first AI model based on the reasoning request, and to determine whether the first reasoning result meets a second reasoning condition; A second reasoning module, configured to generate a second reasoning result by the cloud server through a deployed second AI model based on the reasoning request if the first reasoning result satisfies the second reasoning condition; The second generating module is used for the edge server to generate a target reasoning result based on the reasoning request, the first reasoning result and the second reasoning result.

Images