AI model dynamic optimization and adaptive reasoning method based on client

Abstract

The invention discloses an AI model dynamic optimization and adaptive reasoning method based on a client, and relates to the technical field of model optimization, hardware resources and an operation environment of client equipment are analyzed, then a proper AI model is selected and deployed to the client equipment, and then operation state information of the client equipment is monitored in real time. Therefore, the AI model is dynamically optimized and adjusted to achieve the purpose of efficient reasoning, an adaptive algorithm is obtained through analysis according to the hardware resource information of the client equipment, the resource occupation amount of each reasoning path and the reasoning accuracy rate, the client equipment can adjust the optimal reasoning path in real time according to the adaptive algorithm, and the reasoning efficiency is improved. Therefore, the optimal reasoning performance and efficiency are realized under the condition of limited resources, and finally, each processed task and the corresponding AI model and the optimal reasoning path are stored to the cloud for subsequent direct calling.

Description

Technical Field

[0001] This application relates to the field of model optimization technology, and specifically to a client-based AI model dynamic optimization and adaptive reasoning method. Background Technology

[0002] With the development of mobile devices and edge computing, AI models are gradually being transferred from the cloud to client devices for processing. However, client devices are limited in computing resources, memory, and battery life. How to efficiently run complex AI models under these constraints has become a challenge. Therefore, this application proposes a client-based AI model dynamic optimization and adaptive reasoning method.

[0003] Existing technology, such as the invention application patent with announcement number: CN114035936A, discloses a multi-dimensional parallel processing system and method based on artificial intelligence. Through data parallelism, it automatically manages the data to be processed and distributes the data to be processed to the hardware processor; through sequence parallelism, it divides and distributes the data to be processed and puts each data to be processed into multiple processors; through pipeline parallelism, it divides the model into multiple segments, each segment is deployed on a different hardware processor, and is connected in series according to the model sequence; and through multi-dimensional model parallelism, it performs network model partitioning on the training model of the data to be processed that is scheduled to the processor, and schedules the training model to multiple processors. The optimizer updates the parameters of the model to complete the training process. In the inference process, the above-mentioned resource scheduling and multi-dimensional parallel technology are also used. By introducing multi-dimensional parallel processing in the AI ​​model training and inference process, the consumption of computing resources by AI is reduced, the efficiency of AI deployment is improved, and the deployment cost is minimized.

[0004] For the above solution, there are the following technical problems: 1. The current technology mainly introduces multi-dimensional parallel processing in the AI ​​model training and reasoning process to reduce AI's consumption of computing resources, so as to improve the efficiency of artificial intelligence deployment and minimize the deployment cost. It does not take into account that when the hardware resources of different user-end devices vary greatly, it is impossible to support multiple hardware processors to deploy the model. It also does not take into account the problem that the user-end device dynamically updates the model according to the load and resource changes of the device during operation. Ignoring the above aspects will cause the AI ​​model to be unable to adapt to the resource limitations of the client device, and thus cannot achieve the ideal processing speed and processing effect.

[0005] 2. The current technology lacks real-time monitoring of the operating status of user-side devices during task processing, and thus lacks analysis of the optimal inference path of the model based on the real-time monitoring of the operating status. The above-mentioned neglect results in the inability of the AI ​​model to automatically adjust according to the hardware resources and operating status of the device, and thus cannot guarantee the maximization of the AI ​​model's inference performance and work efficiency. Summary of the invention

[0006] The purpose of this application is to provide a client-based AI model dynamic optimization and adaptive reasoning method to solve the problems existing in the background technology.

[0007] To solve the above technical problems, this application adopts the following technical solutions: This application provides a client-based AI model dynamic optimization and adaptive reasoning method, including: Step 1, model selection: analyzing the hardware resource information and operating environment information of the user-end device, and then selecting the AI ​​model to deploy to the user-end device; Step 2: Model and reasoning path optimization: Real-time monitoring and analysis of the operating status information of the user-end device, and then optimization of the AI ​​model, and analysis to obtain an adaptive algorithm to select the optimal reasoning path.

[0008] Step 3, Task Matching: Store all executed tasks of the user-side device in the cloud. When a task to be executed appears on the user-side device, match the task to be executed with the executed tasks in the cloud, obtain the matching task of the task to be executed, and then directly call the AI ​​model and optimal reasoning path of the matching task to process the task to be executed.

[0009] Preferably, the hardware resource information of the user terminal device includes the available CPU resources, available GPU resources, available memory resources and network status of the user terminal device, and the operating environment information includes the battery power and network bandwidth of the user terminal device.

[0010] Preferably, the resource demand assessment coefficient of each AI model is analyzed based on the basic operation information of the AI ​​model, and the resource demand assessment coefficient of each AI model is compared with the available resource assessment coefficient of the user-end device, thereby obtaining the ratio of the resource demand assessment coefficient of each AI model to the available resource assessment coefficient of the user-end device, and obtaining each AI model corresponding to the ratio less than 1, recording it as each selectable AI model, and obtaining the inference accuracy of each selectable AI model from the cloud, arranging the inference accuracy of each selectable AI model in order from large to small, and selecting the AI ​​model with the highest inference accuracy to deploy to the user-end device.

[0011] Preferably, the analysis obtains an adaptive algorithm, and the specific process is as follows: A1, based on the hardware resource information of the user terminal device, the available CPU resource amount, available GPU resource amount, available memory resource amount and network status of the user terminal device are obtained, and based on the basic information of each reasoning path, the path number, CPU resource occupancy, GPU resource occupancy, memory resource occupancy and network dependency status of each reasoning path are obtained.

[0012] A2. Traverse each inference path. When the CPU resource occupancy, GPU resource occupancy, and memory resource occupancy of a certain inference path are less than or equal to the available CPU resource, GPU resource, and memory resource occupancy of the user-side device, and the network dependency state of the inference path conforms to the network state of the user-side device, the inference path is recorded as an available inference path. Based on this, all available inference paths in the user-side device are obtained and recorded as available inference paths; the CPU resource occupancy, GPU resource occupancy, and memory resource occupancy in each available inference path are added together to obtain the total resource occupancy of each available path.

[0013] A3. Normalize the total resource usage and inference accuracy of each available inference path and record them as and , where is the number corresponding to each available reasoning path, , is any integer greater than 2, according to the calculation formula: Analyze and obtain the preference index of each available reasoning path , where and Represented as the weight factor corresponding to the total resource usage of the inference path and the weight factor corresponding to the inference accuracy.

[0014] Preferably, the specific process of selecting the optimal reasoning path is as follows: sort the priority indexes of the available reasoning paths in descending order, construct a priority selection table of available reasoning paths of the AI ​​model, and record the available reasoning path corresponding to the maximum value of the priority index as the optimal reasoning path.

[0015] Preferably, the task to be executed is matched with each executed task in the cloud to obtain the matching task of the task to be executed. The specific process is as follows: Obtain a function description set of the task to be executed from the user terminal device and resource requirement type collection ; Get the functional description set of each executed task from the cloud and resource requirement type collection , where is the number corresponding to each executed task, , is any integer greater than 2, according to the calculation formula: Analyze the compatibility between the tasks to be executed and the tasks that have been executed , where and Respectively represent the weight factors corresponding to the function matching and resource requirement type matching when the user-side device performs tasks.

[0016] Compare the obtained fitness of the task to be executed and each executed task with the set task fitness threshold. When the fitness of the task to be executed and a certain executed task is greater than or equal to the set task fitness threshold, it is judged that the task to be executed and the executed task are successfully matched. Otherwise, it is judged that the task to be executed and the executed task are not successfully matched. Based on this, each executed task that successfully matches the task to be executed is obtained, and the executed task with the highest fitness with the task to be executed is recorded as the matching task of the task to be executed.

[0017] The beneficial effects of this application are: 1. This application provides a client-based AI model dynamic optimization and adaptive reasoning method, which analyzes the hardware resources and operating environment of the user-side device, selects a suitable AI model to deploy to the user-side device, and then monitors the operating status information of the user-side device in real time, so as to dynamically optimize and adjust the AI ​​model to achieve efficient reasoning. At the same time, according to the hardware resource information of the user-side device, the resource occupancy of each reasoning path and the reasoning accuracy, an adaptive algorithm is analyzed. According to the adaptive algorithm, the user-side device can adjust the optimal reasoning path in real time, thereby ensuring the best reasoning performance and efficiency under limited resource conditions. Finally, each processed task and its corresponding AI model and optimal reasoning path are stored in the cloud for subsequent direct call.

[0018] 2. This application lays the foundation for the subsequent selection of AI models by analyzing the hardware resource information and operating environment information of the user-side device. The hardware resources of different user-side devices vary greatly, which will lead to large differences in the speed and effect of running AI models on different client devices. Therefore, by analyzing the hardware resource information and operating environment information of the user-side device, the appropriate AI model can be selected and deployed to the user-side device, thereby ensuring the running speed and effect of the AI ​​model.

[0019] 3. This application monitors and analyzes the operating status information of the user-end device after the AI ​​model is deployed, and then optimizes the AI ​​model, thereby adjusting the computational complexity and complexity of the AI ​​model in real time according to the operating status information of the user-end device, ensuring that a high-precision large model runs on a user-end device with strong hardware resources, and switching to a lightweight model on a user-end device with limited resources to ensure the efficiency of reasoning, which also greatly improves the flexibility of AI model deployment on the user-end device.

[0020] 4. This application selects the optimal reasoning path by analyzing the hardware resources of the user-end device, the resource requirements of each reasoning path, and the reasoning accuracy, and adjusts the optimal reasoning path in real time according to the changes in the hardware resources of the user-end device to ensure the efficiency and stability of the AI ​​model reasoning process.

[0021] 5. This application saves each executed task of the user-side device to the cloud, and when the user-side device receives a new unexecuted task, matches the unexecuted task with the executed task, obtains the matching task of the unexecuted task, and then directly calls the AI ​​model and optimal reasoning path of the matching task to process the task to be executed, which greatly reduces the workload of the user-side device and the waste of resources caused by unnecessary analysis, thereby ensuring the perfection and flexibility of the method. Brief Description of the Figures

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flowchart of the implementation steps of this application method. Specific implementation method

[0024] The following will combine the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] Reference Figure 1 As shown, the present application provides a client-based AI model dynamic optimization and adaptive reasoning method, including the following steps: Step 1, model selection: analyzing the hardware resource information and operating environment information of the user-end device, and then selecting the AI ​​model to deploy to the user-end device.

[0026] It should be noted that user-end devices include smart phones, smart homes and wearable devices.

[0027] In a specific example, the hardware resource information of the client device includes the available CPU resources, available GPU resources, available memory resources and network status of the client device, and the operating environment information includes the battery power and network bandwidth of the client device.

[0028] In a specific example, the hardware resources and operating environment of the user terminal device are analyzed, and the specific analysis process is as follows: based on the hardware resource information of the user terminal device, the available CPU resources, available GPU resources and available memory resources of the user terminal device are extracted, and then the hardware resource evaluation coefficient of the user terminal device is obtained by analysis .

[0029] Based on the operating environment information of the user-end device, the battery power and network bandwidth of the user-end device are extracted, and then the operating environment evaluation coefficient of the user-end device is obtained by analysis .

[0030] Comprehensive user terminal equipment hardware resource evaluation coefficient and operating environment assessment coefficient , according to the calculation formula: Analyze and obtain the available resource evaluation coefficient of the user-side equipment , where and Respectively represent the weight factor corresponding to the hardware resource evaluation coefficient of the user-end device and the weight factor corresponding to the operating environment evaluation coefficient.

[0031] It should be noted that , , .

[0032] It should be noted that the weight factors corresponding to the hardware resource evaluation coefficients of the user-end device and the weight factors corresponding to the operating environment evaluation coefficients are obtained through factor analysis. First, the information of the spatial path attenuation of the hardware resource evaluation coefficients of the user-end device and the operating environment evaluation coefficients is concentrated, and then the variance explanation rate after rotation is obtained, and the weight is obtained by dividing the cumulative variance explanation rate.

[0033] It should be noted that factor analysis is a well-known technology. It is a multivariate statistical analysis method that starts from studying the internal dependencies of variables and reduces some variables with complex relationships to a few comprehensive factors; information concentration is expressed as calculating the median; the variance explanation rate is the amount of information extracted by the factor, and the variance explanation rate = characteristic root / total number of analysis items; the variance explanation rate after rotation is expressed as the variance explanation rate of the factor after the maximum variance rotation.

[0034] In a specific example, the analysis obtains the hardware resource evaluation coefficient of the user terminal device and operating environment assessment coefficient , the specific analysis process is as follows: the available CPU resources, GPU resources and memory resources of the user-side device are normalized and recorded as 、 and , according to the calculation formula: Analyze and obtain the hardware resource evaluation coefficient of the user-side device , where 、 and They represent the weight factors corresponding to the available amount of CPU resources, the weight factors corresponding to the available amount of GPU resources, and the weight factors corresponding to the available amount of memory resources. Similarly, the operating environment evaluation coefficient of the user-side device can be obtained by analyzing the battery power and network bandwidth of the user-side device. .

[0035] It should be noted that , , , , where 、 and Setting method and 、 The same, so I won't repeat it.

[0036] In a specific example, the AI ​​model is selected and deployed to the user-end device. The specific process is as follows: the resource demand assessment coefficient of each AI model is analyzed based on the basic operation information of the AI ​​model, and the resource demand assessment coefficient of each AI model is compared with the available resource assessment coefficient of the user-end device, thereby obtaining the ratio of the resource demand assessment coefficient of each AI model to the available resource assessment coefficient of the user-end device, obtaining each AI model corresponding to the ratio less than 1, recording it as each selectable AI model, and obtaining the inference accuracy of each selectable AI model from the cloud, arranging the inference accuracy of each selectable AI model in order from large to small, and selecting the AI ​​model with the highest inference accuracy to deploy to the user-end device.

[0037] It should be noted that the basic operation information of the AI ​​model includes the CPU resource usage, GPU resource usage and memory resource usage. The resource demand evaluation coefficient analysis method of the AI ​​model is the same as the hardware resource evaluation coefficient analysis method of the user-end device, so it will not be repeated here.

[0038] Step 3: Model and reasoning path optimization: Real-time monitoring and analysis of the operating status of the user-end device, and then optimization of the AI ​​model, and analysis to obtain an adaptive algorithm to select the optimal reasoning path.

[0039] In a specific example, the real-time monitoring and analysis of the operating status information of the user terminal device is as follows: the CPU temperature, CPU load and battery power consumption per unit time of the user terminal device at the current moment are obtained from the monitored operating data of the user terminal device, and are recorded as 、 and , according to the calculation formula: Analyze and obtain the current operating status evaluation coefficient of the user terminal equipment , where 、 ​and It represents the standard value of CPU temperature, CPU load and battery power consumption per unit time of the user-side device at the current moment. 、 and Represented respectively as the weight factor corresponding to the CPU temperature of the user-end device at the current moment, the weight factor corresponding to the CPU load, and the weight factor corresponding to the battery power consumption per unit time.

[0040] Compare the operating status evaluation coefficient of the user-end device with the threshold of the operating device evaluation coefficient of the user-end device. When the operating status evaluation coefficient of the user-end device is greater than or equal to the threshold of the operating status evaluation coefficient of the user-end device, optimize the AI ​​model; otherwise, do not optimize the AI ​​model.

[0041] It should be noted that the operating data of the user-end device includes CPU temperature, CPU load and battery power consumption per unit time.

[0042] It should be noted that the standard values ​​of the CPU temperature, CPU load and battery power consumption per unit time of the user-side device at the current moment are the CPU temperature, CPU load and battery power consumption per unit time of the user-side device before the AI ​​model is deployed.

[0043] It should be noted that , , , , where 、 and Setting method and 、 The same, so I won't repeat it.

[0044] It should be noted that the threshold of the user-side operating device evaluation coefficient is the user-side device operating evaluation coefficient obtained by analyzing the CPU available resources, GPU available resources, memory available resources, battery power or network bandwidth of the user-side device when it is insufficient to support the continuous operation of the AI ​​model.

[0045] In a specific example, the optimization of the AI ​​model is performed as follows: Obtaining each data in the AI ​​model , record the floating point range of each data as , and record the floating point range of each data in the optimized AI model as , set the quantization step size to , according to the calculation formula: Analyze the optimized data , where is the number corresponding to each data in the AI ​​model,​ , is any integer greater than 2, indicates pair numbered The data is rounded.

[0046] It should be noted that the storage space and computational complexity of the AI ​​model are reduced by rounding off the data in the AI ​​model.

[0047] In a specific example, the analysis obtains an adaptive algorithm, and the specific process is as follows: A1, based on the hardware resource information of the user-end device, the available CPU resource amount, available GPU resource amount, available memory resource amount and network status of the user-end device are obtained, and based on the basic information of each reasoning path, the path number, CPU resource occupancy, GPU resource occupancy, memory resource occupancy and network dependency status of each reasoning path are obtained.

[0048] A2. Traverse each inference path. When the CPU resource occupancy, GPU resource occupancy, and memory resource occupancy of a certain inference path are less than or equal to the available CPU resource, GPU resource, and memory resource occupancy of the user-side device, and the network dependency state of the inference path conforms to the network state of the user-side device, the inference path is recorded as an available inference path. Based on this, all available inference paths in the user-side device are obtained and recorded as available inference paths; the CPU resource occupancy, GPU resource occupancy, and memory resource occupancy in each available inference path are added together to obtain the total resource occupancy of each available path.

[0049] A3. Normalize the total resource usage and inference accuracy of each available inference path and record them as and , where is the number corresponding to each available reasoning path, , is any integer greater than 2, according to the calculation formula: Analyze and obtain the preference index of each available reasoning path , where and Represented as the weight factor corresponding to the total resource usage of the inference path and the weight factor corresponding to the inference accuracy.

[0050] It should be noted that the network status is divided into disconnected and connected, and the network dependency status is divided into network dependent and network independent.

[0051] It should be noted that the network dependency state of the inference path is consistent with the network state of the user-end device, which means that the network dependency state of the inference path is connected and the network state of the user-end device is connected, or the network dependency state of the inference path is disconnected and the network state of the user-end device is any state.

[0052] It should be noted that the inference accuracy of each available inference path is obtained from the cloud.

[0053] It should be noted that , , . Among them and Setting method and 、 The same, so I won't repeat it.

[0054] In a specific example, the specific process of selecting the optimal reasoning path is as follows: sort the priority indexes of the available reasoning paths from large to small, construct a priority table of available reasoning paths for the AI ​​model, and record the available reasoning path corresponding to the maximum value of the priority index as the optimal reasoning path.

[0055] It should be noted that the optimal inference path is updated in real time based on the available CPU resources, GPU resources, memory resources and network status of the user-end device.

[0056] Step 3, Task Matching: Store all executed tasks of the user-side device in the cloud. When a task to be executed appears on the user-side device, match the task to be executed with the executed tasks in the cloud, obtain the matching task of the task to be executed, and then directly call the AI ​​model and optimal reasoning path of the matching task to process the task to be executed.

[0057] In a specific example, the task to be executed is matched with each executed task in the cloud to obtain the matching task of the task to be executed. The specific process is as follows: Obtain a function description set of the task to be executed from the user terminal device and resource requirement type collection ; Get the functional description set of each executed task from the cloud and resource requirement type collection , where is the number corresponding to each executed task, , is any integer greater than 2, according to the calculation formula: Analyze the compatibility between the tasks to be executed and the tasks that have been executed , where and Respectively represent the weight factors corresponding to the function matching and resource requirement type matching when the user-side device performs tasks.

[0058] Compare the obtained fitness of the task to be executed and each executed task with the set task fitness threshold. When the fitness of the task to be executed and a certain executed task is greater than or equal to the set task fitness threshold, it is judged that the task to be executed and the executed task are successfully matched. Otherwise, it is judged that the task to be executed and the executed task are not successfully matched. Based on this, each executed task that successfully matches the task to be executed is obtained, and the executed task with the highest fitness with the task to be executed is recorded as the matching task of the task to be executed.

[0059] It should be noted that , , . Among them and Setting method and 、 The same, so I won't repeat it.

[0060] This application provides a client-based AI model dynamic optimization and adaptive reasoning method, which analyzes the hardware resources and operating environment of the user-side device, selects a suitable AI model to deploy to the user-side device, and then monitors the operating status information of the user-side device in real time, so as to dynamically optimize and adjust the AI ​​model to achieve efficient reasoning. At the same time, according to the hardware resource information of the user-side device, the resource usage of each reasoning path and the reasoning accuracy, an adaptive algorithm is analyzed. According to the adaptive algorithm, the user-side device can adjust the optimal reasoning path in real time, thereby ensuring the best reasoning performance and efficiency under limited resource conditions. Finally, each processed task and its corresponding AI model and optimal reasoning path are stored in the cloud for subsequent direct call.

[0061] The above contents are merely examples and explanations of the concept of the present application. The technical personnel in the relevant technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined in the present application, they shall fall within the protection scope of the present application.

Claims

1. A client-based AI model dynamic optimization and adaptive reasoning method, characterized in that: include: Step 1: Model selection: Analyze the hardware resource information and operating environment information of the user-end device, and then select the AI ​​model to deploy to the user-end device; Step 2: Model and reasoning path optimization: Real-time monitoring and analysis of the operating status information of the user-end device, and then optimization of the AI ​​model, and analysis to obtain an adaptive algorithm to select the optimal reasoning path; Step 3: Task matching: Store all executed tasks of the user-side device in the cloud. When a task to be executed appears on the user-side device, match the task to be executed with the executed tasks in the cloud, obtain the matching task of the task to be executed, and then directly call the AI ​​model and optimal reasoning path of the matching task to process the task to be executed.

2. According to the client-based AI model dynamic optimization and adaptive reasoning method according to claim 1, it is characterized in that: The hardware resource information of the user terminal device includes the available CPU resources, available GPU resources, available memory resources and network status of the user terminal device, and the operating environment information includes the battery power and network bandwidth of the user terminal device.

3. According to claim 2, a client-based AI model dynamic optimization and adaptive reasoning method is characterized in that: The hardware resources and operating environment of the user terminal device are analyzed, and the specific analysis process is as follows: Based on the hardware resource information of the user-end device, the available CPU resources, GPU resources and memory resources of the user-end device are extracted, and then the hardware resource evaluation coefficient of the user-end device is obtained through analysis. ; Based on the operating environment information of the user-end device, the battery power and network bandwidth of the user-end device are extracted, and then the operating environment evaluation coefficient of the user-end device is obtained through analysis. ; Comprehensive user equipment hardware resource evaluation coefficient and operating environment assessment coefficient , according to the calculation formula: Analyze and obtain the available resource evaluation coefficient of the user terminal equipment ,in and They are respectively represented as the weight factor corresponding to the hardware resource evaluation coefficient of the user-end device and the weight factor corresponding to the operating environment evaluation coefficient.

4. According to claim 3, a client-based AI model dynamic optimization and adaptive reasoning method is characterized in that: The analysis obtains the hardware resource evaluation coefficient of the user terminal device and operating environment assessment coefficient , the specific analysis process is as follows: The available CPU resources, GPU resources, and memory resources of the client device are normalized and recorded as , and , according to the calculation formula: Analyze and obtain the hardware resource evaluation coefficient of the user-side device ,in , and They represent the weight factors corresponding to the available CPU resources, the available GPU resources, and the available memory resources respectively; similarly, the operating environment evaluation coefficient of the user-side device can be obtained by analyzing the battery power and network bandwidth of the user-side device. .

5. According to claim 3, a client-based AI model dynamic optimization and adaptive reasoning method is characterized in that: The specific process of selecting an AI model to be deployed to a user-end device is as follows: Based on the basic operation information of the AI ​​model, the resource requirement assessment coefficient of each AI model when it is deployed on the user-side device is analyzed, and the resource requirement assessment coefficient of each AI model when it is deployed on the user-side is compared with the available resource assessment coefficient of the user-side device, so as to obtain the ratio of the resource requirement assessment coefficient of each AI model when it is deployed on the user-side to the available resource assessment coefficient of the user-side device, obtain the AI ​​models corresponding to the ratio less than 1, record them as selectable AI models, obtain the inference accuracy of each selectable AI model from the cloud, arrange the inference accuracy of each selectable AI model in order from large to small, and select the AI ​​model with the highest inference accuracy to deploy to the user-side device.

6. According to claim 1, a client-based AI model dynamic optimization and adaptive reasoning method is characterized in that: The specific process of real-time monitoring and analyzing the operation status information of the user terminal equipment is as follows: The CPU temperature, CPU load and battery power consumption per unit time of the user-end device at the current moment are obtained from the monitored operation data of the user-end device, and recorded as , and , according to the calculation formula: Analyze and obtain the current operating status evaluation coefficient of the user terminal equipment ,in , and It is expressed as the standard value of CPU temperature, CPU load and battery power consumption per unit time of the user-side device at the current moment. , and They are respectively represented as the weight factor corresponding to the CPU temperature of the user terminal device at the current moment, the weight factor corresponding to the CPU load, and the weight factor corresponding to the battery power consumption per unit time; The operating status evaluation coefficient of the user-end device is compared with the threshold of the operating status evaluation coefficient of the user-end device. When the operating status evaluation coefficient of the user-end device is greater than or equal to the threshold of the operating status evaluation coefficient of the user-end device, the AI ​​model is optimized; otherwise, the AI ​​model is not optimized.

7. According to claim 6, a client-based AI model dynamic optimization and adaptive reasoning method is characterized in that: The specific process of optimizing the AI ​​model is as follows: Get the data in the AI ​​model , record the floating point range of each data as , and record the floating point range of each data in the optimized AI model as , set the quantization step size to , according to the calculation formula: Analyze the optimized data ,in It is the number corresponding to each data in the AI ​​model. , is any integer greater than 2, Indicates that the pair number is The data is rounded.

8. According to claim 7, a client-based AI model dynamic optimization and adaptive reasoning method is characterized in that: The analysis results in an adaptive algorithm, the specific process is as follows: A1. Based on the hardware resource information of the user-end device, obtain the available CPU resource amount, available GPU resource amount, available memory resource amount and network status of the user-end device, and based on the basic information of each reasoning path, obtain the path number, CPU resource occupancy, GPU resource occupancy, memory resource occupancy and network dependency status of each reasoning path; A2. Traverse each reasoning path. When the CPU resource occupancy, GPU resource occupancy, and memory resource occupancy of a certain reasoning path are all less than or equal to the available CPU resource, available GPU resource, and memory resource occupancy of the user-end device, and the network dependency state of the reasoning path conforms to the network state of the user-end device, record the reasoning path as an available reasoning path, and thereby obtain all available reasoning paths in the user-end device, which are recorded as each available reasoning path; add the CPU resource occupancy, GPU resource occupancy, and memory resource occupancy in each available reasoning path to obtain the total resource occupancy of each available path; A3. The total resource usage and inference accuracy of each available inference path are normalized and recorded as and ,in is the number corresponding to each available reasoning path, , is any integer greater than 2, according to the calculation formula: Analyze and obtain the preference index of each available reasoning path ,in and They are respectively expressed as the weight factor corresponding to the total resource usage of the inference path and the weight factor corresponding to the inference accuracy.

9. The client-based AI model dynamic optimization and adaptive reasoning method according to claim 8, characterized in that: The specific process of selecting the optimal reasoning path is as follows: Sort the priority index of each available reasoning path from large to small, build a priority table of available reasoning paths for the AI ​​model, and record the available reasoning path corresponding to the maximum value of the priority index as the optimal reasoning path.

10. The client-based AI model dynamic optimization and adaptive reasoning method according to claim 9, characterized in that: The specific process of matching the tasks to be executed with the executed tasks in the cloud to obtain the matching tasks of the tasks to be executed is as follows: Get the function description set of the tasks to be executed from the user end device and resource requirement type collection ; Get the functional description set of each executed task from the cloud and resource requirement type collection ,in is the number corresponding to each executed task, , is any integer greater than 2, according to the calculation formula: Analyze the compatibility between the tasks to be executed and the tasks that have been executed ,in and They respectively represent the weight factor corresponding to the function matching when the user terminal device performs the task and the weight factor corresponding to the resource requirement type matching; The obtained fitness of the task to be executed and each executed task is compared with the set task fitness threshold. When the fitness of the task to be executed and a certain executed task is greater than or equal to the set task fitness threshold, it is judged that the task to be executed and the executed task are matched successfully. Otherwise, it is judged that the task to be executed and the executed task are matched unsuccessfully. Based on this, each executed task that successfully matches the task to be executed is obtained, and the executed task with the highest fitness with the task to be executed is recorded as the matching task of the task to be executed.

Images