AI task collaborative processing method, device, equipment and storage medium for heterogeneous terminals

Abstract

This invention provides a method, apparatus, device, and storage medium for collaborative AI task processing on heterogeneous terminals. The method decouples AI inference tasks into three stages: preprocessing, model inference, and post-processing. The terminal device obtains current resource availability information and task type, generating a three-bit binary operation instruction, with each bit corresponding to a stage, indicating whether that stage should be executed locally on the terminal. The terminal executes the local task according to the instruction and constructs a request data packet containing the data to be processed, operation instructions, and processing parameters, sending it to the server. Upon receiving the request, the server performs bitwise inversion and correction operations on the instruction, determines and executes the remaining stage tasks, and returns an inference result packet to the terminal. The terminal parses the packet to obtain the final inference result. This invention achieves fine-grained dynamic scheduling of edge-cloud tasks, optimizes the utilization of heterogeneous computing resources, reduces end-to-end latency, and improves system flexibility and energy efficiency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for collaborative AI task processing on heterogeneous terminals. Background Technology

[0002] Currently, with the rapid development of artificial intelligence technology, AI inference applications have been widely deployed on various terminal devices. Typical inference tasks usually include three stages: preprocessing (such as image decoding, normalization, and resizing), model inference (such as neural network computation), and post-processing (such as result parsing, coordinate mapping, and confidence filtering). Existing solutions often deploy these three stages as a whole on a single device, for example, completing them all on the terminal's CPU, GPU, or NPU, or offloading the complete model to a cloud server for processing. However, this coarse-grained task allocation method has the following shortcomings: First, different hardware units have significant differences in processing efficiency for computational tasks, and forcing uniform execution can easily lead to a waste of computing resources, such as using the CPU to process model inference suitable for NPU acceleration, or using the NPU to process simple image scaling tasks; second, non-model inference stages may become performance bottlenecks. If the terminal CPU load is too high, even if the model inference speed is fast, the overall end-to-end latency is still limited by the efficiency of preprocessing or post-processing; third, existing solutions lack the ability to perceive and dynamically adapt to the real-time status of the terminal, and cannot flexibly adjust task allocation strategies according to the terminal's current computing load, power consumption, or network conditions. Some edge-cloud collaborative solutions attempt to optimize by unloading the entire model or splitting the network layer, but they have not yet independently scheduled the preprocessing and postprocessing stages. The task division granularity is still relatively coarse, making it difficult to achieve optimal computing collaboration and resource utilization in heterogeneous terminal environments.

[0003] In summary, the shortcomings of the existing technology urgently need to be addressed. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for collaborative AI task processing on heterogeneous terminals, in order to overcome the deficiencies in the prior art, enable tasks at each stage to be executed on the most suitable computing unit, and eliminate the performance bottleneck of non-model inference.

[0005] This invention provides a method for collaborative AI task processing on heterogeneous terminals, comprising: Upload the data to be processed to the terminal device; The system obtains the current resource availability information of the terminal device, and generates a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and executes the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The terminal device constructs a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction, and sends the request data packet to the server. The server determines the tasks to be executed based on the first operation instruction and executes the tasks to be executed to obtain processing result data. The tasks to be executed include the preprocessing stage task, the model inference stage task, and the postprocessing stage task. The server constructs an inference result package based on the processing result data and returns the inference result package to the terminal device. The final AI inference result is determined using the terminal device based on the inference result package.

[0006] According to the AI ​​task collaborative processing method for heterogeneous terminals provided by the present invention, the step of obtaining the current resource availability information of the terminal device and generating a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed specifically includes: Based on the task type, determine the computational resource requirements of the preprocessing stage task, the model inference stage task, and the postprocessing stage task, respectively. The computing resource requirements are matched with the current resource availability information to determine whether each stage of the task should be executed locally by the terminal device. Based on the stage task determined to be executed locally, a corresponding N-bit binary code is generated as the first operation instruction.

[0007] According to the AI ​​task collaborative processing method for heterogeneous terminals provided by the present invention, the step of obtaining the current resource availability information of the terminal device and generating a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed further includes: Based on a preset privacy policy, determine whether the data to be processed contains sensitive information; If sensitive information is contained, the preprocessing stage task will be forcibly set to be executed locally on the terminal device, so that the original sensitive data can be de-identified locally before the processed intermediate data is uploaded to the server.

[0008] According to the AI ​​task collaborative processing method for heterogeneous terminals provided by the present invention, the step of determining the task to be executed based on the first operation instruction through the server and executing the task to be executed specifically includes: The server performs a bitwise NOT operation on the parsed first operation instruction to obtain a first server instruction, which is used to determine the task to be executed in the next stage. The server performs a bitwise OR operation between the first server instruction and a preset baseline instruction to obtain a second server instruction; wherein the baseline instruction is used to ensure that the server executes at least the preprocessing stage task and the model inference stage task. The server executes the task to be performed according to the instructions of the second server.

[0009] According to the present invention, an AI task collaborative processing method for heterogeneous terminals, prior to the step of constructing a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction through the terminal device, and sending the request data packet to the server, the method further includes: Real-time monitoring of the resource utilization rate of the terminal device; If the resource utilization rate exceeds a preset threshold, the first operation instruction is modified to change at least one stage that was originally executed locally by the terminal device to not be executed locally by the terminal device, and the request data packet is updated according to the modified first operation instruction.

[0010] According to the AI ​​task collaborative processing method for heterogeneous terminals provided by the present invention, the step of determining the final AI inference result through the terminal device based on the inference result package specifically includes: Perform a bitwise inversion operation on the server-side operation instructions in the inference result package to obtain the second operation instruction; If the bit value corresponding to the post-processing stage task in the new terminal instruction is a first preset value, then the post-processing parameters in the inference result package are used to post-process the processing result data to obtain the final AI inference result. If the bit in the new terminal instruction corresponding to the post-processing stage task takes the value of the second preset value, then the processing result data is directly used as the final AI inference result.

[0011] According to the present invention, an AI task collaborative processing method for heterogeneous terminals is provided, wherein the binary encoding is a three-bit binary encoding; The least significant bit corresponds to the preprocessing stage, the middle bit corresponds to the model inference stage, and the most significant bit corresponds to the postprocessing stage. When a certain bit is set to a first preset value, it indicates that the corresponding stage is executed locally by the terminal device; when it is set to a second preset value, it indicates that the corresponding stage is not executed locally by the terminal device.

[0012] The present invention also provides an AI task collaborative processing device for heterogeneous terminals, comprising: The data upload module is used to upload data to be processed to the terminal device; The terminal processing module is used to obtain the current resource availability information of the terminal device, and generate a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and execute the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The data forwarding module is used to construct a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction through the terminal device, and send the request data packet to the server. The server processing module is used to determine the task to be executed based on the first operation instruction through the server, and execute the task to be executed to obtain processing result data. The task to be executed includes the preprocessing task, the model inference task, and the postprocessing task. The feedback receiving module is used to construct an inference result package based on the processing result data through the server, and return the inference result package to the terminal device; The result generation module is used to determine the final AI inference result based on the inference result package through the terminal device.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the AI ​​task collaborative processing method for heterogeneous terminals as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the AI ​​task collaborative processing method for heterogeneous terminals as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the AI ​​task collaborative processing method for heterogeneous terminals as described above.

[0016] The AI ​​task collaborative processing method, apparatus, device, and storage medium for heterogeneous terminals provided by this invention introduces a fine-grained task partitioning mechanism based on binary encoding. This mechanism dynamically generates a first operation instruction based on the terminal device's current resource availability and the task type of the data to be processed. The terminal device then executes the corresponding stage task, while the remaining stage tasks are encapsulated into request data packets and sent to the server for processing. Finally, the terminal device determines the final AI inference result based on the inference result packet returned by the server. This method decouples the preprocessing, model inference, and post-processing stages, enabling flexible collaboration between the edge and cloud, and significantly improving resource utilization efficiency in heterogeneous environments. The terminal device can retain tasks suitable for local acceleration based on the real-time load status of computing units such as CPU, GPU, and NPU, while offloading resource-intensive or currently heavily loaded tasks to the cloud, thus avoiding wasted computing power and local overload. Simultaneously, this method effectively reduces end-to-end processing latency, ensuring that tasks at each stage are executed on the most suitable computing unit through on-demand allocation, eliminating performance bottlenecks in non-model inference stages. Furthermore, this method enhances the flexibility and compatibility of system deployment. The same cloud service can adaptively access various heterogeneous devices, ranging from low-power IoT devices to high-performance mobile terminals, without requiring customized deployment solutions for each type of terminal. Regarding energy consumption and bandwidth overhead, powerful terminals can handle non-inference processes locally to reduce data transmission volume, while weak terminals can offload all tasks to reduce local power consumption, achieving overall energy efficiency optimization. In particular, this method uses a bitwise inversion and correction mechanism of binary instructions to ensure that the server can accurately accept tasks not executed by the terminal and avoids the server entering an illegal state where no valid tasks can be executed, thereby ensuring the stable and reliable operation of the system. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the AI ​​task collaborative processing method for heterogeneous terminals provided by the present invention. Figure 2 This is a schematic diagram of the structure of the AI ​​task collaborative processing device for heterogeneous terminals provided by the present invention. Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] To address the problems in existing technologies, this invention proposes a collaborative AI task processing method for heterogeneous terminals, enabling tasks at each stage to be executed on the most suitable computing unit, thus eliminating performance bottlenecks in non-model inference stages. The following describes this collaborative AI task processing method for heterogeneous terminals, as follows: Figure 1 As shown, including but not limited to the following steps: Step 110: Upload the data to be processed to the terminal device; In step 110, the data to be processed can be images, video frames, audio clips, text data, etc., depending on the task type of the AI ​​application (such as object detection, speech recognition, text classification). Data can be uploaded from the user terminal or data source to the terminal device via wired or wireless network. The terminal device can be a heterogeneous terminal such as a smartphone, IoT device, or edge computing node.

[0021] Step 120: Obtain the current resource availability information of the terminal device, and generate a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and execute the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, model inference stage task and postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; In step 120, the terminal device first collects its current resource availability information, including but not limited to: CPU real-time load rate, GPU idle computing power ratio, NPU occupancy status, available memory capacity, encoding / decoding unit operating status, current power consumption, and temperature. This information reflects the computing power that the terminal can currently handle. Simultaneously, the terminal device determines the task type of this AI inference based on the type of data to be processed (e.g., face recognition, image classification), thereby clarifying the computing resource requirements for each stage of preprocessing, model inference, and post-processing (e.g., preprocessing may primarily consume CPU or specific hardware acceleration units, the inference stage may require NPU / GPU, and post-processing is mostly lightweight CPU computation).

[0022] Step 130: Using the terminal device, construct a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction, and send the request data packet to the server; In step 130, the terminal encapsulates the data to be processed, the first operation instruction, and the processing parameters required for each stage into a request data packet. The processing parameters include preprocessing parameters (such as target image size, normalized mean / variance, color space conversion parameters) and post-processing parameters (such as category mapping table, confidence threshold, coordinate transformation rules). Depending on the value of the first operation instruction, if a stage is determined to be executed locally by the terminal, the corresponding parameters can be omitted in the request packet to reduce the data volume; if the stage needs to be executed by the server, the corresponding parameters must be included for server processing. The request data packet may also include terminal identifier, timestamp, and other information to facilitate server matching and response.

[0023] Step 140: Through the server, determine the task to be executed based on the first operation instruction, and execute the task to be executed to obtain processing result data. The task to be executed includes the preprocessing task, the model inference task, and the postprocessing task. In step 140, after receiving the request data packet, the server parses it to obtain the first operation instruction, the data to be processed, and related parameters. To determine which stages the server needs to execute, this embodiment uses a bitwise inversion conversion method: the first operation instruction is bitwise inverted to obtain the first server instruction. This instruction indicates the stages that the terminal has not executed but that the server needs to complete. For example, if the first operation instruction is "001" (the terminal only performs preprocessing), inverting it yields "110," meaning the server needs to perform model inference and post-processing.

[0024] Step 150: Construct an inference result package based on the processing result data through the server, and return the inference result package to the terminal device; In step 150, the server generates server-side operation instructions (i.e., the second server instruction or its simplified version) according to the actual execution stage, and encapsulates them together with the processing result data and post-processing parameters (if the server performs post-processing, the processed result can be omitted or carried) into an inference result package, which is then returned to the corresponding terminal device through the original communication channel.

[0025] Step 160: Using the terminal device, determine the final AI inference result based on the inference result package.

[0026] In step 160, after receiving the inference result packet, the terminal first parses it to obtain the server-side operation instructions and processing result data. Then, it inverts the server-side operation instructions bit by bit to obtain the second operation instructions (i.e., the stage that the terminal needs to execute). If the bit corresponding to the post-processing stage in the second operation instructions is "1", it means that the server has not performed post-processing, and the terminal needs to use the post-processing parameters in the inference result packet to post-process the processing result data to obtain the final AI inference result; if the bit is "0", it means that the server has completed post-processing, and the terminal directly outputs the processing result data as the final result.

[0027] Through the above steps, the terminal and server achieve collaborative processing of AI inference tasks. Each stage can be executed on the most suitable computing unit, and the whole process supports dynamic adjustment and privacy protection, which significantly improves resource utilization efficiency and system flexibility in heterogeneous environments.

[0028] As a further optional embodiment, the step of obtaining the current resource availability information of the terminal device and generating a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed specifically includes: Based on the task type, determine the computational resource requirements of the preprocessing stage task, the model inference stage task, and the postprocessing stage task, respectively. The computing resource requirements are matched with the current resource availability information to determine whether each stage of the task should be executed locally by the terminal device. Based on the stage task determined to be executed locally, a corresponding N-bit binary code is generated as the first operation instruction.

[0029] In this embodiment, firstly, the computational resource requirements of the preprocessing stage, the model inference stage, and the postprocessing stage are determined according to the task type. Different AI task types have significantly different computational resource requirements for each stage. For example, in object detection tasks, the preprocessing stage may involve image decoding, size scaling, color space conversion, and normalization operations, which mainly consume CPU computing power or hardware encoding / decoding units; the model inference stage requires a large amount of matrix convolution and parallel computation, typically relying on GPU or NPU computing power; the postprocessing stage includes bounding box parsing, confidence filtering, and non-maximum suppression, mostly CPU logical operations. In text classification tasks, the preprocessing stage may only involve simple word segmentation and encoding, with low resource requirements; the model inference stage may be a lightweight neural network, which can be handled by a CPU; the postprocessing stage is only category mapping, with negligible resource requirements. The terminal device can read the computational resource requirement parameters for each stage from a locally pre-configured configuration file or dynamically obtain them from the server, including the required computing power type, estimated computational load, and memory usage, according to the task type.

[0030] Subsequently, the computational resource requirements are matched with the current resource availability information to determine whether each stage of the task should be executed locally by the terminal device. The terminal compares the resource requirements of each stage with the currently collected resource availability information item by item, and performs matching based on preset judgment rules: if the type of resource required by a certain stage has sufficient free capacity on the current terminal, and the estimated computational load is lower than the threshold of available computing power, then the stage is determined to be suitable for local execution; conversely, if the required resources are currently heavily occupied, or the available capacity is insufficient to support the task requirements, then the stage is determined to be unsuitable for local execution and should be handled by the server. For certain special tasks, even if resources are sufficient, offloading can be selected according to preset strategies.

[0031] Finally, based on the stage task determined to be executed locally, a corresponding N-bit binary code is generated as the first operation instruction. If the preprocessing stage is determined to be executed locally, the corresponding bit is set to the first preset value; otherwise, it is set to the second preset value. The same applies to the model inference stage and the postprocessing stage. The values ​​of the three bits are combined into an N-bit binary code, which is the first operation instruction. In this embodiment, N is preferably 3, and the encoding format can be agreed as [postprocessing bit, model inference bit, preprocessing bit] or [preprocessing bit, model inference bit, postprocessing bit], and the specific order can be determined according to the actual system design.

[0032] Through the aforementioned matching mechanism, the terminal can make reasonable local execution decisions based on real-time resource status and task requirements. This avoids performance degradation caused by local resource overload and fully utilizes the terminal's idle computing power, achieving optimal resource utilization. Subsequently, the terminal executes the corresponding stage task according to the generated first operation instruction and proceeds to step 130 for further processing.

[0033] As a further optional embodiment, the step of obtaining the current resource availability information of the terminal device and generating a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed further includes: Based on a preset privacy policy, determine whether the data to be processed contains sensitive information; If sensitive information is contained, the preprocessing stage task will be forcibly set to be executed locally on the terminal device, so that the original sensitive data can be de-identified locally before the processed intermediate data is uploaded to the server.

[0034] In this embodiment, the process of generating the first operation instruction in step 120 may also incorporate a privacy protection mechanism. Specifically, based on a preset privacy policy, it is determined whether the data to be processed contains sensitive information. The sensitive information may include, but is not limited to: facial images, iris data, fingerprint information, medical images, ID card photos, and text data involving personal privacy. The privacy policy can be pre-configured by the system administrator or dynamically set by the user according to data security needs.

[0035] If the data to be processed is determined to contain sensitive information, the preprocessing stage task is forcibly set to be executed locally on the terminal device. Through this mechanism, the terminal device completes the desensitization processing of the original sensitive data locally before uploading the processed intermediate data to the server. Desensitization processing may include, but is not limited to: blurring faces or cropping regions in image data, extracting feature vectors instead of the original image, and anonymizing and replacing text data. For example, in facial recognition applications, the terminal can extract features from the original facial image locally, uploading only the abstract feature vectors to the server for subsequent comparison, thereby avoiding the transmission of the original facial image over the network or storage in the cloud, effectively reducing the risk of privacy leaks.

[0036] Based on this, the terminal continues to generate corresponding instruction bits for other stages (model inference and post-processing) according to resource availability information and task type, ultimately combining them into a complete first operation instruction. If the privacy policy requires full local processing, all stages can be forced to be executed locally; if only the preprocessing stage is required to be executed locally, the remaining stages can still be flexibly allocated according to resource conditions. This embodiment ensures data privacy and security while maximizing the flexibility of edge-cloud collaborative scheduling.

[0037] As a further optional embodiment, the step of determining the task to be executed based on the first operation instruction through the server and executing the task to be executed specifically includes: The server performs a bitwise NOT operation on the parsed first operation instruction to obtain a first server instruction, which is used to determine the task to be executed in the next stage. The server performs a bitwise OR operation between the first server instruction and a preset baseline instruction to obtain a second server instruction; wherein the baseline instruction is used to ensure that the server executes at least the preprocessing stage task and the model inference stage task. The server executes the task to be performed according to the instructions of the second server.

[0038] In this embodiment, after receiving the request data packet sent by the terminal, the server first parses it to obtain the first operation instruction contained therein. This first operation instruction is a binary code generated by the terminal based on its own resource availability information and task type, used to indicate which stages of the task the terminal has already executed locally.

[0039] To determine which stages of tasks the server needs to execute, the server performs a bitwise NOT operation on the parsed first operation instruction to obtain a first server instruction. This first server instruction indicates the stages of tasks the server should execute. For example, if the first operation instruction is "001," indicating that the terminal only performed the preprocessing stage locally, then the bitwise NOT operation yields "110," meaning the server needs to execute both the model inference stage and the post-processing stage. If the first operation instruction is "101," indicating that the terminal performed both preprocessing and post-processing locally, then the bitwise NOT operation yields "010," meaning the server only needs to execute the model inference stage.

[0040] Furthermore, to avoid the server falling into an illegal state of "no valid tasks to execute," for example, when the first operation instruction is "111," the terminal has already executed all three stages. After bitwise inversion, it gets "000," at which point the first server instruction is empty, and the server will have no stage tasks to execute. To solve this problem, this embodiment introduces a correction mechanism: the server performs a bitwise OR operation between the first server instruction and a preset baseline instruction to obtain a second server instruction. The baseline instruction is used to ensure that the server executes at least the preprocessing stage task and the model inference stage task. In this embodiment, it is preferably "011," where the least significant bit is 1 to indicate the execution of preprocessing, the middle bit is 1 to indicate the execution of model inference, and the most significant bit is 0 to indicate that the server is not forced to execute postprocessing. By performing a bitwise OR operation, even if the first server instruction is "000", the result of the OR operation with "011" is "011", ensuring that the server at least performs preprocessing and model inference; if the first server instruction is "010", the result of the OR operation with "011" is "011", the server still performs preprocessing and model inference; if the first server instruction is "110", the result of the OR operation with "011" is "111", the server will perform all three stages.

[0041] Finally, the server executes the corresponding pending stage task according to the second server instruction: if the least significant bit of the second server instruction is 1, the data to be processed is preprocessed using the preprocessing parameters in the request data packet; if the middle bit is 1, the deployed AI model is invoked for inference calculation; if the most significant bit is 1, the inference result is post-processed using the post-processing parameters. Through the above inversion and correction mechanism, the server can accurately take over the unfinished tasks from the terminal while ensuring that it is always in an effective execution state, thus improving the stability and reliability of the system.

[0042] As a further optional embodiment, before the step of constructing a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction through the terminal device, and sending the request data packet to the server, the method further includes: Real-time monitoring of the resource utilization rate of the terminal device; If the resource utilization rate exceeds a preset threshold, the first operation instruction is modified to change at least one stage that was originally executed locally by the terminal device to not be executed locally by the terminal device, and the request data packet is updated according to the modified first operation instruction.

[0043] In this embodiment, before the terminal device constructs and sends the request data packet in step 130, a dynamic load adjustment mechanism can be introduced to address real-time changes in resource usage during terminal device operation. Specifically, after generating the first operation instruction but before actually sending the request data packet, the terminal device continuously or periodically monitors its own resource usage in real time. The resource usage may include, but is not limited to, the central processing unit (CPU) usage, graphics processing unit (GPU) usage, neural network processor (NPU) usage, memory usage, and encoding / decoding unit load rate. The specific monitoring targets can be selected based on the terminal hardware configuration and the characteristics of the current task.

[0044] Terminal devices preset one or more resource utilization thresholds. For example, a CPU utilization rate exceeding 90% is considered a high-load state, or an NPU utilization rate exceeding 80% is considered a busy state. These thresholds can be set by system defaults or dynamically adjusted by the user according to the application scenario. When the utilization rate of any critical resource exceeds the corresponding preset threshold, the terminal determines that the current load is too high. If it continues to execute local tasks as originally planned, it may lead to performance degradation or increased response latency. Therefore, it triggers the instruction dynamic adjustment mechanism.

[0045] During the adjustment process, the terminal device modifies the generated first operation instruction, changing at least one stage task that was originally executed locally on the terminal device to be handled by the server instead. The adjustment strategy can be based on various principles: for example, the stage with the highest computational resource requirements or the most significant contribution to the current load can be prioritized for unloading; alternatively, it can be performed according to a preset unloading priority order. For instance, the post-processing stage is usually less sensitive to latency and has a relatively small computational load, so it can be retained first, while the model inference stage, if it depends on specific hardware (such as the NPU) and that hardware is overloaded, can be unloaded first. Specifically, if the first operation instruction was originally "111" (execute all locally), when the CPU utilization exceeds the threshold, it can be modified to "110" (unload only post-processing), "101" (unload only inference), or "011" (retain only pre-processing); if it was originally "101" (execute both pre-processing and post-processing locally), when the NPU is overloaded, the model inference, which was originally unloaded, can be unloaded if either pre-processing or post-processing also consumes CPU, depending on the situation.

[0046] After modification, the terminal reconstructs the request data packet according to the updated first operation instruction, ensuring that the data to be processed and the processing parameters match the new instruction. For example, if the preprocessing stage is changed from local execution to server execution, the preprocessing parameters that might have been omitted before need to be added to the request data packet; conversely, if a stage is changed from server execution to local execution, the corresponding parameters can be removed from the request data packet to reduce transmission overhead. Subsequently, the terminal sends the updated request data packet to the server, and the subsequent processing flow remains unchanged.

[0047] Through the aforementioned dynamic load adjustment mechanism, the terminal can flexibly adjust the task allocation according to the real-time resource status during runtime, avoiding processing delays or system instability caused by instantaneous load peaks, and further improving the robustness and resource utilization efficiency of end-to-cloud collaboration.

[0048] As a further optional embodiment, the step of determining the final AI inference result based on the inference result package through the terminal device specifically includes: Perform a bitwise inversion operation on the server-side operation instructions in the inference result package to obtain the second operation instruction; If the bit value corresponding to the post-processing stage task in the new terminal instruction is a first preset value, then the post-processing parameters in the inference result package are used to post-process the processing result data to obtain the final AI inference result. If the bit in the new terminal instruction corresponding to the post-processing stage task takes the value of the second preset value, then the processing result data is directly used as the final AI inference result.

[0049] In this embodiment, after receiving the inference result packet returned by the server, the terminal device first parses it to obtain the server-side operation instructions and processing result data contained therein. The server-side operation instructions are binary codes generated by the server according to the actual execution stage, used to indicate which stages of tasks the server has completed. For example, if the server performed model inference and post-processing, the server-side operation instruction may be "110"; if the server only performed model inference, it may be "010".

[0050] To determine whether the terminal needs to perform additional tasks for unfinished stages, the terminal device performs a bitwise NOT operation on the server-side operation instructions in the inference result packet to obtain a second operation instruction. This second operation instruction indicates the stage task that the terminal device needs to continue executing. For example, if the server-side operation instruction is "110", indicating that the server has performed model inference and post-processing, then the bitwise NOT operation yields "001", meaning the terminal needs to perform the pre-processing stage; if the server-side operation instruction is "010", indicating that the server has only performed model inference, then the bitwise NOT operation yields "101", meaning the terminal needs to perform both pre-processing and post-processing.

[0051] Subsequently, the terminal device checks the bit value corresponding to the post-processing stage task in the second operation instruction. According to the encoding rules adopted in this embodiment, this bit can be the most significant bit or the least significant bit, depending on the system convention. If the bit value is a first preset value (e.g., "1"), it indicates that the server has not performed the post-processing stage, and the terminal device needs to perform the post-processing operation. At this time, the terminal device uses the post-processing parameters carried in the inference result package to post-process the processing result data to obtain the final AI inference result. The post-processing parameters may include a category mapping table, confidence threshold, coordinate transformation rules, non-maximum suppression parameters, etc., depending on the task type.

[0052] If the bit in the second operation instruction corresponding to the post-processing stage task is set to a second preset value (e.g., "0"), it indicates that the server has already executed the post-processing stage, and the processed result data in the inference result package is already the complete final result. At this time, the terminal device does not need to perform any additional processing and directly outputs the processed result data as the final AI inference result.

[0053] Through the above mechanism, terminal devices can accurately determine whether they need to supplement the post-processing stage based on the server-side operation instructions, avoiding duplicate processing or missing processing, and ensuring the integrity and correctness of end-to-cloud collaborative processing.

[0054] As a further optional embodiment, the binary encoding is a three-bit binary encoding; The least significant bit corresponds to the preprocessing stage, the middle bit corresponds to the model inference stage, and the most significant bit corresponds to the postprocessing stage. When a certain bit is set to a first preset value, it indicates that the corresponding stage is executed locally by the terminal device; when it is set to a second preset value, it indicates that the corresponding stage is not executed locally by the terminal device.

[0055] In this embodiment, when a certain bit is set to a first preset value, it indicates that the corresponding stage is executed locally by the terminal device; when it is set to a second preset value, it indicates that the corresponding stage is not executed locally by the terminal device. Preferably, the first preset value is "1" and the second preset value is "0". That is, if a bit is "1", it indicates that the task of that stage is executed locally on the terminal; if it is "0", it indicates that the task of that stage is not executed locally on the terminal (i.e., executed by the server).

[0056] Based on the above encoding rules, various combinations of operation instructions can be formed to adapt to different terminal capabilities and task requirements. For example, instruction "001" indicates that the terminal only performs the preprocessing stage locally, while model inference and postprocessing are handled by the server; instruction "010" indicates that the terminal only performs the model inference stage locally, while preprocessing and postprocessing are handled by the server; instruction "100" indicates that the terminal only performs the postprocessing stage locally, while preprocessing and model inference are handled by the server. Instruction "011" indicates that the terminal performs preprocessing and model inference locally, while postprocessing is handled by the server; instruction "101" indicates that the terminal performs preprocessing and postprocessing locally, while model inference is handled by the server; instruction "110" indicates that the terminal performs model inference and postprocessing locally, while preprocessing is handled by the server. Instruction "111" indicates that the terminal performs all three stages locally, i.e., completely local processing; instruction "000" indicates that the terminal does not perform any stages, and all tasks are handled by the server.

[0057] This encoding rule is concise and clear, facilitating generation and parsing by terminal devices, and also enabling bitwise inversion and correction operations by the server. Throughout the edge-cloud collaborative processing flow, the terminal device generates the first operation instruction according to the above rule based on its available resources and task type. Upon receiving the instruction, the server determines the stage it needs to execute by bitwise inversion and ensures at least preprocessing and model inference are performed through a bitwise OR operation with the baseline instruction. Finally, based on the server-side operation instruction returned by the server, the terminal device again determines whether a post-processing stage is needed by bitwise inversion. This three-bit encoding design permeates the entire process, providing a concise and efficient technical foundation for fine-grained task partitioning and dynamic scheduling between the edge and cloud.

[0058] The following describes the AI ​​task collaborative processing device for heterogeneous terminals provided by the present invention, such as... Figure 2 As shown, the AI ​​task collaborative processing device for heterogeneous terminals described below and the AI ​​task collaborative processing method for heterogeneous terminals described above can be referred to in correspondence.

[0059] A heterogeneous terminal AI task collaborative processing device, comprising: Data upload module 210 is used to upload data to be processed to the terminal device; The terminal processing module 220 is used to obtain the current resource availability information of the terminal device, and generate a first operation instruction according to the resource availability information and the task type corresponding to the data to be processed, and execute the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The data forwarding module 230 is used to construct a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction through the terminal device, and send the request data packet to the server. Server processing module 240 is used to determine the task to be executed based on the first operation instruction through the server, and execute the task to be executed to obtain processing result data. The task to be executed includes the preprocessing task, the model inference task, and the postprocessing task. The feedback receiving module 250 is used to construct an inference result package based on the processing result data through the server, and return the inference result package to the terminal device; The result generation module 260 is used to determine the final AI inference result based on the inference result package through the terminal device.

[0060] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a heterogeneous terminal AI task collaborative processing method, which includes: Upload the data to be processed to the terminal device; The system obtains the current resource availability information of the terminal device, and generates a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and executes the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The terminal device constructs a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction, and sends the request data packet to the server. The server determines the tasks to be executed based on the first operation instruction and executes the tasks to be executed to obtain processing result data. The tasks to be executed include the preprocessing stage task, the model inference stage task, and the postprocessing stage task. The server constructs an inference result package based on the processing result data and returns the inference result package to the terminal device. The final AI inference result is determined using the terminal device based on the inference result package.

[0061] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the AI ​​task collaborative processing method for heterogeneous terminals provided by the above methods, the method comprising: Upload the data to be processed to the terminal device; The system obtains the current resource availability information of the terminal device, and generates a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and executes the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The terminal device constructs a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction, and sends the request data packet to the server. The server determines the tasks to be executed based on the first operation instruction and executes the tasks to be executed to obtain processing result data. The tasks to be executed include the preprocessing stage task, the model inference stage task, and the postprocessing stage task. The server constructs an inference result package based on the processing result data and returns the inference result package to the terminal device. The final AI inference result is determined using the terminal device based on the inference result package.

[0063] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the AI ​​task collaborative processing method for heterogeneous terminals provided by the above methods, the method comprising: Upload the data to be processed to the terminal device; The system obtains the current resource availability information of the terminal device, and generates a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and executes the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The terminal device constructs a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction, and sends the request data packet to the server. The server determines the tasks to be executed based on the first operation instruction and executes the tasks to be executed to obtain processing result data. The tasks to be executed include the preprocessing stage task, the model inference stage task, and the postprocessing stage task. The server constructs an inference result package based on the processing result data and returns the inference result package to the terminal device. The final AI inference result is determined using the terminal device based on the inference result package.

[0064] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0065] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for collaborative AI task processing on heterogeneous terminals, characterized in that, include: Upload the data to be processed to the terminal device; The system obtains the current resource availability information of the terminal device, and generates a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and executes the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The terminal device constructs a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction, and sends the request data packet to the server. The server determines the tasks to be executed based on the first operation instruction and executes the tasks to be executed to obtain processing result data. The tasks to be executed include the preprocessing stage task, the model inference stage task, and the postprocessing stage task. The server constructs an inference result package based on the processing result data and returns the inference result package to the terminal device. The final AI inference result is determined using the terminal device based on the inference result package. 2.The AI task cooperative processing method of heterogeneous terminals according to claim 1, wherein, The step of obtaining the current resource availability information of the terminal device and generating a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed specifically includes: Based on the task type, determine the computational resource requirements of the preprocessing stage task, the model inference stage task, and the postprocessing stage task, respectively. The computing resource requirements are matched with the current resource availability information to determine whether each stage of the task should be executed locally by the terminal device. Based on the stage task determined to be executed locally, a corresponding N-bit binary code is generated as the first operation instruction. 3.The AI task cooperative processing method of heterogeneous terminals according to claim 1, characterized in that, The step of obtaining the current resource availability information of the terminal device and generating a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed further includes: Based on a preset privacy policy, determine whether the data to be processed contains sensitive information; If sensitive information is contained, the preprocessing stage task will be forcibly set to be executed locally on the terminal device, so that the original sensitive data can be de-identified locally before the processed intermediate data is uploaded to the server. 4.The AI task cooperative processing method of heterogeneous terminals according to claim 1, wherein, The step of determining the task to be executed based on the first operation instruction through the server and executing the task to be executed specifically includes: The server performs a bitwise NOT operation on the parsed first operation instruction to obtain a first server instruction, which is used to determine the task to be executed in the next stage. The server performs a bitwise OR operation between the first server instruction and a preset baseline instruction to obtain a second server instruction; wherein the baseline instruction is used to ensure that the server executes at least the preprocessing stage task and the model inference stage task. The server executes the task to be performed according to the instructions of the second server. 5.The AI task cooperative processing method of heterogeneous terminals according to claim 1, characterized in that, Before the step of constructing a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction via the terminal device, and sending the request data packet to the server, the method further includes: Real-time monitoring of the resource utilization rate of the terminal device; If the resource utilization rate exceeds a preset threshold, the first operation instruction is modified to change at least one stage that was originally executed locally by the terminal device to not be executed locally by the terminal device, and the request data packet is updated according to the modified first operation instruction. 6.The AI task cooperative processing method of heterogeneous terminals according to claim 1, characterized in that, The step of determining the final AI inference result based on the inference result package using the terminal device specifically includes: Perform a bitwise inversion operation on the server-side operation instructions in the inference result package to obtain the second operation instruction; If the bit value corresponding to the post-processing stage task in the new terminal instruction is a first preset value, then the post-processing parameters in the inference result package are used to post-process the processing result data to obtain the final AI inference result. If the bit in the new terminal instruction corresponding to the post-processing stage task takes the value of the second preset value, then the processing result data is directly used as the final AI inference result. 7.The AI task cooperative processing method of heterogeneous terminals according to claim 1, wherein, The binary encoding is a three-bit binary encoding; The least significant bit corresponds to the preprocessing stage, the middle bit corresponds to the model inference stage, and the most significant bit corresponds to the postprocessing stage. When a certain bit is set to a first preset value, it indicates that the corresponding stage is executed locally by the terminal device; when it is set to a second preset value, it indicates that the corresponding stage is not executed locally by the terminal device.

8. An AI task cooperative processing device of a heterogeneous terminal, comprising: include: The data upload module is used to upload data to be processed to the terminal device; The terminal processing module is used to obtain the current resource availability information of the terminal device, and generate a first operation instruction based on the resource availability information and the task type corresponding to the data to be processed, and execute the stage task corresponding to the first operation instruction through the terminal device; wherein, the first operation instruction is a binary code, and different bits in the binary code correspond to the preprocessing stage task, the model inference stage task, and the postprocessing stage task in the AI ​​inference task, respectively, and the value of each bit is used to indicate whether the corresponding stage task is executed locally by the terminal device; The data forwarding module is used to construct a request data packet based on the data to be processed, the first operation instruction, and the processing parameters corresponding to the first operation instruction through the terminal device, and send the request data packet to the server. The server processing module is used to determine the task to be executed based on the first operation instruction through the server, and execute the task to be executed to obtain processing result data. The task to be executed includes the preprocessing task, the model inference task, and the postprocessing task. The feedback receiving module is used to construct an inference result package based on the processing result data through the server, and return the inference result package to the terminal device; The result generation module is used to determine the final AI inference result based on the inference result package through the terminal device.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the AI ​​task collaborative processing method for heterogeneous terminals as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the AI ​​task collaborative processing method for heterogeneous terminals as described in any one of claims 1 to 7.

Images