Mobile platform multi-user edge end cloud collaborative reasoning multi-objective optimization method and system

Abstract

The invention relates to the technical field of edge intelligent computing, and discloses a mobile platform multi-user edge end cloud collaborative reasoning multi-objective optimization method and system, and the method comprises the steps: constructing a multi-user collaborative reasoning mathematical model, obtaining platform performance parameters and neural network information, calculating decision variables through a multi-objective optimization algorithm, and carrying out dynamic segmentation and parameter configuration. And executing cooperative reasoning of the platform and the server. According to the invention, through a dynamic multi-objective optimization algorithm, in combination with the bandwidth, power, dominant frequency and other parameters of the mobile platform, a model segmentation point and a task allocation strategy are optimized, so that the system performance is maximized under the constraint of time and energy consumption, and the method is suitable for unmanned aerial vehicle task execution, intelligent manufacturing, intelligent medical treatment, intelligent monitoring and other scenes needing real-time reasoning, and has a wide application prospect. Through cooperative work of the edge platform and the server, efficient and low-power-consumption deep learning reasoning task execution is realized.

Description

Technical Field

[0001] The present invention relates to the field of edge intelligent computing technology, and in particular to a mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization method and system. Background Art

[0002] With the rapid development of edge computing, the demand for deploying deep learning applications on resource-constrained mobile platform devices is increasing. Mobile platforms are widely used in scenarios such as smart manufacturing, drone control, and smart city management. These applications usually rely on complex neural network reasoning tasks to achieve real-time data processing and intelligent decision-making. However, the computing resources, storage space, and battery capacity of mobile platforms are relatively limited, making it difficult to support independent execution of high-computation, low-latency deep learning tasks. This limitation requires edge devices to work in collaboration with edge servers to offload some computing tasks to edge servers when processing complex reasoning tasks to make up for the lack of computing power of a single device.

[0003] In multi-user scenarios, multiple mobile platforms simultaneously access edge servers to jointly complete reasoning tasks. However, due to the differences in computing power, network bandwidth, power, and communication distance between mobile platforms and servers, how to allocate tasks and optimize resources in multi-user scenarios has become a major challenge. The traditional centralized computing model cannot fully utilize the computing power of edge devices, and due to the limitations of network bandwidth and power, large delays and energy consumption may occur during task transmission, resulting in a decrease in the overall performance of the system.

[0004] Therefore, how to improve the efficiency and performance of the system is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization method and system to solve the resource-constrained reasoning problem, the multi-user task competition problem and the balance problem between energy consumption and reasoning efficiency.

[0006] In order to achieve the above purpose, the technical solutions adopted are as follows: In a first aspect, the present invention provides a multi-objective optimization method for multi-user edge-end cloud collaborative reasoning on a mobile platform, the method comprising: When clients of multiple mobile platforms or edge devices are detected within the communication radius of the multi-user collaborative reasoning server, the mobile platforms or edge devices are automatically added to the edge collaborative reasoning system; wherein the communication radius refers to the effective transmission distance between the server and the mobile platforms or edge devices; When a mobile platform or edge device is added to the edge collaborative reasoning system, platform performance parameters and neural network information are obtained; wherein the platform performance parameters include the computing power of the platform, communication conditions, and the distance between the platform and the server; and the neural network information includes the computational complexity of each layer, the amount of parameters, and the size of the feature graph between each layer; Based on the platform performance parameters and neural network information, and according to the different computing capabilities, communication conditions and distances between the mobile platforms and the server, a multi-objective optimization model is used to determine the optimal Pareto solution set of the neural network decision variables; wherein the decision variables include the model split point, communication bandwidth, transmission power and platform device main frequency; Based on the decision variables, the neural network is dynamically segmented through a collaborative reasoning framework, and corresponding parameter configuration is performed; The mobile platform and the server begin to execute the collaborative reasoning task; during the collaborative reasoning process, the mobile platform and the server continue to interact, the mobile platform transmits part of the calculation results to the server, and the server performs the remaining complex calculations. The entire reasoning process is carried out according to the Pareto optimal conditions set by the multi-objective optimization algorithm.

[0007] Furthermore, when a mobile platform or edge device is added to the edge collaborative reasoning system, platform performance parameters and neural network information are obtained, including: Obtain platform performance parameters through platform self-reporting or central controller query; The model structure of the neural network is obtained, and the model structure of the neural network is layered to extract the neural network information; wherein the model structure of the neural network is layered by the following method: Make sure there are no complex connections between two neural network layers to split the network into multiple sub-networks; Multiple sub-networks run and train independently; Using the distributed computing framework, the sub-networks are assigned to different mobile platforms for processing.

[0008] Furthermore, the multi-objective optimization model is constructed by the following method: Taking time consumption and energy consumption as optimization targets, a time model and an energy consumption model are established respectively; wherein the time consumption includes the computing time and transmission time of the platform end, which are calculated according to the computing power of the mobile platform and the bandwidth conditions of the wireless transmission respectively, and the energy consumption includes the computing energy consumption and transmission energy consumption of the mobile platform, which are calculated according to the energy consumption efficiency and transmission power of the mobile platform respectively; Based on the time model and the energy consumption model, the multi-objective optimization model is constructed.

[0009] Furthermore, the multi-objective optimization model is expressed as: ; In the formula, represents the decision maker’s preference, represents the decision variable corresponding to the minimum objective function value, Indicates the number of mobile platforms, Indicates The total latency of collaborative reasoning on mobile platforms, Indicates The total energy consumption of a mobile platform.

[0010] Furthermore, the time model is expressed as: ; In the formula, represents the total latency of collaborative reasoning, represents the computing latency of the platform, Indicates the transmission delay of the platform; ; In the formula, Indicates the model splitting point of the platform, , Indicates the platform The number of floating point operations of the layer data, Indicates the number of floating-point operations per second of the platform, which is obtained by multiplying the platform device frequency by the number of periodic floating-point operations. Indicates the number of floating-point operations in a cycle. Indicates the device main frequency of the mobile platform; ; In the formula, Indicates the data size of the output feature variables at the model splitting point of the platform, Indicates the transmission rate; ; In the formula, , Represent the bandwidth and power of the platform respectively, represents the noise power spectrum, represents the average channel gain; ; In the formula, represents the weight of the channel gain, , , and represent radio propagation speed, antenna gain, carrier frequency and path loss index respectively, Indicates the distance between the platform and the server.

[0011] Furthermore, the energy consumption model is expressed as: ; In the formula, represents the total energy consumption of collaborative reasoning, represents the computing energy consumption of platform data, represents the transmission energy consumption of the platform intermediate characteristic variables to the server, Represents the processor chip energy efficiency coefficient, Indicates the chip main frequency of the platform. Indicates the power of the platform.

[0012] Furthermore, based on the platform performance parameters and the neural network information, a multi-objective optimization model is used to perform calculations, including: The multi-objective optimization model is solved based on a multi-objective optimization algorithm, and the multi-objective optimization algorithm generates multiple candidate solutions by performing Gaussian sampling in the solution space; each candidate solution represents a configuration, including a split point, bandwidth allocation, power consumption, and operation frequency; and the optimal configuration is selected by evaluating the time consumption and energy consumption of each candidate solution.

[0013] Furthermore, the collaborative reasoning framework includes an edge platform and a cloud server. Based on the decision variables, the collaborative reasoning framework dynamically segments the neural network and performs corresponding parameter configuration, including: According to the current system resource situation, select the model split point and divide the neural network model into two parts, namely M E and M S , M E It is the part of the model that runs on the edge platform. S It is the part of the model that runs on the cloud server; After the edge platform receives the image data, it uses M E Perform forward reasoning on the image to obtain the intermediate feature variable IF. The edge platform compresses and encodes IF and transmits it to the cloud server via the wireless network. After receiving the IF, the cloud server decodes and restores it using M S Perform forward reasoning to obtain the detection result PR, where PR contains the category and location information of the weed target in the image; the cloud server transmits the detection result PR back to the edge platform.

[0014] In a second aspect, the present invention provides a mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization system, the system comprising a processor, and the processor is configured to: When clients of multiple mobile platforms or edge devices are detected within the communication radius of the multi-user collaborative reasoning server, the mobile platforms or edge devices are automatically added to the edge collaborative reasoning system; wherein the communication radius refers to the effective transmission distance between the server and the mobile platforms or edge devices; When a mobile platform or edge device is added to the edge collaborative reasoning system, platform performance parameters and neural network information are obtained; wherein the platform performance parameters include the computing power of the platform, communication conditions, and the distance between the platform and the server; and the neural network information includes the computational complexity of each layer, the amount of parameters, and the size of the feature graph between each layer; Based on the platform performance parameters and the neural network information, the decision variables of each mobile platform are calculated using a multi-objective optimization model; wherein the decision variables include a model split point, a communication bandwidth, a transmission power, and a main frequency of a platform device, and the multi-objective optimization model determines the optimal split point of the neural network and the load of each mobile platform in the reasoning task according to the different computing capabilities, communication conditions, and distances between the mobile platforms and the server; Based on the decision variables, the neural network is dynamically segmented through a collaborative reasoning framework, and corresponding parameter configuration is performed; The execution of collaborative reasoning tasks begins through the mobile platform and the server; during the collaborative reasoning process, the mobile platform and the server continuously interact, the mobile platform transmits part of the calculation results to the server, and the server performs the remaining complex calculations. The entire reasoning process is carried out according to the Pareto optimal conditions set by the multi-objective optimization algorithm.

[0015] Furthermore, the processor is further configured to: Obtain platform performance parameters through platform self-reporting or central controller query; The model structure of the neural network is obtained, and the model structure of the neural network is layered to extract the neural network information; wherein the model structure of the neural network is layered by the following method: Make sure there are no complex connections between two neural network layers to split the network into multiple sub-networks; Multiple sub-networks run and train independently; Using the distributed computing framework, the sub-networks are assigned to different mobile platforms for processing.

[0016] The beneficial effects of the present invention are: The present invention dynamically optimizes the computing resources, transmission bandwidth, power and other parameters of multiple platforms through a multi-objective optimization algorithm to ensure that the system achieves the optimal balance in terms of time, energy consumption and resource utilization. By segmenting and unloading the neural network, the edge device of the mobile platform and the server collaborate to complete the reasoning task, achieving efficient and low-power system performance improvement. The present invention is suitable for smart mobile terminals, Internet of Things devices and other application scenarios that require efficient edge reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of an application scenario according to the prior art is shown.

[0018] Figure 2 A flowchart of a mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization method according to an embodiment of the present invention is shown.

[0019] Figure 3 A diagram showing the optimization principle of a multi-objective optimization model according to an embodiment of the present invention is shown.

[0020] Figure 4 A flow chart of a multi-objective optimization algorithm for dynamic Gaussian sampling according to an embodiment of the present invention is shown.

[0021] Figure 5 A schematic diagram of a cross-border processing strategy according to an embodiment of the present invention is shown.

[0022] Figure 6 A structural diagram of a single-user collaborative reasoning framework according to an embodiment of the present invention is shown.

[0023] Figure 7 A structural diagram of a mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0025] The specific implementation of the present invention is further described in detail below in conjunction with the drawings and examples.

[0026] Multi-user collaborative reasoning refers to the collaborative execution of neural network reasoning tasks by multiple mobile platforms and edge servers in edge computing scenarios. Neural network tasks are usually difficult to complete on a single mobile platform due to their high computational complexity. In order to reduce the computational load and energy consumption of a single device, collaborative reasoning improves reasoning efficiency by offloading part of the layers of the deep learning model to the edge server. Each platform dynamically chooses to execute part of the tasks based on its own computing power and communication conditions, while the remaining tasks are completed by the edge server.

[0027] In actual application scenarios, multiple platforms need to participate in reasoning tasks at the same time. Due to the limited computing resources and communication bandwidth of edge servers, how to reasonably allocate resources among multiple platforms becomes a key issue. In addition, the computing power, bandwidth conditions and communication power of different platforms also vary, and a unified resource allocation strategy is difficult to meet the personalized needs in multi-user scenarios. Therefore, an embodiment of the present invention provides a multi-objective optimization method for collaborative reasoning between multi-user edge terminals and clouds on a mobile platform to solve the following problems: 1) Resource-constrained reasoning problem: The computing resources and communication bandwidth of mobile platforms are limited, and they cannot complete complex deep learning reasoning tasks alone. Reasoning efficiency needs to be improved through reasonable task offloading and resource allocation.

[0028] 2) Multi-user task competition problem: When multiple platforms participate in reasoning tasks at the same time, limited server computing resources and network bandwidth need to be reasonably allocated to avoid system performance degradation caused by resource competition.

[0029] 3) Balance between energy consumption and inference efficiency: The battery capacity of mobile platforms is limited, so the system needs to minimize the platform’s energy consumption while ensuring that inference tasks are completed in a timely manner.

[0030] See also Figure 1 , is a schematic diagram of an application scenario according to the prior art. In this application scenario, there are four devices and a server. The four devices are device 1, device 2, device 3 and device 4. The server can be used as a collaborative reasoning server. Figure 1 The dotted circle in shows the communication range of the collaborative reasoning server. All four devices are within the communication range of the collaborative reasoning server and are able to communicate with the server.

[0031] The embodiment of the present invention provides a mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization method, which realizes multi-objective optimization through mobile platform multi-user edge collaborative reasoning technology based on model offloading to maximize system efficiency. The system is mainly composed of multiple mobile platforms and edge servers, where the mobile platform is responsible for performing the preliminary processing of deep learning tasks and transferring the computationally intensive part to the edge server for further processing through model offloading technology. The mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization method can be applied to Figure 1 In the application scenario shown. Figure 2 A flowchart of a multi-objective optimization method for multi-user edge-end cloud collaborative reasoning on a mobile platform according to an embodiment of the present invention is shown. Figure 2 As shown, the method includes steps S10 to S50, which are described in detail as follows.

[0032] S10: Multiple platforms join the edge collaborative reasoning system.

[0033] In this embodiment, clients of multiple mobile platforms or edge devices are detected within the communication radius of the multi-user collaborative reasoning server and automatically added to the edge collaborative reasoning system. The communication radius refers to the effective transmission distance between the server and the mobile platform or edge device. Each platform establishes a communication link with the edge server to transmit the performance parameters of the platform, such as the platform's main frequency, available computing resources, bandwidth, etc. The server needs to monitor the communication status of the platform in a timely manner to ensure that it is successfully added to the system. The main task of this stage is to provide basic information on platform resources for the allocation of subsequent reasoning tasks.

[0034] S20: Platform performance parameters and neural network information acquisition.

[0035] In this embodiment, once the mobile platform or edge device joins the system, the server begins to collect various performance parameters of the platform and specific information about network communications. These parameters include the computing power of the platform, communication conditions, the distance between the platform and the server, etc. The communication distance between the platform and the server affects the delay and rate of data transmission and is an important indicator of dynamic optimization. The server also obtains the model structure of the deep neural network, including the computational complexity of each layer, the number of parameters, and the size of the feature maps between each layer. This information provides basic data support for the segmentation and offloading strategies of neural network tasks.

[0036] In some embodiments, platform performance parameters and neural network information are obtained by the following method: First, you need to query the server's platform list to obtain detailed performance parameters of each platform in the system. These parameters include the number of platforms, channel gain of wireless transmission, platform cycle floating point operations (FLOPs), platform maximum bandwidth (Bmax), maximum power (Pmax) and maximum frequency (Fmax), etc. These parameters are obtained through platform self-reporting or central controller query, and are used for subsequent model stratification and multi-objective optimization calculations.

[0037] There are several main methods for neural network layering: No complex connections between neural network layers or blocks: Make sure there are no complex connections between two neural network layers, so that the network can be easily divided into multiple sub-networks. For example, multi-layer fully connected neural networks and convolutional neural networks.

[0038] Modular design: Each segmented sub-network can be run and trained independently. This approach helps improve the maintainability and scalability of the neural network model.

[0039] Distributed computing: Using a distributed computing framework, different parts of the neural network are distributed to different mobile platforms for processing. This approach can significantly improve computing efficiency, especially when processing large-scale data.

[0040] Through the above method, the neural network structure and parameters can be represented as a sequence, each element is a neural network block, and the space between elements is the dividing point.

[0041] S30: The multi-objective optimization algorithm solves the Pareto solution set corresponding to the optimization objective.

[0042] After obtaining the platform's performance parameters and neural network structure information, the server uses a multi-objective optimization model to calculate the decision variables of each platform. The decision variables include: model split point (C), communication bandwidth (B), transmission power (P), and platform device main frequency (F).

[0043] The multi-objective optimization model combines the inference time and energy consumption targets of multiple platforms into a multi-objective optimization problem. Its main purpose is to reasonably allocate resources among different platforms to ensure that the time and energy consumption of the inference process reach Pareto optimality. Specifically, the multi-objective optimization model determines the optimal split point of the neural network and the optimal resource allocation solution for each platform in the inference task based on the different computing capabilities, communication conditions, and distances between the platforms and the server.

[0044] In some embodiments, see Figure 3, which is an optimization principle diagram of the multi-objective optimization model provided by an embodiment of the present invention, and the multi-objective optimization model takes time (T) and energy (E) as optimization objectives. Time consumption mainly includes the computing time (T_dc) and transmission time (T_tx) of the platform, and the calculation formula is based on the computing power of the platform and the bandwidth conditions of wireless transmission. Energy consumption includes computing energy consumption (E_dc) and transmission energy consumption (E_tx) of the platform, which also depends on the energy efficiency and transmission power of the platform.

[0045] In the scenario of multi-user collaborative reasoning, the model split point (C), platform device bandwidth (B), platform device power (P) and platform device frequency (F) are determined as decision variables that affect collaborative reasoning performance. Based on these variables, this embodiment further elaborates on how to construct a multi-objective optimization model from the two dimensions of time consumption and energy consumption.

[0046] Specifically, by establishing a time model and an energy consumption model, and then combining the time model and the energy consumption model, the multi-objective optimization model can be obtained.

[0047] For the time model, it is expressed as: ; In the formula, represents the total latency of collaborative reasoning, represents the computing latency of the platform, Indicates the transmission delay of the platform.

[0048] ; In the formula, Represents the model split point of the platform, . Representing the platform The number of floating point operations for the layer data. Indicates the number of floating-point operations in a cycle. Represents the number of floating-point operations per second of the platform, which can be calculated by multiplying the platform device frequency by the number of floating-point operations per cycle (FPC).

[0049] ; In the formula, Indicates the data size of the output feature variables at the model splitting point of the platform, Indicates the transmission rate; ; In the formula, , Represent the bandwidth and power of the platform respectively, represents the noise power spectrum, Represents the average channel gain.

[0050] h represents the average channel gain. ω refers to the weight of the channel gain.

[0051] ; In the formula, represents the weight of the channel gain, , , and represent radio propagation speed, antenna gain, carrier frequency and path loss index respectively, Indicates the distance between the platform and the server.

[0052] For the energy model, it is expressed as: ; In the formula, represents the total energy consumption of collaborative reasoning, Indicates the computing energy consumption of platform data. Represents the transmission energy consumption of the platform intermediate characteristic variables to the server.

[0053] In the energy model, the data computing energy consumption of the edge platform takes into account the impact of the chip energy efficiency coefficient, chip main frequency, and computing latency on data computing energy consumption. The calculation method is shown in the following formula: ; In the formula, The energy efficiency coefficient of the processor chip refers to the energy consumed by the chip when performing a unit computing task. It reflects the energy efficiency level of the chip. f D The main frequency of the chip of a certain platform refers to the frequency at which the chip executes instructions, which directly affects the computing speed of the chip.

[0054] The transmission energy consumption of the edge platform involves the transmission power and the transmission of intermediate characteristic variables. When designing a user collaborative reasoning system, it is necessary to reasonably select the transmission scheme and optimize the transmission strategy to reduce the transmission energy overhead: ; In the formula, P Represents the power of the platform in Watts (W), or joules per second (j / s).

[0055] By comprehensively considering the two objectives of time and energy consumption, the following multi-objective optimization model can be established: ; In the formula, represents the decision maker's preference. When it is greater than 0.5, it means that the decision maker has higher requirements for the real-time performance of the system, otherwise it means that the decision maker hopes for less energy consumption. represents the decision variable corresponding to the minimum objective function value, Indicates the number of mobile platforms, Indicates The total latency of collaborative reasoning on mobile platforms, represents the total energy consumption of the i-th mobile platform.

[0056] In some embodiments, in order to solve the above multi-objective optimization model, a multi-objective optimization algorithm with dynamic Gaussian sampling is proposed. In the process of algorithm design, in order to improve the search efficiency and the quality of the solution, normalization processing, boundary rebound strategy and differential solution update strategy are introduced. Normalization processing ensures that decision variables of different dimensions are considered on the same scale, the boundary rebound strategy prevents the solution from exceeding the predetermined search space, and the differential solution update strategy enhances the global search capability of the algorithm.

[0057] The multi-objective optimization algorithm of dynamic Gaussian sampling generates multiple candidate solutions by performing Gaussian sampling in the solution space. Each candidate solution represents a possible configuration, including split point, bandwidth allocation, power consumption, and operation frequency. The algorithm selects the optimal configuration by evaluating the time consumption and energy consumption of each candidate solution. Figure 4 As shown. The algorithm divides the overall goal of the system into two-objective problems and uses a multi-objective optimization problem based on dynamic Gaussian sampling. In the algorithm, 100 individuals are randomly initialized as a population. In each population evolution, multiple neighbors of the individual are referenced and Gaussian mutation is performed, and the Chebyshev distance between the result of Gaussian sampling and the original individual is compared. If the result of Gaussian mutation is better than the original individual, then the value of Gaussian mutation is used as the new solution and the Pareto front is updated. After several rounds of sampling, the population will gradually move to the Pareto front surface under the action of two objectives. At this time, the decision maker can select the optimal solution according to his or her preferences. Finally, the overall time consumption and energy consumption of the system are minimized, which means that the efficiency of the system has been improved.

[0058] During the optimization process, since the solution update strategy adopts the Gaussian mutation algorithm, the solution is updated as follows: Figure 5 At the same time, in order to accelerate the convergence of the algorithm, when optimizing a solution, its poorly performing neighbors will be updated simultaneously, and the difference solution will be replaced by the mean point.

[0059] After the optimization is completed, the algorithm will generate multiple non-dominated solutions to form the Pareto front. The solutions on the Pareto front represent multiple sets of optimal solutions between time and energy consumption, which corresponds to the multi-objective optimization model. α , allowing users to choose the optimal configuration based on actual needs.

[0060] In one embodiment, the above-mentioned multi-objective optimization model and optimization algorithm can be constructed using mathematical libraries such as Python, PyTorch, and NumPy.

[0061] S40: Perform dynamic segmentation and parameter configuration according to user preferences.

[0062] After calculating the Pareto solution set of the optimization plan, the appropriate optimization plan is selected from the Pareto solution set according to the user's personal preference. The server dynamically divides the neural network according to the optimization plan through the collaborative reasoning framework and configures the corresponding parameters. According to the performance and communication conditions of the platform, the system offloads part of the computing tasks of the neural network to the edge server for execution, and the rest is completed locally by the mobile platform. This process ensures that the computing load of each platform matches its capabilities to avoid overloading a single platform.

[0063] Neural network task segmentation: Depending on the platform performance and communication conditions, the server will choose to execute certain layers of the neural network on the platform and offload more complex layers to the server for processing. For example, for platforms with rich computing resources, more neural network layers can be allocated for local processing, while for resource-constrained platforms, only a small number of shallow networks are processed, and the rest are processed by the server.

[0064] Parameter configuration and optimization: Based on the task segmentation, the system will configure each platform according to the bandwidth, power, main frequency and other parameters obtained by the optimization algorithm. The server dynamically adjusts the bandwidth allocation and transmission power of each platform to improve the efficiency of data transmission. At the same time, the main frequency of the platform will also be appropriately adjusted according to the load of the task to ensure the best balance between energy consumption and computing efficiency.

[0065] In some embodiments, the collaborative reasoning framework includes a server side and a client side.

[0066] The server has three main functions: Record and manage the platform information and model library in the system: The server is responsible for recording and managing the platform information and model library in the system, including the platform's performance parameters, the model's structure and parameters, etc.

[0067] Run a multi-objective optimization algorithm based on dynamic Gaussian sampling: The server runs a multi-objective optimization algorithm based on dynamic Gaussian sampling to calculate the optimal solution for the decision variables of each platform at each moment.

[0068] Complete system reasoning: The server receives the intermediate feature variables, completes the reasoning, and returns the results to the client.

[0069] The client has two functions: Synchronize platform information and receive decision variables distributed by the server: The client is responsible for synchronizing platform information and receiving decision variables distributed by the server.

[0070] System reasoning: The client performs system reasoning and sends intermediate feature variables. The server receives the reasoning results and performs the next operation.

[0071] Through the implementation of the above functions, the collaborative reasoning framework of the present invention can realize multi-user edge collaborative reasoning technology on mobile platforms based on model offloading, maximize system efficiency, and meet the needs of multi-objective optimization.

[0072] In another embodiment, in order to solve the problem of insufficient computing resources on the edge platform, this embodiment proposes a single-user collaborative reasoning framework, see FIG. Figure 6 , which is a structural diagram of a single-user collaborative reasoning framework provided by an embodiment of the present invention. The single-user collaborative reasoning framework is a collaborative reasoning framework. Taking the weed recognition task as an example, the weed recognition task is divided into two parts, one of which is executed on the edge platform and the other is executed on the cloud server, thereby realizing collaborative reasoning between the edge platform and the cloud server. The collaborative reasoning framework includes the following steps: S41, model segmentation. According to the current system resource situation, select the appropriate model segmentation point and divide the deep neural network model into two parts, denoted as M E and M S . M E It is the part of the model that runs on the edge platform. S It is the part of the model that runs on the cloud server. The selection of the model splitting point needs to balance the computing time and transmission time of the edge platform to achieve the optimal collaborative reasoning efficiency.

[0073] S42, edge platform reasoning. After receiving the image data, the edge platform uses M E Perform forward reasoning on the image to obtain the intermediate feature variable IF. The edge platform compresses and encodes IF and transmits it to the cloud server via the wireless network.

[0074] S43, cloud server inference. After receiving the IF, the cloud server decodes and restores it, and then uses M S Perform forward reasoning to obtain the final detection result PR. PR contains the category and location information of the weed target in the image. The cloud server transmits it back to the edge platform to complete the collaborative reasoning process.

[0075] S50: Execution of collaborative reasoning between the platform and the server.

[0076] After the resource allocation and parameter configuration of the aforementioned steps S10 to S40, the platform and the server begin to execute the collaborative reasoning task. During the collaborative reasoning process, the mobile platform and the edge server continue to interact, and the mobile platform transmits part of the calculation results to the server, and the server performs the remaining complex calculations. The execution process of collaborative reasoning requires an efficient data transmission protocol to ensure that the data exchange between the platform and the server is not affected by factors such as network bandwidth and latency. The entire reasoning process is carried out according to the Pareto optimal conditions set by the multi-objective optimization algorithm to ensure that the system energy consumption is minimized while meeting the real-time requirements of the task.

[0077] The server continuously monitors the operating status and communication status of the platform. If a platform experiences performance fluctuations or network instability, the system will automatically adjust task allocation or recalculate optimization parameters to ensure the stability and reliability of the reasoning process.

[0078] In summary, the method provided by the present invention combines the performance parameters of the mobile platform and the network communication conditions, and adopts a multi-objective optimization algorithm to achieve dynamic segmentation and load balancing of neural network reasoning tasks. Compared with the traditional centralized reasoning method, the present invention significantly reduces the total energy consumption of the system, improves the efficiency of reasoning and the resource utilization of the platform. By optimizing the bandwidth, power and main frequency configuration of the platform, it ensures that the collaborative reasoning tasks between the platform and the server can achieve the best in time and energy consumption. In addition, the multi-objective optimization algorithm of this technology effectively solves the problem of resource competition when multiple platforms participate in reasoning tasks at the same time, and improves the overall reasoning performance of the system. Specifically, it is reflected in the following aspects: First, the model split point: determines which part of the neural network is executed by each platform. In edge collaborative reasoning, the neural network is usually divided into different layers, some of which are executed by the mobile platform, and the rest are offloaded to the server for execution. By reasonably selecting the split point, it is ensured that the overall energy consumption and time consumption of the system are minimized while ensuring the reasoning accuracy.

[0079] Second, communication bandwidth: In collaborative reasoning, data transmission between the platform and the server requires network bandwidth. Optimizing bandwidth allocation can effectively reduce transmission delays and ensure efficient data interaction. The server dynamically allocates bandwidth resources based on the communication conditions of each platform to maximize the overall transmission efficiency of the system.

[0080] Third, transmission power: Transmission power determines the signal strength and data transmission rate between the platform and the server. In the case of long transmission distances, appropriately increasing the transmission power can enhance signal quality and reduce data transmission latency. However, an increase in power will also lead to an increase in energy consumption, so during the optimization process, the power parameter needs to be balanced with the energy consumption target.

[0081] Fourth, the main frequency of the platform device: The main frequency of the platform device determines its computing power. The higher the main frequency, the faster the platform's computing speed, but the corresponding energy consumption will also increase. By dynamically adjusting the main frequency of the platform, it is possible to balance inference efficiency and energy consumption. The multi-objective optimization algorithm dynamically adjusts the main frequency of different platforms according to the computing task load to achieve the best inference performance.

[0082] The embodiment of the present invention also provides a mobile platform multi-user edge terminal cloud collaborative reasoning multi-objective optimization system, please refer to Figure 7 , the system includes a processor 701, wherein the processor 701 is configured to: When clients of multiple mobile platforms or edge devices are detected within the communication radius of the multi-user collaborative reasoning server, the mobile platforms or edge devices are automatically added to the edge collaborative reasoning system; wherein the communication radius refers to the effective transmission distance between the server and the mobile platforms or edge devices; When a mobile platform or edge device is added to the edge collaborative reasoning system, platform performance parameters and neural network information are obtained; wherein the platform performance parameters include the computing power of the platform, communication conditions, and the distance between the platform and the server; and the neural network information includes the computational complexity of each layer, the amount of parameters, and the size of the feature graph between each layer; Based on the platform performance parameters and the neural network information, the decision variables of each mobile platform are calculated using a multi-objective optimization model; wherein the decision variables include a model split point, a communication bandwidth, a transmission power, and a main frequency of a platform device, and the multi-objective optimization model determines the optimal split point of the neural network and the load of each mobile platform in the reasoning task according to the different computing capabilities, communication conditions, and distances between the mobile platforms and the server; Based on the decision variables, the neural network is dynamically segmented through a collaborative reasoning framework, and corresponding parameter configuration is performed; The execution of collaborative reasoning tasks begins through the mobile platform and the server; during the collaborative reasoning process, the mobile platform and the server continuously interact, the mobile platform transmits part of the calculation results to the server, and the server performs the remaining complex calculations. The entire reasoning process is carried out according to the Pareto optimal conditions set by the multi-objective optimization algorithm.

[0083] In some embodiments, the processor 701 is further configured to: Obtain platform performance parameters through platform self-reporting or central controller query; The model structure of the neural network is obtained, and the model structure of the neural network is layered to extract the neural network information; wherein the model structure of the neural network is layered by the following method: Make sure there are no complex connections between two neural network layers to split the network into multiple sub-networks; Multiple sub-networks run and train independently; Using the distributed computing framework, the sub-networks are assigned to different mobile platforms for processing.

[0084] It should be noted that the mobile platform multi-user edge-cloud collaborative reasoning multi-objective optimization system belongs to the same technical concept as the previously described method, and has the same technical principles and beneficial effects, so it will not be repeated here.

[0085] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The patent protection scope of the present invention should be defined by the claims.

Claims

1. A multi-objective optimization method for multi-user edge-end cloud collaborative reasoning on a mobile platform, characterized in that: The method comprises: When clients of multiple mobile platforms or edge devices are detected within the communication radius of the multi-user collaborative reasoning server, the mobile platforms or edge devices are automatically added to the edge collaborative reasoning system; wherein the communication radius refers to the effective transmission distance between the server and the mobile platforms or edge devices; When a mobile platform or edge device is added to the edge collaborative reasoning system, platform performance parameters and neural network information are obtained; wherein the platform performance parameters include the computing power of the platform, communication conditions, and the distance between the platform and the server; and the neural network information includes the computational complexity of each layer, the amount of parameters, and the size of the feature graph between each layer; Based on the platform performance parameters and neural network information, and according to the different computing capabilities, communication conditions and distances between the mobile platforms and the server, a multi-objective optimization model is used to determine the optimal Pareto solution set of the neural network decision variables; wherein the decision variables include the model split point, communication bandwidth, transmission power and platform device main frequency; Based on the decision variables, the neural network is dynamically segmented through a collaborative reasoning framework, and corresponding parameter configuration is performed; The mobile platform and the server begin to execute the collaborative reasoning task; during the collaborative reasoning process, the mobile platform and the server continue to interact, the mobile platform transmits part of the calculation results to the server, and the server performs the remaining complex calculations. The entire reasoning process is carried out according to the Pareto optimal conditions set by the multi-objective optimization algorithm.

2. The method according to claim 1, characterized in that When a mobile platform or edge device is added to the edge collaborative inference system, platform performance parameters and neural network information are obtained, including: Obtain platform performance parameters through platform self-reporting or central controller query; The model structure of the neural network is obtained, and the model structure of the neural network is layered to extract the neural network information; wherein the model structure of the neural network is layered by the following method: Make sure there are no complex connections between two neural network layers to split the network into multiple sub-networks; Multiple sub-networks run and train independently; Using the distributed computing framework, the sub-networks are assigned to different mobile platforms for processing.

3. The method according to claim 1, characterized in that The multi-objective optimization model is constructed by the following method: Taking time consumption and energy consumption as optimization targets, a time model and an energy consumption model are established respectively; wherein the time consumption includes the computing time and transmission time of the platform end, which are calculated according to the computing power of the mobile platform and the bandwidth conditions of the wireless transmission respectively, and the energy consumption includes the computing energy consumption and transmission energy consumption of the mobile platform, which are calculated according to the energy consumption efficiency and transmission power of the mobile platform respectively; Based on the time model and the energy consumption model, the multi-objective optimization model is constructed.

4. The method according to claim 3, characterized in that The multi-objective optimization model is expressed as: ； In the formula, represents the decision maker’s preference, represents the decision variable corresponding to the minimum objective function value, Indicates the number of mobile platforms, Indicates The total latency of collaborative reasoning on mobile platforms, Indicates The total energy consumption of a mobile platform.

5. The method according to claim 4, characterized in that The time model is expressed as: ； In the formula, represents the total latency of collaborative reasoning, represents the computing latency of the platform, Indicates the transmission delay of the platform; ； In the formula, Indicates the model splitting point of the platform, , Indicates the platform The number of floating point operations of the layer data, Indicates the number of floating-point operations per second of the platform, which is obtained by multiplying the platform device frequency by the number of periodic floating-point operations. Indicates the number of floating-point operations in a cycle. Indicates the device main frequency of the mobile platform; ； In the formula, Indicates the data size of the output feature variables at the model splitting point of the platform, Indicates the transmission rate; ； In the formula, , Represent the bandwidth and power of the platform respectively, represents the noise power spectrum, represents the average channel gain; ； In the formula, represents the weight of the channel gain, , , and represent radio propagation speed, antenna gain, carrier frequency and path loss index respectively, Indicates the distance between the platform and the server.

6. The method according to claim 5, characterized in that The energy consumption model is expressed as: ； In the formula, represents the total energy consumption of collaborative reasoning, represents the computing energy consumption of platform data, represents the transmission energy consumption of the platform intermediate characteristic variables to the server, Represents the processor chip energy efficiency coefficient, Indicates the chip main frequency of the platform. Indicates the power of the platform.

7. The method according to claim 6, characterized in that Based on the platform performance parameters and the neural network information, the decision variables of each mobile platform are calculated using a multi-objective optimization model, including: The multi-objective optimization model is solved based on a multi-objective optimization algorithm, and the multi-objective optimization algorithm generates multiple candidate solutions by performing Gaussian sampling in the solution space; each candidate solution represents a configuration, including a split point, bandwidth allocation, power consumption, and operation frequency; and the optimal configuration is selected by evaluating the time consumption and energy consumption of each candidate solution.

8. The method according to claim 1, characterized in that The collaborative reasoning framework includes an edge platform and a cloud server. Based on the decision variables, the neural network is dynamically segmented through the collaborative reasoning framework, and corresponding parameter configuration is performed, including: According to the current system resource situation, select the model split point and divide the neural network model into two parts, namely M E and M S , M E It is the part of the model that runs on the edge platform. S It is the part of the model that runs on the cloud server; After the edge platform receives the image data, it uses M E Perform forward reasoning on the image to obtain the intermediate feature variable IF. The edge platform compresses and encodes IF and transmits it to the cloud server via the wireless network. After receiving the IF, the cloud server decodes and restores it using M S Perform forward reasoning to obtain the detection result PR, where PR contains the category and location information of the weed target in the image; the cloud server transmits the detection result PR back to the edge platform.

9. A mobile platform multi-user edge-end cloud collaborative reasoning multi-objective optimization system, characterized in that: The system includes a processor configured to: When clients of multiple mobile platforms or edge devices are detected within the communication radius of the multi-user collaborative reasoning server, the mobile platforms or edge devices are automatically added to the edge collaborative reasoning system; wherein the communication radius refers to the effective transmission distance between the server and the mobile platforms or edge devices; When a mobile platform or edge device is added to the edge collaborative reasoning system, platform performance parameters and neural network information are obtained; wherein the platform performance parameters include the computing power of the platform, communication conditions, and the distance between the platform and the server; and the neural network information includes the computational complexity of each layer, the amount of parameters, and the size of the feature graph between each layer; Based on the platform performance parameters and the neural network information, the decision variables of each mobile platform are calculated using a multi-objective optimization model; wherein the decision variables include a model split point, a communication bandwidth, a transmission power, and a main frequency of a platform device, and the multi-objective optimization model determines the optimal split point of the neural network and the load of each mobile platform in the reasoning task according to the different computing capabilities, communication conditions, and distances between the mobile platforms and the server; Based on the decision variables, the neural network is dynamically segmented through a collaborative reasoning framework, and corresponding parameter configuration is performed; The execution of collaborative reasoning tasks begins through the mobile platform and the server; during the collaborative reasoning process, the mobile platform and the server continuously interact, the mobile platform transmits part of the calculation results to the server, and the server performs the remaining complex calculations. The entire reasoning process is carried out according to the Pareto optimal conditions set by the multi-objective optimization algorithm.

10. The system according to claim 9, characterized in that The processor is further configured to: Obtain platform performance parameters through platform self-reporting or central controller query; The model structure of the neural network is obtained, and the model structure of the neural network is layered to extract the neural network information; wherein the model structure of the neural network is layered by the following method: Make sure there are no complex connections between two neural network layers to split the network into multiple sub-networks; Design the neural network into multiple modules, each of which runs and is trained independently; Using the distributed computing framework, different parts of the neural network are assigned to different computing nodes for processing.

Images