A device-edge collaborative model reasoning method, device, related equipment and storage medium

Abstract

The present application discloses an end-edge collaborative model reasoning method, apparatus and related equipment, which relates to the field of communication technology, in order to solve the problem that the existing end-edge collaborative reasoning scheme does not take into account the difference between the different computing resources respectively owned by the terminal and the network side, resulting in poor model reasoning performance. The method includes: sending an inference request message carrying model information to a network device; receiving an inference reply message sent by the network device; when the inference reply message indicates that the network device agrees to collaborative reasoning, generating a first decision action of the terminal and a second decision action of the network device according to the status information of the network device and the status information of the terminal; allocating computing resources to the first model part that the terminal is responsible for calculating indicated in the first decision action, and performing model reasoning calculations for the first model part; sending a second decision action to the network device. The embodiments of the present application can improve model reasoning performance.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an end-edge collaborative model reasoning method, apparatus, and related equipment. Background Art

[0002] Device-edge collaborative reasoning based on model segmentation refers to the situation where, when a terminal needs to complete an intelligent model inference task but its own computing power is insufficient, network-side computing resources are used to assist in completing the inference task. Currently, network-side assistance typically involves splitting the intelligent model into parts, with the terminal and network each performing a portion of the computation. The network then returns the inference results to the terminal.

[0003] However, existing end-edge collaborative reasoning solutions focus on the decision of the model split point, that is, they only determine which part of the computing tasks are placed on the terminal and which part on the network side, without considering the differences in the different computing resources owned by the terminal and the network side, resulting in poor model reasoning performance. Summary of the Invention

[0004] The embodiments of the present application provide an end-edge collaborative model reasoning method, apparatus, and related equipment to solve the problem that the existing end-edge collaborative reasoning scheme does not take into account the differences in different computing resources owned by the terminal and the network side, resulting in poor model reasoning performance.

[0005] In a first aspect, an embodiment of the present application provides a device-edge collaborative model inference method, which is executed by a terminal. The method includes:

[0006] Sending an inference request message carrying model information to the network device;

[0007] receiving an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference, and when the inference reply message indicates that the network device agrees to the collaborative inference, the inference reply message also carries status information of the network device;

[0008] When the reasoning reply message indicates that the network device agrees to collaborative reasoning, generating a first decision action of the terminal and a second decision action of the network device according to the state information of the network device and the state information of the terminal, the first decision action and the second decision action including a model portion responsible for calculation and corresponding computing resource allocation information, and the state information of the network device and the state information of the terminal including at least computing resource state information;

[0009] Allocating computing resources to the first model portion that the terminal is responsible for calculating in the first decision action, and performing model inference calculation of the first model portion;

[0010] The second decision action is sent to the network device, so that the network device allocates computing resources to the second model part that the network device is responsible for calculating as indicated in the second decision action, and performs model inference calculation of the second model part.

[0011] Optionally, after sending the second decision action to the network device, the method further includes:

[0012] receiving a computing resource allocation update message sent by the network device when the second decision action is changed, wherein the computing resource allocation update message carries a third decision action updated by the network device;

[0013] According to the third decision action, adjusting the first decision action to obtain a fourth decision action;

[0014] According to the fourth decision action, the third model part that the terminal is responsible for calculating is re-determined, and computing resources are allocated to the third model part to perform model inference calculation of the third model part.

[0015] Optionally, the allocating computing resources to the first model portion that the terminal is responsible for computing indicated in the first decision action, and performing model inference calculation of the first model portion, includes:

[0016] Sending a first computing resource allocation suggestion message to the computing resource management module of the terminal, wherein the first computing resource allocation suggestion message carries parameters of the first model part, the computing resource type and quantity of computing resources of each layer in the first model part, and the timing and data transmission relationship of inter-layer calculations;

[0017] The computing resource management module allocates computing tasks of different layers in the first model part to different types of computing resources of the terminal according to the first computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

[0018] Optionally, the method further includes:

[0019] Sending the model reasoning intermediate result of the first model part to the network device, so that the network device integrates the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0020] Receive the model collaborative reasoning result sent by the network device.

[0021] Optionally, the method further includes:

[0022] Acquire model reasoning performance parameters, where the model reasoning performance parameters include at least one of reasoning accuracy, average reasoning latency, and average single reasoning energy consumption;

[0023] Calculating reward information based on the model reasoning performance parameters;

[0024] The state information, decision action and reward information in each collaborative reasoning are collected as training samples of the reinforcement learning model used by the terminal and the network device for model collaborative reasoning, and the reinforcement learning model is trained and optimized.

[0025] Optionally, obtaining the model reasoning performance parameter includes:

[0026] Counting the inference accuracy, average inference latency, and average single inference energy consumption of the terminal within a preset time period;

[0027] Receive the inference energy consumption information sent by the network device, wherein the inference energy consumption information includes the average single inference energy consumption value of the network device within the preset time length or the average single inference computing power value consumed by the network device.

[0028] Optionally, the method further includes:

[0029] receiving a status update message sent by the network device, wherein the status update message includes updated status information of the network device;

[0030] and / or, when detecting a change in the status information of the terminal, sending a status update request message to the network device; and receiving the latest status information sent by the network device;

[0031] regenerating a decision action of the terminal and a decision action of the network device according to the updated status information of the terminal and / or the network device;

[0032] When the regenerated decision action of the network device is updated, the updated decision action of the network device is sent to the network device.

[0033] In a second aspect, an embodiment of the present application further provides a device-edge collaboration model inference method, which is executed by a network device. The method includes:

[0034] Receiving an inference request message carrying model information sent by a terminal;

[0035] Acquiring status information of the network device, wherein the status information at least includes computing resource status information;

[0036] generating, according to the state information of the network device, an inference reply message, wherein the inference reply message indicates whether the network device agrees with the collaborative inference;

[0037] Sending the inference reply message to the terminal, wherein, if the network device agrees to collaborative inference, the inference reply message also carries status information of the network device;

[0038] When the network device agrees to collaborative reasoning, receiving a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal according to the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding computing resource allocation information;

[0039] Allocate computing resources to the second model part that the network device indicated in the second decision action is responsible for calculating, and perform model inference calculation on the second model part.

[0040] Optionally, obtaining the status information of the network device includes:

[0041] Sending a radio channel quality information request message to a radio resource management (RRM) module of the network device; receiving a radio channel quality information reply message returned by the RRM module and carrying uplink and downlink radio channel quality information of the terminal;

[0042] Sending a computing resource information request message carrying computing task information to a computing resource management (CRM) module of the network device; receiving a computing resource information reply message carrying computing resource status information of the network device returned by the CRM module;

[0043] Send a computing performance model request message to the computing performance model management CPMM module of the network device, wherein the computing performance model request message carries the remaining computing resource type information and model layer type information of the network device; receive a computing performance model reply message carrying the computing performance model information returned by the CPMM module.

[0044] Optionally, the allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion, includes:

[0045] determining whether to execute the second decision action according to at least one of the resource status information, user priority, and service priority of the network device;

[0046] In the case of determining to execute the second decision action, allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion;

[0047] If it is determined that the second decision action is not to be performed, changing the second decision action and sending a computing resource allocation update message to the terminal, wherein the computing resource allocation update message carries a third decision action for updating the network device;

[0048] Allocate computing resources to the fourth model portion that the network device indicated in the third decision action is responsible for computing, and perform model inference calculations on the fourth model portion.

[0049] Optionally, the allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion, includes:

[0050] Sending a wireless resource allocation suggestion message to the RRM module of the network device, wherein the wireless resource allocation suggestion message carries the uplink and downlink channel bandwidth allocated for the model inference computing task;

[0051] Sending a second computing resource allocation suggestion message to the CRM module of the network device, wherein the second computing resource allocation suggestion message includes parameters of the second model part, the computing resource type and quantity of computing resources for each layer in the second model part, and the timing and data transmission relationship of inter-layer calculations;

[0052] The CRM module of the network device allocates computing tasks of different layers in the second model part to different types of computing resources of the network device according to the second computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

[0053] Optionally, the method further includes:

[0054] receiving an intermediate result of model inference of the first model part sent by the terminal;

[0055] Integrating the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0056] Sending the model collaborative reasoning result to the terminal.

[0057] Optionally, the method further includes:

[0058] Collecting information on the inference energy consumption of the network device, wherein the inference energy consumption information includes an average single inference energy consumption value of the network device within a preset time period or an average single inference computing power value consumed by the network device;

[0059] The inference energy consumption information is sent to the terminal.

[0060] Optionally, the method further includes:

[0061] When a change in the status information of the network device is detected, sending a status update message to the terminal, wherein the status update message includes the updated status information of the network device;

[0062] And / or, receiving a status update request message sent by the terminal; and sending the latest status information of the network device to the terminal.

[0063] In a third aspect, an embodiment of the present application provides a terminal, including:

[0064] A first sending module, configured to send an inference request message carrying model information to a network device;

[0065] A first receiving module is configured to receive an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to collaborative inference, and when the inference reply message indicates that the network device agrees to collaborative inference, the inference reply message also carries status information of the network device;

[0066] a first generating module, configured to generate, when the inference reply message indicates that the network device agrees to collaborative reasoning, a first decision action for the terminal and a second decision action for the network device based on the state information of the network device and the state information of the terminal, wherein the first decision action and the second decision action include a model portion responsible for calculation and corresponding computing resource allocation information, and the state information of the network device and the state information of the terminal include at least computing resource state information;

[0067] a first processing module, configured to allocate computing resources to the first model portion that the terminal is responsible for calculating in the first decision action, and perform model inference calculation on the first model portion;

[0068] The second sending module is used to send the second decision action to the network device, so that the network device allocates computing resources to the second model part that the network device is responsible for calculating indicated in the second decision action, and performs model inference calculation of the second model part.

[0069] In a fourth aspect, an embodiment of the present application provides a network device, including:

[0070] A second receiving module is used to receive an inference request message carrying model information sent by a terminal;

[0071] A first acquisition module is configured to acquire status information of the network device, wherein the status information at least includes computing resource status information;

[0072] A second generating module is configured to generate an inference reply message according to the status information of the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference;

[0073] A third sending module is configured to send the inference reply message to the terminal, wherein, if the network device agrees to collaborative inference, the inference reply message also carries status information of the network device;

[0074] a third receiving module, configured to receive, when the network device agrees to collaborative reasoning, a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal based on the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding computing resource allocation information;

[0075] The second processing module is used to allocate computing resources to the second model part that the network device indicated in the second decision action is responsible for calculating, and perform model inference calculation of the second model part.

[0076] In the fifth aspect, an embodiment of the present application also provides a terminal, comprising: a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, the steps in the end-edge collaboration model inference method as described in the first aspect are implemented.

[0077] In the sixth aspect, an embodiment of the present application also provides a network device, comprising: a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, the steps in the end-edge collaboration model inference method as described in the second aspect are implemented.

[0078] In the seventh aspect, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the end-edge collaboration model inference method described in the first and second aspects are implemented.

[0079] The end-edge collaborative model inference method of the embodiment of the present application is executed by the terminal, and the terminal sends an inference request message carrying model information to the network device; and receives an inference reply message sent by the network device, wherein the inference reply message also carries the status information of the network device. According to the status information of the network device and the status information of the terminal, a first decision action of the terminal and a second decision action of the network device are generated. In this embodiment, the first decision action and the second decision action are generated based on the status information of the network device and the status information of the terminal. Therefore, when splitting the inference model, the difference in computing resources of the network device and the terminal is taken into account, which makes the segmentation of the inference model more reasonable, thereby improving the model inference performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0081] Figure 1 This is one of the flow charts of the device-edge collaboration model reasoning method provided in an embodiment of the present application;

[0082] Figure 2 This is the second flowchart of the device-edge collaboration model reasoning method provided in an embodiment of the present application;

[0083] Figure 3 This is the third flowchart of the device-edge collaboration model reasoning method provided in an embodiment of the present application;

[0084] Figure 4 This is one of the structural diagrams of the terminal provided in the embodiment of the present application;

[0085] Figure 5 This is one of the structural diagrams of the network device provided in the embodiment of the present application;

[0086] Figure 6 This is the second structural diagram of the terminal provided in the embodiment of the present application;

[0087] Figure 7 This is the second structural diagram of the network device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0088] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0089] See also Figure 1 , Figure 1 This is one of the flow charts of the device-edge collaboration model reasoning method provided in the embodiment of the present application, such as Figure 1 As shown, the edge-device collaboration model reasoning method is executed by the terminal, and the edge-device collaboration model reasoning method includes the following steps:

[0090] Step 101: Send an inference request message carrying model information to a network device.

[0091] First, the terminal sends an inference request message to the network device, which carries model information. In this embodiment, the model information may include at least one of the following: the terminal identifier, the number of model layers, the model type for each layer, and the computational load parameter variable value for each layer. In other embodiments, the model information may also include at least one of the following: the layer number that the terminal recommends the network device calculate, the terminal's latency requirement for inference calculations on the network device, the terminal's latency requirement for uplink and downlink air interface transmission, and the terminal's energy consumption requirement for a single inference on the network device.

[0092] Specifically, the terminal identifier may be a temporary terminal identifier and an identity document (ID) of an existing radio resource control (RRC) connection. The type of each layer model is a neural network layer type, and its specific type is not limited here. For example, in some embodiments, the type of each layer model may be any one of the following: a convolutional layer, an activation layer, a pooling layer, a fully connected layer, a dropout layer, and a local response regularization layer.

[0093] The network device receives an inference request message carrying model information sent by the terminal. In this embodiment, the computing resource status information can be understood as status information of the remaining allocatable computing resources of the network device. The network device obtains the status information of the network device, wherein the status information at least includes computing resource status information.

[0094] It should be understood that in some embodiments, the network device includes a first interaction module. The terminal includes a second interaction module. The first interaction module of the network device can be used to interact with the second interaction module of the terminal. The first interaction module can also interact with other modules of the network device. The second interaction module can also interact with other modules of the terminal.

[0095] Specifically, obtaining the status information of the network device can be understood as the following interaction process: the first interaction module sends a radio channel quality information request message to a radio resource management (Radio Resource Management, RRM) module of the network device.

[0096] It should be understood that the wireless channel quality request message is used to obtain the uplink and downlink wireless channel quality information of the terminal from the RRM module. The wireless channel quality information request message may include a terminal identifier. The specific content of the terminal identifier is no longer limited here. For example, in some embodiments, the terminal identifier may include the terminal temporary identifier and the ID of an existing RRC connection, etc. The statistical indicators of the uplink and downlink wireless channel quality information of the terminal are not limited here. For example, in some embodiments, the uplink and downlink wireless channel quality information of the terminal can be reflected using a channel quality indicator (Channel Quality Indicator, CQI). In other embodiments, the uplink and downlink wireless channel quality information of the terminal can be reflected using a received signal strength indication (Received Signal Strength Indication, RSSI).

[0097] The first interaction module receives a radio channel quality information reply message returned by the RRM module of the network device and carrying the uplink and downlink radio channel quality information of the terminal.

[0098] It should be understood that, in some embodiments, the wireless channel quality information reply message includes the uplink and downlink wireless channel quality information of the terminal, such as CQI. In some embodiments, the wireless channel quality information reply message also includes uplink and downlink wireless channel bandwidth information allocated to the terminal.

[0099] The first interaction module sends a computing resource information request message carrying computing task information to a computing resource management (CRM) module of the network device.

[0100] It should be understood that in some embodiments, the computing resource information request message may include a terminal identifier, a computing task type description, a layer type description, a statistical description of the layer type computing amount, a description of the computing operation type, or a statistical description of the computing operation type computing amount.

[0101] It should be understood that the specific content of the terminal identifier is no longer limited here. For example, in some embodiments, the terminal identifier may include the terminal temporary identifier and the ID of the existing RRC connection, etc. The computing task type can be understood as model reasoning. The layer type can be understood as a convolutional layer, an activation layer, a pooling layer, a fully connected layer, a dropout layer, and a local response regularization layer. The statistical description of the layer type calculation amount can be understood as the calculation amount of each layer type mentioned above. The operation type can be a multiplication and addition operation, an exclusive OR operation, or an activation function operation. The operation operation type calculation amount can be understood as the calculation amount of each operation operation type mentioned above.

[0102] The first interaction module receives a computing resource information reply message returned by the CRM module and carrying computing resource status information of the network device.

[0103] It should be understood that the computing resource information reply message may include the types of computing resources remaining allocatable by the network device and the remaining allocatable quantity of each type of computing resources.

[0104] It should be understood that the type of the allocatable computing resources is not limited herein. For example, in some embodiments, the type of the computing resources may be at least one of the following: a central processing unit (CPU), a graphics processing unit (GPU), or a neural-network processing unit (NPU).

[0105] It should be understood that the remaining allocatable amount of each type of computing resource can be understood as the remaining computing power of each type of computing resource. In some embodiments, the computing power can be described using the utilization rate of each type of computing resource. In other embodiments, the computing power can be described using the number of floating-point operations per second (FLOPS) performed at each quantization level. In other embodiments, the computing power can also be described using the number of basic computing units, such as multiply accumulate (MAC) computing units and computing cores.

[0106] The first interaction module sends a computing performance model request message to a computing performance model management (CPMM) module of the network device, wherein the computing performance model request message carries remaining computing resource type information and model layer type information of the network device.

[0107] The first interaction module receives a computing performance model reply message carrying computing performance model information returned by the CPMM module.

[0108] It should be understood that the computing performance model reply message carrying computing performance model information can be understood as carrying the computing performance model for each computing resource type and each layer type. For example, in some embodiments, there are three computing resource types and three layer types. In this embodiment, there are nine corresponding computing performance models.

[0109] Then, the network device generates an inference reply message according to the state information of the network device, wherein the inference reply message indicates whether the network device agrees with the collaborative inference, and the network device sends the inference reply message to the terminal.

[0110] It should be understood that the indication in the reasoning reply message of whether the network device agrees with the collaborative reasoning can be understood as the reasoning reply message including an indication result, and the indication result can be agreement with the collaborative reasoning or disagreement with the collaborative reasoning.

[0111] It should be understood that the content included in the inference reply message may also vary depending on the indication result. If the network device agrees to collaborative inference, the inference reply message also carries the network device's status information. If the network device disagrees with collaborative inference, the inference reply message only carries the indication result.

[0112] It should be understood that, in some embodiments, the status information of the network device may include at least one of the following: the remaining allocatable computing power of various types of computing resources within the network device, the remaining allocatable uplink and downlink air interface channel transmission resources of the corresponding cell of the network device, and the uplink channel quality.

[0113] It should be understood that, in some embodiments, the remaining allocable uplink and downlink air interface channel transmission resources of the cell corresponding to the network device can be understood or expressed as the number of resource blocks (RBs). The terminal uplink channel quality can be understood or expressed as CQI or RSSI.

[0114] It should be understood that in other embodiments, the network device status information may further include: the transmission bandwidth between various types of computing resources within the network device, where the transmission bandwidth may be expressed in Mbps or Gbps. In other embodiments, the network device status information may further include a computing performance model corresponding to the computing resources that the network device can allocate to the terminal's inference task.

[0115] Step 102: Receive an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to collaborative reasoning. When the inference reply message indicates that the network device agrees to collaborative reasoning, the inference reply message also carries status information of the network device.

[0116] It should be understood that the terminal receives the inference reply message sent by the network device. If the inference reply message indicates that the network device agrees to collaborative inference, the terminal executes subsequent steps based on the inference reply message. If the inference reply message indicates that the network device does not agree to collaborative inference, the terminal may stop executing subsequent steps or resend the inference request message to the network device.

[0117] Step 103: When the reasoning reply message indicates that the network device agrees to collaborative reasoning, a first decision action of the terminal and a second decision action of the network device are generated based on the status information of the network device and the status information of the terminal. The first decision action and the second decision action include the model part responsible for calculation and the corresponding computing resource allocation information. The status information of the network device and the status information of the terminal at least include computing resource status information.

[0118] The terminal may receive an inference reply message sent by the network device. When the inference reply message indicates that the network device agrees to collaborative inference, the inference reply message also carries status information of the network device.

[0119] The terminal may generate a first decision action for the terminal and a second decision action for the network device based on the status information of the network device and the status information of the terminal. The terminal status information is acquired by the terminal. The terminal status information may include at least one of the following: the remaining allocatable computing power of various types of computing resources within the terminal, the remaining allocatable uplink and downlink air interface channel transmission resources of the cell corresponding to the terminal, and the downlink channel quality.

[0120] According to the above content, the status information includes the status information of the network device and the status information of the terminal. Among them, the status information of the network device includes the remaining allocable computing power of each type of computing resource in the network device (such as FLOPS of each quantization level, the number of basic computing units (such as MAC operation units, computing cores), utilization, the above description of physical computing resources or the above description of virtual computing resources), the transmission bandwidth between each type of computing resource in the network device, the remaining allocable uplink and downlink air interface channel transmission resources (such as the number of RBs) of the cell corresponding to the network device, and the terminal uplink channel quality (such as CQI and RSSI).

[0121] The status information of the terminal includes the remaining allocatable computing power of various types of computing resources in the terminal (such as FLOPS of each quantization level, the number of basic computing units (such as MAC operation units, computing cores), utilization, the above description of physical computing resources or the above description of virtual computing resources), the transmission bandwidth between various types of computing resources in the terminal, and the terminal downlink channel quality (such as CQI and RSSI).

[0122] Step 104: Allocate computing resources to the first model portion that the terminal is responsible for calculating in the first decision action, and perform model inference calculations on the first model portion. The specific manner in which the terminal allocates computing resources to the first model portion that the terminal is responsible for calculating in the first decision action and performs model inference calculations on the first model portion is not limited herein.

[0123] For example, optionally, in some embodiments, step 104 includes the following steps:

[0124] Sending a first computing resource allocation suggestion message to the computing resource management module of the terminal, wherein the first computing resource allocation suggestion message carries parameters of the first model part, the computing resource type and quantity of computing resources of each layer in the first model part, and the timing and data transmission relationship of inter-layer calculations;

[0125] The computing resource management module allocates computing tasks of different layers in the first model portion to different types of computing resources of the terminal based on the first computing resource allocation recommendation message, and configures data transmission relationships between the different types of computing resources. The first decision action indicates that the computing resources are allocated to the first model portion for which the terminal is responsible for computing.

[0126] In this embodiment, the terminal sends a first computing resource allocation suggestion message to the computing resource management module of the terminal. The first computing resource allocation suggestion message includes parameters of the first model portion, the computing resource type and quantity of each layer in the first model portion, and the timing and data transmission relationship of inter-layer computations. Using the first computing resource allocation suggestion message, the computing resource management module of the terminal can complete the allocation of computing resources, making the computing resource allocation on the terminal more reasonable.

[0127] It should be understood that the terminal sending the first computing resource allocation recommendation message to the computing resource management module of the terminal can be understood as, based on the first decision action, the second interaction module of the terminal sending the computing resource allocation message carrying the model parameters of the first model part for which it is responsible and the corresponding computing resource allocation information to the computing resource management module of the terminal.

[0128] In this embodiment, the computing resource allocation message includes at least one of the following: the computing resource allocation message includes model parameters, computing resource types responsible for computing at each layer, the number of computing resources, the timing of inter-layer computing, and the data transmission relationship.

[0129] After receiving the computing resource allocation message, the CRM module distributes computing tasks of different layers to computing resources at different locations and configures the data transmission relationship between the computing resources. The model reasoning calculation of the first model part will be completed on its corresponding computing resources.

[0130] It should be understood that the terminal executing the decision action can be understood as the second interaction module of the terminal sending a computing resource allocation recommendation message to the CRM module of the terminal, and the CMR module of the terminal responds to the computing resource allocation recommendation message, assigning computing tasks of different layers to computing resources of terminals in different locations, and configuring the data transmission relationship between computing resources.

[0131] Step 105: Send the second decision action to the network device so that the network device allocates computing resources to the second model part that the network device is responsible for calculating as indicated in the second decision action, and performs model inference calculation on the second model part.

[0132] The terminal sends the generated second decision action to the network device. The second decision action includes at least one of the following: model parameters corresponding to the partial model that the network device is responsible for calculating, the type of computing resources for each layer of calculation, the number of computing resources, the timing of inter-layer calculations, and the data transmission relationship. In other embodiments, the second decision action also includes uplink and downlink bandwidth allocation recommendations, such as uplink and downlink guaranteed data rates. In other embodiments, the second decision action may also include a terminal computing task assignment success indication.

[0133] The network device receives the second decision action sent by the terminal, allocates computing resources to the second model portion that the network device is responsible for computing as indicated in the second decision action, and performs model inference computing on the second model portion.

[0134] It should be understood that the network device allocates computing resources to the second model part that the network device is responsible for calculating as indicated in the second decision action, and the specific method for performing model inference calculation of the second model part is not limited here.

[0135] It should be understood that the network device executing the decision action can be understood as the first interaction module of the network device sending a computing resource allocation recommendation message to the CRM module of the network device, and the CMR module of the network device responds to the computing resource allocation recommendation message, assigning computing tasks of different layers to computing resources of network devices at different locations, and configuring the data transmission relationship between computing resources.

[0136] In some embodiments, after receiving the decision action sent by the terminal, the network device first determines whether to perform the second decision action based on at least one of the resource status information, user priority, and service priority of the network device;

[0137] In the case of determining to execute the second decision action, allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion;

[0138] If it is determined that the second decision action is not to be performed, changing the second decision action and sending a computing resource allocation update message to the terminal, wherein the computing resource allocation update message carries a third decision action for updating the network device;

[0139] Allocate computing resources to the fourth model portion that the network device indicated in the third decision action is responsible for computing, and perform model inference calculations on the fourth model portion.

[0140] In this embodiment, after receiving the decision action sent by the terminal, the network device first determines whether to execute the second decision action based on at least one of the network device's resource status information, user priority, and service priority. With this configuration, the network device determines whether the second decision action is necessary based on its own status before executing it, making the second decision action more consistent with the network device's current status and further improving model inference performance.

[0141] It should be understood that the network device determines whether to execute the second decision action based on at least one of the resource status information, user priority and business priority of the network device. It can be understood that the network device determines whether the second decision action can be executed based on at least one of the resource status information, user priority and business priority of the network device using the decision action evaluation and adjustment module of the network device.

[0142] In a case where the second decision action can be executed, the network device determines to execute the second decision action.

[0143] In other embodiments, after receiving the decision action sent by the terminal, the network device sends a radio resource allocation recommendation message to the RRM module of the network device, wherein the radio resource allocation recommendation message carries the uplink and downlink channel bandwidth allocated for the model inference computing task;

[0144] Sending a second computing resource allocation suggestion message to the CRM module of the network device, wherein the second computing resource allocation suggestion message includes parameters of the second model part, the computing resource type and quantity of computing resources for each layer in the second model part, and the timing and data transmission relationship of inter-layer calculations;

[0145] The CRM module of the network device allocates computing tasks of different layers in the second model part to different types of computing resources of the network device according to the second computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

[0146] Specifically, the first interaction module of the network device sends a wireless resource allocation suggestion message carrying the uplink and downlink bandwidth allocation in the second decision action to the RRM model of the network device, wherein the wireless resource allocation suggestion message includes the uplink and downlink channel bandwidths recommended to be allocated for the collaborative reasoning task of the terminal model.

[0147] The first interaction module of the network device sends a computing resource allocation suggestion message carrying model parameters of the model portion for which the network device is responsible and corresponding computing resource allocation information to the CRM module of the network device. The computing resource allocation suggestion message includes the model portion parameters, the type of computing resources responsible for each layer of computing, the number of computing resources, the timing of inter-layer computing, and the data transmission relationship.

[0148] The CMR module of the network device distributes the computing tasks of different model layers to computing resources at different locations and configures the data transmission relationship between the computing resources. The model reasoning calculation of the second model part will be completed on its corresponding computing resources.

[0149] If the second decision action cannot be executed, the network device determines not to execute the second decision action. In this case, the network device changes the second decision action and sends a computing resource allocation update message to the terminal, wherein the computing resource allocation update message carries the updated third decision action of the network device.

[0150] The network device allocates computing resources to the fourth model portion that the network device is responsible for computing, as indicated in the third decision action, and performs model inference calculations on the fourth model portion. The process for performing model inference calculations on the fourth model portion is the same as the process for performing model inference calculations on the second model portion described above and is not further described here.

[0151] Optionally, in some embodiments, the terminal receives a computing resource allocation update message sent by the network device when the second decision action is changed, wherein the computing resource allocation update message carries a third decision action updated by the network device;

[0152] According to the third decision action, adjusting the first decision action to obtain a fourth decision action;

[0153] According to the fourth decision action, the third model part that the terminal is responsible for calculating is re-determined, and computing resources are allocated to the third model part to perform model inference calculation of the third model part.

[0154] In this embodiment, since the network device changes the second decision action, the terminal can adjust the first decision action accordingly to obtain the fourth decision action. Through the above settings, after the network device changes the second decision action, the terminal can also adjust the first decision action accordingly to match the third decision action of the network device.

[0155] Specifically, after receiving the computing resource allocation update message, the terminal uses the decision action evaluation and adjustment module of the terminal to decide whether to adjust the first decision action according to the third decision action.

[0156] It should be understood that the specific method by which the terminal adjusts the first decision action based on the third decision action is not limited herein. For example, in some embodiments, the terminal may use an algorithm based on the first decision action to calculate the portion of the terminal in the new action based on the original decision action previously made, in accordance with the principle of minimizing the distance between the new action and the original action.

[0157] It should be understood that after the terminal decides to adjust the first decision action based on the third decision action, the process of executing the adjusted fourth decision action is the same as the above-mentioned model inference calculation process of the first model part, and will not be repeated here.

[0158] Optionally, in some embodiments, the device-edge collaborative model inference method executed by the terminal further includes the following steps:

[0159] Sending the model reasoning intermediate result of the first model part to the network device, so that the network device integrates the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0160] Receive the model collaborative reasoning result sent by the network device.

[0161] In this embodiment, the terminal sends the intermediate model inference results of the first model portion to the network device, and the network device obtains the intermediate model inference results of the second model portion. By integrating the intermediate model inference results of the first model portion with the intermediate model inference results of the second model portion, the network device obtains the model collaborative inference results and shares the model collaborative inference results with the terminal. This configuration allows for testing the effectiveness of model collaborative inference and adjusting the collaboration strategy in real time based on the model collaborative inference results to further improve model inference performance.

[0162] It should be understood that the intermediate result of the model reasoning of the first model part is obtained after the terminal performs the model reasoning calculation of the first model part and completes the model reasoning calculation of the first model part.

[0163] It should be understood that the inference intermediate result of the second model part is obtained after the network device performs the model inference calculation of the second model part and completes the model inference calculation of the second model part. The terminal sends the model inference intermediate result of the first model part to the network device. The network device receives the model inference intermediate result of the first model part sent by the terminal, and then integrates the model inference intermediate result of the first model part and the model inference intermediate result of the second model part to obtain the model collaborative reasoning result, and sends the model collaborative reasoning result to the terminal. The effect of the model collaborative reasoning can be tested through the model collaborative reasoning result.

[0164] Optionally, in some embodiments, the device-edge collaborative model inference method executed by the terminal further includes the following steps:

[0165] Acquire model reasoning performance parameters, where the model reasoning performance parameters include at least one of reasoning accuracy, average reasoning latency, and average single reasoning energy consumption;

[0166] Calculating reward information based on the model reasoning performance parameters;

[0167] The state information, decision action and reward information in each collaborative reasoning are collected as training samples of the reinforcement learning model used by the terminal and the network device for model collaborative reasoning, and the reinforcement learning model is trained and optimized.

[0168] It should be understood that the reward information may include at least one of the following: a terminal reasoning latency reward, a terminal reasoning accuracy reward, a terminal energy consumption reward, and a network device energy consumption reward. In some embodiments, the reward information is determined based on the terminal reasoning latency reward, the terminal reasoning accuracy reward, the terminal energy consumption reward, and the network device energy consumption reward. The specific method for calculating the reward information based on the terminal reasoning latency reward, the terminal reasoning accuracy reward, the terminal energy consumption reward, and the network device energy consumption reward is not limited herein.

[0169] In this embodiment, the state information, decision actions, and reward information collected from each collaborative reasoning session are used as sample data to train the terminal's generative value estimation neural network and decision generation neural network. When making a decision, the real-time state is input into the decision generation neural network, which then outputs the decision action. During reasoning, real-time state information is acquired, decision actions are generated, and after being applied to the wireless system, corresponding reward information is collected. If the state information changes, the value estimation neural network and decision generation neural network must promptly be informed of the new state information, provide new action information, and then collect the corresponding reward information.

[0170] In this embodiment, reward information is calculated based on model reasoning performance parameters. State information, decision actions, and reward information from each collaborative reasoning session are collected and used as training samples for the reinforcement learning model used by the terminal and network device for model collaborative reasoning. This training optimizes the reinforcement learning model. By optimizing and training the reinforcement learning model, the decision model can be continuously optimized, further improving model reasoning performance.

[0171] It should be understood that the model inference performance parameters obtained by the terminal are model performance inference parameters within a preset time length. In specific implementations, the model performance parameters can be statistical values ​​of the average single inference energy consumption of the network device, or can also include statistical values ​​of the average computing power consumed by the network device per single inference.

[0172] It should be understood that the specific method for obtaining the model reasoning performance parameter is not limited herein. For example, optionally, in some embodiments, obtaining the model reasoning performance parameter specifically includes the following steps:

[0173] Counting the inference accuracy, average inference latency, and average single inference energy consumption of the terminal within a preset time period;

[0174] Receive the inference energy consumption information sent by the network device, wherein the inference energy consumption information includes the average single inference energy consumption value of the network device within the preset time length or the average single inference computing power value consumed by the network device.

[0175] When the terminal counts the inference accuracy, average inference delay and average single inference energy consumption of the terminal within a preset time length, the network device counts the inference energy consumption information of the network device, wherein the inference energy consumption information includes the average single inference energy consumption value of the network device within the preset time length or the computing power value consumed by the average single inference of the network device.

[0176] The network device sends the inference energy consumption information to the terminal. The terminal receives the inference energy consumption information sent by the network device.

[0177] It should be understood that the model reasoning performance parameters include the reasoning accuracy, average reasoning delay and average single reasoning energy consumption of the terminal within a preset time period and the reasoning energy consumption information statistically collected by the network device.

[0178] It should be understood that the specific method for calculating the reward information based on the model reasoning performance parameters is not limited here.

[0179] It should be understood that the state information in each collaborative reasoning includes the state information of the terminal and the state information of the network device. The decision action in each collaborative reasoning includes the decision action executed by the terminal and the decision action executed by the network device. For example, in some embodiments, the decision action in each collaborative reasoning may include at least one of the following: a first decision action, a second decision action, a third decision action, and a fourth decision action.

[0180] Optionally, in some embodiments, the device-edge collaborative model inference method executed by the terminal further includes the following steps:

[0181] receiving a status update message sent by the network device, wherein the status update message includes updated status information of the network device;

[0182] and / or, when detecting a change in the status information of the terminal, sending a status update request message to the network device; and receiving the latest status information sent by the network device;

[0183] regenerating a decision action of the terminal and a decision action of the network device according to the updated status information of the terminal and / or the network device;

[0184] When the regenerated decision action of the network device is updated, the updated decision action of the network device is sent to the network device.

[0185] In this embodiment, if the state of at least one of the network device and the terminal changes, the terminal can regenerate its decision-making actions and those of the network device based on the updated state information of the terminal and / or the network device. This arrangement allows the decision-making actions to be adjusted based on the real-time state of the network device and the terminal, making them more adaptable to their current states and further improving model reasoning performance.

[0186] It should be understood that the change in the state information of the network device can be understood as the change value of any variable in the state information of the network device exceeds a threshold, or the type of computing resources allocated to the inference computing task of the network device changes.

[0187] It should be understood that the change in the terminal's status information can be understood as a change in the value of any variable in the terminal's status information exceeding a threshold, or a change in the type of computing resources allocated to the terminal's inference computing task.

[0188] It should be understood that the specific content of the network device's updated status information is not limited herein. For example, in some embodiments, the network device's updated status information includes the network device's updated variables and the variable's changed value. In other embodiments, if the computing resource type assigned to the network device's inference computing task changes from a CPU to a GPU, the network device's updated status information includes the computing performance model corresponding to the GPU.

[0189] It should be understood that the specific content of the terminal's updated status information is not limited herein. For example, in some embodiments, the terminal's updated status information includes variables updated by the network device and the changed values ​​of the variables. In other embodiments, if the computing resource type assigned to the terminal's inference computing task changes from a CPU to a GPU, the terminal's updated status information includes a computing performance model corresponding to the GPU.

[0190] In one case, when the network device detects a change in the state information of the network device, the network device sends a state update message to the terminal, wherein the state update message includes the updated state information of the network device.

[0191] In this case, the terminal regenerates the decision action of the terminal and the decision action of the network device based on the updated state information of the network device. If the regenerated decision action of the network device is updated, the terminal sends the updated decision action of the network device to the network device.

[0192] In another embodiment, when the terminal detects a change in the terminal's status information, the terminal sends a status update request message to the network device. The network device receives the status update request message sent by the terminal. In response to the status update request, the network device sends the latest status information of the network device to the terminal.

[0193] In this case, the terminal regenerates the decision action of the terminal and the decision action of the network device according to the updated state information of the terminal. If the regenerated decision action of the network device is updated, the terminal sends the updated decision action of the network device to the network device.

[0194] It should be understood that, in a specific implementation, if at least one of the status information of the terminal and the status information of the network device changes, that is, if at least one of the above two situations occurs, the terminal may update the decision information.

[0195] It should be understood that, in the above two cases, if the regenerated decision action of the network device is not updated, the terminal does not need to send the updated decision action of the network device to the network device.

[0196] According to the above content, the decision-making action includes the decision-making action performed by the network device and the decision-making action performed by the terminal. The decision-making action performed by the network device includes the model part (model parameters) that the network device is responsible for calculating, the uplink bandwidth allocation suggestion for this inference task (such as the number of uplink RBs and the uplink guaranteed data rate), the downlink bandwidth allocation suggestion (such as the number of downlink RBs and the downlink guaranteed data rate), and the computing resource allocation of the network device for model calculation (in the model part allocated to the network device, which type of computing resource calculation is placed on each layer, and the number of computing resources allocated to the layer (such as FLOPS, number of computing units or utilization)).

[0197] The decision-making actions performed by the terminal include the model part (model parameters) that the terminal is responsible for calculating, the allocation of computing resources for model calculation (for example, in the model part allocated to the terminal, which type of computing resources each layer is placed on for calculation, and the amount of computing resources allocated to the layer (for example, FLOPS, number of computing units or utilization)).

[0198] Specifically, the decision action may be the first decision action, the second decision action, the third decision action, or the fourth decision action.

[0199] The end-edge collaborative model inference method of the embodiment of the present application is executed by the terminal, and the terminal sends an inference request message carrying model information to the network device; and receives an inference reply message sent by the network device, wherein the inference reply message also carries the status information of the network device. According to the status information of the network device and the status information of the terminal, a first decision action of the terminal and a second decision action of the network device are generated. In this embodiment, the first decision action and the second decision action are generated based on the status information of the network device and the status information of the terminal. Therefore, when splitting the inference model, the difference in computing resources of the network device and the terminal is taken into account, which makes the segmentation of the inference model more reasonable, thereby improving the model inference performance.

[0200] See also Figure 2 , Figure 2 This is the second flow chart of the device-edge collaboration model reasoning method provided in the embodiment of the present application. Figure 2 As shown, the end-edge collaboration model reasoning method performed by the network device includes the following steps:

[0201] Step 201: receiving an inference request message carrying model information sent by a terminal;

[0202] Step 202: Acquire status information of the network device, wherein the status information at least includes computing resource status information;

[0203] Step 203: Generate an inference reply message based on the state information of the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference;

[0204] Step 204: Send the inference reply message to the terminal, wherein, if the network device agrees to the collaborative inference, the inference reply message also carries the status information of the network device;

[0205] Step 205: If the network device agrees to collaborative reasoning, receiving a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal based on the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding computing resource allocation information;

[0206] Step 206: Allocate computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and perform model inference calculations on the second model portion.

[0207] Optionally, step 202 includes:

[0208] Sending a radio channel quality information request message to the RRM module of the network device; receiving a radio channel quality information reply message returned by the RRM module and carrying the uplink and downlink radio channel quality information of the terminal;

[0209] Sending a computing resource information request message carrying computing task information to the CRM module of the network device; receiving a computing resource information reply message carrying computing resource status information of the network device returned by the CRM module;

[0210] Send a computing performance model request message to the CPMM module of the network device, wherein the computing performance model request message carries the remaining computing resource type information and model layer type information of the network device; and receive a computing performance model reply message carrying the computing performance model information returned by the CPMM module.

[0211] Optionally, step 206 includes:

[0212] determining whether to execute the second decision action according to at least one of the resource status information, user priority, and service priority of the network device;

[0213] In the case of determining to execute the second decision action, allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion;

[0214] If it is determined that the second decision action is not to be performed, changing the second decision action and sending a computing resource allocation update message to the terminal, wherein the computing resource allocation update message carries a third decision action for updating the network device;

[0215] Allocate computing resources to the fourth model portion that the network device indicated in the third decision action is responsible for computing, and perform model inference calculations on the fourth model portion.

[0216] In this embodiment, since the network device changes the second decision action, the terminal can adjust the first decision action accordingly to obtain the fourth decision action. Through the above settings, after the network device changes the second decision action, the terminal can also adjust the first decision action accordingly to match the third decision action of the network device.

[0217] Optionally, step 206 includes:

[0218] Sending a wireless resource allocation suggestion message to the RRM module of the network device, wherein the wireless resource allocation suggestion message carries the uplink and downlink channel bandwidth allocated for the model inference computing task;

[0219] Sending a second computing resource allocation suggestion message to the CRM module of the network device, wherein the second computing resource allocation suggestion message includes parameters of the second model part, the computing resource type and quantity of computing resources for each layer in the second model part, and the timing and data transmission relationship of inter-layer calculations;

[0220] The CRM module of the network device allocates computing tasks of different layers in the second model part to different types of computing resources of the network device according to the second computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

[0221] Optionally, the method further includes:

[0222] receiving an intermediate result of model inference of the first model part sent by the terminal;

[0223] Integrating the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0224] Sending the model collaborative reasoning result to the terminal.

[0225] Optionally, the method further includes:

[0226] Collecting information on the inference energy consumption of the network device, wherein the inference energy consumption information includes an average single inference energy consumption value of the network device within a preset time period or an average single inference computing power value consumed by the network device;

[0227] The inference energy consumption information is sent to the terminal.

[0228] In this embodiment, reward information is calculated based on model reasoning performance parameters. State information, decision actions, and reward information from each collaborative reasoning session are collected and used as training samples for the reinforcement learning model used by the terminal and network device for model collaborative reasoning. This training optimizes the reinforcement learning model. By optimizing and training the reinforcement learning model, the decision model can be continuously optimized, further improving model reasoning performance.

[0229] Optionally, the method further includes:

[0230] When a change in the status information of the network device is detected, sending a status update message to the terminal, wherein the status update message includes the updated status information of the network device;

[0231] And / or, receiving a status update request message sent by the terminal; and sending the latest status information of the network device to the terminal.

[0232] In this embodiment, if the state of at least one of the network device and the terminal changes, the terminal can regenerate its decision-making actions and those of the network device based on the updated state information of the terminal and / or the network device. This arrangement allows the decision-making actions to be adjusted based on the real-time state of the network device and the terminal, making them more adaptable to their current states and further improving model reasoning performance.

[0233] It should be noted that this embodiment is as Figure 1 The embodiment of the network device corresponding to the example shown can also be applied to Figure 1 In the corresponding embodiments and to achieve the same beneficial effects, the specific implementation methods can be found in Figure 1 To avoid repetition, the relevant introduction in the illustrated embodiments will not be repeated here.

[0234] See Figure 3 , Figure 3 This is the third flow chart of the edge-end collaboration model reasoning method provided in the embodiment of this application. Figure 3 As shown, the specific process of the end-edge collaboration model reasoning method will be introduced below by taking a specific embodiment as an example.

[0235] First, it should be noted that in this embodiment, the terminal has three types of computing resources: CPU, GPU, and DSP. The network device has two types of computing resources: CPU and GPU. The inference model is a Deep Neural Networks (DNN) model, specifically MobileNet v1. This model consists of 14 convolutional layers and 1 fully connected layer, and the 15 network layers of the model are numbered from 1 to 15.

[0236] In this embodiment, the hardware information of the computing resources of the terminal is shown in the following table:

[0237] Table 1 Hardware information of terminal computing resources

[0238]

[0239]

[0240] In this embodiment, the network structure of MobileNet v1 is a depthwise separable convolution (DSC) network. Specifically, the DSC network consists of depthwise convolution (DW) and pointwise convolution (PW), where DW and PW should be considered as one layer. The structural information of the MobileNet v1 neural network model and the calculation parameters of each layer are shown in the figure below:

[0241] First, the terminal sends an inference request message carrying model information to the network device. The inference request message includes the terminal identifier or the terminal's existing connection identifier, the structure of the MobileNet v1 neural network model, the layer number that the terminal recommends the network device calculate, the terminal's latency requirement for inference calculations on the network device, the terminal's transmission latency requirement for uplink and downlink air interfaces, and the terminal's single-inference energy consumption requirement for the network device. In this embodiment, the terminal recommends that the network device calculate layer numbers 5 to 15.

[0242] The network device receives the inference request message carrying the model information sent by the terminal, and then obtains the status information of the network device. In this embodiment, the status information of the network device includes the computing resource status information of the network device, the channel quality information of the network device, the uplink and downlink air interface resources that can be allocated by the network device, and the bandwidth of the network device, which are specifically described as follows:

[0243] The computing resource status information of the network device is shown in the following table:

[0244] Table 2 Computing resource status information of network devices

[0245] Resource Number Resource Type Parameter Type Parameter value 1 Virtual CPU resources Utilization 50％ 3 Virtual GPU resources Number of computing units 100 4 Virtual GPU resources Number of computing units 50

[0246] The channel quality information for the network device is as follows: the uplink signal CQI is 5, and the downlink signal CQI is 10. The network device can allocate uplink and downlink air interface resources as follows: 10 RBs for uplink and 30 RBs for downlink. The bandwidth (BW) information for the network device is as follows: Resource numbers 1 and 3: 10 Gbps; Resource numbers 1 and 4: 40 Gbps; Resource numbers 3 and 4: 100 Gbps.

[0247] Based on the status information of the network device, the network device generates an inference reply message and sends the inference reply message to the terminal. The inference reply message indicates that the network device agrees to collaborative inference. In this embodiment, the inference reply message includes the above-mentioned status information of the network device and the performance calculation model of the network device. The input parameters of the performance calculation model of the network device include the amount of allocable computing resources and layer computing parameter information, which are specifically described as follows:

[0248] The information on the number of allocatable computing resources of the network device is as follows: the allocatable CPU utilization is 0-50%; the number of allocatable computing units of the virtual GPU resource with resource number 3 is 0-100; the number of allocatable computing units of the virtual GPU resource with resource number 4 is 0-50.

[0249] The network device's layer computational parameters are as follows: convolutional layer computational parameters (number of bytes per pixel (stride), filter shape, input size), fully connected layer computational parameters (1024, 1000), pooling layer computational parameters (7x7x1024, 1024), and softmax layer computational parameters (1000, 1).

[0250] The terminal receives an inference reply message sent by the network device. Because the inference reply message indicates that the network device agrees to collaborative inference, the terminal generates a first decision action for the terminal and a second decision action for the network device based on the state information of the network device and the state information of the terminal. The decision action includes the model portion responsible for calculation and corresponding computing resource allocation information, and the state information includes at least computing resource state information.

[0251] In this embodiment, the terminal status information includes computing resource status information of the terminal, channel quality information of the terminal, and BW information of the terminal, which are specifically described as follows:

[0252] The computing resource status information of the terminal is shown in the following table:

[0253] Table 3 Computing resource status information of the terminal

[0254] Resource Number Resource Type Parameter Type Parameter value 1 Physical CPU resources Utilization 50％ 3 Physical GPU resources GFlops 100 6 Physical DSP resources GFlops 40

[0255] The channel quality information of the terminal is as follows: the downlink CQI is 10.

[0256] The BW information of the terminal is: resource number 1, 3: 10 Gbps; resource number 1, 6: 40 Gbps; resource number 3, 6: 50 Gbps.

[0257] The input parameters of the performance calculation model of the terminal include information on the amount of allocable computing resources and layer computing parameter information, as described below:

[0258] The terminal's allocatable computing resources are as follows: The CPU's allocatable utilization is 0-50%. The physical GPU resource numbered 3 has an allocatable GFLOPs of 0-100. The physical DSP resource numbered 6 has an allocatable GFLOPs of 0-40.

[0259] The terminal's layer computational parameter information is as follows: convolutional layer computational parameters (stride, filter shape, input size).

[0260] The terminal sends a first computing resource allocation suggestion message to the computing resource management module of the terminal, wherein the first computing resource allocation suggestion message carries the parameters of the first model part, the computing resource type and the number of computing resources of each layer in the first model part, the timing of inter-layer calculations and the data transmission relationship; the computing resource management module allocates the computing tasks of different layers in the first model part to different types of computing resources of the terminal according to the first computing resource allocation suggestion message, and configures the data transmission relationship between various types of computing resources.

[0261] Then, the terminal sends the second decision action to the network device. The network device receives the second decision action sent by the terminal, where the second decision action includes the model portion that the network device is responsible for calculating and corresponding computing resource allocation information. The network device determines whether to execute the second decision action based on at least one of the resource status information of the network device, user priority, and service priority;

[0262] If it is determined that the second decision action is to be executed, the network device allocates computing resources to the second model portion that the network device is responsible for computing in the second decision action, and performs model inference calculations on the second model portion. Upon completion of the model inference calculations on the second model portion, the network device sends a configuration completion message to the terminal.

[0263] If it is determined that the second decision action is not to be performed, the network device changes the second decision action and allocates computing resources to the fourth model portion that the network device is responsible for computing as indicated in the third decision action, and performs model inference calculation on the fourth model portion.

[0264] If it is determined that the second decision action is not to be performed, the network device sends a computing resource allocation update message to the terminal. The terminal receives the computing resource allocation update message. The computing resource allocation update message includes: the model portion that the network device is responsible for calculating, the guaranteed bit rate (GBR) requirement, and a mapping of computing resources to layer numbers.

[0265] In this embodiment, since the terminal recommends that the network device calculate layer numbers 5 to 15, the network device is responsible for calculating the model parameters corresponding to network layers 5 to 15. The GBR requirements include the following: an uplink guaranteed rate of 100 Mbps; a downlink guaranteed rate of 150 Mbps. The mapping between computing resources and layer numbers is as follows:

[0266] Table 4 Mapping relationship between computing resources and layer numbers of network devices

[0267]

[0268] In this embodiment, since the network device is responsible for calculating the model parameters corresponding to the network layers numbered 5 to 15, the terminal is responsible for calculating the model parameters corresponding to the network layers numbered 1 to 4. The mapping relationship between the computing resources and the layer numbers is as follows:

[0269] Table 5 Mapping relationship between terminal computing resources and layer numbers

[0270] Layer number 1 2 3 4 Resource Type GPU GPU+DSP DSP DSP Resource Allocation 80 80+20 20 30

[0271] The terminal adjusts the first decision action according to the third decision action to obtain a fourth decision action; based on the fourth decision action, the terminal redetermines the third model part that the terminal is responsible for calculating, allocates computing resources to the third model part, and performs model inference calculations on the third model part.

[0272] The network device collects the inference energy consumption information of the network device and sends the inference energy consumption information to the terminal. In this embodiment, the energy consumption-related reward A1 of the network device can be obtained based on the inference energy consumption information of the network device, which satisfies:

[0273]

[0274] Wherein, P1 is the threshold value of the energy consumption of the network device, and P2 is the actual measured value of the energy consumption of the network device.

[0275] The terminal obtains the model inference performance parameters and calculates the reward information based on the model inference performance parameters. According to the inference energy consumption information of the terminal, the inference delay-related reward A2 of the terminal can be calculated to satisfy:

[0276]

[0277] Among them, T1 is the threshold value of the inference delay of the terminal, and T2 is the actual measured value of the inference delay of the terminal.

[0278] The terminal's reasoning accuracy-related reward A3 satisfies:

[0279]

[0280] Here, K1 is the threshold value of the inference accuracy of the terminal, and K2 is the actual measurement value of the inference accuracy of the terminal.

[0281] The energy consumption related reward A4 of the terminal satisfies:

[0282]

[0283] Wherein, P3 is the threshold value of the energy consumption of the terminal, and P4 is the actual measured value of the energy consumption of the terminal.

[0284] Finally, the terminal obtains reward information A based on the model reasoning performance parameters, satisfying:

[0285] A=2*A1+A4+0.5*A2+0.5*A3.

[0286] The terminal collects reward information, terminal status information, network device status information, terminal decision actions and network device decision actions in each collaborative reasoning, and uses the collected content as training samples of the reinforcement learning model used by the terminal and the network device for model collaborative reasoning, and trains and optimizes the reinforcement learning model.

[0287] In the first case, when a change in the state information of the network device is detected, a state update message is sent to the terminal, and the terminal receives the state update message sent by the network device.

[0288] It should be understood that the change in the status information of the network device can be understood as a change in any of the following: computing resource status information of the network device, channel quality information of the network device, allocatable uplink and downlink air interface resources of the network device, and BW of the network device.

[0289] In the second case, upon detecting a change in the terminal's status information, the terminal sends a status update request message to the network device. The network device receives the status update request message sent by the terminal. In response to the status update request, the network device sends the terminal its latest status information.

[0290] When the terminal sends a status update request message to the network device, it may indicate which type or types of information status to update. For example, in this embodiment, the network device may be instructed to update resource status information, channel status information, and computing resource connection bandwidth status information.

[0291] The change in the terminal's status information may be understood as a change in any one of the following: computing resource status information of the terminal, channel quality information of the terminal, and bandwidth of the terminal.

[0292] The state information of at least one of the terminal and the network device is updated. The terminal regenerates the decision action of the terminal and the decision action of the network device based on the updated state information of at least one of the terminal and the network device. If the regenerated decision action of the network device is updated, the terminal sends the updated decision action of the network device to the network device.

[0293] The embodiment of the present application also provides a terminal. Figure 4 , Figure 4 This is a structural diagram of the terminal 400 provided in the embodiment of the present application. Since the principle of the terminal 400 in solving the problem is similar to the edge-end collaboration model reasoning method in the embodiment of the present application, the implementation of the terminal 400 can be referred to as Figure 1 The implementation of the method shown will not be repeated any more.

[0294] like Figure 4 As shown, terminal 400 includes:

[0295] A first sending module 401 is configured to send an inference request message carrying model information to a network device;

[0296] A first receiving module 402 is configured to receive an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to collaborative inference, and when the inference reply message indicates that the network device agrees to collaborative inference, the inference reply message also carries status information of the network device;

[0297] A first generating module 403 is configured to generate, when the inference reply message indicates that the network device agrees to collaborative inference, a first decision action for the terminal and a second decision action for the network device based on the state information of the network device and the state information of the terminal, wherein the first decision action and the second decision action include a model portion responsible for calculation and corresponding computing resource allocation information, and the state information of the network device and the state information of the terminal include at least computing resource state information;

[0298] A first processing module 404 is configured to allocate computing resources to the first model portion that the terminal is responsible for computing, as indicated in the first decision action, and perform model inference calculation on the first model portion;

[0299] The second sending module 405 is used to send the second decision action to the network device, so that the network device allocates computing resources to the second model part that the network device is responsible for calculating indicated in the second decision action, and performs model inference calculation of the second model part.

[0300] Optionally, the terminal 400 further includes:

[0301] a fourth receiving module, configured to receive a computing resource allocation update message sent by the network device when the second decision action is changed, wherein the computing resource allocation update message carries a third decision action updated by the network device;

[0302] an adjustment module, configured to adjust the first decision action according to the third decision action to obtain a fourth decision action;

[0303] The third processing module is used to redetermine the third model part that the terminal is responsible for calculating according to the fourth decision action, allocate computing resources to the third model part, and perform model inference calculation of the third model part.

[0304] Optionally, the first processing module 404 includes:

[0305] A first sending unit is configured to send a first computing resource allocation suggestion message to a computing resource management module of the terminal, wherein the first computing resource allocation suggestion message carries parameters of the first model part, a computing resource type and a computing resource quantity of each layer in the first model part, a timing of inter-layer computing, and a data transmission relationship;

[0306] The first processing unit is used to allocate computing tasks of different layers in the first model part to different types of computing resources of the terminal through the computing resource management module according to the first computing resource allocation recommendation message, and configure the data transmission relationship between various types of computing resources.

[0307] Optionally, the terminal 400 further includes:

[0308] a fourth sending module, configured to send the model reasoning intermediate result of the first model part to the network device, so that the network device integrates the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0309] The fifth receiving module is used to receive the model collaborative reasoning result sent by the network device.

[0310] Optionally, the terminal 400 further includes:

[0311] A second acquisition module is used to obtain model reasoning performance parameters, where the model reasoning performance parameters include at least one of reasoning accuracy, average reasoning latency, and average single reasoning energy consumption;

[0312] A calculation module, configured to calculate reward information based on the model reasoning performance parameters;

[0313] A training optimization module is used to collect state information, decision actions and reward information in each collaborative reasoning as training samples of the reinforcement learning model used by the terminal and the network device for model collaborative reasoning, and to train and optimize the reinforcement learning model.

[0314] Optionally, the second acquisition module includes:

[0315] A statistical unit, configured to calculate the inference accuracy, average inference latency, and average single inference energy consumption of the terminal within a preset time period;

[0316] The first receiving unit is used to receive the inference energy consumption information sent by the network device, wherein the inference energy consumption information includes the average single inference energy consumption value of the network device within the preset time period or the average computing power value consumed by the network device for a single inference.

[0317] Optionally, the terminal 400 further includes:

[0318] a sixth receiving module, configured to receive a status update message sent by the network device, wherein the status update message includes updated status information of the network device;

[0319] and / or, a fifth sending module, configured to send a status update request message to the network device when a change in the status information of the terminal is detected; and receive the latest status information sent by the network device;

[0320] A third generating module is used to regenerate the decision action of the terminal and the decision action of the network device according to the updated status information of the terminal and / or the network device;

[0321] A sixth sending module is used to send the updated decision action of the network device to the network device when the regenerated decision action of the network device is updated.

[0322] The terminal 400 provided in the embodiment of the present application can execute the above-mentioned Figure 1 The implementation principle and technical effect of the method embodiment shown are similar, and this embodiment will not be repeated here.

[0323] The terminal 400 of the embodiment of the present application can be used to send an inference request message carrying model information to a network device; receive an inference reply message sent by the network device, wherein the inference reply message also carries the status information of the network device. Based on the status information of the network device and the status information of the terminal, a first decision action of the terminal and a second decision action of the network device are generated. In this embodiment, the first decision action and the second decision action are generated based on the status information of the network device and the status information of the terminal. Therefore, when splitting the inference model, the difference in computing resources of the network device and the terminal is taken into account, making the segmentation of the inference model more reasonable, thereby improving the model inference performance.

[0324] The embodiment of the present application also provides a network device. Figure 5 , Figure 5 is a structural diagram of the network device 500 provided in the embodiment of the present application. Since the principle of solving the problem by the network device 500 is similar to the end-edge collaboration model reasoning method in the embodiment of the present application, the implementation of the network device 500 can be referred to as Figure 2 The implementation of the method shown will not be repeated any more.

[0325] like Figure 5 As shown, the network device 500 includes:

[0326] The second receiving module 501 is used to receive an inference request message carrying model information sent by a terminal;

[0327] A first acquisition module 502 is configured to acquire status information of the network device, wherein the status information at least includes computing resource status information;

[0328] A second generating module 503 is configured to generate an inference reply message according to the state information of the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference;

[0329] A third sending module 504 is configured to send the inference reply message to the terminal, wherein, if the network device agrees to collaborative inference, the inference reply message also carries the status information of the network device;

[0330] A third receiving module 505 is configured to receive, when the network device agrees to collaborative reasoning, a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal based on the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding computing resource allocation information;

[0331] The second processing module 506 is used to allocate computing resources to the second model part that the network device indicated in the second decision action is responsible for calculating, and perform model inference calculation on the second model part.

[0332] Optionally, the first obtaining module 502 includes:

[0333] The first transceiver unit is configured to send a radio channel quality information request message to a radio resource management RRM module of the network device; and receive a radio channel quality information reply message returned by the RRM module and carrying uplink and downlink radio channel quality information of the terminal;

[0334] The second transceiver unit is configured to send a computing resource information request message carrying computing task information to a computing resource management CRM module of the network device; and receive a computing resource information reply message carrying computing resource status information of the network device returned by the CRM module;

[0335] The third transceiver unit is used to send a computing performance model request message to the computing performance model management CPMM module of the network device, wherein the computing performance model request message carries the remaining computing resource type information and model layer type information of the network device; and receive the computing performance model reply message carrying the computing performance model information returned by the CPMM module.

[0336] Optionally, the second processing module 506 includes:

[0337] a determining unit, configured to determine whether to execute the second decision action based on at least one of the resource status information, user priority, and service priority of the network device;

[0338] a second processing unit, configured to, when determining to execute the second decision action, allocate computing resources to the second model portion that the network device is responsible for calculating indicated in the second decision action, and perform model inference calculation on the second model portion;

[0339] a third processing unit, configured to, if it is determined that the second decision action is not to be performed, change the second decision action and send a computing resource allocation update message to the terminal, wherein the computing resource allocation update message carries the third decision action updated by the network device;

[0340] The fourth processing unit is used to allocate computing resources to the fourth model part that the network device indicated in the third decision action is responsible for calculating, and perform model inference calculation of the fourth model part.

[0341] Optionally, the second processing module 506 includes:

[0342] A second sending unit is configured to send a wireless resource allocation recommendation message to the RRM module of the network device, wherein the wireless resource allocation recommendation message carries the uplink and downlink channel bandwidth allocated for the model inference computing task;

[0343] a third sending unit, configured to send a second computing resource allocation suggestion message to the CRM module of the network device, wherein the second computing resource allocation suggestion message includes parameters of the second model part, a computing resource type and a computing resource quantity of each layer in the second model part, a timing of inter-layer computing, and a data transmission relationship;

[0344] The fifth processing unit is used to allocate the computing tasks of different layers in the second model part to different types of computing resources of the network device through the CRM module of the network device according to the second computing resource allocation recommendation message, and configure the data transmission relationship between different types of computing resources.

[0345] Optionally, the network device 500 further includes:

[0346] a seventh receiving module, configured to receive an intermediate result of model inference of the first model part sent by the terminal;

[0347] an integration module, configured to integrate the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0348] The seventh sending module is used to send the model collaborative reasoning result to the terminal.

[0349] Optionally, the network device 500 further includes:

[0350] A statistics module, configured to collect statistics on the inference energy consumption information of the network device, wherein the inference energy consumption information includes an average single inference energy consumption value of the network device within a preset time period or an average single inference computing power value consumed by the network device;

[0351] An eighth sending module is used to send the inference energy consumption information to the terminal.

[0352] Optionally, the network device 500 further includes:

[0353] a ninth sending module, configured to send a status update message to the terminal when detecting a change in the status information of the network device, wherein the status update message includes the updated status information of the network device;

[0354] And / or, an eighth receiving module, configured to receive a status update request message sent by the terminal; and send the latest status information of the network device to the terminal.

[0355] The network device provided in the embodiment of the present application can perform the above-mentioned Figure 2 The implementation principle and technical effects of the method embodiment are similar, and will not be repeated here in this embodiment.

[0356] The embodiment of the present application also provides a terminal. Since the principle of the terminal to solve the problem is similar to the end-edge collaboration model reasoning method in the embodiment of the present application, the implementation of the terminal can refer to the implementation of the method, and the repeated parts will not be repeated. Figure 6 As shown, the terminal of the embodiment of the present application includes: a processor 600, which is used to read the program in the memory 620 and perform the following processes:

[0357] When the reasoning reply message indicates that the network device agrees to collaborative reasoning, generating a first decision action of the terminal and a second decision action of the network device according to the state information of the network device and the state information of the terminal, the first decision action and the second decision action including a model portion responsible for calculation and corresponding computing resource allocation information, and the state information of the network device and the state information of the terminal including at least computing resource state information;

[0358] Allocating computing resources to the first model portion that the terminal is responsible for calculating in the first decision action, and performing model inference calculation of the first model portion;

[0359] The processor 600 is configured to read the program in the memory 620 and execute the following process:

[0360] Sending an inference request message carrying model information to the network device through the transceiver 610;

[0361] Receiving, through the transceiver 610, an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference, and when the inference reply message indicates that the network device agrees to the collaborative inference, the inference reply message also carries status information of the network device;

[0362] The second decision action is sent to the network device via the transceiver 610, so that the network device allocates computing resources to the second model part that the network device is responsible for calculating indicated in the second decision action, and performs model inference calculation of the second model part.

[0363] The transceiver 610 is configured to receive and send data under the control of the processor 600 .

[0364] Among them, Figure 6 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically various circuits of one or more processors represented by processor 600 and memory represented by memory 620, which are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 610 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 630 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

[0365] The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 can store data used by the processor 600 when performing operations.

[0366] Optionally, the processor 600 is further configured to read a program in the memory 620 and execute the following steps:

[0367] receiving, through the transceiver 610, a computing resource allocation update message sent by the network device when the second decision action is changed, wherein the computing resource allocation update message carries a third decision action updated by the network device;

[0368] According to the third decision action, adjusting the first decision action to obtain a fourth decision action;

[0369] According to the fourth decision action, the third model part that the terminal is responsible for calculating is re-determined, and computing resources are allocated to the third model part to perform model inference calculation of the third model part.

[0370] Optionally, the processor 600 is further configured to read a program in the memory 620 and execute the following steps:

[0371] Sending a first computing resource allocation suggestion message to the computing resource management module of the terminal through the transceiver 610, wherein the first computing resource allocation suggestion message carries parameters of the first model part, the computing resource type and quantity of each layer in the first model part, the timing of inter-layer calculations, and the data transmission relationship;

[0372] The computing resource management module allocates computing tasks of different layers in the first model part to different types of computing resources of the terminal according to the first computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

[0373] Optionally, the processor 600 is further configured to read a program in the memory 620 and execute the following steps:

[0374] Sending the model reasoning intermediate result of the first model part to the network device through the transceiver 610, so that the network device integrates the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0375] The model collaborative reasoning result sent by the network device is received through the transceiver 610.

[0376] Optionally, the processor 600 is further configured to read a program in the memory 620 and execute the following steps:

[0377] Acquire model reasoning performance parameters, where the model reasoning performance parameters include at least one of reasoning accuracy, average reasoning latency, and average single reasoning energy consumption;

[0378] Calculating reward information based on the model reasoning performance parameters;

[0379] The state information, decision action and reward information in each collaborative reasoning are collected as training samples of the reinforcement learning model used by the terminal and the network device for model collaborative reasoning, and the reinforcement learning model is trained and optimized.

[0380] Optionally, the processor 600 is further configured to read a program in the memory 620 and execute the following steps:

[0381] Counting the inference accuracy, average inference latency, and average single inference energy consumption of the terminal within a preset time period;

[0382] The inference energy consumption information sent by the network device is received through the transceiver 610, wherein the inference energy consumption information includes the average single inference energy consumption value of the network device within the preset time period or the average single inference computing power consumed by the network device.

[0383] Optionally, the processor 600 is further configured to read a program in the memory 620 and execute the following steps:

[0384] Receiving, through the transceiver 610, a status update message sent by the network device, wherein the status update message includes updated status information of the network device;

[0385] and / or, when detecting a change in the status information of the terminal, sending a status update request message to the network device via the transceiver 610; and receiving the latest status information sent by the network device;

[0386] regenerating a decision action of the terminal and a decision action of the network device according to the updated status information of the terminal and / or the network device;

[0387] In the case that the regenerated decision action of the network device is updated, the updated decision action of the network device is sent to the network device via the transceiver 610 .

[0388] The terminal provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0389] The embodiment of the present application also provides a network device. Since the principle of the network device in solving the problem is similar to the edge-end collaboration model reasoning method in the embodiment of the present application, the implementation of the network device can refer to the implementation of the method, and the repeated parts will not be repeated. Figure 7 As shown, the network device of the embodiment of the present application includes: a processor 700, which is used to read the program in the memory 720 and execute the following process:

[0390] Acquiring status information of the network device, wherein the status information at least includes computing resource status information;

[0391] generating, according to the state information of the network device, an inference reply message, wherein the inference reply message indicates whether the network device agrees with the collaborative inference;

[0392] Allocate computing resources to the second model part that the network device indicated in the second decision action is responsible for calculating, and perform model inference calculation on the second model part.

[0393] The processor 700 is configured to read the program in the memory 720 and execute the following process:

[0394] Receiving, through the transceiver 710, an inference request message carrying model information sent by the terminal;

[0395] Sending the inference reply message to the terminal through the transceiver 710, wherein, if the network device agrees to the collaborative inference, the inference reply message also carries the status information of the network device;

[0396] When the network device agrees to collaborative reasoning, receiving, through the transceiver 710, a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal according to the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding calculation resource allocation information;

[0397] The transceiver 710 is configured to receive and send data under the control of the processor 700 .

[0398] Among them, Figure 7 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 700 and memory represented by memory 720. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 710 may be a plurality of components, i.e., a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. The processor 700 is responsible for managing the bus architecture and general processing, and the memory 720 may store data used by the processor 700 when performing operations.

[0399] Optionally, the processor 700 is further configured to read a program in the memory 720 and execute the following steps:

[0400] Sending a radio channel quality information request message to the radio resource management RRM module of the network device through the transceiver 710; receiving a radio channel quality information reply message carrying the uplink and downlink radio channel quality information of the terminal returned by the RRM module through the transceiver 710;

[0401] Sending a computing resource information request message carrying computing task information to the computing resource management CRM module of the network device through the transceiver 710; receiving a computing resource information reply message carrying computing resource status information of the network device returned by the CRM module through the transceiver 710;

[0402] A computing performance model request message is sent to the computing performance model management CPMM module of the network device through the transceiver 710, wherein the computing performance model request message carries the remaining computing resource type information and model layer type information of the network device; and a computing performance model reply message carrying the computing performance model information returned by the CPMM module is received through the transceiver 710.

[0403] Optionally, the processor 700 is further configured to read a program in the memory 720 and execute the following steps:

[0404] determining whether to execute the second decision action according to at least one of the resource status information, user priority, and service priority of the network device;

[0405] In the case of determining to execute the second decision action, allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion;

[0406] If it is determined that the second decision action is not to be performed, change the second decision action and send a computing resource allocation update message to the terminal through the transceiver 710, wherein the computing resource allocation update message carries the third decision action updated by the network device;

[0407] Allocate computing resources to the fourth model portion that the network device indicated in the third decision action is responsible for computing, and perform model inference calculations on the fourth model portion.

[0408] Optionally, the processor 700 is further configured to read a program in the memory 720 and execute the following steps:

[0409] Sending a radio resource allocation recommendation message to the RRM module of the network device through the transceiver 710, wherein the radio resource allocation recommendation message carries the uplink and downlink channel bandwidth allocated for the model inference computing task;

[0410] Sending a second computing resource allocation suggestion message to the CRM module of the network device through the transceiver 710, wherein the second computing resource allocation suggestion message includes parameters of the second model part, the computing resource type and quantity of each layer in the second model part, and the timing and data transmission relationship of inter-layer calculations;

[0411] The CRM module of the network device allocates computing tasks of different layers in the second model part to different types of computing resources of the network device according to the second computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

[0412] Optionally, the processor 700 is further configured to read a program in the memory 720 and execute the following steps:

[0413] Receiving, through the transceiver 710, the model inference intermediate result of the first model part sent by the terminal;

[0414] Integrating the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result;

[0415] The model collaborative reasoning result is sent to the terminal through the transceiver 710.

[0416] Optionally, the processor 700 is further configured to read a program in the memory 720 and execute the following steps:

[0417] Collecting information on the inference energy consumption of the network device, wherein the inference energy consumption information includes an average single inference energy consumption value of the network device within a preset time period or an average single inference computing power value consumed by the network device;

[0418] The inferred energy consumption information is sent to the terminal through the transceiver 710.

[0419] Optionally, the processor 700 is further configured to read a program in the memory 720 and execute the following steps:

[0420] When a change in the status information of the network device is detected, a status update message is sent to the terminal via the transceiver 710, wherein the status update message includes the updated status information of the network device;

[0421] And / or, receiving a status update request message sent by the terminal through the transceiver 710; and sending the latest status information of the network device to the terminal through the transceiver 710.

[0422] The network device provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, so this embodiment will not be repeated here.

[0423] The embodiment of the present application further provides a computer-readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the following is achieved: Figure 1 or Figure 2The various processes of the method embodiment shown can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0424] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0425] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may be physically included separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0426] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute some steps of the sending and receiving methods described in various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0427] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A device-edge collaborative model reasoning method, characterized by: Executed by a terminal, the method includes: Sending an inference request message carrying model information to the network device; receiving an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference, and when the inference reply message indicates that the network device agrees to the collaborative inference, the inference reply message also carries status information of the network device; When the reasoning reply message indicates that the network device agrees to collaborative reasoning, generating a first decision action of the terminal and a second decision action of the network device according to the state information of the network device and the state information of the terminal, the first decision action and the second decision action including a model portion responsible for calculation and corresponding computing resource allocation information, and the state information of the network device and the state information of the terminal including at least computing resource state information; Allocating computing resources to the first model portion that the terminal is responsible for calculating in the first decision action, and performing model inference calculation of the first model portion; The second decision action is sent to the network device, so that the network device allocates computing resources to the second model part that the network device is responsible for calculating as indicated in the second decision action, and performs model inference calculation of the second model part.

2. The method according to claim 1, characterized in that After sending the second decision action to the network device, the method further includes: receiving a computing resource allocation update message sent by the network device when the second decision action is changed, wherein the computing resource allocation update message carries a third decision action updated by the network device; According to the third decision action, adjusting the first decision action to obtain a fourth decision action; According to the fourth decision action, the third model part that the terminal is responsible for calculating is re-determined, and computing resources are allocated to the third model part to perform model inference calculation of the third model part.

3. The method according to claim 1, characterized in that The allocating computing resources to the first model portion that the terminal is responsible for computing indicated in the first decision action, and performing model inference calculation on the first model portion, includes: Sending a first computing resource allocation suggestion message to the computing resource management module of the terminal, wherein the first computing resource allocation suggestion message carries parameters of the first model part, the computing resource type and quantity of computing resources of each layer in the first model part, and the timing and data transmission relationship of inter-layer calculations; The computing resource management module allocates computing tasks of different layers in the first model part to different types of computing resources of the terminal according to the first computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

4. The method according to claim 1, wherein The method further comprises: Sending the model reasoning intermediate result of the first model part to the network device, so that the network device integrates the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result; Receive the model collaborative reasoning result sent by the network device.

5. The method according to claim 1, wherein The method further comprises: Acquire model reasoning performance parameters, where the model reasoning performance parameters include at least one of reasoning accuracy, average reasoning latency, and average single reasoning energy consumption; Calculating reward information based on the model reasoning performance parameters; The state information, decision action and reward information in each collaborative reasoning are collected as training samples of the reinforcement learning model used by the terminal and the network device for model collaborative reasoning, and the reinforcement learning model is trained and optimized.

6. The method according to claim 5, characterized in that The obtaining of model reasoning performance parameters includes: Counting the inference accuracy, average inference latency, and average single inference energy consumption of the terminal within a preset time period; Receive the inference energy consumption information sent by the network device, wherein the inference energy consumption information includes the average single inference energy consumption value of the network device within the preset time length or the average single inference computing power value consumed by the network device.

7. The method according to claim 1, characterized in that The method further comprises: receiving a status update message sent by the network device, wherein the status update message includes updated status information of the network device; and / or, when detecting a change in the status information of the terminal, sending a status update request message to the network device; and receiving the latest status information sent by the network device; regenerating a decision action of the terminal and a decision action of the network device according to the updated status information of the terminal and / or the network device; When the regenerated decision action of the network device is updated, the updated decision action of the network device is sent to the network device.

8. A device-edge collaborative model reasoning method, characterized in that: Executed by a network device, the method includes: Receiving an inference request message carrying model information sent by a terminal; Acquiring status information of the network device, wherein the status information at least includes computing resource status information; generating, according to the state information of the network device, an inference reply message, wherein the inference reply message indicates whether the network device agrees with the collaborative inference; Sending the inference reply message to the terminal, wherein, if the network device agrees to collaborative inference, the inference reply message also carries status information of the network device; When the network device agrees to collaborative reasoning, receiving a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal according to the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding computing resource allocation information; Allocate computing resources to the second model part that the network device indicated in the second decision action is responsible for calculating, and perform model inference calculation on the second model part.

9. The method according to claim 8, characterized in that The acquiring of the status information of the network device includes: Sending a radio channel quality information request message to a radio resource management (RRM) module of the network device; receiving a radio channel quality information reply message returned by the RRM module and carrying uplink and downlink radio channel quality information of the terminal; Sending a computing resource information request message carrying computing task information to a computing resource management (CRM) module of the network device; receiving a computing resource information reply message carrying computing resource status information of the network device returned by the CRM module; Send a computing performance model request message to the computing performance model management CPMM module of the network device, wherein the computing performance model request message carries the remaining computing resource type information and model layer type information of the network device; receive a computing performance model reply message carrying the computing performance model information returned by the CPMM module.

10. The method according to claim 8, characterized in that The allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation on the second model portion, includes: determining whether to execute the second decision action according to at least one of the resource status information, user priority, and service priority of the network device; In the case of determining to execute the second decision action, allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation of the second model portion; If it is determined that the second decision action is not to be performed, changing the second decision action and sending a computing resource allocation update message to the terminal, wherein the computing resource allocation update message carries a third decision action for updating the network device; Allocate computing resources to the fourth model portion that the network device indicated in the third decision action is responsible for computing, and perform model inference calculations on the fourth model portion.

11. The method according to claim 8, characterized in that The allocating computing resources to the second model portion that the network device indicated in the second decision action is responsible for computing, and performing model inference calculation on the second model portion, includes: Sending a wireless resource allocation suggestion message to the RRM module of the network device, wherein the wireless resource allocation suggestion message carries the uplink and downlink channel bandwidth allocated for the model inference computing task; Sending a second computing resource allocation suggestion message to the CRM module of the network device, wherein the second computing resource allocation suggestion message includes parameters of the second model part, the computing resource type and quantity of computing resources for each layer in the second model part, and the timing and data transmission relationship of inter-layer calculations; The CRM module of the network device allocates computing tasks of different layers in the second model part to different types of computing resources of the network device according to the second computing resource allocation recommendation message, and configures the data transmission relationship between different types of computing resources.

12. The method according to claim 8, characterized in that The method further comprises: receiving an intermediate result of model inference of the first model part sent by the terminal; Integrating the model reasoning intermediate result of the first model part and the model reasoning intermediate result of the second model part to obtain a model collaborative reasoning result; Sending the model collaborative reasoning result to the terminal.

13. The method according to claim 8, characterized in that The method further comprises: Collecting information on the inference energy consumption of the network device, wherein the inference energy consumption information includes an average single inference energy consumption value of the network device within a preset time period or an average single inference computing power value consumed by the network device; The inference energy consumption information is sent to the terminal.

14. The method according to claim 8, characterized in that The method further comprises: When a change in the status information of the network device is detected, sending a status update message to the terminal, wherein the status update message includes the updated status information of the network device; And / or, receiving a status update request message sent by the terminal; and sending the latest status information of the network device to the terminal.

15. A terminal, characterized in that: include: A first sending module, configured to send an inference request message carrying model information to a network device; A first receiving module is configured to receive an inference reply message sent by the network device, wherein the inference reply message indicates whether the network device agrees to collaborative inference, and when the inference reply message indicates that the network device agrees to collaborative inference, the inference reply message also carries status information of the network device; a first generating module, configured to generate, when the inference reply message indicates that the network device agrees to collaborative reasoning, a first decision action for the terminal and a second decision action for the network device based on the state information of the network device and the state information of the terminal, wherein the first decision action and the second decision action include a model portion responsible for calculation and corresponding computing resource allocation information, and the state information of the network device and the state information of the terminal include at least computing resource state information; a first processing module, configured to allocate computing resources to the first model portion that the terminal is responsible for calculating in the first decision action, and perform model inference calculation on the first model portion; The second sending module is used to send the second decision action to the network device, so that the network device allocates computing resources to the second model part that the network device is responsible for calculating indicated in the second decision action, and performs model inference calculation of the second model part.

16. A network device, characterized in that: include: A second receiving module is used to receive an inference request message carrying model information sent by a terminal; A first acquisition module is configured to acquire status information of the network device, wherein the status information of the network device and the status information of the terminal include at least computing resource status information; A second generating module is configured to generate an inference reply message according to the status information of the network device, wherein the inference reply message indicates whether the network device agrees to the collaborative inference; A third sending module is configured to send the inference reply message to the terminal, wherein, if the network device agrees to collaborative inference, the inference reply message also carries status information of the network device; a third receiving module, configured to receive, when the network device agrees to collaborative reasoning, a second decision action sent by the terminal, wherein the second decision action is a decision action of the network device generated by the terminal based on the state information of the network device and the state information of the terminal, and the second decision action includes a model portion responsible for calculation and corresponding computing resource allocation information; The second processing module is used to allocate computing resources to the second model part that the network device indicated in the second decision action is responsible for calculating, and perform model inference calculation of the second model part.

17. A terminal comprising: A transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: The processor is configured to read a program in a memory to implement the steps in the method according to any one of claims 1 to 7.

18. A network device comprising: A transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: The processor is configured to read a program in a memory to implement the steps in the method according to any one of claims 8 to 14.

19. A computer-readable storage medium for storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented; or the steps of the method according to any one of claims 8 to 14 are implemented.

Images