Model distribution method for edge-cloud collaborative reasoning and edge-cloud collaborative reasoning method

Abstract

The application relates to the technical field of artificial intelligence and the field of digital medicine, and provides a model allocation method for edge-cloud cooperative reasoning and an edge-cloud cooperative reasoning method. After model allocation is performed on an edge device and a cloud server, an allocation evaluation result corresponding to the edge device is acquired. For the edge device whose allocation evaluation result does not satisfy an allocation end condition, a new target network model is randomly selected from multiple heterogeneous models for the edge device, a new round of model allocation is performed based on the new target network model, the model allocation scheme is continuously optimized, and the overall reasoning effect of edge-cloud cooperative reasoning is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and the field of digital medicine, and in particular to a model allocation method for edge-cloud collaborative reasoning and an edge-cloud collaborative reasoning method. BACKGROUND

[0002] With the increasing demand for human-computer interaction, artificial intelligence (AI) models are gradually migrating from the cloud to the edge. However, due to the limited computing power of devices, the edge can only deploy smaller models, resulting in low overall reasoning accuracy.

[0003] Currently, the industry has proposed an edge-cloud collaborative reasoning method, which deploys models on the cloud and the edge respectively, and collaborates with the cloud and the edge to complete reasoning. For example, in the field of digital medicine, models are deployed on the cloud and the edge in an auxiliary diagnosis system, and the cloud and the edge collaborate to complete auxiliary diagnosis, which can meet the characteristics of large demand and high real-time requirements of auxiliary diagnosis, and break free from the shackles of high privacy requirements of medical data and low computing power of the edge.

[0004] However, due to the varying computing power of the edge, if models are not reasonably allocated to the cloud and the edge, the edge-cloud collaborative reasoning will result in large reasoning latency and affect the overall reasoning accuracy. SUMMARY

[0005] The main purpose of the embodiments of the present application is to propose a model allocation method for edge-cloud collaborative reasoning, an edge-cloud collaborative reasoning method, a model allocation device for edge-cloud collaborative reasoning, an electronic device and a computer readable storage medium, which can optimize model allocation in edge-cloud collaborative reasoning and improve edge-cloud collaborative reasoning effect.

[0006] To achieve the above purpose, the first aspect of the embodiments of the present application proposes a model allocation method for edge-cloud collaborative reasoning, which comprises:

[0007] obtaining device resource information of a plurality of edge devices;

[0008] randomly selecting a corresponding target network model for each edge device from a plurality of heterogeneous models constructed in advance;

[0009] determining a model split point of the target network model according to the device resource information of the edge device, wherein the model split point is used to divide the target network model into a first sub-network module and a second sub-network module, the first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model;

[0010] For each target network model, the first sub-network module is assigned to the edge device corresponding to the target network model, and the second sub-network module is assigned to the cloud server, so that the edge device and the cloud server respectively perform collaborative inference through the first sub-network module and the second sub-network module.

[0011] After the edge device and the cloud server perform collaborative inference, an assignment evaluation result corresponding to each edge device is obtained.

[0012] For the edge device whose assignment evaluation result does not satisfy the assignment end condition, a new target network model is randomly selected from the plurality of heterogeneous models for the edge device, so as to obtain a new assignment evaluation result corresponding to the edge device, until the new assignment evaluation result satisfies the assignment end condition.

[0013] To achieve the above object, a second aspect of the embodiment of the present application proposes a method for edge-cloud collaborative inference, applied to an edge device, and the method comprises:

[0014] Receiving to-be-inferred data;

[0015] Inputting the to-be-inferred data into a first sub-network module to obtain intermediate feature values;

[0016] Sending the intermediate feature values to a cloud server, so that the cloud server performs inference calculation on the intermediate feature values through a second sub-network module to obtain an inference result;

[0017] Receiving the inference result returned by the cloud server;

[0018] The first sub-network module and the second sub-network module on the cloud server are obtained through the model allocation method for edge-cloud collaborative inference according to any one of the first aspect.

[0019] To achieve the above object, a third aspect of the embodiment of the present application proposes a method for edge-cloud collaborative inference, applied to a cloud server, and the method comprises:

[0020] Receiving intermediate feature values sent by an edge device, wherein the intermediate feature values are obtained by the edge device through a first sub-network module performing inference calculation on to-be-inferred data;

[0021] Inputting the intermediate feature values into a second sub-network module to obtain an inference result;

[0022] Sending the inference result to the edge device;

[0023] The second sub-network module and the first sub-network module on the edge device are distributed by the model distribution method for edge-cloud collaborative inference of any one of the first aspect.

[0024] To achieve the above-mentioned purpose, the fourth aspect of the embodiment of the present application provides a model distribution device for edge-cloud collaborative inference, the device comprises:

[0025] A resource information acquisition module is configured to acquire device resource information of a plurality of edge devices.

[0026] A network model selection module is configured to randomly select a corresponding target network model for each edge device from a plurality of heterogeneous models constructed in advance.

[0027] A network model segmentation module is configured to determine a model segmentation point of the target network model according to the device resource information of the edge device, wherein the model segmentation point is used to divide the target network model into a first sub-network module and a second sub-network module, the first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model.

[0028] A first model distribution module is configured to, for each target network model, distribute the first sub-network module to the edge device corresponding to the target network model and distribute the second sub-network module to a cloud server, so that the edge device and the cloud server respectively perform collaborative inference through the first sub-network module and the second sub-network module.

[0029] An evaluation result acquisition module is configured to acquire an allocation evaluation result corresponding to each edge device after the edge device and the cloud server perform collaborative inference.

[0030] A second model distribution module is configured to, for the edge device whose allocation evaluation result does not satisfy an allocation end condition, randomly select a new target network model for the edge device from a plurality of heterogeneous models, so as to obtain a new allocation evaluation result corresponding to the edge device, until the new allocation evaluation result satisfies the allocation end condition.

[0031] According to the model distribution device for edge-cloud collaborative inference provided by the embodiment of the present application, the device resource information comprises a memory capacity of the edge device, and the network model segmentation module further comprises:

[0032] A first network model segmentation module is configured to acquire a memory occupation amount of each network layer in the target network model.

[0033] A second network model segmentation module is configured to determine a model segmentation point of the target network model according to memory occupancy of each network layer in the target network model and memory capacity of the edge device.

[0034] According to the model allocation device for edge-cloud collaborative reasoning provided in the embodiments of the present application, the second network model segmentation module comprises:

[0035] A memory occupancy accumulation module is configured to accumulate memory occupancy of each network layer to obtain memory cumulative occupancy of a current network layer, starting from a first network layer in the target network model.

[0036] A model segmentation point judgment module is configured to determine a position between the current network layer and a previous network layer of the current network layer as the model segmentation point of the target network model if the memory cumulative occupancy of the current network layer is greater than the memory capacity of the edge device.

[0037] According to the model allocation device for edge-cloud collaborative reasoning provided in the embodiments of the present application, the allocation evaluation result comprises model reasoning accuracy of the target network model, the allocation end condition is that the model reasoning accuracy is greater than or equal to a preset accuracy threshold, and the evaluation result acquisition module comprises:

[0038] An inference result acquisition module is configured to acquire, for each edge device, an inference result obtained by the cloud server through inference calculation of the second sub-network module.

[0039] A reasoning accuracy calculation module is configured to determine, according to the inference result, model reasoning accuracy of the target network model corresponding to each edge device.

[0040] According to the model allocation device for edge-cloud collaborative reasoning provided in the embodiments of the present application, the allocation evaluation result comprises model reasoning delay of the target network model, the allocation end condition is that the model reasoning delay is less than or equal to a preset time threshold, and the evaluation result acquisition module comprises:

[0041] A first delay time acquisition module is configured to acquire, for each edge device, a first delay time of the edge device in inference calculation through the first sub-network module.

[0042] A second delay time acquisition module is configured to acquire a second delay time of the edge device in sending an intermediate feature value to the cloud server, the intermediate feature value being obtained by the edge device through inference calculation of the first sub-network module.

[0043] a third delay time obtaining module, configured to obtain a third delay time of the cloud server in performing inference calculation through the second sub-network module;

[0044] a model inference delay calculation module, configured to determine a model inference delay of the target network model corresponding to each of the edge devices according to the first delay time, the second delay time and the third delay time.

[0045] To achieve the above object, a fifth aspect of embodiments of the present application provides an electronic device, which comprises a memory, a processor, a computer program stored in the memory and executable on the processor, and the computer program is executed by the processor to implement the model allocation method for edge-cloud collaborative inference of the first aspect, the edge-cloud collaborative inference method of the second aspect or the third aspect.

[0046] To achieve the above object, a sixth aspect of embodiments of the present application provides a storage medium, which is a computer readable storage medium, for computer readable storage, and the storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the model allocation method for edge-cloud collaborative inference of the first aspect, the edge-cloud collaborative inference method of the second aspect or the third aspect.

[0047] The application provides a model allocation method for edge-cloud collaborative reasoning, an edge-cloud collaborative reasoning method, a model allocation device for edge-cloud collaborative reasoning, an electronic device and a computer readable storage medium. The model allocation method first acquires device resource information of a plurality of edge devices, and randomly selects a corresponding target network model for each edge device from a plurality of heterogeneous models constructed in advance. Then, a model cutting point of the target network model is determined according to the device resource information of the edge device. The model cutting point is used to divide the target network model into a first sub-network module and a second sub-network module. The first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model. For each target network model, the first sub-network module is allocated to the edge device corresponding to the target network model, and the second sub-network module is allocated to a cloud server, so that the edge device and the cloud server perform collaborative reasoning through the first sub-network module and the second sub-network module, respectively. After the edge device and the cloud server perform collaborative reasoning, an allocation evaluation result corresponding to each edge device is acquired. For an edge device whose allocation evaluation result does not satisfy an allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, so as to obtain a new allocation evaluation result corresponding to the edge device, until the new allocation evaluation result satisfies the allocation end condition. After the model allocation is performed for the edge device and the cloud server, the allocation evaluation result corresponding to the edge device is acquired. For an edge device whose allocation evaluation result does not satisfy the allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, and a new round of model allocation is performed based on the new target network model. The model allocation scheme is continuously optimized, and the overall reasoning effect of edge-cloud collaborative reasoning is improved.

[0048] The model distribution method for edge-cloud collaborative reasoning provided in the application can be applied to the field of digital medicine. In a medical application scenario, the edge device can be a medical terminal device, and the cloud server can be a medical cloud server. The medical terminal device and the medical cloud server perform edge-cloud collaborative reasoning. The model distribution method first acquires device resource information of a plurality of medical terminal devices, and randomly selects a corresponding target network model for each medical terminal device from a plurality of heterogeneous models constructed in advance. Then, the model distribution method determines a model cutting point of the target network model according to the device resource information of the medical terminal device. The model cutting point is used to divide the target network model into a first sub-network module and a second sub-network module. The first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model. For each target network model, the first sub-network module is distributed to the medical terminal device corresponding to the target network model, and the second sub-network module is distributed to the medical cloud server, so that the medical terminal device and the medical cloud server perform collaborative reasoning through the first sub-network module and the second sub-network module, respectively. After the medical terminal device and the medical cloud server perform collaborative reasoning, an allocation evaluation result corresponding to each edge device is acquired. For an edge device whose allocation evaluation result does not satisfy an allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, so as to obtain a new allocation evaluation result corresponding to the edge device, until the new allocation evaluation result satisfies the allocation end condition. After the model distribution for the edge device and the cloud server is performed, the allocation evaluation result corresponding to the edge device is acquired. For an edge device whose allocation evaluation result does not satisfy the allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, and a new round of model distribution is performed based on the new target network model. The model distribution scheme is continuously optimized, the reasoning effect of the edge-cloud collaborative reasoning is improved, the medical terminal device and the medical cloud server can perform edge-cloud collaborative reasoning by using the target network model, and under the conditions of meeting the characteristics of a large amount of medical auxiliary diagnosis demand and high real-time requirement, and breaking away from the shackles of high privacy requirement of medical data and low computing power of the medical terminal device, a better edge-cloud collaborative reasoning result can be obtained, the doctor is effectively assisted in medical diagnosis, and the doctor's work efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 FIG. 1 is a flowchart of a model distribution method for edge-cloud collaborative reasoning provided by an embodiment of the application;

[0050] Figure 2 is Figure 1 FIG. 2 is a flowchart of a substep of step S130 in FIG. 1;

[0051] Figure 3 is Figure 2 FIG. 3 is a flowchart of a substep of step S220 in FIG. 2;

[0052] Figure 4 is Figure 1 a sub-step flowchart of step S150 in

[0053] Figure 5 is Figure 1 a sub-step flowchart of step S150 in

[0054] Figure 6 is a flowchart of a side cloud collaborative reasoning method provided by an embodiment of the present application;

[0055] Figure 7 is a flowchart of a side cloud collaborative reasoning method provided by another embodiment of the present application;

[0056] Figure 8 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0057] Figure 9 is a schematic diagram of an implementation environment provided by another embodiment of the present application;

[0058] Figure 10 is a structural schematic diagram of a plurality of heterogeneous models provided by an embodiment of the present application;

[0059] Figure 11 is a flowchart of a model allocation method for side cloud collaborative reasoning provided by another embodiment of the present application;

[0060] Figure 12 is a structural schematic diagram of a model allocation device for side cloud collaborative reasoning provided by an embodiment of the present application;

[0061] Figure 13 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0063] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0064] First, a model allocation method for side cloud collaborative reasoning in the embodiments of the present application is described:

[0065] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. The artificial intelligence (AI) is the theory, method, technology and application system for using digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0066] The artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0067] The model allocation method for edge-cloud collaborative reasoning provided by the embodiments of the present application can be applied in a terminal, can also be applied in a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as an independent physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform, etc.; and the software can be an application program implementing the model allocation method for edge-cloud collaborative reasoning, etc., but is not limited to the above forms.

[0068] The embodiments of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0069] In one specific embodiment, referring to Figure 8 , Figure 8 An implementation environment schematic diagram provided by the present application is shown as follows:Figure 8 As shown, the implementation environment includes a cloud server and multiple edge devices, wherein the cloud server is directly or indirectly connected to the multiple edge devices via wired or wireless communication.

[0070] The cloud server is used to execute a model allocation method for edge-cloud collaborative reasoning provided in the embodiments of this application. During the execution of the model allocation method, the cloud server obtains device resource information of multiple edge devices, allocates the first network sub-module of the target network model to the corresponding edge device, and performs edge-cloud collaborative reasoning with the edge device through the second network sub-module of the target network model.

[0071] In another specific embodiment, see Figure 9 , Figure 9 This application provides a schematic diagram of an implementation environment, such as... Figure 9 As shown, the implementation environment includes a cloud server, a database server, and multiple edge devices. The cloud server is directly or indirectly connected to the database server and multiple edge devices via wired or wireless communication.

[0072] A heterogeneous model library is built in the database server, which stores multiple heterogeneous models. The cloud server is used to execute the model allocation method for edge-cloud collaborative reasoning provided in this application embodiment. During the execution of the model allocation method, the cloud server obtains multiple heterogeneous models from the heterogeneous model library of the database server, and randomly selects a corresponding target network model for each edge device from the multiple heterogeneous models. The first network sub-module of the target network model is then allocated to the corresponding edge device, and edge-cloud collaborative reasoning is performed with the edge device through the second network sub-module of the target network model.

[0073] In another specific embodiment, the model allocation method for edge-cloud collaborative inference provided in this application can also be applied to an implementation environment including a database server, a cloud server, multiple edge devices, and an independent model allocation device. The model allocation device is directly or indirectly connected to the database server, cloud server, and multiple edge devices via wired or wireless communication.

[0074] A heterogeneous model library is constructed in the database server, the heterogeneous model library stores a plurality of heterogeneous models, and a model distribution device is used to execute the model distribution method for edge-cloud collaborative reasoning provided in the embodiments of the present application. In the process of executing the model distribution method, the model distribution device acquires device resource information of a plurality of edge devices, acquires a plurality of heterogeneous models from the heterogeneous model library of the database server, and randomly selects a corresponding target network model for each edge device from the plurality of heterogeneous models, so as to distribute a first sub-network module of the target network model to the corresponding edge device, and distribute a second sub-network module of the target network model to a cloud server, so that the edge device and the cloud server can perform edge-cloud collaborative reasoning through the first sub-network module and the second sub-network module, respectively.

[0075] Please refer to Figure 1 , Figure 1 A flowchart of a model distribution method for edge-cloud collaborative reasoning provided in the embodiments of the present application is shown, as shown in Figure 1 , the model distribution method for edge-cloud collaborative reasoning includes but is not limited to steps S110 to S160.

[0076] Step S110, acquiring device resource information of a plurality of edge devices.

[0077] It can be understood that before step S110, a model distribution task can also be received, the model distribution task including an edge-cloud collaborative reasoning task and a plurality of edge devices and a cloud server participating in the edge-cloud collaborative reasoning, and then the device resource information of the plurality of edge devices is acquired according to the model distribution task.

[0078] Step S120, randomly selecting a corresponding target network model for each of the edge devices from a plurality of heterogeneous models constructed in advance.

[0079] It can be understood that the heterogeneous model refers to a network model with different network structures for the same reasoning task, as shown in Figure 10 , Figure 10 A structural diagram of a plurality of heterogeneous models provided in the embodiments of the present application is shown, as shown in Figure 10 , the heterogeneous model library stores M network structure different models, wherein the first model includes i layers of network layers, the second model includes j layers of network layers, and the Mth model includes k layers of network layers. For example, the edge-cloud collaborative reasoning task is an image classification task in the field of computer vision, and a plurality of heterogeneous models with different network structures for image classification tasks are acquired from the heterogeneous model library, so as to randomly select a corresponding target network model for a plurality of edge devices.

[0080] It should be noted that the number of heterogeneous models is greater than the number of edge devices, and enough heterogeneous models are provided for the edge devices in the model allocation process, and a better model allocation scheme is found for the edge devices as much as possible.

[0081] In one specific embodiment, a corresponding target network model is randomly selected for each edge device from a plurality of pre-constructed heterogeneous models, including:

[0082] According to the edge cloud collaborative inference task, a plurality of heterogeneous models are obtained from a pre-constructed heterogeneous model library.

[0083] A corresponding target network model is randomly selected for each edge device from a plurality of the heterogeneous models.

[0084] It can be understood that the pre-constructed heterogeneous model library includes a plurality of heterogeneous models corresponding to different edge cloud collaborative inference tasks, which can improve the generality of the model allocation method. For different edge cloud collaborative inference tasks, a plurality of corresponding heterogeneous models can be obtained from the heterogeneous model library according to the current edge cloud collaborative inference task, so that a corresponding target network model is randomly selected for each edge device from a plurality of heterogeneous models.

[0085] In step S130, the model cut point of the target network model is determined according to the device resource information of the edge device, wherein the model cut point is used to divide the target network model into a first sub-network module and a second sub-network module, the first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model.

[0086] It can be understood that the model cut point of the target network model is determined according to the device resource information of the edge device, which can divide the first sub-network module suitable for the edge device.

[0087] Referring to Figure 11 , Figure 11 The model allocation method for edge cloud collaborative inference provided by the embodiments of the present application is shown in FIG. 1. Figure 11 As shown in FIG. 1, for model M, the first sub-network module is the first network layer, which is the shallow network module of model M, and the second sub-network module is the second to kth network layer, which is the deep network module of model M.

[0088] It can be understood that the shallow network refers to the network relatively close to the input end of the model, and the deep network refers to the network relatively close to the output end of the model. The shallow network of the target network model is deployed to the edge device, the edge device takes the local data as the input of the target network model, inputs the local data into the shallow network for inference calculation to obtain the intermediate feature value, and then uploads the intermediate feature value to the cloud server. The cloud server performs inference calculation on the intermediate feature value through the deep network to obtain the inference result. The shallow network is deployed in the edge device, and the edge device uploads the intermediate feature value to the cloud server, instead of the original local data, which can protect the user privacy.

[0089] In some embodiments, the device resource information includes a memory capacity of the edge device, referring to Figure 2 , Figure 2 A sub-step flow diagram of step S130 in Figure 1 is shown, as shown in Figure 2 The determination of the model split point of the target network model according to the device resource information of the edge device includes but is not limited to steps S210 and S220.

[0090] Step S210, obtaining the memory occupation of each network layer in the target network model.

[0091] Step S220, determining the model split point of the target network model according to the memory occupation of each network layer in the target network model and the memory capacity of the edge device.

[0092] It can be understood that the content occupation of the network layer refers to the capacity of the random access memory (RAM) occupied by the network layer when running. The memory occupation of the network layer can be determined according to the network parameter amount of the network layer. Taking the memory occupation of the network layer as a reference index for splitting the model can adjust the memory occupation of the edge device when running the first sub-network module for inference calculation to meet the actual application requirements.

[0093] In some embodiments, referring to Figure 3 , Figure 3 A sub-step flow diagram of step S220 in Figure 2 is shown, as shown in Figure 3 The determination of the model split point of the target network model according to the memory occupation of each network layer in the target network model and the memory capacity of the edge device includes but is not limited to steps S310 and S320.

[0094] Step S310, starting from the first network layer in the target network model, the memory occupation of each network layer is accumulated to obtain the memory cumulative occupation of the current network layer.

[0095] Step S320: If the cumulative memory usage of the current network layer is greater than the memory capacity of the edge device, the position between the current network layer and the network layer above the current network layer is determined as the model splitting point of the target network model.

[0096] It is understandable that, starting from the first network layer in the target network model, the memory usage of each network layer is accumulated to obtain the cumulative memory usage of the current network layer. For example, the memory usage of the first network layer is 2MB, the memory usage of the second network layer is 4MB, the memory usage of the third network layer is 6MB, and so on. Then the cumulative memory usage of the second network layer is 6MB, the cumulative memory usage of the third network layer is 12MB. If the memory capacity of the edge device is 8MB, and the cumulative memory usage of the third network layer is 12MB, which is greater than the memory capacity of the edge device, then the position between the third network layer and the network layer above the third network layer (i.e., the second network layer) is determined as the model splitting point of the target network.

[0097] Assume the i-th edge device (with memory capacity S) i The j-th target network model is randomly selected from multiple heterogeneous models. j From the target network model j Starting from the first layer L1, the search continues up to the kth layer L. k The following conditions must be met:

[0098]

[0099] Here, size(*) represents the size of the network layer, that is, the amount of memory occupied by the network layer during runtime.

[0100] For example, such as Figure 11 As shown, the model split point of model M is the position between the first-layer network model and the second-layer network model. This model split point divides model M into a first sub-network module including the first-layer network layer and a second sub-network module including the second-layer network layer to the Kth-layer network layer.

[0101] It should be understood that when the cumulative memory usage of the current network layer exceeds the memory capacity of the edge device, determining the position between the current network layer and the network layer above it as the model splitting point of the target network model can maximize the utilization of memory resources when the edge device performs inference calculations through the first sub-network module.

[0102] In step S140, for each target network model, the first sub-network module is assigned to the edge device corresponding to the target network model, and the second sub-network module is assigned to the cloud server, so that the edge device and the cloud server respectively perform collaborative inference through the first sub-network module and the second sub-network module.

[0103] It can be understood that, if the model allocation method provided in the embodiments of the present application is executed by an independent model allocation device, for each target network model, the model allocation device sends the first sub-network module to the edge device corresponding to the target network model, and sends the second sub-network module to the cloud server participating in the edge-cloud collaborative inference. If the model allocation method provided in the embodiments of the present application is executed by the cloud server participating in the edge-cloud collaborative inference, for each target network model, the cloud server sends the first sub-network module to the edge device corresponding to the target network model, and the cloud server saves the second sub-network module.

[0104] In some possible embodiments, the assigning the first sub-network module to the edge device corresponding to the target network model comprises:

[0105] sending assignment indication information to the edge device corresponding to the target network model, the assignment indication information being used to instruct the edge device to acquire the first sub-network module in the target network model from the heterogeneous model library according to the model split point.

[0106] It can be understood that the model allocation method for edge-cloud collaborative inference provided in the embodiments of the present application acquires a plurality of heterogeneous models from a heterogeneous model library, and after determining the model split point of a target network model, sends corresponding assignment indication information to an edge device, so that the edge device acquires a corresponding first sub-network module from the heterogeneous model library. In the case that the cloud server executes the model allocation method and the heterogeneous model library is deployed on an independent database server, the transmission pressure of the cloud server can be reduced.

[0107] In step S150, after the edge device and the cloud server perform collaborative inference, an assignment evaluation result corresponding to each edge device is acquired.

[0108] It can be understood that the assignment evaluation result represents the inference effect of the edge device and the cloud server performing collaborative inference through the first sub-network module and the second sub-network module, and the model allocation is re-performed according to the inference effect to obtain a better model allocation scheme.

[0109] In some embodiments, the assignment evaluation result includes model inference accuracy of the target network model, and the assignment end condition is that the model inference accuracy is greater than or equal to a preset accuracy threshold, see Figure 4 ,Figure 4 Fig. 1 shows a flowchart of a method for model distribution according to an embodiment of the present disclosure. Figure 1 Fig. 2 shows a flowchart of a method for model distribution according to an embodiment of the present disclosure. Figure 4 As shown in Fig. 2, the obtaining of the distribution evaluation result corresponding to each edge device comprises but is not limited to steps S410 and S420.

[0110] In step S410, for each edge device, an inference result obtained by the cloud server through inference calculation of the second sub-network module is obtained.

[0111] In step S420, the model inference accuracy of the target network model corresponding to each edge device is determined according to the inference result.

[0112] It should be understood that taking the model inference accuracy as one of the indicators for evaluating the model inference effect, for the edge device whose model inference accuracy does not satisfy the distribution end condition, a new target network model is randomly selected from the plurality of heterogeneous models, a new distribution evaluation result is obtained, and the process is repeated until the model inference accuracy is greater than or equal to the preset accuracy threshold, so that the final model distribution scheme is obtained, which can improve the overall inference accuracy of the edge-cloud collaborative inference.

[0113] For example, for the edge-cloud collaborative inference task of image segmentation, the model inference accuracy of the i-th target network model on the validation set (Z pictures) can be:

[0114]

[0115] Wherein, Acc i represents the model inference accuracy of the j-th validation picture in the validation set.

[0116] In some embodiments, the distribution evaluation result comprises the model inference delay of the target network model, and the distribution end condition is that the model inference delay is less than or equal to a preset time threshold, as shown in Fig. 3. Figure 5 Figure 5 Fig. 3 shows a flowchart of a method for model distribution according to an embodiment of the present disclosure. Figure 1 Fig. 4 shows a flowchart of a method for model distribution according to an embodiment of the present disclosure. Figure 5 As shown in Fig. 4, the obtaining of the distribution evaluation result corresponding to each edge device comprises but is not limited to steps S510 to S540.

[0117] In step S510, for each edge device, a first delay time of the edge device in inference calculation through the first sub-network module is obtained.

[0118] In step S520, a second delay time of the edge device in sending intermediate feature values to the cloud server is obtained, wherein the intermediate feature values are obtained by the edge device through inference calculation of the first sub-network module. ​

[0119] Step S530, acquiring a third delay time of the cloud server in inference calculation through the second sub-network module.

[0120] Step S540, determining a model inference delay of the target network model corresponding to each edge device according to the first delay time, the second delay time and the third delay time.

[0121] It should be understood that taking the model inference delay as one of the indicators for evaluating the model inference effect, for the edge device whose model inference delay does not meet the distribution end condition, a new target network model is randomly selected from the plurality of heterogeneous models, a new distribution evaluation result is obtained, and the process is repeated until the model inference delay is less than or equal to the preset time threshold, so as to obtain the final model distribution scheme, which can reduce the overall inference delay of edge cloud collaborative inference.

[0122] It should be noted that the target network model is divided into two modules: the first sub-network module and the second sub-network module, so the inference delay of each target network model includes three parts: the first delay time L edge of the edge device in inference calculation through the first sub-network module, the second delay time L trans of transmitting the intermediate feature value obtained by the first sub-network module to the cloud server, and the third delay time L cloud of the cloud server in inference calculation through the second sub-network module, so as to determine the model inference delay L i of the target network model corresponding to each edge device according to the first delay time, the second delay time and the third delay time, which is represented as follows:

[0123] L i = L edge + L trans + L cloud ;

[0124] In some possible embodiments, after the edge cloud collaborative inference is performed by the edge device and the cloud server, a new target network model is determined again for the edge device with the lowest model inference precision and the edge device with the slowest model inference delay, until the lowest model inference precision in the plurality of edge devices is greater than a preset precision threshold, and the slowest model inference delay in the plurality of edge devices is less than a preset time threshold, to obtain the model distribution scheme.

[0125] Step S160, for the edge device whose distribution evaluation result does not meet the distribution end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, to obtain a new distribution evaluation result corresponding to the edge device, until the new distribution evaluation result meets the distribution end condition.

[0126] It can be understood that for the edge device whose allocation evaluation result does not satisfy the allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, a model split point of the new target network model is determined according to the device resource information of the edge device, the new target network model is split into a new first sub-network model and a new second sub-network model according to the model split point of the new target network model, the new first sub-network model is allocated to the edge device, and the new second sub-network model is allocated to the cloud server. After the edge device and the cloud server perform collaborative reasoning, a new allocation evaluation result corresponding to the edge device is obtained, and the process is repeated until the new allocation evaluation result satisfies the allocation end condition.

[0127] After the model allocation for the edge device and the cloud server, an allocation evaluation result corresponding to the edge device is obtained. For the edge device whose allocation evaluation result does not satisfy the allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, and a new round of model allocation is performed based on the new target network model. The model allocation scheme can be optimized, the overall reasoning effect of edge-cloud collaborative reasoning can be improved, and the model allocation method provided in the embodiment of the application has stronger universality compared with the model allocation optimization method in which only the network model split point is changed and the network model structure on the edge device and the cloud server is not changed. The model allocation method can quickly optimize the model allocation in an implementation environment in which the computing capabilities of edge devices are different, and finally find a better model allocation scheme. Further, since the model split point depends only on the device resource information of the edge device, the first sub-network module after division can be ensured to adapt to the edge device in terms of device resources when a better model allocation scheme for the edge device and the cloud server is found. For example, the first sub-network module after division can maximize the use of the memory resources of the edge device during inference calculation, improve the resource utilization rate of the edge device, avoid the situation that the edge device only runs a small part of the network layers of the target network model, and thus most of the resources are idle, and at the same time, the inference calculation burden of the cloud server is increased.

[0128] The application provides a model allocation method for edge cloud collaborative reasoning. The model allocation method first acquires device resource information of a plurality of edge devices, and randomly selects a corresponding target network model for each edge device from a plurality of heterogeneous models constructed in advance. Then, a model cutting point of the target network model is determined according to the device resource information of the edge device, wherein the model cutting point is used to divide the target network model into a first sub-network module and a second sub-network module, the first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model. For each target network model, the first sub-network module is allocated to the edge device corresponding to the target network model, and the second sub-network module is allocated to a cloud server, so that the edge device and the cloud server perform collaborative reasoning through the first sub-network module and the second sub-network module respectively. After the edge device and the cloud server perform collaborative reasoning, an allocation evaluation result corresponding to each edge device is acquired. For an edge device whose allocation evaluation result does not satisfy an allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, so as to obtain a new allocation evaluation result corresponding to the edge device, until the new allocation evaluation result satisfies the allocation end condition. After the model allocation for the edge device and the cloud server is performed, the allocation evaluation result corresponding to the edge device is acquired. For an edge device whose allocation evaluation result does not satisfy the allocation end condition, a new target network model is randomly selected for the edge device from the plurality of heterogeneous models, and a new round of model allocation is performed based on the new target network model, so as to continuously optimize the model allocation scheme and improve the overall reasoning effect of edge cloud collaborative reasoning.

[0129] Referring to Figure 6 , Figure 6 A method for edge cloud collaborative reasoning is shown, which is applied to an edge device, as shown in Figure 6 The method includes but is not limited to steps S610 to S640.

[0130] Step S610, receiving to-be-reasoned data.

[0131] Step S620, inputting the to-be-reasoned data into the first sub-network module to obtain intermediate feature values.

[0132] Step S630, sending the intermediate feature values to a cloud server, so that the cloud server performs reasoning calculation on the intermediate feature values through the second sub-network module to obtain a reasoning result.

[0133] Step S640, receiving the reasoning result returned by the cloud server.

[0134] The first sub-network module and the second sub-network module on the cloud server are allocated by the model allocation method for edge cloud collaborative reasoning provided in any of the above embodiments.

[0135] It should be noted that the edge cloud collaborative reasoning method belongs to the same inventive concept as the model allocation method for edge cloud collaborative reasoning provided in the embodiments of the present application. For specific steps and technical effects, please refer to the model allocation method embodiment part, which will not be repeated here.

[0136] Referring to Figure 7 , Figure 7 An edge cloud collaborative reasoning method provided by another embodiment of the present application is shown. As shown in Figure 7 , the method includes but is not limited to steps S710 to S730.

[0137] Step S710, receiving intermediate feature values sent by an edge device, wherein the intermediate feature values are obtained by the edge device through a first sub-network module for inference calculation on to-be-reasoned data.

[0138] Step S720, inputting the intermediate feature values to a second sub-network module to obtain a reasoning result.

[0139] Step S730, sending the reasoning result to the edge device.

[0140] Wherein the second sub-network module and the first sub-network module on the edge device are allocated by the model allocation method for edge cloud collaborative reasoning provided in any of the above embodiments.

[0141] It should be noted that the edge cloud collaborative reasoning method belongs to the same inventive concept as the model allocation method for edge cloud collaborative reasoning provided in the embodiments of the present application. For specific steps and technical effects, please refer to the model allocation method embodiment part, which will not be repeated here.

[0142] The application scenarios of the embodiments of the present application are described below through specific embodiments:

[0143] The model distribution method for edge-cloud collaborative reasoning provided by the embodiments of the present application is applied to an auxiliary diagnosis system based on artificial intelligence technology in the field of digital medical treatment. The auxiliary diagnosis system includes a medical cloud server and a plurality of medical terminal devices. In a medical application scenario, a sample image is a medical image, and an object type contained in the sample image is a lesion, i.e., a part of the body where a pathological change occurs. The medical image refers to an internal tissue obtained in a non-invasive manner for medical treatment or medical research, for example, images of the stomach, abdomen, heart, knee, and brain generated by medical instruments such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), US (ultrasonic), X-ray images, electroencephalograms, and optical photography lamps.

[0144] The medical cloud server is configured to execute the model distribution method for edge-cloud reasoning according to the embodiments of the present application. For example, when facing a medical image segmentation task, the medical cloud server performs model distribution in advance according to the model distribution method for edge-cloud reasoning provided by the embodiments of the present application. After the model distribution is completed, when the medical terminal device receives a to-be-segmented medical image sent by a user, the medical terminal device performs inference calculation on the to-be-segmented medical image by using the first sub-network module obtained through the model distribution to obtain intermediate feature values of the to-be-segmented medical image, and uploads the intermediate feature values of the to-be-segmented image to the medical cloud server. The medical cloud server performs inference calculation on the received intermediate feature values by using the second sub-network module to obtain a segmentation result of the to-be-segmented medical image, and returns the segmentation result of the to-be-segmented medical image to the medical terminal device, thereby completing the medical image segmentation task.

[0145] Please refer to Figure 12 The embodiments of the present application also provide a model distribution device 100 for edge-cloud collaboration, which comprises:

[0146] A resource information acquisition module 110 is configured to acquire device resource information of a plurality of edge devices.

[0147] A network model selection module 120 is configured to randomly select a corresponding target network model for each edge device from a plurality of heterogeneous models constructed in advance.

[0148] A network model segmentation module 130 is configured to determine a model segmentation point of the target network model according to the device resource information of the edge device, wherein the model segmentation point is used to divide the target network model into a first sub-network module and a second sub-network module, the first sub-network module is a shallow network of the target network model, and the second sub-network module is a deep network of the target network model.

[0149] The first model allocation module 140 is configured to, for each target network model, allocate the first sub-network module to an edge device corresponding to the target network model and allocate the second sub-network module to a cloud server, so that the edge device and the cloud server respectively perform collaborative inference through the first sub-network module and the second sub-network module.

[0150] The evaluation result acquisition module 150 is configured to acquire an allocation evaluation result corresponding to each edge device after the edge device and the cloud server perform collaborative inference.

[0151] The second model allocation module 160 is configured to, for the edge device whose allocation evaluation result does not satisfy an allocation end condition, randomly select a new target network model for the edge device from the plurality of heterogeneous models, so as to obtain a new allocation evaluation result corresponding to the edge device, until the new allocation evaluation result satisfies the allocation end condition.

[0152] In some embodiments, the device resource information includes a memory capacity of the edge device, and the network model segmentation module further includes:

[0153] The first network model segmentation module is configured to acquire a memory occupancy of each network layer in the target network model.

[0154] The second network model segmentation module is configured to determine a model segmentation point of the target network model according to the memory occupancy of each network layer in the target network model and the memory capacity of the edge device.

[0155] In some embodiments, the second network model segmentation module includes:

[0156] The memory occupancy accumulation module is configured to, starting from a first network layer in the target network model, accumulate the memory occupancy of each network layer to obtain a memory cumulative occupancy of a current network layer.

[0157] The model segmentation point judgment module is configured to, in a case where the memory cumulative occupancy of the current network layer is greater than the memory capacity of the edge device, determine a position between the current network layer and a previous network layer of the current network layer as the model segmentation point of the target network model.

[0158] In some embodiments, the allocation evaluation result includes a model inference accuracy of the target network model, and the allocation end condition is that the model inference accuracy is greater than or equal to a preset accuracy threshold, and the evaluation result acquisition module includes:

[0159] An inference result obtaining module is configured to obtain, for each edge device, an inference result obtained by the cloud server through the second sub-network module.

[0160] An inference accuracy calculating module is configured to determine, according to the inference result, a model inference accuracy of the target network model corresponding to each edge device.

[0161] In some embodiments, the allocation evaluation result includes a model inference delay of the target network model, the allocation end condition is that the model inference delay is less than or equal to a preset time threshold, and the evaluation result obtaining module includes:

[0162] A first delay time obtaining module is configured to obtain, for each edge device, a first delay time of the edge device in performing inference calculation through the first sub-network module.

[0163] A second delay time obtaining module is configured to obtain a second delay time of the edge device in sending an intermediate feature value to the cloud server, the intermediate feature value being obtained by the edge device through the first sub-network module.

[0164] A third delay time obtaining module is configured to obtain a third delay time of the cloud server in performing inference calculation through the second sub-network module.

[0165] A model inference delay calculating module is configured to determine, according to the first delay time, the second delay time and the third delay time, a model inference delay of the target network model corresponding to each edge device.

[0166] It should be noted that the information interaction between the modules of the above apparatus, the execution process and the like, since based on the same conception as the method embodiments of the present application, the specific functions and the technical effects brought by the same can be referred to the method embodiments part, and will not be repeated here.

[0167] Please refer to Figure 13 , Figure 13 A hardware structure of an electronic device provided by an embodiment of the present application is shown, and the electronic device includes:

[0168] The processor 210 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute a related computer program to implement the technical solutions provided by the embodiments of the present application.

[0169] The memory 220 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 220 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 220 and are called and executed by the processor 210 to implement the model allocation method for edge-cloud collaborative reasoning and the edge-cloud collaborative reasoning method of the embodiments of the present application;

[0170] The input / output interface 230 is configured to realize information input and output.

[0171] The communication interface 240 is configured to realize the communication interaction between the device and other devices, and the communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, a WI-FI, Bluetooth, etc.). The bus 250 is configured to transmit information between each component (for example, the processor 210, the memory 220, the input / output interface 230, and the communication interface 240) of the device. The processor 210, the memory 220, the input / output interface 230, and the communication interface 240 are connected to each other in the device through the bus 250.

[0172] The embodiments of the present application also provide a storage medium, which is a computer readable storage medium, and is configured to store computer readable information. The storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the above-mentioned model allocation method for edge-cloud collaborative reasoning and the edge-cloud collaborative reasoning method.

[0173] The memory, as a computer readable storage medium, can be used to store software programs and computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0174] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0175] The apparatus embodiments described above are merely exemplary, and units described as separate components may or may not be physically separate, i.e., may be located in one place, or may be distributed over multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand that all or some of the steps in the above method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations. In the above embodiments, the description of each embodiment is focused on, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0176] The terms "first", "second", "third", "fourth" etc. (if any) in the specification of this application and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0177] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0178] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0179] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed to a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme. In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0180] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0181] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A model allocation method for edge-cloud collaborative reasoning, characterized in that, The method includes: Obtain device resource information from multiple edge devices; For each edge device, a corresponding target network model is randomly selected from a plurality of pre-built heterogeneous models; The model splitting point of the target network model is determined based on the device resource information of the edge device. The model splitting point is used to divide the target network model into a first sub-network module and a second sub-network module. The first sub-network module is the shallow network of the target network model, and the second sub-network module is the deep network of the target network model. For each target network model, the first sub-network module is assigned to the edge device corresponding to the target network model, and the second sub-network module is assigned to the cloud server, so that the edge device and the cloud server can perform collaborative inference through the first sub-network module and the second sub-network module respectively; After the edge device and the cloud server perform collaborative reasoning, the allocation evaluation result corresponding to each edge device is obtained; For edge devices whose allocation evaluation results do not meet the allocation termination condition, a new target network model is randomly selected from the multiple heterogeneous models to obtain a new allocation evaluation result for the edge device, until the new allocation evaluation result meets the allocation termination condition.

2. The model allocation method for edge-cloud collaborative reasoning according to claim 1, characterized in that, The device resource information includes the memory capacity of the edge device; The step of determining the model segmentation points of the target network model based on the device resource information of the edge device includes: Obtain the memory usage of each network layer in the target network model; The model splitting point of the target network model is determined based on the memory usage of each network layer in the target network model and the memory capacity of the edge device.

3. The model allocation method for edge-cloud collaborative reasoning according to claim 2, characterized in that, The step of determining the model segmentation point of the target network model based on the memory usage of each network layer in the target network model and the memory capacity of the edge device includes: Starting from the first network layer in the target network model, the cumulative memory usage of each network layer is calculated to obtain the cumulative memory usage of the current network layer. If the cumulative memory usage of the current network layer is greater than the memory capacity of the edge device, the position between the current network layer and the network layer above the current network layer is determined as the model splitting point of the target network model.

4. The model allocation method for edge-cloud collaborative reasoning according to claim 1, characterized in that, The allocation evaluation result includes the model inference accuracy of the target network model, and the allocation termination condition is that the model inference accuracy is greater than or equal to a preset accuracy threshold. The step of obtaining the allocation evaluation result corresponding to each edge device includes: For each edge device, obtain the inference result obtained by the cloud server through the second sub-network module; The model inference accuracy of the target network model corresponding to each edge device is determined based on the inference results.

5. The model allocation method for edge-cloud collaborative reasoning according to claim 1, characterized in that, The allocation evaluation result includes the model inference latency of the target network model, and the allocation termination condition is that the model inference latency is less than or equal to a preset time threshold. The step of obtaining the allocation evaluation result corresponding to each edge device includes: For each edge device, obtain the first delay time of the edge device when performing inference calculations through the first sub-network module; The second delay time for the edge device to send intermediate feature values ​​to the cloud server is obtained, wherein the intermediate feature values ​​are obtained by the edge device through inference calculation via the first sub-network module; Obtain the third delay time of the cloud server performing inference calculations through the second sub-network module; The model inference latency of the target network model corresponding to each edge device is determined based on the first latency, the second latency, and the third latency.

6. An edge-cloud collaborative reasoning method, characterized in that, Applied to edge devices, the method includes: Receive the data to be inferred; The data to be inferred is input into the first sub-network module to obtain intermediate feature values; The intermediate feature value is sent to the cloud server so that the cloud server can perform inference calculations on the intermediate feature value through the second sub-network module to obtain the inference result. Receive the inference result returned by the cloud server; The first sub-network module and the second sub-network module on the cloud server are allocated using the model allocation method for edge-cloud collaborative reasoning as described in any one of claims 1 to 5.

7. An edge-cloud collaborative reasoning method, characterized in that, Applied to a cloud server, the method includes: Receive intermediate feature values ​​sent by the edge device, wherein the intermediate feature values ​​are obtained by the edge device through the first sub-network module for inference calculation of the data to be inferred; The intermediate feature values ​​are input into the second sub-network module to obtain the inference result; The inference result is sent to the edge device; The second sub-network module and the first sub-network module on the edge device are allocated using the model allocation method for edge-cloud collaborative reasoning as described in any one of claims 1 to 5.

8. A model allocation device for edge-cloud collaborative reasoning, characterized in that, The device includes: The resource information acquisition module is used to acquire device resource information from multiple edge devices. The network model selection module is used to randomly select a corresponding target network model for each edge device from a plurality of pre-built heterogeneous models; The network model segmentation module is used to determine the model segmentation point of the target network model based on the device resource information of the edge device. The model segmentation point is used to divide the target network model into a first sub-network module and a second sub-network module. The first sub-network module is the shallow network of the target network model, and the second sub-network module is the deep network of the target network model. The first model allocation module is used to allocate the first sub-network module to the edge device corresponding to the target network model and allocate the second sub-network module to the cloud server for each target network model, so that the edge device and the cloud server can perform collaborative inference through the first sub-network module and the second sub-network module respectively. The evaluation result acquisition module is used to acquire the allocation evaluation result corresponding to each edge device after the edge device and the cloud server perform collaborative reasoning. The second model allocation module is used to randomly select a new target network model from multiple heterogeneous models for the edge device whose allocation evaluation result does not meet the allocation termination condition, so as to obtain a new allocation evaluation result for the edge device, until the new allocation evaluation result meets the allocation termination condition.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that is executed by the at least one processor to enable the at least one processor to perform the model allocation method for edge-cloud collaborative reasoning as described in any one of claims 1 to 5, or the edge-cloud collaborative reasoning method as described in claim 6 or 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the model allocation method for edge-cloud collaborative reasoning as described in any one of claims 1 to 5, or the edge-cloud collaborative reasoning method as described in claim 6 or 7.

Images