Model-based data processing method, device, equipment, system and medium

Abstract

The invention discloses a model-based data processing method, device, equipment, system and medium, relates to the technical field of artificial intelligence and optical communication, and can be applied to an FTTR scene. The method comprises the following steps: a first device receives a data processing request sent by a terminal, wherein the data processing request is used for requesting to process target data; the first device sends the target data to the second device; the second device receives target data sent by the first device; the second device performs first processing on the target data based on the data processing model to obtain a first data processing result; the second device sends the first data processing result to the first device; the first device receives a first data processing result sent by the second device; the first device performs second processing on the first data processing result based on the data processing model to obtain a second data processing result; and the first device sends the second data processing result to the terminal.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and optical communication technology, and in particular to a model-based data processing method, device, equipment, system and medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, large models have shown good performance in natural language processing, computer vision processing, etc., such as large language models (LLM). Large models refer to deep learning models with tens of billions or even hundreds of billions of model parameters.

[0003] Currently, the big model can be deployed on the terminal so that the terminal performs data processing based on the big model, or the big model can also be deployed on the cloud so that the cloud performs data processing based on the big model.

[0004] However, in the solution where the terminal performs data processing based on a large model, since the large model requires the support of high-performance computing power / memory resources, and the computing power / memory resources provided by the terminal are limited, it is difficult to support data processing based on the large model. In the solution where the cloud performs data processing based on a large model, since the data needs to be transmitted in the network that provides cloud services, it is difficult to ensure the security of the transmitted data. Summary of the invention

[0005] The present application provides a model-based data processing method, apparatus, device, system and medium, which are used to provide a model-based data processing method while taking into account high computing power and high security.

[0006] In order to achieve the above objectives, this application provides the following technical solutions.

[0007] In the first aspect, a model-based data processing method is provided, which can be executed by a first device or by a component of the first device, such as a processor, chip, or chip system of the first device, or by a logic module or software that can implement all or part of the functions of the first device.

[0008] The first device is a network device connected to the terminal. In the present application, the first device refers to a terminal side device, that is, a network device deployed close to the terminal side. The method includes the following contents.

[0009] The first device receives a data processing request sent by the terminal.

[0010] The data processing request is used to request processing of target data. In this application, target data is used to refer to data to be processed.

[0011] The first device sends target data to the second device.

[0012] The second device is a network device connected to the cloud. In the present application, the second device refers to an edge device, that is, a network device deployed at the edge of the network. In the present application, the second device stores a data processing model, such as an LLM.

[0013] The first device receives the first data processing result sent by the second device.

[0014] The first data processing result is a result obtained by the data processing model performing a first processing on the target data. It is understandable that the second device can use the stored data processing model to perform a first processing on the target data to obtain the first data processing result.

[0015] The first device performs second processing on the first data processing result based on the data processing model to obtain a second data processing result.

[0016] The second data processing result is the processing result of the target data.

[0017] The first device sends the second data processing result to the terminal.

[0018] In the above technical solution, a scheme for performing model-based data processing in combination with a first device and a second device is provided, that is, a model reasoning scheme for collaboration between a terminal device and an edge device (such as edge collaboration) is provided. Among them, by deploying the first processing based on the data processing model on the edge device, and deploying the second processing based on the data processing model on the terminal device, the separate deployment of the model reasoning service is realized. On the one hand, considering that the edge device is usually deployed with powerful hardware and can provide strong computing / memory capabilities, the data processing model is deployed on the edge device, which can provide sufficient computing power / memory resources for the first processing of the data processing model, and the terminal device only needs to perform the second processing of the first data processing result, which also reduces the computing power / memory pressure of the terminal device. On the other hand, for the second processing of the terminal device, since it is performed locally on the terminal device, the data does not need to be transmitted in the network, and the security is higher. For the first processing of the edge device, since the edge device is a network device located at the edge of the network, it is closer to the terminal than the cloud, so it can reduce the transmission of data in the network, thereby effectively improving the security of the data.

[0019] In some possible implementations, the first processing is a prefill calculation, and the second processing is a decode calculation. In the present application, the computing power / memory requirement of the first processing is greater than the computing power / memory requirement of the second processing. It is understandable that in other possible implementations, the first processing and the second processing can also be other types of calculations, such as the first processing is the calculation of a part of the network layer of the data processing model. The second processing is the calculation of another part of the network layer of the data processing model, which is not limited in the present application.

[0020] In the above implementation, a separate deployment architecture for Prefill calculation and Decode calculation is provided, which can improve the overall throughput of data processing. Among them, by deploying the Prefill calculation with high computing power / memory requirements on the edge device, which is an upper-level device that provides high computing power / memory, the computing power / memory resources of the edge device can be effectively utilized, thereby improving the efficiency of the Prefill calculation. By deploying the Decode calculation with low computing power / memory requirements on the end-side device, which is a lower-level device that provides low computing power / memory, the computing power / memory pressure of the end-side device can be reduced, thereby reducing the computing power / memory cost of the end-side device.

[0021] In some possible implementations, the second process includes at least one third process, wherein each third process includes the following process.

[0022] The first device obtains target model parameters.

[0023] The target model parameters are model parameters of the data processing model used to perform the third processing, such as model weights.

[0024] The first device performs the third processing on the data to be processed based on the target model parameters.

[0025] The data to be processed in the first third processing is the first data processing result, and the data to be processed in other third processing is the data processing result obtained in the last third processing. The data processing result obtained in the last third processing is the second data processing result.

[0026] In the above implementation, a method for obtaining target model parameters in batches is provided, so that the amount of data to be processed by the first device during each third processing is small, reducing the computing power / memory pressure of the first device, thereby improving the processing efficiency of the first device.

[0027] In some possible implementations, the first device stores some model parameters of the data processing model.

[0028] The first device acquires the target model parameter, including: if the first device stores the target model parameter, acquiring the stored target model parameter. Or, if the first device does not store the target model parameter, sending a first request to the second device. The first request is used to request to acquire the target model parameter. The first device receives the target model parameter sent by the second device.

[0029] In the above implementation, on the one hand, the first device requests part of the model parameters from the second device in batches, so that the amount of data to be processed by the first device during each third processing is small, reducing the computing power / memory pressure of the first device, thereby improving the processing efficiency of the first device. On the other hand, by storing part of the model parameters of the data processing model in the first device, the amount of data requested by the first device for the model parameters from the second device is reduced, thereby saving the consumption of network bandwidth.

[0030] In some possible implementations, the first request carries performance parameters of the first device. The performance parameters are used to indicate the computing power and / or memory capacity of the first device. The target model parameters include: target model parameters of the target number of layers of the data processing model that match the performance parameters. The target number of layers is the number of layers that match the performance parameters. It is understandable that the higher the computing power and / or memory capacity indicated by the performance parameters, the more target layers, and the lower the computing power and / or memory capacity indicated by the performance parameters, the fewer target layers.

[0031] In the above implementation, by carrying the performance parameters of the first device in the first request, the second device can adaptively deliver model parameters of the corresponding number of layers, such as 1 layer or 2 layers, according to the computing power and / or memory capacity of the first device, thereby improving the flexibility of delivering model parameters.

[0032] In some possible implementations, the first device receiving the target model parameter sent by the second device includes: The first device receives broadcast information from the second device, wherein the broadcast information includes target model parameters.

[0033] In the above embodiment, the target model parameters are transmitted by broadcasting, so that the second device only needs to send the target model parameters once when facing multiple first devices, which greatly reduces the consumption of network bandwidth.

[0034] In some possible implementations, before the first device receives the data processing request sent by the terminal, the method further includes: the first device receives all model parameters of the data processing model sent by the second device, or some model parameters of the data processing model. The first device stores some model parameters of the data processing model.

[0035] In some possible implementations, after the first device receives the data processing request sent by the terminal, the method further includes: the first device sends a query request to the second device. The query request is used to request a query of model parameters of the target version of the data processing model. The target version of the data processing model may be the latest version of the data processing model. The first device receives the model parameters of the target version of the data processing model sent by the second device. The first device updates the stored model parameters based on the model parameters of the target version of the data processing model.

[0036] In the above embodiment, the first device updates some of the model parameters stored locally by querying the latest version of the model.

[0037] In some possible implementations, the first device performs a second process on the first data processing result based on the data processing model to obtain the second data processing result, including: the first device sends a second request to the second device. The second request is used to request to obtain a model parameter of the data processing model for performing the second process. The first device receives the model parameter sent by the second device. The first device performs a second process on the first data processing result based on the model parameter to obtain the second data processing result.

[0038] In the above embodiment, a solution for sending all model parameters at one time is provided. For example, when the computing power / memory of the first device is sufficient, a request can be made to the second device to send all model parameters.

[0039] In some possible implementations, the data processing request carries target data.

[0040] Or, in some other possible implementations, the data processing request carries a data identifier of the target data. The method further includes: acquiring data corresponding to the data identifier from a data set associated with the first device as the target data.

[0041] In the above implementation, two solutions for acquiring target data are provided, both of which can quickly and efficiently acquire the target data, thereby smoothly completing the subsequent data processing process.

[0042] In some possible implementations, before the first device sends the target data to the second device, the method further includes: The first device encrypts the target data.

[0043] In the above implementation, by encrypting the target data, the security of the target data during transmission from the first device to the second device can be ensured.

[0044] In some possible implementations, the first device is an optical network unit (ONU) device or a main fiber to the room unit (MFU) device, and the second device is an optical line terminal (OLT) device.

[0045] Or, in some other possible implementations, the first device is a customer premise equipment (CPE) device in a network based on an optical transport network (OTN) dedicated line, and the second device is an edge cloud device in a network based on an OTN dedicated line.

[0046] On the second aspect, a model-based data processing method is provided, which can be executed by a second device, or by a component of the second device, such as a processor, chip, or chip system of the second device, or by a logic module or software that can implement all or part of the functions of the second device.

[0047] The second device stores a data processing model, such as an LLM.

[0048] The method includes: The second device receives the target data sent by the first device. The second device performs a first process on the target data based on the data processing model to obtain a first data processing result. The second device sends the first data processing result to the first device. The first data processing result is used to perform a second process based on the data processing model to obtain a second data processing result. The second data processing result is a processing result of the target data.

[0049] In the above technical solution, a scheme for performing model-based data processing in combination with a first device and a second device is provided, that is, a model reasoning scheme for collaboration between a terminal device and an edge device (such as edge collaboration) is provided. Among them, by deploying the first processing based on the data processing model on the edge device, and deploying the second processing based on the data processing model on the terminal device, the separate deployment of the model reasoning service is realized. On the one hand, considering that the edge device is usually deployed with powerful hardware and can provide strong computing / memory capabilities, the data processing model is deployed on the edge device, which can provide sufficient computing power / memory resources for the first processing of the data processing model, and the terminal device only needs to perform the second processing of the first data processing result, which also reduces the computing power / memory pressure of the terminal device. On the other hand, for the second processing of the terminal device, since it is performed locally on the terminal device, the data does not need to be transmitted in the network, and the security is higher. For the first processing of the edge device, since the edge device is a network device located at the edge of the network, it is closer to the terminal than the cloud, so it can reduce the transmission of data in the network, thereby effectively improving the security of the data.

[0050] In some possible implementations, the second processing includes at least one third processing. During each third processing, the method further includes: the second device receives a first request sent by the first device. The first request is used to request to obtain target model parameters of the data processing model, and the target model parameters are model parameters of the data processing model used to perform the third processing. Based on the first request, the second device sends the target model parameters to the first device.

[0051] In the above implementation, a method for obtaining target model parameters in batches is provided, so that the amount of data to be processed by the first device during each third processing is small, reducing the computing power / memory pressure of the first device, thereby improving the processing efficiency of the first device.

[0052] In some possible implementations, the first request carries performance parameters of the first device. The performance parameters are used to indicate the computing power and / or memory capacity of the first device. The target model parameters include: target model parameters of the target number of layers of the data processing model that match the performance parameters.

[0053] In the above implementation, by carrying the performance parameters of the first device in the first request, the second device can adaptively deliver model parameters of the corresponding number of layers, such as 1 layer or 2 layers, according to the computing power and / or memory capacity of the first device, thereby improving the flexibility of delivering model parameters.

[0054] In some possible implementations, the second device sends the target model parameter to the first device, including: The second device sends broadcast information, where the broadcast information includes target model parameters.

[0055] In the above embodiment, the target model parameters are transmitted by broadcasting, so that the second device only needs to send the target model data once when facing multiple first devices, which greatly reduces the consumption of network bandwidth.

[0056] In some possible implementations, the first device stores some model parameters of the data processing model; before receiving the target data sent by the first device, the method further includes: The second device sends all model parameters of the data processing model, or part of the model parameters of the data processing model to the first device.

[0057] In the above implementation, by storing part of the model parameters of the data processing model in the first device, the amount of data that the first device requests for the model parameters from the second device is reduced, thereby saving the consumption of network bandwidth.

[0058] In some possible implementations, before sending all model parameters of the data processing model or some model parameters of the data processing model to the first device, the method further includes: The second device receives the model update information sent by the cloud, wherein the model update information includes the model parameters of the updated data processing model. That is, the second device can send all or part of the model parameters of the data processing model to the first device when the model is updated.

[0059] Or, the second device receives a parameter request sent by the first device. The parameter request is used to request to obtain model parameters of the data processing model. That is, the first device can actively request all or part of the model parameters of the data processing model from the second device.

[0060] In some possible implementations, before the second device receives the target data sent by the first device, the method further includes: The second device receives the query request sent by the first device, wherein the query request is used to request to query the model parameters of the target version of the data processing model. Based on the query request, the second device sends the model parameters of the target version of the data processing model to the first device.

[0061] In the above implementation, the first device updates some of the model parameters stored locally by querying the latest version of the model.

[0062] In some possible implementations, after the second device sends the first data processing result to the first device, the method further includes: The second device receives the second request sent by the first device, wherein the second request is used to request to obtain the model parameters of the data processing model for performing the second processing, and the second device sends the model parameters to the first device.

[0063] In the above embodiment, a solution for sending all model parameters at one time is provided. For example, when the computing power / memory of the first device is sufficient, a request can be made to the second device to send all model parameters.

[0064] In some possible implementations, after the second device receives the target data sent by the first device, the method further includes: The second device decrypts the target data.

[0065] In the above implementation, by encrypting the target data, the security of the target data during transmission from the first device to the second device can be ensured.

[0066] In some possible implementations, the first process is Prefill calculation, and the second process is Decode calculation.

[0067] In some possible implementations, the first device is an ONU device or an MFU device, and the second device is an OLT device; or, the first device is a CPE device in an OTN dedicated line-based network, and the second device is an edge cloud device in an OTN dedicated line-based network.

[0068] In a third aspect, a model-based data processing device is provided for implementing any one of the methods provided in the first aspect. The model-based data processing device includes a module, unit or means corresponding to the above method, and the action performed by the module, unit or means can be implemented by hardware, software or by hardware executing the corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0069] In a possible implementation, the device may include a receiving module, a sending module, and a processing module; wherein: A receiving module, used for receiving a data processing request sent by a terminal, wherein the data processing request is used for requesting processing of target data; A sending module, used for sending target data to a second device, where the second device is a network device connected to the cloud, and the second device stores a data processing model; The receiving module is further used to receive a first data processing result sent by the second device, where the first data processing result is a result obtained by the data processing model performing a first processing on the target data; A processing module, used for performing a second processing on the first data processing result based on the data processing model to obtain a second data processing result, where the second data processing result is a processing result of the target data; The sending module is also used to send the second data processing result to the terminal.

[0070] In a fourth aspect, a model-based data processing device is provided for implementing any one of the methods provided in the second aspect. The model-based data processing device includes a module, unit or means corresponding to the above method, and the action performed by the module, unit or means can be implemented by hardware, software or by hardware executing the corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0071] In a possible implementation, the device may include a receiving module, a processing module, and a sending module; wherein: A receiving module, used for receiving target data sent by a first device, where the first device is a network device connected to the terminal; A processing module, used for performing a first processing on the target data based on the data processing model to obtain a first data processing result; The sending module is used to send a first data processing result to the first device, the first data processing result is used to perform a second processing based on the data processing model to obtain a second data processing result, and the second data processing result is a processing result of the target data.

[0072] In a fifth aspect, a first device is provided, comprising: a memory and a processor, the memory and the processor are connected; the memory is used to store computer execution instructions; the processor is used to call computer execution instructions, thereby implementing the method of the above-mentioned first aspect or any of its implementation methods.

[0073] Among them, the first device of the fifth aspect mentioned above can be: the first device in any implementation manner of the first aspect mentioned above, or a device including the first device mentioned above, or a device included in the first device mentioned above, such as a chip.

[0074] In the sixth aspect, a second device is provided, comprising: a memory and a processor, the memory and the processor are connected; the memory is used to store computer execution instructions; the processor is used to call the computer execution instructions, thereby implementing the method of the above-mentioned second aspect or any of its implementation methods.

[0075] Among them, the second device of the sixth aspect mentioned above can be: the second device in any implementation manner of the second aspect mentioned above, or a device including the second device mentioned above, or a device included in the second device mentioned above, such as a chip.

[0076] In a seventh aspect, a data processing system is provided, comprising the first device as shown in the fifth aspect and the second device as shown in the sixth aspect.

[0077] In an eighth aspect, a chip is provided, comprising: a processor and an interface circuit; the interface circuit is used to receive computer execution instructions and transmit them to the processor; the processor is used to run the computer execution instructions to execute the method of the first aspect, the second aspect or any implementation method thereof.

[0078] Wherein, when the chip is deployed on the first device, the processor is used to run the computer execution instruction to execute the method of the first aspect or any implementation thereof. When the chip is deployed on the second device, the processor is used to run the computer execution instruction to execute the method of the second aspect or any implementation thereof.

[0079] In the ninth aspect, a computer-readable storage medium is provided, comprising computer execution instructions, which, when the computer execution instructions are executed on a first device, cause the first device to execute the method of the above-mentioned first aspect or any implementation thereof; and, when the computer execution instructions are executed on a second device, cause the second device to execute the method of the above-mentioned second aspect or any implementation thereof.

[0080] In the tenth aspect, a computer program product is provided, comprising computer execution instructions, which, when the computer execution instructions are executed on a first device, cause the first device to execute the method of the above-mentioned first aspect or any implementation thereof; and, when the computer execution instructions are executed on a second device, cause the second device to execute the method of the above-mentioned second aspect or any implementation thereof.

[0081] Among them, the technical effects brought about by any implementation method in the third aspect to the tenth aspect can refer to the technical effects brought about by the corresponding implementation method in the first aspect or the second aspect, and will not be repeated here.

[0082] Various possible implementations of any of the above aspects can be combined under the premise that there is no contradiction between the solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 An example schematic diagram of a large model inference calculation provided for related technologies; Figure 2 A schematic diagram of a large-scale PD separation architecture provided for related technologies; Figure 3 A schematic diagram of the architecture of a data processing system provided in an embodiment of the present application; Figure 4 A schematic diagram of the hardware structure of an ONU device provided in an embodiment of the present application; Figure 5A schematic diagram of the hardware structure of an OLT device provided in an embodiment of the present application; Figure 6 A flowchart of a model-based data processing method provided in an embodiment of the present application; Figure 7 A schematic diagram of a process for obtaining target model parameters provided in an embodiment of the present application; Figure 8 A schematic diagram of an FTTR-B scenario provided in an embodiment of the present application; Fig. 9 A schematic diagram of an FTTR-H scenario provided in an embodiment of the present application; Fig.10 A schematic diagram of an enterprise campus OTN dedicated line scenario provided in an embodiment of the present application; Fig.11 A schematic diagram of the structure of a model-based data processing device provided in an embodiment of the present application; Fig.12 A schematic diagram of the structure of another model-based data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] In the description of this application, unless otherwise specified, "plurality" means two or more than two. At least one of the following or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and (or) c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0085] In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0086] In the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0087] It is understood that the "embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, throughout the specification, each embodiment does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It is understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0088] It can be understood that some optional features in the embodiments of the present application may not rely on other features, such as the solution on which they are currently based, in certain scenarios, and may be implemented independently to solve corresponding technical problems and achieve corresponding effects. They may also be combined with other features according to needs in certain scenarios. Accordingly, the devices provided in the embodiments of the present application may also realize these features or functions accordingly, which will not be elaborated here.

[0089] In this application, unless otherwise specified, the same or similar parts between the various embodiments can refer to each other. In the various embodiments of this application, if there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. The following implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0090] The following is an exemplary introduction to the application scenarios of the embodiments of the present application.

[0091] With the rapid development of artificial intelligence technology, large models have performed well in natural language processing, computer vision processing and other aspects.

[0092] For example, taking the star technology of Large Language Model LLM as an example, with the widespread application of transformer architecture to the emergence of intelligent chat tools (chat generative pre-trained transformer, Chat GPT), LLM is leading an unprecedented intelligent revolution with its powerful data processing capabilities and profound insights, and has promoted the application of artificial intelligence in all walks of life. Among them, LLM has strong language understanding and generation capabilities, and performs well in a variety of natural language processing tasks, such as text generation, machine translation, sentiment analysis, etc. Chat GPT is an advanced artificial intelligence tool, a tool for processing natural language based on LLM.

[0093] With the in-depth application of big models, for big model service providers, how to deploy big models and perform data processing based on big models to provide big model services to users is a key issue. The following introduces three implementation solutions in related technologies.

[0094] Related technology 1: A solution in which a terminal performs data processing based on a large model.

[0095] The large model can be deployed on the terminal so that the terminal performs data processing based on the large model. For example, the user can input the data to be processed to the terminal and trigger the data processing request by operating on the terminal. The terminal receives the data processing request triggered by the user, and processes the data to be processed locally on the terminal based on the large model deployed on the terminal to obtain the processing result.

[0096] However, in the solution where the terminal performs data processing based on a large model, although the terminal has advantages such as low latency, high security, and offline availability, the large model requires the support of high-performance computing power / memory resources, and the terminal is limited by computing power, memory resources, and battery life. The computing power / memory resources provided by the terminal are limited, and performance bottlenecks may be encountered when running large models, making it difficult to support data processing based on large models.

[0097] Related technology 2: Cloud-based data processing solution based on large models.

[0098] Among them, the large model can be deployed on the cloud so that the cloud performs data processing based on the large model. Exemplarily, the user can input the data to be processed into the terminal and trigger the data processing request by operating on the terminal, so that the terminal sends the data processing request to the cloud. The cloud receives the data processing request sent by the terminal, processes the data to be processed on the cloud based on the large model deployed on the cloud, obtains the processing result, and sends the processing result to the terminal. It is understandable that all data is forwarded to the cloud for processing.

[0099] Since the data processing of the large model is an autoregressive process, the data processing process based on the large model can be divided into Prefill calculation (P calculation for short) and Decode calculation (D calculation for short). The autoregressive process refers to the large model generating output statements in an autoregressive manner, that is, outputting tokens one by one. In each token generation step, the large model takes the original input token and the token sequence (all tokens) output in the past as input to output the next token.

[0100] Among them, Prefill calculation is the first stage of large model reasoning. The main task is to calculate and output the first token based on the original input token (that is, the data to be processed input to the large model), and store the original input token and the intermediate calculation results to obtain the key-value pair cache (KV cache) data. Decode calculation is the second stage of large model reasoning. The main task is to calculate based on the previous token and KV cache data to output tokens one by one, and store the newly output tokens as KV cache data. It can be understood that Prefill calculation is computationally intensive and Decode calculation is memory intensive.

[0101] For ease of understanding, the following is based on Figure 1 The Prefill calculation and Decode calculation of a large model are exemplified. For example, Figure 1 An example schematic diagram of a large model inference calculation provided for related technologies. Figure 1 , assuming that the data to be processed is "Which side does the sun rise from" input into the large model, in the process of generating the first token "East", the KV cache data of the input "Which side does the sun rise from" is generated. Then, the first token "East" is re-input into the large model, and the second token "edge" is output in combination with the KV cache data, and the KV cache data of "East" is generated. Then, the second token "edge" is re-input into the large model, and the terminator (end of sequence, EOS) is output in combination with the KV cache data, indicating the end of the sequence, and the KV cache data of "edge" is generated. Among them, the process of generating "East" is the Prefill calculation, and the process of generating "edge" and "EOS" is the Decode calculation.

[0102] In the cloud-based data processing solution based on a large model, based on the differences in computing power and memory requirements for Prefill and Decode calculations, Prefill and Decode calculations can be deployed on two different computing nodes to implement PD separation (prefill and decode seperation) deployment calculations, thereby making full use of device resources. For example, considering that Prefill calculations are computationally intensive, Prefill calculations can be deployed on devices with high computing power, while Decode calculations are memory intensive, so Decode calculations can be deployed on devices with large memory.

[0103] For example, Figure 2 A schematic diagram of a large-scale PD separation architecture provided for related technologies. Figure 2, the controller sends the data to be processed to the Prefill node (such as Figure 2 The P node shown in the figure) performs Prefill calculation, and the Prefill node transmits the generated KV cache data to the Decode node (as shown in the figure). Figure 2 The D node shown in the figure is used to perform subsequent Decode calculations on the Decode node, and the calculations are iterative until a terminator is obtained or the length is exceeded and the data is truncated. Among them, both the Prefill node and the Decode node can be deployed with an XPU, such as a neural processing unit (NPU) or a graphics processing unit (GPU), to perform data processing based on a large model.

[0104] In order to effectively support large models with tens of billions or even hundreds of billions of model parameters, cloud server clusters can be deployed in the cloud. Correspondingly, cloud server clusters can be divided into Prefill clusters (corresponding to the above-mentioned Prefill nodes) and Decode clusters (corresponding to the above-mentioned Decode nodes). In other words, the Prefill calculation of large models is performed based on the Prefill cluster, and the Decode calculation of large models is performed based on the Decode cluster.

[0105] In the process of executing Prefill calculations of large models based on the Prefill cluster, the large model can be divided into multiple parts and distributed on dozens to hundreds of computing units, and the calculation is accelerated through pipeline parallelism and tensor parallelism. In the process of executing Decode calculations of large models based on the Decode cluster, the large model can also be divided into multiple parts and distributed on dozens to hundreds of computing units, and the calculation is accelerated through pipeline parallelism and tensor parallelism.

[0106] However, in the cloud-based data processing solution based on large models, since the data needs to be transmitted in the network that provides cloud services, it is difficult to ensure the security of the transmitted data, such as privacy leakage, etc. In addition, large model service providers also need to purchase public network bandwidth and build a cloud service network, which has high construction and operation and maintenance costs.

[0107] Related technology three: A solution for end-cloud collaboration to perform data processing based on large models.

[0108] Among them, end-cloud collaboration refers to allocating computing resources and model tasks between the cloud and the terminal, combining the advantages of cloud computing and terminal computing, and giving full play to the powerful computing power of the cloud and the high responsiveness of the terminal, thereby achieving more efficient data processing.

[0109] For example, end-cloud collaboration can be achieved using a variety of technologies such as speculative inference and distributed hybrid deployment. Among them, speculative inference refers to optimizing resource allocation and task scheduling by predicting the execution path of model tasks in advance in the end-cloud collaboration architecture, thereby improving the overall efficiency of the system. Distributed hybrid deployment refers to distributing model tasks on multiple nodes (such as terminals and the cloud) to complete complex model tasks through collaborative work.

[0110] However, in the solution where the end-cloud collaborates to perform data processing based on a large model, the software and hardware chain between the terminal and the cloud is too long, and the requirements for network stability are high. It is easy to have a single point error, that is, the failure of a component in the system causes the entire system to fail to operate normally.

[0111] In view of this, the embodiment of the present application provides a model-based data processing method, and provides a combination of a first device and a second device to perform a model-based data processing scheme, that is, to provide a model reasoning scheme for collaboration between a terminal device and an edge device (such as called edge collaboration). Among them, by deploying the first processing based on the data processing model on the edge device, and deploying the second processing based on the data processing model on the terminal device, the separate deployment of the model reasoning service is realized. On the one hand, considering that the edge device is usually deployed with powerful hardware and can provide strong computing / memory capabilities, the data processing model is deployed on the edge device, which can provide sufficient computing power / memory resources for the first processing of the data processing model, and the terminal device only needs to perform the second processing of the first data processing result, which also reduces the computing power / memory pressure of the terminal device. On the other hand, for the second processing of the terminal device, since it is performed locally on the terminal device, the data does not need to be transmitted in the network, and the security is higher. For the first processing of the edge device, since the edge device is a network device located at the edge of the network, it is closer to the terminal than the cloud, so it can reduce the transmission of data in the network, thereby effectively improving the security of the data.

[0112] The model-based data processing method provided in the embodiment of the present application can be applied to fiber to the room (FTTR) scenarios, such as FTTR scenarios based on point-to-multipoint networking (P2MP) architecture.

[0113] Among them, FTTR is a networking method for indoor fiber-optic access networks. Communication services are transmitted to users' rooms through optical fibers, enabling direct connection between users and broadband networks, thereby providing high-speed and stable network connections. In some embodiments, through FTTR technology, operators can provide users with highly intelligent and customized services, such as smart campus management, smart home services, and smart data storage. In addition, given that users attach great importance to privacy protection, FTTR can support the processing and storage of user data within the enterprise / home, avoiding data transmission on the public network or storage on the cloud server, thereby ensuring the security and privacy of user information.

[0114] The P2MP architecture is a network communication architecture that is mainly used to implement a source point (sender) to send data to multiple target points (receivers) at the same time, such as broadcasting. For example, in the P2MP architecture, the device on the trunk can be an OLT device, and the device on the branch can be an ONU device. Among them, the signal can be transmitted from the OLT device to each ONU device through a 1:N splitter (N is a positive integer greater than 1) or a series of splitters.

[0115] In order to facilitate understanding of the embodiments of the present application, the following points are explained before introducing the embodiments of the present application.

[0116] 1. In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, directly indicating the information to be indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated may also be indirectly indicated by indicating other information, wherein the other information and the information to be indicated have an association relationship. It is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance.

[0117] 2. In the embodiments of the present application, descriptions such as "in the case of...", "if" and "if" all mean that under certain objective circumstances, the device (for example, the first device or the second device) will make corresponding processing, which does not limit the time, nor does it require the device (for example, the first device or the second device) to have a judgment action when implementing it, nor does it mean that there are other limitations.

[0118] In addition, the system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0119] In addition, the actions, terms, etc. involved in the various embodiments of the present application can refer to each other without limitation. The message name or parameter name in the message exchanged between the various devices in the embodiments of the present application is only an example, and other names can also be used in the specific implementation without limitation.

[0120] The following is an exemplary introduction to the system architecture of the embodiments of the present application.

[0121] In some embodiments, the model-based data processing method provided in the embodiments of the present application can be applied to Figure 3 In the data processing system shown. By way of example, Figure 3 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of the present application. Figure 3 , the data processing system includes: one or more first devices 301 and second devices 302.

[0122] The first device 301 is a network device connected to the terminal. It is understandable that the first device 301 is also a terminal-side device, that is, a device located at the terminal side.

[0123] In some embodiments, the first device 301 may be an ONU device, an MFU device, a CPE device in an OTN dedicated line-based network, etc.

[0124] In some embodiments, the terminal may be at least one of a smart phone, a smart watch, a printer, a desktop computer, a laptop computer, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop computer.

[0125] In an embodiment of the present application, the first device 301 is used to: receive a data processing request sent by a terminal, the data processing request is used to request processing of target data; send target data to the second device 302; receive a first data processing result sent by the second device 302; based on a data processing model, perform a second processing on the first data processing result to obtain a second data processing result; and send the second data processing result to the terminal.

[0126] The second device 302 is a network device connected to the cloud. It is understandable that the second device 302 is also a side device, that is, a device located at the edge of the network. In the embodiment of the present application, the second device 302 can be deployed with a large model, such as a data processing model.

[0127] In some embodiments, the second device 302 may be an OLT device or an edge cloud device in an OTN dedicated line-based network, etc.

[0128] In some embodiments, the cloud may be at least one of cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data or artificial intelligence platforms, but the embodiments of the present application are not limited to this.

[0129] In an embodiment of the present application, the second device 302 is used to: receive target data sent by the first device 301; perform a first processing on the target data based on a data processing model to obtain a first data processing result; and send the first data processing result to the first device 301.

[0130] In an example of the present application, taking the first device as an ONU device as an example, the hardware structure diagram of the first device is as follows: Figure 4 shown. Figure 4 A schematic diagram of the hardware structure of an ONU device provided in an embodiment of the present application.

[0131] See also Figure 4 , Figure 4 The ONU device 400 shown may include: a processor 401 , a memory 402 , a communication module 403 , and a bus 404 . The processor 401 , the memory 402 , and the communication module 403 may be connected via the bus 404 .

[0132] The processor 401 may serve as a control center of the ONU device 400. In some embodiments, the processor 401 may include an NPU, a GPU or other general-purpose processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0133] In the embodiment of the present application, the first device is Figure 4 Taking the ONU device 400 as an example, the content executed by the first device in the model-based data processing method can be executed by the processor 401 in the ONU device 400.

[0134] The memory 402 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0135] In a possible implementation, the memory 402 may exist independently of the processor 401. The memory 402 may be connected to the processor 401 via a bus 404 and is used to store data, instructions or program codes.

[0136] In another possible implementation, the memory 402 may also be integrated with the processor 401 .

[0137] In the embodiment of the present application, the first device is Figure 4 Taking the ONU device 400 shown as an example, when the processor 401 calls and executes the instructions or program codes stored in the memory 402, the content executed by the first device in the model-based data processing method provided in the embodiment of the present application can be implemented.

[0138] The communication module 403 is used to connect the ONU device 400 with other devices (such as OLT devices) through a communication network, and the communication network may be P2MP or the like.

[0139] The communication module 403 may include a receiving unit for receiving data and a sending unit for sending data.

[0140] In some embodiments, the communication module 403 includes an ONU media access control (MAC) 4031 , an ONU optical physical layer (PHY) 4032 , a laser 4033 , and a photodetector 4034 .

[0141] In the sending direction, ONU MAC4031 can send the data to be processed, such as the target data mentioned in the embodiment of the present application, to ONU optical PHY4032. ONU optical PHY4032 is also called the driver of laser 4033, which is used to drive laser 4033 to generate optical signals according to the instructions of ONU MAC4031. Laser 4033 modulates the data to be processed into the optical signal under the control of ONU optical PHY4032, and sends the uplink optical signal carrying the data to be processed through the optical fiber, such as to the OLT device.

[0142] In the receiving direction, the photodetector 4034 receives the downstream optical signal from other devices such as the OLT device and converts the downstream optical signal into an electrical signal. The ONU optical PHY 4032 transparently transmits the electrical signal, and the ONU MAC 4031 analyzes the electrical signal to obtain the data content, such as the model parameters mentioned in the embodiment of the present application.

[0143] In some embodiments, the communication module 403 may further include a wavelength division multiplexer 4035 for sending the uplink optical signal generated by the laser 4033 to the optical fiber, and sending the downlink optical signal received from the optical fiber to the photodetector 4034 .

[0144] The bus 404 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0145] It should be pointed out that Figure 4 The structure shown in the figure does not constitute a limitation on the ONU device. Figure 4 In addition to the components shown, the ONU device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0146] In an example of the present application, taking the second device as an OLT device as an example, the hardware structure diagram of the second device is as follows: Figure 5 shown. Figure 5 A schematic diagram of the hardware structure of an OLT device provided in an embodiment of the present application.

[0147] See also Figure 5 , Figure 5 The OLT device 500 shown may include: a processor 501 , a memory 502 , a communication module 503 , and a bus 504 . The processor 501 , the memory 502 , and the communication module 503 may be connected via the bus 504 .

[0148] The processor 501 may serve as a control center of the OLT device 500. In some embodiments, the processor 501 may include an NPU, a GPU or other general-purpose processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0149] In the embodiment of the present application, the second device is Figure 5 Taking the OLT device 500 as an example, the content executed by the second device in the model-based data processing method can be executed by the processor 501 in the OLT device 500 .

[0150] The memory 502 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0151] In a possible implementation, the memory 502 may exist independently of the processor 501. The memory 502 may be connected to the processor 501 via a bus 504 and is used to store data, instructions or program codes.

[0152] In another possible implementation, the memory 502 may also be integrated with the processor 501 .

[0153] In the embodiment of the present application, the second device is Figure 5 Taking the OLT device 500 shown as an example, when the processor 501 calls and executes the instructions or program codes stored in the memory 502, the content executed by the second device in the model-based data processing method provided in the embodiment of the present application can be implemented.

[0154] The communication module 503 is used to connect the OLT device 500 with other devices (such as one or more ONU devices) through a communication network, and the communication network may be a P2MP network or the like.

[0155] The communication module 503 may include a receiving unit for receiving data and a sending unit for sending data.

[0156] In some embodiments, the communication module 503 may include: an OLT MAC 5031 , a signal processing module 5032 , an OLT optical PHY 5033 , a photodetector 5034 , and a laser 5035 .

[0157] In the receiving direction, the photodetector 5034 receives the uplink optical signal from other devices such as ONU devices, and converts the uplink optical signal into an electrical signal. The electrical signal can be an analog electrical signal or a digital electrical signal. The signal processing module 5032 can be implemented by an analog device (such as an amplifier) ​​or a digital device (such as a digital signal processor). Therefore, the signal processing module 5032 can perform analog correlation processing or digital electrical signal processing. The OLT MAC5031 analyzes the electrical signal passing through the signal processing module 5032 to obtain the data content.

[0158] In the sending direction, OLT MAC5031 generates service data, such as the model parameters mentioned in the embodiment of the present application, and the signal processing module 5032 performs analog or digital related processing on the service data. The laser 5035 modulates the service data into an optical signal under the control of the OLT optical PHY5033, and sends the downlink optical signal carrying the service data to other devices such as ONU devices through optical fibers.

[0159] The communication module 503 may further include a wavelength division multiplexer 5036 for sending the downlink optical signal generated by the laser 5035 to the optical fiber, and sending the uplink optical signal received from the optical fiber to the photodetector 5034 .

[0160] Bus 504 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0161] It should be pointed out that Figure 5 The structure shown in the figure does not constitute a limitation on the OLT device. Figure 5 In addition to the components shown, the OLT device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0162] For ease of understanding, the model-based data processing method provided in the embodiment of the present application is exemplarily described below in combination with the above system architecture and accompanying drawings.

[0163] It can be understood that in the embodiments of the present application, the first device or the second device can perform part or all of the steps in the embodiments of the present application. These steps or operations are only examples. The embodiments of the present application can also perform other operations or variations of various operations.

[0164] Figure 6 A flow chart of a model-based data processing method provided in an embodiment of the present application. In some possible implementations, the model-based data processing method can be completed by the terminal, the first device and the second device in the above system architecture, see Figure 6 Taking the interaction process between the terminal, the first device and the second device as an example, the method includes the following S601 to S608.

[0165] S601. A first device receives a data processing request sent by a terminal.

[0166] In some embodiments, the first device may provide a network service interface, and connect to the terminal through the network service interface, so that the terminal communicates with the first device through the network service interface.

[0167] Exemplarily, a model application (APP) may be run on the terminal, and the model application provides a model service, that is, a model-based data processing service, to the user. The user may trigger a data processing request by operating in the model application, so that the terminal sends the data processing request to the first device.

[0168] In another exemplary embodiment, a web application may be run on the terminal, and the web application includes a web page that provides model services to the user. The user may trigger a data processing request by operating in the web application, so that the terminal sends the data processing request to the first device.

[0169] As another example, an audio and video sensor acquisition device is installed on the terminal, so that the user can trigger a data processing request through voice control or action control, so that the terminal sends a data processing request to the first device.

[0170] It is worth noting that the above embodiment exemplifies the process of the terminal sending a data processing request to the first device. In other embodiments, the user can also trigger the data processing request in other ways, so that the terminal sends the data processing request to the first device. This embodiment of the application does not limit this.

[0171] S602: The first device sends target data to the second device.

[0172] The target data refers to the data to be processed. For example, the target data may be text, pictures, videos, documents, etc.

[0173] In some embodiments, the data processing request carries the target data. Accordingly, the above S602 can be replaced by: the first device obtains the target data from the data processing request, and sends the target data to the second device.

[0174] In some other embodiments, the data processing request carries a data identifier of the target data. Accordingly, the above S602 can be replaced by: the first device obtains the data identifier of the target data from the data processing request. And, obtains the data corresponding to the data identifier from the data set associated with the first device as the target data. Then, the first device sends the target data to the second device.

[0175] The data set associated with the first device may be peripheral data of the first device. The peripheral data of the first device refers to data stored in a peripheral connected to the first device, such as a network attached storage (NAS).

[0176] The above embodiment describes the process of the first device sending the target data to the second device. In some embodiments, before the first device sends the target data to the second device, the first device also encrypts the target data. In this way, the security of the target data during network transmission is guaranteed by data encryption, thereby avoiding problems such as privacy leakage.

[0177] It is worth noting that in the scenario where the user requests the model service from the first device through the terminal, the original data processing request triggered by the user may carry the target data or may not carry the target data, such as only carrying the data identifier of the target data.

[0178] It is understandable that, when the original data processing request triggered by the user carries target data, the terminal can directly send a data processing request carrying the target data to the first device, and then the first device can directly send the target data to the second device.

[0179] It is understandable that, in the case where the original data processing request triggered by the user carries only the data identifier of the target data, the terminal can directly forward the data processing request to the first device, and then the first device needs to obtain the target data from the associated data set before sending the target data to the second device. Alternatively, the terminal can also obtain the data corresponding to the data identifier from the data set associated with the terminal as the target data, and then send a data processing request carrying the target data to the first device, and then the first device can directly send the target data to the second device.

[0180] The data set associated with the terminal may be the peripheral data of the terminal. The peripheral data of the terminal refers to the data stored in the peripheral connected to the terminal, such as the peripheral may be a terminal disk or a USB flash drive.

[0181] That is to say, when the original data processing request triggered by the user only carries the data identifier of the target data, the target data can be obtained from the peripheral device connected to the first device, or from the peripheral device connected to the terminal. This embodiment of the present application does not limit this.

[0182] In the embodiment of the present application, the second device stores a data processing model, wherein the data processing model may be a large model, such as an LLM.

[0183] S603: The second device receives target data sent by the first device.

[0184] In some embodiments, when the target data has been encrypted, the second device further decrypts the target data after receiving the target data sent by the first device.

[0185] S604: The second device performs a first process on the target data based on the data processing model to obtain a first data processing result.

[0186] The first data processing result is a result obtained by the data processing model performing a first processing on the target data.

[0187] In the embodiment of the present application, the first processing may be a Prefill calculation. Accordingly, the first data processing result is the result obtained by performing the Prefill calculation on the target data.

[0188] S605: The second device sends the first data processing result to the first device.

[0189] S606: The first device receives the first data processing result sent by the second device.

[0190] S607: The first device performs a second process on the first data processing result based on the data processing model to obtain a second data processing result.

[0191] The second data processing result is the processing result of the target data. In the embodiment of the present application, the second processing may be a Decode calculation. Accordingly, the second data processing result is the result obtained by performing a Decode calculation on the first data processing result.

[0192] In some embodiments, the first device performs iterative decoding calculations on the first data processing result based on the data processing model until a terminator is obtained or the length exceeds and the result is truncated, thereby obtaining a second data processing result.

[0193] S608. The first device sends the second data processing result to the terminal.

[0194] In some embodiments, the first device may send the second data processing result to the terminal, so that the terminal displays the second data processing result for the user, or plays the second data processing result for the user.

[0195] The technical solution provided by the embodiment of the present application provides a method for combining a first device and a second device to perform a model-based data processing solution, that is, a model reasoning solution for collaboration between a terminal device and an edge device (such as edge collaboration) is provided. Among them, by deploying the first processing based on the data processing model on the edge device, and deploying the second processing based on the data processing model on the terminal device, the separate deployment of the model reasoning service is realized. On the one hand, considering that the edge device is usually deployed with powerful hardware and can provide strong computing / memory capabilities, the data processing model is deployed on the edge device, which can provide sufficient computing power / memory resources for the first processing of the data processing model, and the terminal device only needs to perform the second processing of the first data processing result, which also reduces the computing power / memory pressure of the terminal device. On the other hand, for the second processing of the terminal device, since it is performed locally on the terminal device, the data does not need to be transmitted in the network, and the security is higher. For the first processing of the edge device, since the edge device is a network device located at the edge of the network, it is closer to the terminal than the cloud, so it can reduce the transmission of data in the network, thereby effectively improving the security of the data.

[0196] In some embodiments of the present application, the second processing includes at least one third processing. With respect to the second processing in S607 above, each third processing includes the following process.

[0197] ① Obtain target model parameters.

[0198] The target model parameters are model parameters of the data processing model used to perform the third processing.

[0199] ② Based on the target model parameters, the third processing is performed on the data to be processed.

[0200] The data to be processed in the first third processing is the first data processing result. The data to be processed in other third processings is the data processing result obtained in the last third processing. The data processing result obtained in the last third processing is the second data processing result.

[0201] The following is based on Figure 7 The process shown in FIG. 1 is used to explain in detail the process of obtaining the target model parameters mentioned above. Figure 7 A schematic diagram of a process for obtaining target model parameters provided in an embodiment of the present application. Figure 7 , taking the interaction process between the first device and the second device as an example, the method includes the following S701 to S706.

[0202] S701: Determine whether the first device stores the target model parameters. If the first device stores the target model parameters, execute S702. If the first device does not store the target model parameters, execute S703 to S706.

[0203] S702: The first device obtains stored target model parameters.

[0204] In some embodiments, the first device stores some model parameters of the data processing model, and the first device can obtain the target model parameters from the local stored part of the model parameters.

[0205] It is understandable that before implementing this solution, the second device may send all model parameters of the data processing model, or some model parameters of the data processing model to the first device. The first device receives all model parameters of the data processing model, or some model parameters of the data processing model sent by the second device. Furthermore, the first device stores some model parameters of the data processing model.

[0206] In an exemplary scenario, the second device may actively send model parameters of the data processing model to the first device, such as all model parameters or part of the model parameters. For example, the second device may send model parameters of the data processing model to the first device when the model is updated. The corresponding process may be: the second device receives the model update information sent by the cloud. The model update information includes the model parameters of the updated data processing model. The second device sends the updated model parameters of the data processing model to the first device.

[0207] It is understandable that the model parameters of the updated data processing model may be all the model parameters of the updated data processing model. Furthermore, the first device receives all the model parameters of the updated data processing model sent by the second device, and selectively stores some of the model parameters from all the model parameters of the updated data processing model. Alternatively, the model parameters of the updated data processing model may be some of the model parameters of the updated data processing model, such as sending only the updated model parameters without sending the unupdated model parameters. Furthermore, the first device receives some of the model parameters of the updated data processing model sent by the second device, and selectively stores some or all of the model parameters from some of the model parameters of the updated data processing model.

[0208] In another exemplary scenario, the first device may actively request the model parameters of the data processing model from the second device, such as all model parameters or part of the model parameters. The corresponding process may be: the first device sends a parameter request to the second device, and the parameter request is used to request to obtain the model parameters of the data processing model. The second device receives the parameter request sent by the first device. The second device sends the model parameters of the data processing model to the first device based on the parameter request. Furthermore, the first device receives the model parameters of the data processing model sent by the second device.

[0209] It is understandable that the parameter request may request to obtain all model parameters of the data processing model. Furthermore, the first device receives all model parameters of the data processing model sent by the second device, and selectively stores some model parameters from all model parameters of the data processing model. Alternatively, the parameter request may request to obtain some model parameters of the data processing model, such as a parameter request sent in a past data processing event to request model parameters for performing a certain calculation. Furthermore, the first device receives some model parameters of the data processing model sent by the second device, and selectively stores some or all of the model parameters from some model parameters of the data processing model.

[0210] It is worth noting that the above embodiment exemplifies the process of the first device storing some model parameters before implementing this solution. In other embodiments, the first device can also use other methods to obtain and store some model parameters of the data processing model, which is not limited in this embodiment of the application.

[0211] In addition, in the case where the first device stores some model parameters of the data processing model, the first device also sends a query request to the second device after receiving the data processing request sent by the terminal based on S501. The query request is used to request to query the model parameters of the target version of the data processing model. The target version of the data processing model refers to the latest version of the data processing model. The second device receives the query request sent by the first device. Based on the query request, the second device sends the model parameters of the target version of the data processing model to the first device. The first device receives the model parameters of the target version of the data processing model sent by the second device. The first device updates the stored model parameters based on the model parameters of the target version of the data processing model. In this way, by querying the latest version of the model, the first device can update some of the model parameters stored locally.

[0212] S703: The first device sends a first request to the second device.

[0213] The first request is used to request to obtain the target model parameters.

[0214] In some embodiments, the first request may carry relevant information of the third processing to obtain model parameters for executing the third processing. For example, the relevant information of the third processing may be calculation progress, or the relevant information of the third processing may be network layer information. Of course, in other embodiments, the relevant information of the third processing may also be other information that can indicate the third processing, which is not limited in the embodiments of the present application.

[0215] In some embodiments, the first request also carries the performance parameters of the first device to obtain the target model parameters of the target number of layers of the data processing model that match the performance parameters. The performance parameters are used to indicate the computing power and / or memory capacity of the first device. The target number of layers is a pre-set number of network layers, such as 1 or 2 layers. In this way, by carrying the performance parameters of the first device in the first request, the second device can adaptively send the model parameters of the corresponding number of layers, such as 1 or 2 layers, according to the computing power and / or memory capacity of the first device, thereby improving the flexibility of sending the model parameters. It is understandable that in the case where the first request does not carry the performance parameters of the first device, the second device can send the model parameters of 1 or 2 layers of the network by default.

[0216] S704: The second device receives the first request sent by the first device.

[0217] S705: The second device sends the target model parameters to the first device based on the first request.

[0218] In some embodiments, the second device sends broadcast information, wherein the broadcast information includes target model parameters.

[0219] In the embodiment of the present application, the second device can communicate with multiple first devices in a P2MP manner. After receiving the first request sent by the multiple first devices, the target model parameters can be sent by broadcasting. In this way, the target model parameters are transmitted by broadcasting, so that the second device only needs to send the target model parameters once when facing multiple first devices, which greatly reduces the consumption of network bandwidth.

[0220] S706: The first device receives the target model parameters sent by the second device.

[0221] In some embodiments, the first device receives broadcast information from the second device and obtains the target model parameters from the broadcast information.

[0222] Above Figure 7The illustrated embodiment takes the example of the first device acquiring the model parameters of the data processing model in batches based on at least one third processing included in the second processing to illustrate the solution. In this way, a method of acquiring the target model parameters in batches is provided, so that the amount of data to be processed by the first device during each third processing is small, reducing the computing power / memory pressure of the first device, thereby improving the processing efficiency of the first device. In addition, by storing some model parameters of the data processing model in the first device, the amount of data requested by the first device for the model parameters from the second device is reduced, thereby saving the consumption of network bandwidth.

[0223] In other embodiments, the first device may also obtain all model parameters of the data processing model for performing the second processing at one time. The corresponding process may include: the first device sends a second request to the second device. The second request is used to request to obtain the model parameters of the data processing model for performing the second processing. The second device receives the second request sent by the first device. The second device sends the model parameters to the first device. The first device receives the model parameters sent by the second device. Furthermore, the first device performs a second processing on the first data processing result based on the model parameters to obtain the second data processing result. In this way, a solution for sending all model parameters at one time is provided. For example, when the computing power / memory of the first device is sufficient, all model parameters can be requested to be sent to the second device.

[0224] Based on the three scenarios in which the embodiments of the present application may be applied, the model-based data processing method is exemplarily described below in combination with the following three possible embodiments.

[0225] Embodiment 1: Fiber to the room for business (FTTR-B) scenario.

[0226] Among them, FTTR-B is an optical network connection method designed specifically for enterprise parks, which extends the network connection to every room or office area of ​​the enterprise through optical fiber.

[0227] For example, Figure 8 A schematic diagram of a FTTR-B scenario provided in an embodiment of the present application. Figure 8 In the FTTR-B scenario, the first device may be an ONU device or an MFU device deployed on the end side, such as Figure 8 The second device may be an OLT device deployed at the edge, such as Figure 8 The enterprise OLT is shown.

[0228] The enterprise OLT may be deployed in an enterprise access terminal room, which is used to access the operator's cloud computing center. In some embodiments, the enterprise OLT provides a high-speed fiber access service for accessing the operator's cloud computing center to utilize the cloud computing services provided by the operator's cloud computing center. Exemplarily, the cloud connected to the enterprise OLT may be a cloud server of the operator's cloud computing center, hereinafter referred to as the operator cloud.

[0229] The enterprise ONU / MFU is used to provide Internet access and various personalized value-added services to enterprise users. In some embodiments, the enterprise ONU / MFU may provide a human-machine interface for connecting a terminal to realize interaction between the enterprise user and the enterprise ONU / MFU.

[0230] In the embodiment of the present application, the enterprise OLT and the enterprise ONU / MFU communicate based on P2MP.

[0231] The enterprise OLT is connected to multiple enterprise ONUs / MFUs via optical fibers to provide enterprise users with high-speed and stable optical fiber access services. The following embodiments of the present application take the interaction process between the enterprise OLT and an enterprise ONU / MFU as an example to illustrate the solution.

[0232] In the FTTR-B scenario, based on the model-based data processing method provided in the embodiment of the present application, the enterprise OLT performs the process of the first processing (such as Prefill calculation), and the enterprise ONU / MFU performs the process of the second processing (such as Decode calculation), which can provide a low-cost and high-energy-efficiency model service for FTTR based on P2MP for the enterprise park. In some embodiments, the enterprise OLT may provide one or more first processing service interfaces for connecting one or more enterprise ONU / MFU, so that the enterprise ONU / MFU triggers the enterprise OLT to perform the first processing process through the first processing service interface.

[0233] In the embodiment of the present application, the enterprise OLT stores a complete data processing model. In some embodiments, the enterprise OLT can manage the data of each layer of the data processing model through a certain data structure, that is, the model parameters of each network layer, such as weight parameters, etc. In some embodiments, the operator cloud can push the latest version of the data processing model to the enterprise OLT for the enterprise OLT to synchronize the model. In some embodiments, the enterprise OLT can be deployed with an XPU, such as an NPU or a GPU, to perform the first processing based on the data processing model.

[0234] In an embodiment of the present application, the enterprise ONU / MFU may store some model parameters of the data processing model. For example, when the operator cloud pushes the latest version of the data processing model to the enterprise OLT, the enterprise OLT can actively send it to the enterprise ONU / MFU once for the enterprise ONU / MFU to selectively store. For another example, the enterprise ONU / MFU may also actively request the latest version of the data processing model from the enterprise OLT, and then selectively store it. For another example, the enterprise ONU / MFU may also cache the model parameters of the data processing model during daily reasoning. In this way, the amount of data transmitted in communication can be reduced. In some embodiments, the enterprise ONU / MFU may be deployed with an XPU, such as an NPU or a GPU, to perform a second process based on the data processing model.

[0235] The following is based on Figure 8 The process shown in the figure illustrates the data processing process of the enterprise OLT and an enterprise ONU / MFU edge collaboration. Figure 8 The data processing process based on the edge collaboration between the enterprise OLT and the enterprise ONU / MFU may include the following ① to ⑩.

[0236] ① Data processing request: The enterprise user sends a data processing request to the enterprise ONU / MFU through the terminal to request model service.

[0237] ② Obtain target data: The enterprise ONU / MFU obtains target data based on the data processing request.

[0238] See also Figure 8 In some embodiments, the enterprise ONU / MFU obtains target data from the peripheral device based on the data processing request.

[0239] It is worth noting that Figure 8 Take the enterprise ONU / MFU obtaining target data from the peripheral device as an example. In other embodiments, the data processing request may carry the target data, and ② may be replaced by: the enterprise ONU / MFU obtains the target data carried by the data processing request.

[0240] ③Query and synchronize model versions: The enterprise ONU / MFU sends a query request to the enterprise OLT to obtain the target version of the model, such as the latest version of the model. The enterprise OLT receives the query request sent by the enterprise ONU / MFU and returns a model list set of the target version to the enterprise ONU / MFU for the enterprise ONU / MFU to synchronize the model.

[0241] The target version model list set is, for example, the latest version model list set. It is understandable that one or more models may be used in a data processing flow, and the model list set includes one or more models to be used.

[0242] It is worth noting that ③ is an optional step.

[0243] Understandably, Figure 8 Take the case where the enterprise ONU / MFU can store some model parameters of the data processing model as an example. In other embodiments, if the enterprise ONU / MFU does not store some model parameters of the data processing model, ③ is not executed.

[0244] Understandably, Figure 8 Take the enterprise ONU / MFU actively querying the model version for model synchronization when performing data processing as an example. In other embodiments, when the operator cloud pushes the latest version of the data processing model to the enterprise OLT, the enterprise OLT can actively send the latest version of the model list set to the enterprise ONU / MFU for the enterprise ONU / MFU to synchronize the model, and ③ is not executed at this time.

[0245] ④ Target data: The enterprise ONU / MFU sends target data to the enterprise OLT.

[0246] ⑤Prefill calculation (such as Figure 8 P calculation shown): The enterprise OLT performs Prefill calculation on the target data based on the data processing model to obtain the Prefill calculation result and the KV cache data of the target data.

[0247] ⑥Calculation results: The enterprise OLT returns the Prefill calculation results and the KV cache data of the target data to the enterprise ONU / MFU. The enterprise ONU / MFU receives the Prefill calculation results and the KV cache data of the target data returned by the enterprise OLT.

[0248] The enterprise ONU / MFU starts the Decode calculation process, which includes the following steps ⑦ to ⑨.

[0249] ⑦ Parameter request: The enterprise ONU / MFU sends a parameter request to the enterprise OLT based on the current calculation process to request to obtain some model parameters of the data processing model for performing the next step of calculation.

[0250] ⑧ Broadcast parameters: The enterprise OLT receives the parameter request sent by the enterprise ONU / MFU, broadcasts based on the P2MP architecture, and sends some model parameters to the enterprise ONU / MFU.

[0251] ⑨Decode calculation (such as Figure 8 D calculation shown): the enterprise ONU / MFU receives some model parameters and starts calculation, and caches the intermediate calculation results locally in the enterprise ONU / MFU.

[0252] Repeat steps ⑦ to ⑨ above until the Decode calculation of the entire model is completed, and the processing result of the target data is obtained.

[0253] ⑩Processing result: The enterprise ONU / MFU returns the processing result of the target data to the enterprise user through the terminal.

[0254] In the above exemplary scenario, an end-to-end collaborative PD separation architecture based on P2MP communication between the enterprise OLT and the enterprise ONU / MFU is provided. The enterprise OLT performs the Prefill calculation and broadcasts the model parameters to the enterprise ONU / MFU to perform the Decode calculation on the enterprise ONU / MFU, which can provide the enterprise park with a low-cost and high-efficiency model service of FTTR based on the P2MP architecture.

[0255] Moreover, since the enterprise OLT and enterprise ONU / MFU are both the enterprise users' own equipment, the application of the above-mentioned edge-to-edge collaborative data processing solution in the enterprise campus can ensure the user's data privacy.

[0256] In addition, by storing some model parameters of the data processing model in the enterprise ONU / MFU, the amount of data requested by the enterprise ONU / MFU for model parameters from the enterprise OLT can be reduced in the above ⑦ to ⑨, thereby further saving network bandwidth consumption.

[0257] Embodiment 2: Fiber to the room for home (FTTR-H) scenario.

[0258] Among them, FTTR-H is an optical network connection method designed specifically for homes, which extends the network connection to every room in the home through optical fiber.

[0259] For example, Fig. 9 A schematic diagram of a FTTR-H scenario provided in an embodiment of the present application. Fig. 9 In the FTTR-H scenario, the first device may be an ONU device or an MFU device deployed on the end side, such as Fig. 9 The second device may be an OLT device deployed under the jurisdiction of the operator, such as Fig. 9 The operator OLT is shown.

[0260] The operator OLT may be deployed in an operator access terminal room, which is used to access the operator cloud computing center. In some embodiments, the operator OLT provides a high-speed fiber access service for accessing the operator cloud computing center to utilize the cloud computing services provided by the operator cloud computing center. Exemplarily, the cloud connected to the operator OLT may be the operator's cloud server, hereinafter referred to as the operator cloud.

[0261] The home ONU / MFU is used to provide home users with Internet access and various personalized value-added services. In some embodiments, the home ONU / MFU may provide a human-machine interface for connecting a terminal to achieve interaction between the home user and the home ONU / MFU.

[0262] In the embodiment of the present application, the operator OLT and the home ONU / MFU communicate based on P2MP.

[0263] The operator OLT is connected to multiple home ONUs / MFUs via optical fibers to provide home users with high-speed and stable optical fiber access services. The embodiment of the present application takes the interaction process between the operator OLT and a home ONU / MFU as an example to illustrate the solution.

[0264] In the FTTR-H scenario, based on the model-based data processing method provided in the embodiment of the present application, the operator OLT performs the first processing (such as Prefill calculation) process, and the home ONU / MFU performs the second processing (such as Decode calculation) process, which can provide low-cost and high-energy-efficiency model services based on P2MP FTTR for the home. In some embodiments, the operator OLT may provide one or more first processing service interfaces for connecting one or more home ONU / MFU, so that the home ONU / MFU triggers the operator OLT to perform the first processing process through the first processing service interface.

[0265] In the embodiment of the present application, the operator OLT stores a complete data processing model. In some embodiments, the operator OLT can manage the data of each layer of the data processing model through a certain data structure, that is, the model parameters of each network layer, such as weight parameters, etc. In some embodiments, the operator cloud can push the latest version of the data processing model to the operator OLT for the operator OLT to synchronize the model. In some embodiments, the operator OLT can be deployed with an XPU, such as an NPU or a GPU, to perform the first processing based on the data processing model.

[0266] In an embodiment of the present application, the home ONU / MFU may store some model parameters of the data processing model. For example, when the operator cloud pushes the latest version of the data processing model to the operator OLT, the operator OLT can actively send it to the home ONU / MFU once for the home ONU / MFU to selectively store. For another example, the home ONU / MFU can also actively request the latest version of the data processing model from the operator OLT, and then selectively store it. For another example, the home ONU / MFU can also cache the model parameters of the data processing model during daily reasoning. In this way, the amount of data transmitted in communication can be reduced. In some embodiments, the home ONU / MFU can be deployed with an XPU, such as an NPU or a GPU, to perform a second process based on the data processing model.

[0267] The following is based on Fig. 9 The process shown in the figure illustrates the data processing process of the operator's OLT and the edge of a home ONU / MFU. Fig. 9 The data processing process based on the edge collaboration between the operator OLT and the home ONU / MFU may include the following ① to ⑩.

[0268] ① Data processing request: The home user sends a data processing request to the home ONU / MFU through the terminal to request the model service.

[0269] ② Obtain target data: The home ONU / MFU obtains target data based on the data processing request.

[0270] Among them, the relevant contents of the above ② are consistent with the above Figure 8 The content of ② is similar and will not be repeated here.

[0271] ③Query and synchronize model versions: The home ONU / MFU sends a query request to the operator OLT to obtain the target version of the model, such as the latest version of the model. The operator OLT receives the query request sent by the home ONU / MFU and returns a model list set of the target version to the home ONU / MFU for model synchronization.

[0272] Among them, the relevant contents of the above ③ are consistent with the above Figure 8 The content of ③ is similar and will not be repeated here.

[0273] ④ Encrypted target data: The home ONU / MFU sends the encrypted target data to the operator's OLT.

[0274] ⑤Prefill calculation (such as Fig. 9P calculation shown): The operator OLT decrypts the target data and performs Prefill calculation on the decrypted target data based on the data processing model to obtain the Prefill calculation result and KV cache data of the target data.

[0275] ⑥Calculation results: The operator OLT returns the Prefill calculation results and the KV cache data of the target data to the home ONU / MFU. The home ONU / MFU receives the Prefill calculation results and the KV cache data of the target data returned by the operator OLT.

[0276] The home ONU / MFU starts the decoding calculation process, and the decoding calculation process includes the following ⑦ to ⑨.

[0277] ⑦ Parameter request: The home ONU / MFU sends a parameter request to the operator OLT based on the current calculation process to request to obtain some model parameters of the data processing model for performing the next step of calculation.

[0278] ⑧ Broadcast parameters: The operator OLT receives the parameter request sent by the home ONU / MFU and broadcasts it based on the P2MP architecture to send some model parameters to the home ONU / MFU.

[0279] ⑨Decode calculation (such as Fig. 9 D calculation shown): The home ONU / MFU receives some model parameters and starts calculation, and caches the intermediate calculation results locally in the home ONU / MFU.

[0280] Repeat steps ⑦ to ⑨ above until the Decode calculation of the entire model is completed, and the processing result of the target data is obtained.

[0281] ⑩Processing result: The home ONU / MFU returns the processing result of the target data to the home user through the terminal.

[0282] In the above exemplary scenario, an end-to-end collaborative PD separation architecture based on P2MP communication between the operator OLT and the home ONU / MFU is provided. The operator OLT performs the Prefill calculation and broadcasts the model parameters to the home ONU / MFU to perform the Decode calculation on the home ONU / MFU, which can provide the home with a low-cost and high-efficiency model service of FTTR based on the P2MP architecture.

[0283] In addition, since the home ONU / MFU is the home user's own equipment, the application of the above-mentioned edge-to-end collaborative data processing solution in the home can protect the home user's data privacy on the home ONU / MFU side, that is, protect the home user's data privacy in the Decode stage. Since the operator OLT is not the home user's own equipment, the target data is encrypted through the home ONU / MFU and then sent to the operator OLT. This can also protect the home user's data privacy on the operator OLT side, that is, protect the home user's data privacy in the Prefill stage.

[0284] In addition, by storing some model parameters of the data processing model in the home ONU / MFU, the amount of data requested by the home ONU / MFU for model parameters from the operator's OLT can be reduced in the above ⑦ to ⑨, thereby further saving network bandwidth consumption.

[0285] Embodiment 3: Enterprise campus OTN private line scenario

[0286] For example, Fig.10 A schematic diagram of an enterprise campus OTN dedicated line scenario provided in an embodiment of the present application. Fig.10 In the enterprise campus OTN dedicated line scenario, the first device can be a CPE device deployed on the end side, such as Fig.10 The second device may be an edge cloud server deployed in the enterprise, such as Fig.10 Enterprise edge cloud shown.

[0287] The enterprise edge cloud is used to access the operator's cloud computing center. In some embodiments, the enterprise edge cloud provides a high-speed fiber optic access service for accessing the operator's cloud computing center to utilize the cloud computing services provided by the operator's cloud computing center. Exemplarily, the cloud connected to the enterprise edge cloud can be the operator's cloud server, hereinafter referred to as the operator cloud.

[0288] The enterprise CPE is used to provide Internet access and various personalized value-added services to enterprise users. In some embodiments, the enterprise CPE may provide a human-machine interface for connecting a terminal to realize interaction between the enterprise user and the enterprise CPE.

[0289] In an embodiment of the present application, the enterprise edge cloud and the enterprise CPE communicate based on P2MP.

[0290] The enterprise edge cloud is connected to multiple enterprise CPEs through one or more OTN devices to provide high-speed and stable fiber access services to enterprise users. It also involves devices that forward data based on the Internet protocol (IP), such as Fig.10The IP device shown in the embodiment of the present application takes the interaction process between an enterprise edge cloud and an enterprise CPE as an example to illustrate the solution.

[0291] In the enterprise campus OTN dedicated line scenario, based on the model-based data processing method provided in the embodiment of the present application, the enterprise edge cloud performs the first processing (such as Prefill calculation) process, and the enterprise CPE performs the second processing (such as Decode calculation) process, which can provide a low-cost and energy-efficient model service based on P2MP for the enterprise campus. In some embodiments, the enterprise edge cloud can provide one or more first processing service interfaces for connecting one or more enterprise CPEs, so that the enterprise CPE triggers the enterprise edge cloud to perform the first processing process through the first processing service interface.

[0292] In an embodiment of the present application, the enterprise edge cloud stores a complete data processing model. In some embodiments, the enterprise edge cloud can manage the data of each level of the data processing model through a certain data structure, that is, the model parameters of each network layer, such as weight parameters, etc. In some embodiments, the operator cloud can push the latest version of the data processing model to the enterprise edge cloud for the enterprise edge cloud to synchronize the model. In some embodiments, the enterprise edge cloud can be deployed with an XPU, such as an NPU or a GPU, to perform the first processing based on the data processing model.

[0293] In an embodiment of the present application, the enterprise CPE may store some model parameters of the data processing model. For example, when the operator cloud pushes the latest version of the data processing model to the enterprise edge cloud, the enterprise edge cloud can actively send it to the enterprise CPE once for the enterprise CPE to selectively store. As another example, the enterprise CPE can also actively request the latest version of the data processing model from the enterprise edge cloud, and then selectively store it. As another example, the enterprise CPE can also cache the model parameters of the data processing model during daily reasoning. In this way, the amount of data transmitted in communication can be reduced. In some embodiments, the enterprise CPE can be deployed with an XPU, such as an NPU or a GPU, to perform a second process based on the data processing model.

[0294] The following is based on Fig.10 The process shown in the figure illustrates the data processing process of the enterprise edge cloud and an enterprise CPE edge end collaboration. Fig.10 The data processing process based on the edge collaboration of the enterprise edge cloud and the enterprise CPE can include the following ① to ⑩.

[0295] ① Data processing request: Enterprise users send data processing requests to enterprise CPE through terminals to request model services.

[0296] ② Obtain target data: The enterprise CPE obtains target data based on the data processing request.

[0297] Among them, the relevant contents of the above ② are consistent with the above Figure 8 The content of ② is similar and will not be repeated here.

[0298] ③Query and synchronize model versions: Enterprise CPE sends a query request to the enterprise edge cloud to obtain the target version of the model, such as the latest version of the model. The enterprise edge cloud receives the query request sent by the enterprise CPE and returns a list of target versions of the model to the enterprise CPE for model synchronization.

[0299] Among them, the relevant contents of the above ③ are consistent with the above Figure 8 The content of ③ is similar and will not be repeated here.

[0300] ④Target data: The enterprise CPE sends target data to the enterprise edge cloud.

[0301] ⑤Prefill calculation (such as Fig.10 P calculation shown): The enterprise edge cloud performs Prefill calculation on the target data based on the data processing model to obtain the Prefill calculation results and KV cache data of the target data.

[0302] ⑥Calculation results: The enterprise edge cloud returns the Prefill calculation results and the KV cache data of the target data to the enterprise CPE. The enterprise CPE receives the Prefill calculation results and the KV cache data of the target data returned by the enterprise edge cloud.

[0303] The enterprise CPE starts the decoding calculation process, and the decoding calculation process includes the following ⑦ to ⑨.

[0304] ⑦ Parameter request: Based on the current calculation process, the enterprise CPE sends a parameter request to the enterprise edge cloud to request some model parameters of the data processing model for performing the next step of calculation.

[0305] ⑧ Broadcast parameters: The enterprise edge cloud receives parameter requests sent by the enterprise CPE and broadcasts them based on the P2MP architecture to send some model parameters to the enterprise CPE.

[0306] ⑨Decode calculation (such as Fig.10 D calculation shown): The enterprise CPE receives some model parameters and starts calculation, and caches the intermediate calculation results locally in the enterprise CPE.

[0307] Repeat steps ⑦ to ⑨ above until the Decode calculation of the entire model is completed, and the processing result of the target data is obtained.

[0308] ⑩Processing result: The enterprise CPE returns the processing result of the target data to the enterprise user through the terminal.

[0309] In the above exemplary scenario, an end-to-end collaborative PD separation architecture based on P2MP communication between the enterprise edge cloud and the enterprise CPE is provided. The enterprise edge cloud performs the Prefill calculation and broadcasts the model parameters to the enterprise CPE to perform the Decode calculation on the enterprise CPE, which can provide the enterprise campus with a low-cost and high-efficiency model service based on the P2MP architecture.

[0310] Moreover, since both the enterprise edge cloud and the enterprise CPE are the enterprise users' own equipment, the application of the above-mentioned edge-end collaborative data processing solution in the enterprise campus OTN dedicated line scenario can ensure the user's data privacy.

[0311] In addition, by storing some model parameters of the data processing model in the enterprise CPE, the amount of data requested by the enterprise CPE for model parameters from the enterprise edge cloud can be reduced in the above ⑦ to ⑨, thereby further saving network bandwidth consumption.

[0312] The technical solution provided in the embodiment of the present application utilizes edge devices and end devices to realize the separate deployment of model inference services based on the PD separation architecture. In this way, by separating the calculation through PD deployment and optimizing the two calculation stages separately, not only can the cost of device deployment be optimized, but also the overall performance can be improved. Among them, the edge devices are used to improve the Prefill calculation performance, which also reduces the computing power / memory pressure of the end-side devices. In addition, by deploying the complete large model on the edge device, the edge device is used as a common memory for the subordinate end-side devices. When the end-side device needs it, the large model is split to send part of the model parameters, and the end-side device does not need to deploy the complete large model, which further reduces the computing power / memory pressure of the end-side device. In addition, the edge device and the end-side device are based on P2MP communication, so that the edge device only needs to send the model parameters once when facing multiple end-side devices, which can reduce the consumption of network bandwidth.

[0313] It should be pointed out that the above description is for the purpose of more clearly illustrating the model-based data processing method described in the embodiments of the present application, and should not be understood as a limitation on the specific implementation method of the present application.

[0314] The above mainly introduces the scheme provided by the embodiment of the present application from the perspective of the processing flow. Accordingly, the embodiment of the present application also provides a model-based data processing device, which is used to implement the above-mentioned various methods. The model-based data processing device can be in the above-mentioned method embodiment, or include the above-mentioned device, or be a component that can be used for. It can be understood that the model-based data processing device includes a hardware structure and / or software module corresponding to each function in order to realize the above-mentioned functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0315] The embodiment of the present application can divide the functional modules of the model-based data processing device according to the above method embodiment. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be understood that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0316] For example, Fig.11 A schematic diagram of the structure of a model-based data processing device provided in an embodiment of the present application. Fig.11 The model-based data processing device includes a receiving module 1101, a sending module 1102 and a processing module 1103. Among them: Receiving module 1101, used to execute the above Figure 6 S601 and S606 shown or Figure 7 S706 shown; The sending module 1102 is used to execute the above Figure 6 S602 and S608 shown or Figure 7 S703 shown; Processing module 1103, used to execute the above Figure 6 S607 shown.

[0317] In some possible implementations, the device further includes a judgment module for executing the above Figure 7 S701 shown.

[0318] In some possible implementations, the device further includes an acquisition module for executing the above Figure 7 S702 shown.

[0319] For example, Fig.12 A schematic diagram of the structure of another model-based data processing device provided in an embodiment of the present application. Fig.12 The model-based data processing device includes a receiving module 1201, a processing module 1202 and a sending module 1203. Receiving module 1201, used to execute the above Figure 6 S603 or Figure 7 S704 shown; Processing module 1202, used to execute the above Figure 6 S604 shown; The sending module 1203 is used to execute the above Figure 6 S605 or Figure 7 S705 shown.

[0320] For the detailed description of the above optional methods, please refer to the above method embodiments, which will not be repeated here. In addition, the explanation of any of the above model-based data processing devices and the description of the beneficial effects can refer to the above corresponding method embodiments, which will not be repeated here.

[0321] As an example, combining Figure 4 , the above Fig.11 The functions partially or completely implemented in the receiving module 1101, the sending module 1102 and the processing module 1103 in the model-based data processing device can be Figure 4 Processor 401 in the execution Figure 4 The computer executes instructions in the memory 402 in the embodiment.

[0322] In the embodiment of the present application, the model-based data processing device is presented in the form of dividing various functional modules in an integrated manner. The "module" here can refer to a specific application specific integrated circuit (ASIC), a circuit, a processor and a memory that executes one or more software or firmware programs, an integrated logic circuit, and / or other devices that can provide the above functions. In a simple embodiment, a person skilled in the art can imagine that the model-based data processing device can be used Figure 4 The form of the ONU device is shown.

[0323] for example, Figure 4The processor 401 in the ONU device shown can call the computer execution instructions stored in the memory 402 to enable the ONU device to execute the model-based data processing method in the above method embodiment.

[0324] As an example, combining Figure 5 , the above Fig.12 The functions partially or completely implemented in the receiving module 1201, the processing module 1202 and the sending module 1203 in the model-based data processing device can be Figure 5 Processor 501 in the Figure 5 The computer executes instructions in the memory 502 in the embodiment.

[0325] In the embodiment of the present application, the model-based data processing device is presented in the form of dividing various functional modules in an integrated manner. The "module" here can refer to a specific ASIC, circuit, processor and memory that executes one or more software or firmware programs, integrated logic circuit, and / or other devices that can provide the above functions. In a simple embodiment, a person skilled in the art can imagine that the model-based data processing device can be used Figure 5 The form of the OLT equipment is shown.

[0326] for example, Figure 5 The processor 501 in the OLT device shown can call the computer execution instructions stored in the memory 502 to enable the OLT device to execute the model-based data processing method in the above method embodiment.

[0327] Since the model-based data processing device provided in the embodiment of the present application can execute the above-mentioned model-based data processing method, the technical effects that can be obtained can be referred to the above-mentioned method embodiment and will not be repeated here.

[0328] It should be understood that one or more of the above modules or units can be implemented by software, hardware or a combination of the two. When any of the above modules or units is implemented by software, the software exists in the form of computer program instructions and is stored in the memory, and the processor can be used to execute the program instructions and implement the above method flow. The processor can be built into a system on chip (SoC) or ASIC, or it can be an independent semiconductor chip. In addition to the core used to execute software instructions for calculation or processing in the processor, it can further include necessary hardware accelerators, such as field programmable gate arrays (FPGA), programmable logic devices (PLD), or logic circuits that implement dedicated logic operations.

[0329] When the above modules or units are implemented in hardware, the hardware can be any one or any combination of a microprocessor, a digital signal processing (DSP) chip, a microcontroller unit (MCU), an artificial intelligence processor, an ASIC, a SoC, an FPGA, a PLD, a dedicated digital circuit, a hardware accelerator or a non-integrated discrete device, which can run the necessary software or not rely on the software to execute the above method flow.

[0330] Optionally, an embodiment of the present application further provides a first device (for example, the first device may be a chip or a chip system), the first device including a processor for implementing the method executed by the first device in any of the above method embodiments. In one possible design, the first device also includes a memory. The memory is used to store necessary program instructions and data, and the processor can call the program code stored in the memory to instruct the first device to execute the method in any of the above method embodiments. Of course, the memory may not be in the first device. When the first device is a chip system, it may be composed of a chip, or it may include a chip and other discrete devices, which is not specifically limited in the embodiment of the present application.

[0331] Optionally, an embodiment of the present application further provides a second device (for example, the second device may be a chip or a chip system), and the second device includes a processor for implementing the method executed by the second device in any of the above method embodiments. In one possible design, the second device also includes a memory. The memory is used to store necessary program instructions and data, and the processor can call the program code stored in the memory to instruct the second device to execute the method in any of the above method embodiments. Of course, the memory may not be in the second device. When the second device is a chip system, it may be composed of a chip, or it may include a chip and other discrete devices, which is not specifically limited in the embodiment of the present application.

[0332] An embodiment of the present application also provides a computer-readable storage medium having computer execution instructions stored thereon. When the computer execution instructions are executed on a first device, the first device executes any method executed by the model-based data processing apparatus provided above. When the computer execution instructions are executed on a second device, the second device executes any method executed by the model-based data processing apparatus provided above.

[0333] For the explanation of the relevant contents and description of the beneficial effects in any of the above-mentioned computer-readable storage media, reference may be made to the above-mentioned corresponding embodiments, which will not be repeated here.

[0334] The embodiment of the present application also provides a chip. The chip integrates a control circuit and one or more ports for implementing the functions of the above-mentioned model-based data processing device. Optionally, the functions supported by the chip can be referred to above and will not be repeated here. A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned embodiment can be completed by instructing the relevant hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The above-mentioned processing unit or processor can be a central processing unit, a general-purpose processor, an ASIC, a DSP, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof.

[0335] The embodiment of the present application also provides a computer program product including computer-executable instructions, which, when executed on a first device, causes the first device to execute any one of the methods in the above-mentioned embodiments. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on the first device, the process or function according to the embodiment of the present application is generated in whole or in part. The first device may be a network device or other programmable device. When the computer-executable instructions are executed on a second device, the second device executes any one of the methods in the above-mentioned embodiments. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on the second device, the process or function according to the embodiment of the present application is generated in whole or in part. The second device may be a network device or other programmable device.

[0336] Computer-executable instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer-executable instructions may be transmitted from one website, computer, network device, or data center to another website, computer, network device, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer-readable storage media may be any available medium that can be accessed by a network device or a data storage device such as a network device, data center, etc. that includes one or more available media. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., digital versatile discs (DVD)), or semiconductor media (e.g., solid state disks (SSD)), etc.

[0337] It should be noted that the above-mentioned devices for storing computer instructions or computer programs provided in the embodiments of the present application, such as but not limited to the above-mentioned memories, computer-readable storage media, and communication chips, etc., are all non-transitory.

[0338] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software programs, all or part of the embodiments may be implemented in the form of computer program products.

[0339] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0340] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method based on a model, characterized in that: Applied to a first device, where the first device is a network device connected to a terminal; the method includes: receiving a data processing request sent by the terminal, wherein the data processing request is used to request processing of target data; Sending the target data to a second device, where the second device is a network device connected to the cloud, and the second device stores a data processing model; receiving a first data processing result sent by the second device, where the first data processing result is a result obtained by the data processing model performing a first processing on the target data; Based on the data processing model, performing a second processing on the first data processing result to obtain a second data processing result, where the second data processing result is a processing result of the target data; Send the second data processing result to the terminal.

2. The method according to claim 1, characterized in that The second processing includes at least one third processing; the second processing is performed on the first data processing result based on the data processing model to obtain the second data processing result, including: Each time the third processing includes the following process: Acquire target model parameters, where the target model parameters are model parameters of the data processing model used to perform the third processing; Based on the target model parameters, performing the third processing on the data to be processed; Among them, the data to be processed in the first third processing is the first data processing result, and the data to be processed in other third processing is the data processing result obtained by the previous third processing; the data processing result obtained by the last third processing is the second data processing result.

3. The method according to claim 2, characterized in that The first device stores some model parameters of the data processing model; and obtaining the target model parameters includes: In the case where the target model parameters are stored in the first device, acquiring the stored target model parameters; or, In a case where the first device does not store the target model parameters, a first request is sent to the second device, where the first request is used to request to obtain the target model parameters; and the target model parameters sent by the second device are received.

4. The method according to claim 3, characterized in that The first request carries performance parameters of the first device, where the performance parameters are used to indicate computing power and / or memory capacity of the first device; The target model parameters include: target model parameters of a target number of layers of the data processing model that match the performance parameters.

5. The method according to claim 3, characterized in that: The receiving the target model parameter sent by the second device includes: The broadcast information of the second device is received, where the broadcast information includes the target model parameter.

6. The method according to claim 3, characterized in that Before receiving the data processing request sent by the terminal, the method further includes: receiving all model parameters of the data processing model, or the part of model parameters of the data processing model, sent by the second device; The portion of model parameters of the data processing model is stored.

7. The method according to claim 3, characterized in that After receiving the data processing request sent by the terminal, the method further includes: Sending a query request to the second device, where the query request is used to query the model parameters of the data processing model of the target version; receiving the model parameters of the data processing model of the target version sent by the second device; Based on the model parameters of the data processing model of the target version, the stored model parameters are updated.

8. The method according to claim 1, characterized in that The performing a second processing on the first data processing result based on the data processing model to obtain a second data processing result includes: Sending a second request to the second device, where the second request is used to request to obtain a model parameter of the data processing model used to perform the second processing; receiving the model parameters sent by the second device; Based on the model parameters, the second processing is performed on the first data processing result to obtain the second data processing result.

9. The method according to claim 1, characterized in that: The data processing request carries the target data; or, The data processing request carries a data identifier of the target data; the method further includes: acquiring data corresponding to the data identifier from a data set associated with the first device as the target data.

10. The method according to claim 1, characterized in that Before sending the target data to the second device, the method further includes: The target data is encrypted.

11. The method according to claim 1, characterized in that: The first processing is a Prefill calculation, and the second processing is a Decode calculation.

12. The method according to claim 1, characterized in that The first device is an optical network unit ONU device or a main fiber to the room MFU device, and the second device is an optical line terminal OLT device; or, The first device is a customer terminal CPE device in a network based on an optical transport network OTN dedicated line, and the second device is an edge cloud device in a network based on the OTN dedicated line.

13. A data processing method based on a model, characterized in that: Applied to a second device, the second device is a network device connected to the cloud, and the second device stores a data processing model; the method includes: Receiving target data sent by a first device, where the first device is a network device connected to the terminal; Performing a first process on the target data based on the data processing model to obtain a first data processing result; The first data processing result is sent to the first device, where the first data processing result is used to perform a second processing based on the data processing model to obtain a second data processing result, where the second data processing result is a processing result of the target data.

14. The method according to claim 13, characterized in that The second process includes at least one third process; during each third process, the method further includes: receiving a first request sent by the first device, where the first request is used to request to obtain a target model parameter, where the target model parameter is a model parameter of the data processing model used to perform the third processing; Based on the first request, the target model parameters are sent to the first device.

15. The method according to claim 14, characterized in that The first request carries performance parameters of the first device, where the performance parameters are used to indicate computing power and / or memory capacity of the first device; The target model parameters include: target model parameters of a target number of layers of the data processing model that match the performance parameters.

16. The method according to claim 14, characterized in that The sending the target model parameter to the first device includes: Broadcast information is sent, where the broadcast information includes the target model parameters.

17. The method according to claim 13, characterized in that The first device stores some model parameters of the data processing model; before receiving the target data sent by the first device, the method further includes: All model parameters of the data processing model, or the part of model parameters of the data processing model, are sent to the first device.

18. The method according to claim 17, characterized in that Before sending all the model parameters of the data processing model or the part of the model parameters of the data processing model to the first device, the method further includes: Receiving model update information sent by the cloud, wherein the model update information includes updated model parameters of the data processing model; or, A parameter request sent by the first device is received, where the parameter request is used to request to obtain model parameters of the data processing model.

19. The method according to claim 17, characterized in that Before receiving the target data sent by the first device, the method further includes: receiving a query request sent by the first device, wherein the query request is used to query a model parameter of the data processing model of a target version; Based on the query request, model parameters of the target version of the data processing model are sent to the first device.

20. The method according to claim 13, characterized in that After sending the first data processing result to the first device, the method further includes: receiving a second request sent by the first device, where the second request is used to request to obtain a model parameter of the data processing model for performing the second processing; The model parameters are sent to the first device.

21. The method according to claim 13, characterized in that After receiving the target data sent by the first device, the method further includes: The target data is decrypted.

22. The method according to claim 13, characterized in that The first processing is a Prefill calculation, and the second processing is a Decode calculation.

23. The method according to claim 13, characterized in that The first device is an optical network unit ONU device or a main fiber to the room MFU device, and the second device is an optical line terminal OLT device; or, The first device is a customer terminal CPE device in a network based on an optical transport network OTN dedicated line, and the second device is an edge cloud device in a network based on the OTN dedicated line.

24. A data processing device based on a model, characterized in that: Applied to a first device, the first device is a network device connected to a terminal; the apparatus comprises: A receiving module, used for receiving a data processing request sent by the terminal, wherein the data processing request is used for requesting processing of target data; A sending module, used for sending the target data to a second device, where the second device is a network device connected to the cloud, and the second device stores a data processing model; The receiving module is further used to receive a first data processing result sent by the second device, where the first data processing result is a result obtained by the data processing model performing a first processing on the target data; A processing module, configured to perform a second processing on the first data processing result based on the data processing model to obtain a second data processing result, wherein the second data processing result is a processing result of the target data; The sending module is further used to send the second data processing result to the terminal.

25. A data processing device based on a model, characterized in that: Applied to a second device, the second device is a network device connected to the cloud, and the second device stores a data processing model; the apparatus comprises: A receiving module, configured to receive target data sent by a first device, where the first device is a network device connected to the terminal; A processing module, configured to perform a first processing on the target data based on the data processing model to obtain a first data processing result; A sending module is used to send the first data processing result to the first device, the first data processing result is used to perform a second processing based on the data processing model to obtain a second data processing result, and the second data processing result is the processing result of the target data.

26. A first device, characterized in that: The first device is a network device connected to the terminal, including a memory and a processor, the memory and the processor are connected; the memory is used to store computer execution instructions; the processor is used to call the computer execution instructions to execute the method according to any one of claims 1-12.

27. A second device, characterized in that: The second device is a network device connected to the cloud, including a memory and a processor, the memory and the processor are connected; the memory is used to store computer execution instructions; the processor is used to call the computer execution instructions to execute the method described in any one of claims 13-23.

28. A data processing system, characterized in that: Comprises the first device as claimed in claim 26 and the second device as claimed in claim 27.

29. A computer-readable storage medium, characterized in that: It includes computer execution instructions, which, when running on a first device, enable the first device to execute the method as described in any one of claims 1-12, and when running on a second device, enable the second device to execute the method as described in any one of claims 13-23.

30. A computer program product, characterized in that It includes computer execution instructions, which, when the computer execution instructions are executed on a first device, cause the first device to execute the method as described in any one of claims 1-12, and when the computer execution instructions are executed on a second device, cause the second device to execute the method as described in any one of claims 13-23.

Images