Large strip coding calculation optimization method for prediction service system
By using the Set Transformer corrector and perturbation Chebyshev points in the prediction service system, the calculation of large stripe coding is optimized, which solves the problem of low decoding accuracy and improves the decoding accuracy of single and multiple missing nodes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-06
- Publication Date
- 2026-03-31
AI Technical Summary
Existing approximate coding techniques have low decoding accuracy when performing large-strip coding calculations, especially when one or more nodes fall behind, resulting in insufficient decoding accuracy or accuracy oscillations.
A Set Transformer-based corrector is used to correct the decoding results of the decoder, and perturbated Type II Chebyshev points are used in the encoding and decoding process. This improves the decoding accuracy for single-falling-node scenarios and avoids Chebyshev point conflicts through small random perturbations for multi-falling-node scenarios.
It improves the decoding accuracy in scenarios with a single missing node and resolves the accuracy oscillation phenomenon in scenarios with multiple missing nodes, thereby improving the overall decoding accuracy.
Smart Images
Figure CN121770534A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of distributed machine learning technology, and more specifically, relates to a method for optimizing large stripe coding computation for prediction service systems. Background Technology
[0002] Large-strip coding computation has become a research hotspot in the storage field in recent years. Large stripes refer to the use of a large coding parameter K (generally K>10), which can significantly improve the system's resource utilization. In recent years, learning-based coding computation methods have been proposed to improve accuracy. These methods abandon the use of specific encoding / decoding formulas and instead use neural networks as encoders and decoders. They leverage the learnability of neural networks to achieve encoding and decoding functions. However, due to the high computational complexity of neural networks, learning-based coding computation is not suitable for large-strip coding computation scenarios. Approximate coding techniques no longer pursue a precise encoding and decoding process but use an approximate fitting method, replacing the precise value with an approximate value of the encoding result. This method has lower complexity. Although this method is suitable for handling large-strip coding computation, its decoding accuracy is relatively low for the following reasons: In large-strip scenarios, existing approximate coding schemes exhibit insufficient accuracy when a single lagging node appears. This is because increasing the coding parameter K requires the interpolation function to traverse more points, significantly increasing the fitting error of the function interpolation. In large-strip scenarios, the probability of multiple lagging nodes appearing simultaneously increases considerably. Although existing approximate coding schemes can tolerate multiple lagging nodes, the decoding accuracy drops sharply when the number of lagging nodes reaches certain specific values. We call this phenomenon accuracy oscillation. This is because during encoding, some Type I Chebyshev points used as interpolation points and some Type II Chebyshev points used as sampling points overlap, resulting in a zero denominator for their corresponding basis functions, rendering some encoding requests unavailable. Summary of the Invention
[0003] To address the shortcomings of existing technologies, the purpose of this application is to provide an optimization method for large-strip coding computation in prediction service systems, aiming to solve the problem of low decoding accuracy of existing approximate coding techniques when processing large-strip coding computation.
[0004] To achieve the above objectives, in a first aspect, this application provides a method for optimizing large-strip coding computation in a prediction service system, comprising: If there is one lagging node in the prediction service system cluster, the first optimization process is performed on the large stripe coding calculation based on approximate coding; if there are multiple lagging nodes in the prediction service system cluster, the second optimization process is performed on the large stripe coding calculation based on approximate coding. The first optimization process includes: correcting the decoding result of the decoder using a pre-trained Set Transformer-based corrector; The second optimization process includes using perturbed second-type Chebyshev points in the codec to complete the encoding and decoding process.
[0005] This application trains a Set Transformer-based corrector for single-drop-node scenarios. Since the decoder output contains K inference results that conform to the definition of a set, namely determinism, independence, and unorderedness, the Set Transformer's powerful modeling capability for set data is utilized. The decoder's decoding result is input, and the output is a verified inference result with higher accuracy, thus improving the decoding accuracy in single-drop-node scenarios. For multi-drop-node scenarios, a small random perturbation is applied to the second type of Chebyshev points to avoid conflicts with the first type of Chebyshev points, thus resolving the accuracy oscillation phenomenon when multiple drops occur and improving the decoding accuracy in multi-drop-node scenarios.
[0006] According to the large-strip coding computation optimization method for prediction service systems provided in this application, the step of correcting the decoding result of the decoder using a pre-trained Set Transformer-based corrector includes: Construct a corresponding Set Transformer-based corrector based on the strip length; The Set Transformer-based corrector is trained based on a preset dataset to obtain the trained Set Transformer-based corrector. The decoding result of the decoder is input into the trained Set Transformer-based corrector to obtain the corrected decoding result.
[0007] According to the large-strip coding computation optimization method for prediction service systems provided in this application, the encoding and decoding process is completed using perturbed Chebyshev points of the second kind in the encoder and decoder, including: For each of the generated second-type Chebyshev points, a corresponding perturbation value is randomly generated within a preset perturbation range; Each Chebyshev point of the second kind is added to its corresponding perturbation value to obtain the perturbation of the Chebyshev point of the second kind; In the encoder, N encoding requests are generated using perturbed Type II Chebyshev points as sampling points, where N is the number of Type II Chebyshev points. In the decoder, K perturbed second-type Chebyshev points are used as interpolation points to generate a rational interpolation function, where K is the strip length.
[0008] Secondly, this application provides a large-strip coding computation optimization apparatus for prediction service systems, comprising: The optimization module is used to perform a first optimization process on the large stripe coding calculation based on approximate coding if there is one lagging node in the prediction service system cluster, and to perform a second optimization process on the large stripe coding calculation based on approximate coding if there are multiple lagging nodes in the prediction service system cluster. The first optimization process includes: correcting the decoding result of the decoder using a pre-trained Set Transformer-based corrector; The second optimization process includes using perturbed second-type Chebyshev points in the codec to complete the encoding and decoding process.
[0009] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the large stripe coding computation optimization method for a prediction service system described in the first aspect or any possible implementation of the first aspect.
[0010] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the large stripe coding computation optimization method for a prediction service system described in the first aspect or any possible implementation of the first aspect.
[0011] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to execute the large stripe coding computation optimization method for a prediction service system described in the first aspect or any possible implementation of the first aspect.
[0012] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.
[0013] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: This application trains a Set Transformer-based corrector for single-drop-node scenarios. Since the decoder output contains K inference results that conform to the definition of a set, namely determinism, independence, and unorderedness, the Set Transformer's powerful modeling capability for set data is utilized. The decoder's decoding result is input, and the output is a verified inference result with higher accuracy, thus improving the decoding accuracy in single-drop-node scenarios. For multi-drop-node scenarios, a small random perturbation is applied to the second type of Chebyshev points to avoid conflicts with the first type of Chebyshev points, thus resolving the accuracy oscillation phenomenon when multiple drops occur and improving the decoding accuracy in multi-drop-node scenarios. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 This is a flowchart illustrating the large stripe coding calculation optimization method for prediction service systems provided in this application embodiment; Figure 2 This is a schematic diagram illustrating the processing procedure for a fallen node provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the processing procedure for multiple lagging nodes provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the large stripe coding calculation optimization device for a prediction service system provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0017] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.
[0018] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0019] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.
[0020] First, let's introduce the following: Machine learning, as the cornerstone of artificial intelligence, has been widely applied in various fields, including image recognition, speech recognition, and natural language processing. To provide users with more stable AI prediction services, prediction service systems (PSSes) have been created in large numbers. Like other latency-sensitive systems, prediction service systems should also provide low-latency services. However, as the number of users increases, a single node can no longer guarantee low-latency service. To solve this problem, most prediction service systems adopt a distributed architecture to improve the overall throughput of the system. However, this design introduces a new problem: some nodes in the system may fail to return prediction results to the client due to hardware failure, resource contention, or network transmission latency. This phenomenon is called node deprecation, and nodes that fail to return inference results on time are called deprecated nodes.
[0021] In a prediction service system based on approximate encoding computation, for each stripe, rational interpolation is used to encode all K inference requests, generating NK redundant inference requests. These N inference requests form a stripe. The inference model in the prediction service system infers from all N requests, obtaining N inference results. Of these N results, S will fail to return to the client due to being out of order. The decoder can then use the remaining NS successfully returned inference results to decode the inference results of the K original requests. The encoding parameters K and N can be reasonably selected according to the actual usage of the system.
[0022] Next, combined Figures 1-3 This application introduces a method for optimizing large stripe coding calculations for prediction service systems.
[0023] Figure 1 This is a flowchart illustrating the large stripe coding calculation optimization method for prediction service systems provided in this application embodiment, as shown below. Figure 1 As shown, the method includes the following steps: Step S1: If there is one lagging node in the prediction service system cluster, perform the first optimization process for the large stripe coding calculation based on approximate coding; if there are multiple lagging nodes in the prediction service system cluster, perform the second optimization process for the large stripe coding calculation based on approximate coding. The first optimization process includes: correcting the decoding results of the decoder using a pre-trained Set Transformer-based corrector; The second optimization process includes using perturbed second-type Chebyshev points in the encoder and decoder to complete the encoding and decoding process.
[0024] The large stripe coding computation optimization method provided in this application for prediction service systems trains a Set Transformer-based corrector for single-falling-node scenarios. Since the decoder output contains K inference results that conform to the definition of a set, namely determinism, independence, and unorderedness, the Set Transformer's powerful modeling capability for set data is utilized. The decoder's decoding result is input, and the output is a verified inference result with higher accuracy, thus improving the decoding accuracy in single-falling-node scenarios. For multi-falling-node scenarios, a small random perturbation is applied to the second type of Chebyshev points to avoid conflicts with the first type of Chebyshev points, thus resolving the accuracy oscillation phenomenon when multiple falls occur and improving the decoding accuracy in multi-falling-node scenarios.
[0025] In some embodiments, the first optimization process in step S1 specifically includes: Construct a corresponding Set Transformer-based corrector based on the strip length; The Set Transformer-based corrector is trained based on a preset dataset to obtain the trained Set Transformer-based corrector. The decoding result of the decoder is input into the trained Set Transformer-based corrector to obtain the corrected decoding result.
[0026] Optionally, the input data for the Set Transformer-based corrector is a two-dimensional tensor with K rows and 1 column of class_num, where class_num is the number of classes. Therefore, different correctors need to be trained for different strip lengths K.
[0027] Figure 2 This is a schematic diagram illustrating the processing procedure for a fallen node provided in an embodiment of this application, as shown below. Figure 2 As shown in one embodiment of this application, For the 20 inference requests submitted by the prediction service system client, These are 21 encoded requests obtained after the original inference request is approximated by the encoder. After these encoded requests are processed by the inference model deployed in the prediction service system, 21 corresponding inference results will be obtained. Because one node may fall behind, only 20 inference results can be successfully returned, and their indices are... Therefore, the reasoning result that was successfully returned is Specifically, it includes the following steps: 1a. The user submits K inference requests on the client, such as Figure 2 As shown, in this embodiment, K=20 is used, and the inference request is denoted as... ; 2a. The encoder in the client generates 20 Chebyshev points of the first kind. and 21 second-type Chebyshev points ; 3a. Interpolate the original inference request using Berrut rational functions to obtain rational functions. The basis functions are ; 4a. Using the second type of Chebyshev points, sample the rational function obtained in step 3a to obtain the encoding request, i.e. ; 5a. Using the models already deployed in the prediction service system, inference calculations were performed on the encoding requests, resulting in 21 corresponding inference results. And return to the client; 6a. The client will only wait for the first 20 inference results to be returned; the rest are considered to have fallen behind. The first 20 inference results are denoted as... ; 7a. The decoder in the client also uses Berrut rational functions to interpolate the 20 inference results from step 6a to obtain rational functions. ; 8a. Substitute the same type I Chebyshev points as in step 2a. In the process, the reasoning result of the original request is obtained. ; 9a. Input the 20 inference results into the Set Transformer-based corrector, and after correction calculation, obtain 20 inference results with higher accuracy. Return to the client.
[0028] In some embodiments, the second optimization process in step S1 specifically includes: For each of the generated second-type Chebyshev points, a corresponding perturbation value is randomly generated within a preset perturbation range; Each Chebyshev point of the second kind is added to its corresponding perturbation value to obtain the perturbation of the Chebyshev point of the second kind; In the encoder, N encoding requests are generated using perturbed Type II Chebyshev points as sampling points, where N is the number of Type II Chebyshev points. In the decoder, K perturbed Chebyshev points of the second kind are used as interpolation points to generate a rational interpolation function, where K is the strip length.
[0029] Figure 3 This is a schematic diagram illustrating the processing procedure for multiple lagging nodes provided in an embodiment of this application, as shown below. Figure 3 As shown, in one embodiment of this application, there are 2 lagging nodes, N=22, K=20, and the 20 inference requests generated by the client are denoted as... The specific steps are as follows: 1b. The user submits K inference requests on the client, such as Figure 3 As shown, in this embodiment, K=20 is used, and the inference request is denoted as... ; 2b. The encoder in the client generates 20 Chebyshev points of the first kind. and 22 Chebyshev points of the second kind And a random minimal perturbation is applied to the second type of Chebyshev points, resulting in the following second type of Chebyshev points: ; 3b. Interpolate the original inference request using Berrut rational functions to obtain rational functions. The basis functions are ; 4b. Using the perturbed Chebyshev points of the second kind, sample the rational function obtained in step 3b to obtain the encoded inference request, i.e. ; 5b. Using the models already deployed in the prediction service system, perform inference calculations on the coded inference requests to obtain 22 corresponding inference results. And return to the client; 6b. The client will only wait for the first 20 inference results to return; any subsequent returns are considered to have fallen behind. The first 20 inference results are denoted as... ; 7b. The decoder in the client also uses Berrut rational functions to interpolate the 20 inference results from step 6b to obtain rational functions. ; 8b. Substitute the same first-type Chebyshev points as in step 2b. In the process, the reasoning result of the original request is obtained. .
[0030] The following describes the large-strip coding computation optimization apparatus for prediction service systems provided in this application. The large-strip coding computation optimization apparatus for prediction service systems described below can be referred to in correspondence with the large-strip coding computation optimization method for prediction service systems described above.
[0031] Figure 4 This is a schematic diagram of the structure of a large stripe coding computation optimization device for a prediction service system provided in an embodiment of this application, as shown below. Figure 4 As shown, the device 400 includes: The optimization module 410 is used to perform a first optimization process on the large stripe coding calculation based on approximate coding if there is one lagging node in the prediction service system cluster, and to perform a second optimization process on the large stripe coding calculation based on approximate coding if there are multiple lagging nodes in the prediction service system cluster. The first optimization process includes: correcting the decoding results of the decoder using a pre-trained Set Transformer-based corrector; The second optimization process includes using perturbed second-type Chebyshev points in the encoder and decoder to complete the encoding and decoding process.
[0032] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.
[0033] Based on the methods in the above embodiments, Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown in the illustration, this application provides an electronic device that may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540. The processor 510, communication interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions from the memory 530 to execute the large stripe coding computation optimization method for prediction service systems described in the above embodiment.
[0034] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the large stripe coding computation optimization method for predictive service systems described in the various embodiments of this application.
[0035] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program. When the computer program runs on a processor, it causes the processor to execute the large stripe coding calculation optimization method for prediction service systems described in the above embodiments.
[0036] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the large stripe coding calculation optimization method for prediction service systems described in the above embodiments.
[0037] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.
[0038] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.
[0039] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0040] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.
[0041] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for optimizing large-strip coding computation in a predictive service system, characterized in that, include: If there is one lagging node in the prediction service system cluster, the first optimization process is performed on the large stripe coding calculation based on approximate coding; if there are multiple lagging nodes in the prediction service system cluster, the second optimization process is performed on the large stripe coding calculation based on approximate coding. The first optimization process includes: correcting the decoding result of the decoder using a pre-trained Set Transformer-based corrector; The second optimization process includes using perturbed second-type Chebyshev points in the codec to complete the encoding and decoding process.
2. The method for optimizing large stripe coding calculations for predictive service systems according to claim 1, characterized in that, The step of correcting the decoding result of the decoder using a pre-trained Set Transformer-based corrector includes: Construct a corresponding Set Transformer-based corrector based on the strip length; The Set Transformer-based corrector is trained based on a preset dataset to obtain the trained Set Transformer-based corrector. The decoding result of the decoder is input into the trained Set Transformer-based corrector to obtain the corrected decoding result.
3. The method for optimizing large-strip coding calculations for predictive service systems according to claim 1, characterized in that, The use of perturbed Chebyshev points of type II in the encoder-decoder process includes: For each of the generated second-type Chebyshev points, a corresponding perturbation value is randomly generated within a preset perturbation range; Each Chebyshev point of the second kind is added to its corresponding perturbation value to obtain the perturbation of the Chebyshev point of the second kind; In the encoder, N encoding requests are generated using perturbed Type II Chebyshev points as sampling points, where N is the number of Type II Chebyshev points. In the decoder, K perturbed second-type Chebyshev points are used as interpolation points to generate a rational interpolation function, where K is the strip length.
4. A large-strip coding computation optimization device for predictive service systems, characterized in that, include: The optimization module is used to perform a first optimization process on the large stripe coding calculation based on approximate coding if there is one lagging node in the prediction service system cluster, and to perform a second optimization process on the large stripe coding calculation based on approximate coding if there are multiple lagging nodes in the prediction service system cluster. The first optimization process includes: correcting the decoding result of the decoder using a pre-trained Set Transformer-based corrector; The second optimization process includes using perturbed second-type Chebyshev points in the codec to complete the encoding and decoding process.
5. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the large stripe coding computation optimization method for a prediction service system as described in any one of claims 1-3.
6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on a processor, the processor performs the large stripe coding computation optimization method for prediction service systems as described in any one of claims 1-3.
7. A computer program product, characterized in that, When the computer program product is run on a processor, the processor performs the large stripe coding computation optimization method for prediction service systems as described in any one of claims 1-3.