Data resource generation method and device, storage medium and electronic equipment

Abstract

The invention relates to a data resource generation method and device, a storage medium and electronic equipment, and relates to the technical field of natural language processing. The method comprises the following steps: receiving a data resource generation request reported by a user terminal, and matching a target speculation sampling model for the data resource generation request based on a preset service gating network model; performing speculation sampling processing on the data resource generation request based on the target speculation sampling model to obtain a speculation sampling result; constructing a key value storage result corresponding to the data resource generation request, and pushing the key value storage result and a speculation sampling result to a cloud server; and receiving a target data resource obtained after the cloud server verifies the speculation sampling result according to the key value storage result, and feeding back the target data resource to the user terminal. The data resource generation efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of natural language processing, and in particular, to a method for generating data resources, a device for generating data resources, a computer-readable storage medium, and an electronic device. Background Art

[0002] In some related technical solutions, large model reasoning can be divided into two stages: Prefill and Decoding to achieve distributed high-speed reasoning of large models. However, this method cannot perform targeted policy diversion according to business scenarios, which makes the generation efficiency of data resources low.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a method for generating data resources, a device for generating data resources, a computer-readable storage medium and an electronic device, thereby overcoming the problem of low efficiency in generating data resources due to limitations and defects of related technologies, at least to a certain extent.

[0005] According to one aspect of the present disclosure, a method for generating data resources is provided, comprising:

[0006] Receiving a data resource generation request reported by a user terminal, and matching a target speculative sampling model for the data resource generation request based on a preset service gating network model;

[0007] Performing speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result;

[0008] Constructing a key-value storage result corresponding to the data resource generation request, and pushing the key-value storage result and the speculative sampling result to a cloud server;

[0009] Receive the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feed back the target data resource to the user terminal.

[0010] In an exemplary embodiment of the present disclosure, the preset service gating network model includes an embedding mapping layer, a coding layer and a service gating network layer;

[0011] Wherein, generating a request matching target speculative sampling model for the data resource based on a preset business gating network model includes:

[0012] Parsing the data resource generation request to obtain a business request identifier, a business request type, task computing resources required to execute the data resource generation task, and text to be processed;

[0013] Generate task basic information according to the business request identifier, business request information and task computing resources, and generate task context information according to the text to be processed;

[0014] Based on the embedding mapping layer, word embedding processing is performed on the task basic information to obtain a first task feature, and based on the embedding mapping layer, embedding mapping processing is performed on the task context information to obtain a context marker sequence;

[0015] The first task feature and the context flag sequence are encoded based on the encoding layer to obtain an overall context representation, and the overall context representation is predicted based on the business gating network layer to obtain the target speculative sampling model.

[0016] In an exemplary embodiment of the present disclosure, the service gating network layer includes a gating network model and a candidate speculative sampling model;

[0017] The overall context representation is predicted based on the service gating network layer to obtain the target speculative sampling model, including:

[0018] Based on the gating network model and according to the overall context representation, a first model weight of the candidate speculative sampling model when performing a data resource generation task is determined, and a target speculative sampling model is determined from the candidate speculative sampling models based on the first model weight.

[0019] In an exemplary embodiment of the present disclosure, the target speculative sampling model includes a first embedding mapping layer, a position encoding module, a feature encoding module, a feature decoding module, a linear transformation layer, and a Softmax layer;

[0020] Wherein, performing speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result includes:

[0021] Performing embedding mapping processing on the task context information of the data resource generation request based on the first embedding mapping layer to obtain a first word vector, and performing encoding processing on the task context information based on the position encoding module to obtain a first position vector;

[0022] Superimposing the first word vector and the first position vector to obtain a first input vector, and encoding the first input vector based on a feature encoding module to obtain a first encoding result;

[0023] Decoding the first encoding result based on a feature decoding module to obtain a first encoding matrix, and transforming the first encoding matrix based on a linear transformation layer to obtain a first logic matrix;

[0024] The first logic matrix is ​​mapped based on the Softmax layer to obtain a speculative sampling result corresponding to the first audio data to be compared.

[0025] In an exemplary embodiment of the present disclosure, the feature encoding module includes a first multi-head self-attention module, a first residual connection and normalization module, a first feedforward neural network, and a second residual connection and normalization module;

[0026] The first input vector is encoded based on the feature encoding module to obtain a first encoding result, including:

[0027] Calculating a first attention mechanism of the first input vector based on a first multi-head self-attention module, and performing residual connection and normalization processing on the first attention mechanism based on a first residual connection and normalization module to obtain a first normalization processing result;

[0028] The first normalization processing result is linearized based on the first feedforward neural network to obtain a first linearization processing result, and the first linearization processing result is residually connected and normalized based on the second residual connection and normalization module to obtain a first encoding result.

[0029] In an exemplary embodiment of the present disclosure, calculating a first attention mechanism of the first input vector based on a first multi-head self-attention module includes:

[0030] Performing linear fusion on the first input vector to obtain a first query vector, a first key vector, and a first value vector, and calculating a first outer product vector of the first query vector and the first key vector to obtain a first similarity between the first query vector and the first key vector;

[0031] The first similarity is normalized to obtain a first weight matrix, and a second outer product vector of the first weight matrix and the first value vector is calculated to obtain the first attention mechanism.

[0032] In an exemplary embodiment of the present disclosure, constructing a key-value storage result corresponding to the data resource generation request includes:

[0033] Performing hash processing on the business request identifier, business request information, and task computing resources of the data resource generation request to obtain an associated primary key corresponding to the data resource generation request;

[0034] Determining the number of context tags included in the context tag sequence of the data resource generation request, and determining whether it is necessary to perform text block processing on the task context information according to the number of tags;

[0035] When it is determined that the task context information needs to be processed by text block, the task context information is processed by block to obtain a context block result, and the context block result and the associated primary key are associated and stored to obtain the key value storage result;

[0036] When it is determined that the task context information does not need to be processed into text blocks, the task context information and the associated primary key are associated and stored to obtain the key-value storage result.

[0037] In an exemplary embodiment of the present disclosure, the target data resource is obtained in the following manner:

[0038] Identifying a context marker sequence included in the speculative sampling result to obtain a target context marker corresponding to the speculative sampling result;

[0039] Based on the target context flag and the key-value storage result, determine the task context information or context block result required to generate the speculative sampling result;

[0040] The accuracy of the speculative sampling result is verified based on the task context information or the context segmentation result through a preset task reasoning big model to obtain an accuracy verification result, and the target data resource is determined from the speculative sampling result according to the accuracy verification result.

[0041] In an exemplary embodiment of the present disclosure, the target speculative sampling model is obtained in the following manner:

[0042] The number of layers of the encoding module and the number of layers of the decoding module included in the preset task reasoning big model are trimmed, and a target speculative sampling model is generated based on the task reasoning big model after the number of layers is trimmed.

[0043] According to one aspect of the present disclosure, there is provided a device for generating data resources, comprising:

[0044] A target speculative sampling model matching module is used to receive a data resource generation request reported by a user terminal, and match a target speculative sampling model for the data resource generation request based on a preset service gating network model;

[0045] A speculative sampling result determination module, configured to perform speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result;

[0046] A key-value storage result construction module, used to construct a key-value storage result corresponding to the data resource generation request, and push the key-value storage result and the speculative sampling result to the cloud server;

[0047] The target data resource generation module is used to receive the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feed back the target data resource to the user terminal.

[0048] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for generating data resources described in any one of the above is implemented.

[0049] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0050] Processor; and

[0051] A memory, configured to store executable instructions of the processor;

[0052] The processor is configured to execute any one of the above-mentioned methods for generating data resources by executing the executable instructions.

[0053] A method for generating data resources provided by an embodiment of the present disclosure includes, on the one hand, receiving a data resource generation request reported by a user terminal, and matching a target speculative sampling model for the data resource generation request based on a preset business gating network model; then speculative sampling is performed on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result; a key-value storage result corresponding to the data resource generation request is constructed, and the key-value storage result and the speculative sampling result are pushed to a cloud server; finally, the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feeds back the target data resource to the user terminal. Since the target speculative sampling model can be matched for the data resource generation request, and a specific data prediction process is performed in the target speculative sampling model, and only the verification process needs to be performed on the cloud server side, the problem of low data resource generation efficiency due to the inability to perform targeted policy diversion according to business scenarios in the prior art is solved, and the data resource generation efficiency can be improved on the basis of reducing the computing burden on the cloud server side; on the other hand, the problem of data resource generation failure caused by the loss and timeout of many requests during concurrent peaks can also be avoided.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0056] Figure 1 The following schematically shows a flow chart of a method for generating data resources according to an exemplary embodiment of the present disclosure.

[0057] Figure 2 A diagram schematically illustrates a structural example of a service gating network model according to an exemplary embodiment of the present disclosure.

[0058] Figure 3 The following schematically shows an example structure diagram of a service gating network layer according to an example embodiment of the present disclosure.

[0059] Figure 4 An example diagram of a speculative sampling model according to an example embodiment of the present disclosure is schematically shown.

[0060] Figure 5 An example diagram of a feature encoding module according to an example embodiment of the present disclosure is schematically shown.

[0061] Figure 6 The following schematically shows an example architecture diagram of a system for generating data resources according to an example embodiment of the present disclosure.

[0062] Figure 7 The structure block diagram schematically shows a device for generating data resources according to an exemplary embodiment of the present disclosure.

[0063] Figure 8 An electronic device for implementing a method for generating data resources according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0064] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, the known technical solutions are not shown or described in detail to avoid obscuring the various aspects of the present disclosure.

[0065] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0066] Currently, AIGC (Artificial Intelligence Generated Content) is developing rapidly, and the forms of large models are diverse. For example, AIGC can already generate videos and pictures from the initial text, and the generation capabilities of large models are also under continuous development. With the development of AIGC, all walks of life are using AIGC to improve productivity and work efficiency. In this scenario, the field of automatic question answering has always been the forefront of large model applications. However, the real-time, accuracy, and resource usage costs of the content generated during the automatic question answering process have always been typical problems that plague the large-scale application of large models.

[0067] In actual applications, large models consume a lot of memory resources and generate content slowly due to their large size. To solve this problem, targeted diversion strategies can be formulated based on different user requests, multimodal data types, and requested text lengths. Traffic distribution and calculations can be performed based on the data center hardware resource type and cloud-edge computing architecture to achieve distributed high-speed reasoning of large models.

[0068] In some related technical solutions, large-model distributed high-speed reasoning can be achieved in the following way: the large-model reasoning is divided into two stages: Prefill and Decoding; however, this solution can only achieve parallel computing in the Prefill stage, and a single Token still needs to be generated in a sequential manner in the Decoding stage; and large-model reasoning cannot fully utilize hardware resources such as GPU (Graphics Processing Unit), resulting in a lot of time wasted in the Prefill stage; further, large-model reasoning cannot perform targeted strategy diversion according to business scenarios, resulting in the loss and timeout of many requests during peak concurrency.

[0069] Based on this, this exemplary embodiment first provides a method for generating data resources, which can be run on a server, a server cluster or a cloud server, etc. Of course, those skilled in the art can also run the method disclosed in this disclosure on other platforms as required, and this exemplary embodiment does not specifically limit this. Figure 1 As shown, the method for generating data resources described in the exemplary embodiment of the present disclosure may include the following steps:

[0070] Step S110: receiving a data resource generation request reported by a user terminal, and matching a target speculative sampling model for the data resource generation request based on a preset service gating network model;

[0071] Step S120. Perform speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result;

[0072] Step S130: construct a key-value storage result corresponding to the data resource generation request, and push the key-value storage result and the speculative sampling result to the cloud server;

[0073] Step S140: Receive the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feed back the target data resource to the user terminal.

[0074] In the above-mentioned data resource generation method, on the one hand, by receiving the data resource generation request reported by the user terminal, and matching the target speculative sampling model for the data resource generation request based on the preset business gating network model; then speculative sampling is performed on the data resource generation request based on the target speculative sampling model to obtain the speculative sampling result; then the key-value storage result corresponding to the data resource generation request is constructed, and the key-value storage result and the speculative sampling result are pushed to the cloud server; finally, the cloud server verifies the speculative sampling result according to the key-value storage result to obtain the target data resource, and feeds back the target data resource to the user terminal. Since the target speculative sampling model can be matched for the data resource generation request, and the specific data prediction process is performed in the target speculative sampling model, and only the verification process needs to be performed on the cloud server side, the problem of low data resource generation efficiency in the prior art due to the inability to perform targeted policy diversion according to the business scenario is solved, and the data resource generation efficiency can be improved on the basis of reducing the computing burden on the cloud server side; on the other hand, the problem of data resource generation failure caused by the loss and timeout of many requests during concurrency peak can also be avoided.

[0075] Hereinafter, the method for generating data resources described in the exemplary embodiments of the present disclosure will be further explained and illustrated in conjunction with the accompanying drawings.

[0076] First, the terms involved in the exemplary embodiments of the present disclosure are explained and illustrated.

[0077] Prefill: The first stage of large model inference. In the early prefill stage of Prompt, operations such as operator matrix multiplication are performed on the user's input to generate the corresponding KV (Key Value) and cache it.

[0078] Decoding: The second stage of large model inference is to decode the KV cache generated in the Prefill stage. Each decoding process generates a character.

[0079] Secondly, the technical implementation principle of the example embodiment of the present disclosure is explained and illustrated. Specifically, in the process of actual application, on the one hand, due to the large number of parameters and large size of dense large models, in the process of actual implementation, there are problems such as high requirements for computing resources, slow reasoning and calculation speed of large models, low accuracy, and a large amount of time required for data transmission and exchange; on the other hand, in the actual training and reasoning calculation of large models under the traditional reasoning architecture, the actual training and reasoning calculation of large models still rely on the entire Transformer network structure of the large model. All weights and neural units will participate in each calculation process, and there is redundancy and waste in the occupation and use of video memory; on the other hand, the multimodal data large model has problems such as high resource consumption, long duration, and low efficiency of data interaction communication in the reasoning process; therefore, the example embodiment of the present disclosure provides a large model distributed reasoning acceleration solution based on policy diversion, which divides the reasoning process of the large model into Prefill and Decoding. The test results in the actual application process show that the Prefill stage mainly relies on a large amount of computing resources and is a computing-intensive task; the Decoding stage is mainly frequent KV cache interaction, which is a memory-intensive task; therefore, in the diversion strategy of the business scenario, the user's multimodal data can be embedded based on the multimodal gating routing of the business, and the computing resources that best match the data type can be selected through business gating (that is, matching the target speculative sampling model); further, for the long text request input by the user (that is, the text to be processed), it can be filtered according to the text length gating mechanism, and the user request can be distributed to the corresponding long text Prefill logic processing module, the long text is chunked, and the corresponding KV cache is generated in a pipeline parallel manner, which can greatly improve the processing time of the long text. At the same time, based on the scheme recorded in the example embodiments of the present disclosure, the type of multimodal data, the length of user requested data, and business scenarios can also be gated and distributed; at the same time, a speculative sampling reasoning scheme based on a cloud-edge heterogeneous resource network architecture is proposed. Based on this scheme, various resources can be used in a targeted manner to perform distributed parallel reasoning on different data, greatly improving the reasoning speed.

[0080] Further, the preset service gating network model involved in the exemplary embodiment of the present disclosure is explained and illustrated. Figure 2 As shown, the service gating network model may include a first input layer 210, an embedding mapping layer 220, a coding layer 230, a service gating network layer 240 and a first output layer 250; further, refer to Figure 3As shown, the service gating network layer recorded here may include a gating network model and a candidate speculative sampling model. At the same time, the specific functions that each module needs to perform in the actual application process will be described in detail later, and no further details will be given here.

[0081] In an exemplary embodiment, referring to Figure 4 As shown, the speculative sampling model described in the exemplary embodiment of the present disclosure may include a second input layer 410, a first word embedding layer 420, a position encoding module 430, a feature encoding module 440, a feature decoding module 450, a linear transformation layer 460, a Softmax layer 470, and a second output layer 480; wherein the specific functions of each module will be introduced later, and will not be listed one by one here. Further, refer to Figure 5 As shown, the feature encoding module 440 recorded here may include a first multi-head self-attention module 501, a first residual connection and normalization module 502, a first feedforward neural network 503, and a second residual connection and normalization module 504; wherein, the specific functions of each module will be introduced later and will not be listed one by one here.

[0082] It should be noted that the speculative sampling model described in the exemplary embodiment of the present disclosure is obtained based on the task reasoning model. In the actual application process, the number of layers of the encoding module and the number of layers of the decoding module included in the preset task reasoning model can be trimmed, and the target speculative sampling model (or candidate speculative sampling model) is generated based on the task reasoning model with the number of layers trimmed. In other words, the speculative sampling model is actually a small-scale task reasoning model. Based on this method, the efficiency of generating speculative sampling results can be further improved.

[0083] The following will explain and illustrate the data resource generation system involved in the exemplary embodiment of the present disclosure. Figure 6 As shown, the data resource generation system may include a user terminal 610, an edge server 620, and a cloud server 630. The user terminal may be connected to the edge server through a wired network or a wireless network, and the edge server may be connected to the cloud server through a local area network. In the actual application process, the user terminal is used to report a data resource generation request to the edge server, the edge server is used to implement the data resource generation method described in the exemplary embodiment of the present disclosure, and the cloud server is used to verify the speculative sampling results to determine the target data resource from the speculative sampling results.

[0084] Will combine Figure 2-Figure 6 right Figure 1 The generation method of the data resources shown in the figure is further explained and illustrated. Specifically:

[0085] In step S110, a data resource generation request reported by a user terminal is received, and a target speculative sampling model is matched for the data resource generation request based on a preset service gating network model.

[0086] In this example embodiment, first, a data resource generation request reported by a user terminal is received; specifically, the data resources recorded here may include but are not limited to text resources, video resources or image resources, etc.; at the same time, when generating a data resource generation request, the user will enter the corresponding text information as the text to be processed in the form of voice or text according to the requirements of the data resources to be generated, and of course, the corresponding business request type needs to be entered; when the terminal receives the information entered by the user, it can determine the business request identifier and generate a data resource generation request based on the business request identifier, the text to be processed and the business request type, and report it to the user terminal.

[0087] Secondly, after the user terminal receives the data resource generation request, it can match the target speculative sampling model for the data resource generation request based on the preset business gating network model; specifically, it can be achieved in the following ways: parsing the data resource generation request to obtain the business request identifier, business request type, task computing resources required to execute the data resource generation task, and the text to be processed; generating task basic information according to the business request identifier, business request information and task computing resources, and generating task context information according to the text to be processed; performing word embedding processing on the task basic information based on the embedding mapping layer to obtain the first task feature, and performing embedding mapping processing on the task context information based on the embedding mapping layer to obtain the context flag sequence; encoding the first task feature and the context flag sequence based on the encoding layer to obtain the overall context representation, and predicting the overall context representation based on the business gating network layer to obtain the target speculative sampling model. Specifically, the embedding mapping layer recorded here may include an Embedding embedding mapping layer and a Bert embedding mapping layer. In actual application, the basic task information can be word embedded based on the Embedding embedding mapping layer to obtain the first task feature, and then the task context information can be embedded based on the Bert embedding mapping layer to obtain a context marker sequence; further, the encoding layer recorded here can be a multi-layer bidirectional Transformer, and of course it can also be other structures, and this example does not impose any special restrictions on this.

[0088] In an example embodiment, the target speculative sampling model is obtained by predicting the overall context representation based on the business gating network layer, which can be achieved in the following manner: based on the gating network model and according to the overall context representation, the first model weight of the candidate speculative sampling model when executing the data resource generation task is determined, and the target speculative sampling model is determined from the candidate speculative sampling models based on the first model weight; specifically, in the actual application process, after obtaining the first model weight, the candidate speculative sampling model with the largest weight value can be selected from the first model weight as the target speculative sampling model. It should be noted here that the candidate speculative sampling models recorded here are set in parallel in the edge server in the form of chips; in the process of predicting the speculative sampling results, each chip will report its own model attribute information and computing power resource information. In the specific matching process, the target speculative sampling model can be matched directly based on the data resource generation requirements and the attribute information of each chip.

[0089] In step S120, speculative sampling processing is performed on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result.

[0090] Specifically, the specific prediction process of the speculative sampling result is: based on the first embedding mapping layer, embedding and mapping processing is performed on the task context information of the data resource generation request to obtain a first word vector, and based on the position encoding module, encoding processing is performed on the task context information to obtain a first position vector; the first word vector and the first position vector are superimposed to obtain a first input vector, and the first input vector is encoded based on the feature encoding module to obtain a first encoding result; the first encoding result is decoded based on the feature decoding module to obtain a first encoding matrix, and the first encoding matrix is ​​converted based on the linear transformation layer to obtain a first logical matrix; the first logical matrix is ​​mapped based on the Softmax layer to obtain a speculative sampling result corresponding to the first audio data to be compared.

[0091] In an exemplary embodiment, encoding the first input vector based on the feature encoding module to obtain a first encoding result can be achieved in the following manner: calculating the first attention mechanism of the first input vector based on the first multi-head self-attention module, and performing residual connection and normalization processing on the first attention mechanism based on the first residual connection and normalization module to obtain a first normalization processing result; linearizing the first normalization processing result based on the first feedforward neural network to obtain a first linearization processing result, and performing residual connection and normalization processing on the first linearization processing result based on the second residual connection and normalization module to obtain a first encoding result.

[0092] In an exemplary embodiment, a first attention mechanism for calculating a first input vector based on a first multi-head self-attention module can be implemented in the following manner: linearly fuse the first input vector to obtain a first query vector, a first key vector and a first value vector, and calculate the first outer product vector of the first query vector and the first key vector to obtain a first similarity between the first query vector and the first key vector; normalize the first similarity to obtain a first weight matrix, and calculate the second outer product vector of the first weight matrix and the first value vector to obtain the first attention mechanism.

[0093] It should be noted here that, in the process of determining the speculative sampling results, multiple tasks can be performed based on the same chip, or the same task can be performed based on multiple chips; in the actual application process, it can be configured according to actual needs, and this example does not impose any special restrictions on this.

[0094] In step S130, a key-value storage result corresponding to the data resource generation request is constructed, and the key-value storage result and the speculative sampling result are pushed to the cloud server.

[0095] In this example embodiment, first, a key-value storage result corresponding to a data resource generation request is constructed; specifically, this can be achieved in the following manner: a business request identifier, business request information, and task computing resources of the data resource generation request are hashed to obtain an associated primary key corresponding to the data resource generation request; the number of context flags included in the context flag sequence of the data resource generation request is determined, and based on the number of flags, it is determined whether text segmentation processing of the task context information is required; when it is determined that text segmentation processing of the task context information is required, the task context information is segmented to obtain a context segmentation result, and the context segmentation result and the associated primary key are associated and stored to obtain the key-value storage result; when it is determined that text segmentation processing of the task context information is not required, the task context information and the associated primary key are associated and stored to obtain the key-value storage result. That is, in the actual application process, the business request identifier res_id + business request type busi_id + task computing resources can be hashed, and identified and diverted according to the business request type and business request identifier res_id, and the same user request is distributed to the same computing resource in the cloud service cluster, so as to ensure that the KV cache in the prefill stage is reused by the same user. Among them, the specific calculation process of the associated primary key can be implemented by the following formula (1):

[0096] G j (X i)=Hash(res_id+busi_id+task computing resource); Formula (1)

[0097] Among them, G j (X i ) represents the associated primary key of the i-th data resource generation request of the j-th user.

[0098] Furthermore, after obtaining the key-value storage result, the key-value storage result and the speculative sampling result can be pushed to the cloud server, so that the cloud server verifies the speculative sampling result according to the key-value storage result, and then determines the target data resource.

[0099] In step S140, the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result is received, and the target data resource is fed back to the user terminal.

[0100] Specifically, after the server receives the key-value storage result and the speculative sampling result, the speculative sampling result can be verified; wherein the specific verification process can be implemented in the following manner: identifying the context flag sequence included in the speculative sampling result to obtain the target context flag corresponding to the speculative sampling result; determining the task context information or context block result required to generate the speculative sampling result based on the target context flag and the key-value storage result; verifying the accuracy of the speculative sampling result based on the task context information or context block result through a preset task reasoning large model to obtain an accuracy verification result, and determining the target data resource from the speculative sampling result according to the accuracy verification result. That is, in the actual application process, the multimodal data can be speculatively sampled by deploying a speculative small model trained based on the same data on the edge side, and the sampled data can be sent to the cloud side large model for reasoning verification, and the Token with a high probability is adopted, and the Token with a low probability can be returned to the edge server for regeneration; based on this method, the reasoning speed can be greatly improved.

[0101] It should also be noted here that, in the process of verifying the accuracy of the speculative sampling results based on the task context information or context segmentation results through the preset task reasoning big model to obtain the accuracy verification results, its specific implementation process is consistent with the specific determination process of the speculative sampling results, and no further details will be given here.

[0102] So far, the data resource generation method recorded in the example embodiment of the present disclosure has been fully realized. Based on the above-mentioned contents, it can be known that the data resource generation method recorded in the example embodiment of the present disclosure, on the one hand, by adopting the strategy-based large model distributed inference acceleration technology, can generate data resources in the current proprietary computer room distribution, thereby achieving the purpose of saving physical hardware costs and accelerating the industry application of large models; on the other hand, the example embodiment of the present disclosure proposes a Transformer gating network based on multimodal data types to divert large model inference applications in image and video recognition business scenarios, and for Data Type is used to control the gating function, which can aggregate different multimodal business systems and reduce the cost of system construction. On the other hand, the example embodiment of the present disclosure innovatively applies the large model distributed reasoning acceleration technology based on policy diversion, and distributes multimodal data to different heterogeneous resources for calculation through the business data type gating module, thereby improving the reasoning speed of the large model. At the same time, the cloud-edge heterogeneous computing power architecture can also be adopted, and a small model trained with the same data can be deployed on the edge to perform speculative sampling reasoning on user input, and the speculative sampling data can be reasoned, verified and adopted on the cloud side. Furthermore, the example embodiment of the present disclosure is based on a large model distributed reasoning acceleration framework based on policy diversion, The user request is identified based on the res_id hash, which can ensure that the same user request is calculated using the same server resources, and that the KVCache is calculated once and reused. In addition, the inference acceleration module based on the long text strategy diversion can be used to identify the processed text in chunks, and the long text blocks can be distributedly calculated in a pipeline parallel manner. The logical ID is used to save each chunk KVCache in the CPU memory database, so that the cloud server can obtain the corresponding text from the database to verify the speculative sampling results, thereby achieving the purpose of improving the verification efficiency of the speculative sampling results.

[0103] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0104] The exemplary embodiment of the present disclosure also provides a device for generating data resources. Specifically, refer to Figure 7 As shown, the data resource generation device may include a target speculative sampling model matching module 710, a speculative sampling result determination module 720, a key-value storage result construction module 730, and a target data resource generation module 740. Among them:

[0105] The target speculative sampling model matching module 710 may be used to receive a data resource generation request reported by a user terminal, and match a target speculative sampling model for the data resource generation request based on a preset service gating network model;

[0106] The speculative sampling result determination module 720 may be used to perform speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result;

[0107] The key-value storage result construction module 730 may be used to construct a key-value storage result corresponding to the data resource generation request, and push the key-value storage result and the speculative sampling result to the cloud server;

[0108] The target data resource generation module 740 may be configured to receive the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feed back the target data resource to the user terminal.

[0109] In an exemplary embodiment of the present disclosure, the preset business gating network model includes an embedding mapping layer, an encoding layer and a business gating network layer; wherein, matching the target speculative sampling model for the data resource generation request based on the preset business gating network model includes: parsing the data resource generation request to obtain a business request identifier, a business request type, task computing resources required to execute the data resource generation task and a text to be processed; generating task basic information according to the business request identifier, business request information and task computing resources, and generating task context information according to the text to be processed; performing word embedding processing on the task basic information based on the embedding mapping layer to obtain a first task feature, and performing embedding mapping processing on the task context information based on the embedding mapping layer to obtain a context flag sequence; encoding the first task feature and the context flag sequence based on the encoding layer to obtain a context overall representation, and predicting the context overall representation based on the business gating network layer to obtain the target speculative sampling model.

[0110] In an exemplary embodiment of the present disclosure, the business gating network layer includes a gating network model and a candidate speculative sampling model; wherein, the context overall representation is predicted based on the business gating network layer to obtain the target speculative sampling model, including: based on the gating network model and the context overall representation, determining the first model weight of the candidate speculative sampling model when performing a data resource generation task, and determining the target speculative sampling model from the candidate speculative sampling models based on the first model weight.

[0111] In an exemplary embodiment of the present disclosure, the target speculative sampling model includes a first embedding mapping layer, a position encoding module, a feature encoding module, a feature decoding module, a linear transformation layer and a Softmax layer; wherein, based on the target speculative sampling model, the data resource generation request is speculatively sampled to obtain a speculative sampling result, including: embedding and mapping the task context information of the data resource generation request based on the first embedding mapping layer to obtain a first word vector, and encoding the task context information based on the position encoding module to obtain a first position vector; superimposing the first word vector and the first position vector to obtain a first input vector, and encoding the first input vector based on the feature encoding module to obtain a first encoding result; decoding the first encoding result based on the feature decoding module to obtain a first encoding matrix, and converting the first encoding matrix based on the linear transformation layer to obtain a first logical matrix; mapping the first logical matrix based on the Softmax layer to obtain a speculative sampling result corresponding to the first audio data to be compared.

[0112] In an exemplary embodiment of the present disclosure, the feature encoding module includes a first multi-head self-attention module, a first residual connection and normalization module, a first feedforward neural network, and a second residual connection and normalization module; wherein, encoding the first input vector based on the feature encoding module to obtain a first encoding result includes: calculating the first attention mechanism of the first input vector based on the first multi-head self-attention module, and performing residual connection and normalization processing on the first attention mechanism based on the first residual connection and normalization module to obtain a first normalization processing result; linearizing the first normalization processing result based on the first feedforward neural network to obtain a first linearization processing result, and performing residual connection and normalization processing on the first linearization processing result based on the second residual connection and normalization module to obtain a first encoding result.

[0113] In an exemplary embodiment of the present disclosure, a first attention mechanism for calculating the first input vector is calculated based on a first multi-head self-attention module, including: linearly fusing the first input vector to obtain a first query vector, a first key vector and a first value vector, and calculating a first outer product vector of the first query vector and the first key vector to obtain a first similarity between the first query vector and the first key vector; normalizing the first similarity to obtain a first weight matrix, and calculating a second outer product vector of the first weight matrix and the first value vector to obtain the first attention mechanism.

[0114] In an exemplary embodiment of the present disclosure, a key-value storage result corresponding to the data resource generation request is constructed, including: performing hash processing on the business request identifier, business request information and task computing resources of the data resource generation request to obtain an associated primary key corresponding to the data resource generation request; determining the number of context flags included in the context flag sequence of the data resource generation request, and determining whether it is necessary to perform text segmentation processing on the task context information according to the number of flags; when it is determined that text segmentation processing is required for the task context information, segmenting the task context information to obtain a context segmentation result, and associatively storing the context segmentation result and the associated primary key to obtain the key-value storage result; when it is determined that text segmentation processing is not required for the task context information, associatively storing the task context information and the associated primary key to obtain the key-value storage result.

[0115] In an exemplary embodiment of the present disclosure, the target data resource is obtained in the following manner: identifying the context flag sequence included in the speculative sampling result to obtain the target context flag corresponding to the speculative sampling result; determining the task context information or context block result required to generate the speculative sampling result based on the target context flag and the key-value storage result; verifying the accuracy of the speculative sampling result based on the task context information or context block result through a preset task reasoning big model to obtain an accuracy verification result, and determining the target data resource from the speculative sampling result according to the accuracy verification result.

[0116] In an exemplary embodiment of the present disclosure, the target speculative sampling model is obtained in the following manner: the number of layers of the encoding module and the number of layers of the decoding module included in the preset task reasoning big model are trimmed, and the target speculative sampling model is generated based on the task reasoning big model with the trimmed number of layers.

[0117] The specific details of each module in the above-mentioned data resource generation device have been described in detail in the corresponding data resource generation method, so they will not be repeated here.

[0118] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0119] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0120] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. A person skilled in the art will appreciate that various aspects of the present disclosure may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a circuit, module, or system.

[0121] Refer to the following Figure 8 The electronic device 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0123] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform the following steps: Figure 1 The step S110 shown in the figure is: receiving a data resource generation request reported by a user terminal, and matching a target speculative sampling model for the data resource generation request based on a preset business gating network model; step S120: performing speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result; step S130: constructing a key-value storage result corresponding to the data resource generation request, and pushing the key-value storage result and the speculative sampling result to a cloud server; step S140: receiving a target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feeding back the target data resource to the user terminal.

[0124] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0125] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0126] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0127] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0128] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0129] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to perform the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification. A program product for implementing the above-mentioned method according to an implementation of the present disclosure is described, which may adopt a portable compact disk read-only memory (CD-ROM) and include a program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0130] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] Computer readable signal media may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0132] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0133] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0134] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the inventions invented herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not invented by the present disclosure. The specification and examples are to be considered merely exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for generating data resources, characterized in that: include: Receiving a data resource generation request reported by a user terminal, and matching a target speculative sampling model for the data resource generation request based on a preset service gating network model; Performing speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result; Constructing a key-value storage result corresponding to the data resource generation request, and pushing the key-value storage result and the speculative sampling result to a cloud server; Receive the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feed back the target data resource to the user terminal.

2. The method for generating data resources according to claim 1, characterized in that: The preset service gating network model includes an embedding mapping layer, a coding layer and a service gating network layer; Wherein, generating a request matching target speculative sampling model for the data resource based on a preset business gating network model includes: Parsing the data resource generation request to obtain a business request identifier, a business request type, task computing resources required to execute the data resource generation task, and text to be processed; Generate task basic information according to the business request identifier, business request information and task computing resources, and generate task context information according to the text to be processed; Based on the embedding mapping layer, word embedding processing is performed on the task basic information to obtain a first task feature, and based on the embedding mapping layer, embedding mapping processing is performed on the task context information to obtain a context marker sequence; The first task feature and the context flag sequence are encoded based on the encoding layer to obtain an overall context representation, and the overall context representation is predicted based on the business gating network layer to obtain the target speculative sampling model.

3. The method for generating data resources according to claim 2, characterized in that: The service gating network layer includes a gating network model and a candidate speculative sampling model; The process of predicting the overall context representation based on the service gating network layer to obtain the target speculative sampling model includes: Based on the gating network model and according to the overall context representation, a first model weight of the candidate speculative sampling model when performing a data resource generation task is determined, and a target speculative sampling model is determined from the candidate speculative sampling models based on the first model weight.

4. The method for generating data resources according to claim 2, characterized in that: The target speculative sampling model includes a first embedding mapping layer, a position encoding module, a feature encoding module, a feature decoding module, a linear transformation layer and a Softmax layer; Wherein, performing speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result includes: Based on the first embedding mapping layer, embedding mapping is performed on the task context information of the data resource generation request to obtain a first word vector, and based on the position encoding module, encoding is performed on the task context information to obtain a first position vector; Superimposing the first word vector and the first position vector to obtain a first input vector, and encoding the first input vector based on a feature encoding module to obtain a first encoding result; Decoding the first encoding result based on a feature decoding module to obtain a first encoding matrix, and transforming the first encoding matrix based on a linear transformation layer to obtain a first logic matrix; The first logic matrix is ​​mapped based on the Softmax layer to obtain a speculative sampling result corresponding to the first audio data to be compared.

5. The method for generating data resources according to claim 4, characterized in that: The feature encoding module includes a first multi-head self-attention module, a first residual connection and normalization module, a first feedforward neural network, and a second residual connection and normalization module; The first input vector is encoded based on the feature encoding module to obtain a first encoding result, including: Calculating a first attention mechanism of the first input vector based on a first multi-head self-attention module, and performing residual connection and normalization processing on the first attention mechanism based on a first residual connection and normalization module to obtain a first normalization processing result; The first normalization processing result is linearized based on the first feedforward neural network to obtain a first linearization processing result, and the first linearization processing result is residually connected and normalized based on the second residual connection and normalization module to obtain a first encoding result.

6. The method for generating data resources according to claim 5, characterized in that: Calculating a first attention mechanism for the first input vector based on a first multi-head self-attention module includes: Performing linear fusion on the first input vector to obtain a first query vector, a first key vector, and a first value vector, and calculating a first outer product vector of the first query vector and the first key vector to obtain a first similarity between the first query vector and the first key vector; The first similarity is normalized to obtain a first weight matrix, and a second outer product vector of the first weight matrix and the first value vector is calculated to obtain the first attention mechanism.

7. The method for generating data resources according to claim 2, characterized in that: Constructing a key-value storage result corresponding to the data resource generation request, including: Performing hash processing on the business request identifier, business request information, and task computing resources of the data resource generation request to obtain an associated primary key corresponding to the data resource generation request; Determining the number of context tags included in the context tag sequence of the data resource generation request, and determining whether it is necessary to perform text block processing on the task context information according to the number of tags; When it is determined that the task context information needs to be processed by text block, the task context information is processed by block to obtain a context block result, and the context block result and the associated primary key are associated and stored to obtain the key value storage result; When it is determined that the task context information does not need to be processed into text blocks, the task context information and the associated primary key are associated and stored to obtain the key-value storage result.

8. The method for generating data resources according to claim 2, characterized in that: The target data resource is obtained in the following manner: Identifying a context marker sequence included in the speculative sampling result to obtain a target context marker corresponding to the speculative sampling result; Based on the target context flag and the key-value storage result, determine the task context information or context block result required to generate the speculative sampling result; The accuracy of the speculative sampling result is verified based on the task context information or the context segmentation result through a preset task reasoning big model to obtain an accuracy verification result, and the target data resource is determined from the speculative sampling result according to the accuracy verification result.

9. The method for generating data resources according to claim 8, characterized in that: The target speculative sampling model is obtained as follows: The number of layers of the encoding module and the number of layers of the decoding module included in the preset task reasoning big model are trimmed, and a target speculative sampling model is generated based on the task reasoning big model after the number of layers is trimmed.

10. A data resource generation device, characterized in that: include: A target speculative sampling model matching module is used to receive a data resource generation request reported by a user terminal, and match a target speculative sampling model for the data resource generation request based on a preset service gating network model; A speculative sampling result determination module, configured to perform speculative sampling processing on the data resource generation request based on the target speculative sampling model to obtain a speculative sampling result; A key-value storage result construction module, used to construct a key-value storage result corresponding to the data resource generation request, and push the key-value storage result and the speculative sampling result to the cloud server; The target data resource generation module is used to receive the target data resource obtained after the cloud server verifies the speculative sampling result according to the key-value storage result, and feed back the target data resource to the user terminal.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating data resources described in any one of claims 1 to 9 is implemented.

12. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the method for generating data resources according to any one of claims 1 to 9 by executing the executable instructions.

Images