Content generation methods, apparatus, storage media and electronic devices

By integrating feature extraction layers and decoding networks into a large language model, the problem of low acceptance rate of candidate word segmentation generated by the draft model is solved, thus improving the efficiency of content generation.

CN123070102APending Publication Date: 2026-11-10TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510588702.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-08
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

The low content generation efficiency of existing large language models is mainly due to the low acceptance rate of candidate word segments generated by the draft model, which leads to low inference efficiency of large language models.

Method used

By obtaining the target word segment and its corresponding first predicted feature predicted by the preset large language model during the content generation process, the feature extraction layer is called to extract features and fuse them with the first predicted feature. The preset decoding network is used to predict features, generate and verify the predicted word segment, and finally generate the inference content.

Benefits of technology

It improves the content generation efficiency of large language models, provides more prior knowledge by fusing features, and increases the acceptance rate of candidate word segments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070102A_ABST
    Figure CN123070102A_ABST
Patent Text Reader

Abstract

This disclosure provides a content generation method, apparatus, storage medium, and electronic device. The method includes acquiring a target word segment predicted by a preset large language model and acquiring a first predicted feature corresponding to the target word segment. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. The method involves calling the feature extraction layer to extract features from the target word segment, obtaining embedded features, and fusing the embedded features and the first predicted feature to obtain fused features. A preset decoding network is used to predict features from the fused features to obtain a second predicted feature. The inference output layer is called to infer the second predicted feature to obtain a predicted word segment, and the predicted word segment is verified based on the preset large language model. Based on the verification result, inferred content is generated. This method can improve the content generation efficiency of the large language model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a content generation method, apparatus, storage medium, and electronic device. Background Technology

[0002] In recent years, the continuous development and progress of artificial intelligence technology has brought great convenience to people's lives. Among them, the application of large language models based on artificial intelligence technology is particularly widespread, such as in various chatbots, intelligent assistants, and various content generation products. The widespread application of large language models has improved people's experience in acquiring knowledge in multiple fields.

[0003] In related technologies, in order to improve the reasoning efficiency of large language models, a speculative sampling method is proposed. First, multiple candidate words are quickly generated through a draft model. Then, the correctness of these candidate words is verified in parallel through a large language model. Finally, reasoning content is generated based on the correct candidate words.

[0004] However, the current large language models have a low acceptance rate for candidate word segments generated by the draft model, resulting in low content generation efficiency for large language models. Summary of the Invention

[0005] This disclosure provides a content generation method, apparatus, storage medium, and electronic device, which can improve the content generation efficiency of large language models.

[0006] The first aspect of this disclosure provides a content generation method, including:

[0007] The target word predicted by the preset large language model during the content generation process is obtained, and the first predicted feature corresponding to the target word is obtained. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted feature is the feature output by the feature prediction layer.

[0008] The feature extraction layer is invoked to extract features from the target word segmentation to obtain embedded features. The embedded features and the first predicted features are then fused to obtain fused features.

[0009] Based on a preset decoding network, feature prediction is performed on the fused features to obtain the second predicted feature;

[0010] The inference output layer is called to infer the second predicted feature to obtain the predicted word segmentation, and the predicted word segmentation is verified based on the preset large language model. The inferred content is generated based on the verification result.

[0011] A second aspect of this disclosure provides a content generation apparatus, comprising:

[0012] The acquisition unit is used to acquire the target word segment predicted by the preset large language model during the content generation process and to acquire the first predicted feature corresponding to the target word segment. The preset large language model includes a feature extraction layer, a feature prediction layer and an inference output layer. The first predicted feature is the feature output by the feature prediction layer.

[0013] The fusion unit is used to call the feature extraction layer to extract features from the target word segmentation, obtain embedded features, and fuse the embedded features with the first predicted features to obtain fused features.

[0014] The prediction unit is used to predict the fused features based on a preset decoding network to obtain the second predicted features;

[0015] The generation unit is used to call the inference output layer to infer the second prediction feature to obtain the predicted word segmentation, and to verify the predicted word segmentation based on the preset large language model, and generate the inference content based on the verification result.

[0016] Optionally, in one feasible implementation, the fusion unit includes:

[0017] The first acquisition subunit is used to acquire the fusion weights corresponding to the embedded features and the first predicted features;

[0018] The fusion subunit is used to fuse the embedded features and the first predicted features according to the fusion weights to obtain the fused features.

[0019] Optionally, in one feasible implementation, the first acquisition subunit includes:

[0020] The acquisition module is used to acquire the content generation scenario corresponding to the target word segmentation predicted by the preset large language model, and to acquire the mapping relationship table between the scenario and the weight.

[0021] The first determining module is used to determine the fusion weights corresponding to the embedded features and the first predicted features from the mapping relationship table based on the content generation scenario.

[0022] Optionally, in one feasible implementation, the fusion unit includes:

[0023] The dimension alignment subunit is used to perform dimension alignment between the embedded features and the first predicted features;

[0024] The first splicing subunit is used to splice the dimension-aligned embedded features and the first predicted features to obtain the fused features.

[0025] Optionally, in one feasible implementation, the prediction unit further includes:

[0026] The update subunit is used to update the cache based on the key value corresponding to the preset decoding network according to the second prediction feature;

[0027] The generation unit includes:

[0028] The first inference subunit is used to call the inference output layer to infer the predicted feature sequence in the key-value pair cache to obtain the predicted word segmentation.

[0029] Optionally, in one feasible implementation, the generation unit includes:

[0030] The second acquisition subunit is used to acquire the first confidence level corresponding to the predicted word segmentation;

[0031] The second splicing subunit is used to splice together the predicted word segmentation, the target word segmentation, and the content corresponding to the target word segmentation to generate prompt words, thus obtaining the spliced ​​content.

[0032] The first determining subunit is used to call a preset large language model to reason about the concatenated content and determine the second confidence level corresponding to the predicted word segmentation based on the reasoning results;

[0033] The first generation subunit is used to determine the verification result of the predicted word segmentation based on the first confidence level and the second confidence level, and to generate inference content based on the verification result.

[0034] Optionally, in one feasible implementation, the generation unit includes:

[0035] The second inference subunit is used to call the inference output layer to infer the second prediction feature and obtain multiple predicted word segments;

[0036] The verification subunit is used to verify multiple predicted word segments based on a pre-defined large language model and obtain the verification results;

[0037] The second generation subunit is used to determine the effective word segment from multiple predicted word segments based on the verification result of each predicted word segment in multiple predicted word segments, and generate inference content based on the effective word segments.

[0038] Optionally, in one feasible implementation, the generation unit includes:

[0039] The third inference subunit is used to call the inference output layer to infer the predicted features and obtain the word segmentation probability distribution.

[0040] The second determining subunit is used to determine the predicted word segmentation based on the word segmentation probability distribution.

[0041] Optionally, in one feasible implementation, the second determining subunit includes:

[0042] The sorting module is used to sort words based on their probability distribution.

[0043] The second determination module is used to determine the first number of word segments that are ranked first as predicted word segments.

[0044] A third aspect of this disclosure provides a parameter update apparatus for a preset decoding network, comprising:

[0045] The acquisition unit is used to acquire sample word segments and predict the sample word segments based on a preset large language model to obtain the sample predicted word segments and the third prediction feature corresponding to the sample predicted word segments.

[0046] The fusion unit is used to call the feature extraction layer to extract features from the sample prediction word segmentation, obtain sample embedding features, and fuse the sample embedding features with the third prediction features to obtain sample fusion features.

[0047] The prediction unit is used to call the feature prediction layer to predict the embedded features of the sample and obtain the fourth predicted feature.

[0048] The update unit is used to perform feature prediction on the sample fusion features based on the preset decoding network to obtain the fifth predicted feature, and to update the parameters of the preset decoding network based on the difference between the fourth and fifth predicted features.

[0049] Optionally, in one feasible implementation, the updating unit includes:

[0050] The calculation subunit is used to calculate the loss value based on the fourth and fifth prediction features;

[0051] The update sub-unit is used to determine the gradient value based on the loss value and update the parameters of the preset decoding network based on the gradient value.

[0052] The fourth aspect of this disclosure provides a storage medium storing a computer program that, when executed by a processor, implements the content generation method as described in the first aspect.

[0053] The fifth aspect of this disclosure provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the content generation method as described in the first aspect.

[0054] The sixth aspect of this disclosure provides a computer program product comprising a computer program that is read and executed by a processor of an electronic device, causing the electronic device to perform the content generation method as described in the first aspect.

[0055] Therefore, the content generation method disclosed herein includes obtaining target word segments predicted by a preset large language model during the content generation process and obtaining the first predicted features corresponding to the target word segments. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted features are the features output by the feature prediction layer. The feature extraction layer is invoked to extract features from the target word segments to obtain embedded features, and the embedded features and the first predicted features are fused to obtain fused features. Based on a preset decoding network, the fused features are used to predict features to obtain second predicted features. The inference output layer is invoked to infer the second predicted features to obtain predicted word segments, and the predicted word segments are verified based on the preset large language model. Based on the verification results, inferred content is generated.

[0056] This method fuses the features predicted by the language model with the features of word segmentation inferred based on those features, then uses a pre-defined decoding network to predict the fusion result, and infers new word segments based on the prediction result for verification. Because the fusion of predicted features provides more prior knowledge, it improves the acceptance rate of the predicted candidate word segments, thereby improving the generation efficiency.

[0057] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0058] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0059] Figure 1a This is a schematic diagram of the speculative sampling model in an embodiment of this disclosure;

[0060] Figure 1b This is a schematic diagram of a prospective model in an embodiment of this disclosure;

[0061] Figure 1c This is a schematic diagram of the decoding model in an embodiment of this disclosure;

[0062] Figure 2 A system architecture diagram for the content generation method used in this embodiment of the disclosure;

[0063] Figure 3 This is a flowchart illustrating a content generation method in an embodiment of this disclosure;

[0064] Figure 4 This is a schematic diagram illustrating the method for determining the word sequence in multiple inferences in this embodiment of the present disclosure;

[0065] Figure 5 This is a tree structure of the word segmentation set in the embodiments of this disclosure;

[0066] Figure 6 This is another flowchart illustrating the content generation method in this disclosure embodiment;

[0067] Figure 7 This is an architecture diagram of the drafting token for the draft model in this embodiment of the disclosure;

[0068] Figure 8 This is a schematic diagram of the structure of a content generation device according to an embodiment of the present disclosure;

[0069] Figure 9 This is a terminal structure diagram of each method in one embodiment of this disclosure;

[0070] Figure 10 This is a server structure diagram of various methods in one embodiment of this disclosure. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0072] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0073] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0074] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0075] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0077] To facilitate understanding, the terms used in the embodiments of this application will be explained below.

[0078] Speculative sampling is a technique for accelerating inference in Large Language Models (LLMs). It achieves efficient text generation by combining a lightweight draft model and a large target model. The draft model quickly generates multiple candidate tokens, while the target model evaluates and verifies these candidate tokens, thereby reducing the number of sequence calls to the target model.

[0079] The target model can be a large, computationally accurate but slow model used in speculative sampling, responsible for generating high-quality output. The main function of the target model is to validate the candidate tokens generated by the draft model and decide whether to accept them, thus ensuring that the final generated text conforms to the target distribution.

[0080] A draft model can be a small language model that is computationally fast but relatively inaccurate. Its main function is to quickly generate multiple candidate tokens, which are then validated by the target model. It generates text through autoregression, but at a much faster speed than the target model. The output of a draft model typically has some uncertainty, but it can quickly explore multiple possibilities, providing the target model with a rich set of candidate options. This rapid generation characteristic allows draft models to significantly improve overall inference efficiency in speculative sampling.

[0081] The decoder is based on the previously generated word segmentation sequence or the given context, and then uses its internal hidden layer state and attention mechanism to regressively generate candidate sequences.

[0082] An embedding layer maps each word in the vocabulary to a point in a continuous vector space, capturing the semantic and syntactic features of the words. The embedding layer can represent each word as a fixed-dimensional vector.

[0083] Key-value cache (KV cache) is a technique used in the Transformer architecture to accelerate autoregressive generation. It stores previously computed keys and values, which are calculated during the self-attention mechanism. When generating new tokens, the model can reuse these stored key-value pairs instead of recompiling the entire attention matrix, significantly improving generation speed.

[0084] An n-gram is a simple language model that predicts the probability of the next word based on the preceding n-1 words. For example, a bigram predicts the next word based on the previous word, while a trigram predicts the next word based on the previous two words.

[0085] Memory bandwidth refers to the maximum rate at which data is transferred between the computer's memory and processor, typically measured in bytes per second (B / s). It is a critical performance metric because even with a fast processor, insufficient memory bandwidth and slow data transfer rates will limit overall computing performance.

[0086] The Language Modeling Head (LM Head) is an additional output layer in a Transformer model that transforms the model's hidden states into a probability distribution for each word in the vocabulary. This probability distribution represents the probability of each possible next word given the context. The LM Head typically contains a linear layer and a softmax function to generate the final prediction. During training, the output of the LM Head is compared with the actual next word, and the model's parameters are updated using a loss function.

[0087] To address the issue of high resource consumption in large language model inference, in the speculative sampling domain, multiple candidate word segments can be quickly generated using a draft model, and then the correctness of these candidate word segments can be verified in parallel using the large language model. This reduces the number of calls to the large language model and its computational burden. Please refer to [link / reference]. Figures 1a-1c The diagram shown illustrates speculative sampling. Figure 1a Region 101 illustrates the speculative sampling model. In this strategy, a draft model predicts the next token (t4) based on previous tokens (e.g., t1, t2, t3). Subsequently, the draft model again predicts the next token (t5) based on the token (t4) in the expanded sequence of predicted tokens (t1, t2, t3, t4). Each prediction can generate multiple candidate tokens, which can be verified in parallel by the large language model. Figure 1b Region 102 shows the lookahead model using a 2-gram Jacobi model to predict the next token. This method starts with a previous token (e.g., t3), predicts the next token (t4), and then predicts the subsequent token (t5) based on the newly generated token (t4). By progressively predicting the next token, the lookahead model ensures the consistency of the generated text in terms of syntax and context. It can validate multiple tokens through parallel computation, making full use of computing power and avoiding inference bottlenecks caused by limited memory bandwidth. However, the current draft model has a low acceptance rate for new tokens predicted based on prior tokens. Figure 1c Region 103 is the decoding (Medusa) model, which trains multiple Medusa heads to draft multiple tokens in parallel, as shown in the figure with two parallel output heads, Medusa Head1 and Medusa Head2. Both heads receive the same input feature f2 and then generate their respective outputs t4 and t5. A mechanism can then select the most suitable output, increasing the diversity and robustness of the generation process. However, since each Medusa head contains an LM Head layer, the number of parameters in the draft model will also be large if the vocabulary is large.

[0088] In view of the above-mentioned technical problems, this disclosure provides a content generation method in order to solve the technical problems.

[0089] System architecture and scenario description applied in the embodiments of this disclosure

[0090] Figure 2 This is a system architecture diagram applied to the content generation method according to embodiments of the present disclosure. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.

[0091] Terminal 140 includes various forms of devices with display screens, such as desktop computers, laptops, PDAs (personal digital assistants), mobile phones, in-vehicle terminals, home theater terminals, dedicated terminals, intelligent voice interaction devices, smart home appliances, or aircraft. Furthermore, it can be a single device or a collection of multiple devices. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.

[0092] Server 110 refers to a computer system that can provide certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines).

[0093] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.

[0094] The content generation method provided in this embodiment can be implemented alone in terminal 140, alone in server 110, or partially in terminal 140 and partially in server 110.

[0095] When the content generation method provided in this embodiment is implemented alone in terminal 140, terminal 140 obtains the target word predicted by the preset large language model during the content generation process and obtains the first predicted feature corresponding to the target word. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. Terminal 140 calls the feature extraction layer to extract features from the target word to obtain embedded features, and fuses the embedded features and the first predicted features to obtain fused features. Terminal 140 performs feature prediction on the fused features based on a preset decoding network to obtain second predicted features. Terminal 140 calls the inference output layer to infer the second predicted features to obtain predicted word segments, and verifies the predicted word segments based on the preset large language model, and generates inferred content based on the verification results.

[0096] When the content generation method provided in this embodiment is implemented solely in server 110, server 110 obtains the target word segment predicted by a preset large language model during the content generation process and obtains the first predicted feature corresponding to the target word segment. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. Server 110 calls the feature extraction layer to extract features from the target word segment to obtain embedded features, and fuses the embedded features and the first predicted features to obtain fused features. Server 110 performs feature prediction on the fused features based on a preset decoding network to obtain second predicted features. Server 110 calls the inference output layer to infer the second predicted features to obtain predicted word segments, and verifies the predicted word segments based on the preset large language model, and generates inferred content based on the verification results.

[0097] When the content generation method provided in this embodiment is partially implemented in the terminal 140 and partially implemented in the server 110, the terminal 140 obtains a content generation request and sends the content generation request to the server 110. The server 110 obtains the target word predicted by the preset large language model during the content generation process and obtains the first predicted feature corresponding to the target word according to the content generation request. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. The server 110 calls the feature extraction layer to extract features from the target word to obtain embedded features, and fuses the embedded features and the first predicted features to obtain fused features. The server 110 performs feature prediction on the fused features based on a preset decoding network to obtain a second predicted feature. The server 110 calls the inference output layer to infer the second predicted feature to obtain predicted words, and verifies the predicted words based on the preset large language model, and generates inferred content based on the verification results.

[0098] The electronic devices for content generation provided in this disclosure can be various types of terminal devices or servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal devices and servers can be directly or indirectly connected via wired or wireless communication, and this disclosure does not impose any limitations on this.

[0099] Taking servers as an example, such as server clusters deployed in the cloud, AI as a Service (AIaaS) is provided to objects. The AIaaS platform will break down several common AI services and provide independent or packaged services in the cloud. This service model is similar to an AI-themed marketplace. All objects can access and use one or more artificial intelligence services provided by the AIaaS platform through application programming interfaces.

[0100] This disclosure can be applied to various scenarios. For example, taking a medical question-answering model, language models in the medical field require a high degree of professionalism and accuracy, especially when dealing with complex medical terminology and logical reasoning. Predicted tokens generated by the draft model are often rejected if they do not conform to the distribution of the target model. This is because medical text needs to strictly adhere to medical logic and professional knowledge; any content that does not conform to medical common sense or clinical practice may lead to a low acceptance rate. The content generation method of this disclosure allows the draft model to combine predicted features with predicted word segmentation to predict new features. Based on the predicted features, more prior knowledge can be provided, improving the accuracy of new feature predictions and increasing the target model's acceptance rate of predicted word segmentation.

[0101] The above examples do not limit the scope of protection in this case.

[0102] General Description of Embodiments in this Disclosure

[0103] The content generation method provided in the embodiments of this disclosure will be described below with reference to the accompanying drawings. The execution subject of the content generation method described below is a terminal device, which can be implemented by the terminal device by running the various computer programs described above. Of course, based on the understanding of the following text, it is not difficult to see that the content generation method provided in the embodiments of this disclosure can also be implemented by the terminal device and the server in collaboration.

[0104] Please see Figure 3 ,like Figure 3 The diagram shown is a flowchart illustrating a content generation method provided in an embodiment of this disclosure. The method includes:

[0105] Step 301: Obtain the target word predicted by the preset large language model during the content generation process and obtain the first predicted feature corresponding to the target word.

[0106] A pre-defined large language model can be a large-scale language model trained on massive amounts of text data through unsupervised or semi-supervised learning. It adopts a Transformer architecture and efficiently processes sequential data through a self-attention mechanism, enabling it to generate natural and fluent text and understand complex language expressions. The pre-defined large language model can include a feature extraction layer, a feature prediction layer, and an inference output layer.

[0107] The feature extraction layer can be a neural network layer that extracts useful feature representations from the input text. It can consist of multiple neural networks, each of which encodes the input text through a self-attention mechanism. The output of each layer is a feature vector. These vectors can capture multi-dimensional information of the text, representing the semantic, syntactic and contextual information of the text. It can be a feature vector that converts the original text into a feature vector that the model can further process.

[0108] The feature prediction layer can be a neural network layer that predicts features related to a specific task based on the output of the feature extraction layer. These features can be used for tasks such as classification, regression, and generation. In other words, the feature prediction layer can predict relevant features based on the feature representation output by the feature extraction layer.

[0109] The inference output layer can be a neural network layer that generates the final prediction result based on the predicted features. Here, the predicted features output by the feature prediction layer can be converted into word representations.

[0110] Word segmentation refers to the basic unit after text is divided, also known as a lexical unit. The content generation process can be understood as a series of operations performed by a pre-defined large language model after receiving a content generation request. The target word segmentation can be the word representation generated by the inference output layer, or the first lexical unit predicted by the pre-defined large language model when generating text or making predictions based on the content generation request.

[0111] The first predicted feature can be the predicted feature that the inference output layer uses to generate the target word segmentation. That is, the feature prediction layer outputs the first predicted feature, and the inference output layer outputs the target word segmentation based on the first predicted feature.

[0112] Step 302: Call the feature extraction layer to extract features from the target word segmentation, obtain embedded features, and fuse the embedded features and the first predicted features to obtain fused features.

[0113] Embedded features can refer to continuous, fixed-dimensional vector representations transformed from discrete tokens or words. In other words, the feature extraction layer can convert the target word segmentation into fixed-dimensional vector representations; these vector representations are the embedded features. For example, assuming the feature extraction layer has a dimension of 768, the target word segmentation will be converted into a 768-dimensional vector.

[0114] Feature fusion refers to merging feature vectors from multiple sources to generate a richer and more expressive feature representation. Here, it could be a new feature generated by merging embedded features with the first predicted feature. Feature fusion methods can include adding different features with certain weights, concatenating different features into a longer vector, fusing different features through a small neural network, or dynamically selecting and fusing different features using an attention mechanism.

[0115] In one implementation, the embedded features and the first predicted features are fused to obtain fused features, including:

[0116] Obtain the fusion weights corresponding to the embedded features and the first predicted features;

[0117] The embedded features and the first predicted features are fused according to the fusion weights to obtain the fused features.

[0118] In this implementation, the fusion weight refers to the weights used to adjust the relative importance of different feature sources (such as embedded features and first predicted features) during the feature fusion process. These weights determine the degree of contribution of each feature source to the final fused feature during the fusion process. The fusion weight can be a fixed weight manually set based on experience and prior knowledge, or it can be selected based on different types of embedded features and first predicted features.

[0119] Based on the defined fusion weights, a weighted summation method can be used to combine the embedded features and the first predicted feature. Specifically, the embedded features and the first predicted feature are multiplied by their respective weights in the fusion weights, and then summed to obtain the final fused feature. In some tasks, the basic semantic information of the lexical units (embedded features) may be more critical, while in other tasks, contextual information (first predicted features) may be more important. By learning the fusion weights, the model can automatically adjust the weights of these features to adapt to different task requirements.

[0120] In one implementation, obtaining the fusion weights corresponding to the embedded features and the first predicted features includes:

[0121] Obtain the content generation scenario corresponding to the target word segmentation predicted by the preset large language model, and obtain the mapping relationship table between the scenario and the weight;

[0122] Based on the content generation scenario, determine the fusion weights corresponding to the embedded features and the first predicted features from the mapping table.

[0123] In this implementation, the content generation scenario refers to the specific context and task requirements of the text generated by the model in a natural language processing task. This content generation scenario can include different text types, themes, styles, and purposes. The current text type, theme, and style can be identified based on contextual information and prompts from the input text in a pre-defined large language model to determine the content generation scenario.

[0124] The mapping table can be a predefined data structure used to record the correspondence between different content generation scenarios and their corresponding fusion weights. It can be a dictionary or a table, where the key is a description of the content generation scenario and the value is the corresponding fusion weight. Each fusion weight is typically a tuple or list containing the embedded feature weight and the first predicted feature weight. For example, "News Report": (0.3, 0.7); "Story Creation": (0.5, 0.5); "Poetry Creation": (0.7, 0.3).

[0125] It is understandable that embedding features are extracted features from word segmentation, containing semantic information, while the first predictive feature is a predictive feature, containing contextual information. News reports typically require a high degree of coherence and logic. The first predictive feature (the output of the first layer encoder) contains rich contextual information, helping the model better understand the structure and logical relationships of sentences. Therefore, assigning a higher weight (0.7) to the first predictive feature can enhance the model's dependence on context, generating more coherent news content. Story creation requires a balance between the basic semantics and contextual information of word units. Embedding features provide basic semantic information of word units, while the first predictive feature provides contextual information. In story creation, both are equally important, so they can be given the same weight (0.5). Poetry creation requires rich semantics and rhythm. Embedding features (word embeddings) provide basic semantic and rhythmic information of word units, which is crucial for poetry creation. Therefore, assigning a higher weight (0.7) to embedding features can enhance the model's dependence on the basic semantics of word units.

[0126] By appropriately allocating the weights of embedded features and the first predicted feature, the model can dynamically adjust the importance of features according to different content generation scenarios, enabling the model to better adapt to different task requirements and generate higher quality text that is more consistent with the context logic.

[0127] In one implementation, the embedded features and the first predicted features are fused to obtain fused features, including:

[0128] Perform dimensional alignment between the embedded features and the first predicted features;

[0129] The dimension-aligned embedded features and the first predicted features are concatenated to obtain the fused features.

[0130] In this implementation, dimension alignment refers to ensuring that the dimensions of different feature sources (such as embedded features and first predicted features) are consistent during feature fusion so that they can be effectively fused. Dimension alignment can be achieved by using linear layers (fully connected layers) to adjust the feature dimensions to the same size (this can be done by learning weight matrices and bias vectors to map features from one dimension to another), performing average pooling on high-dimensional features to reduce feature dimensions (this can be done by calculating the average of certain dimensions in the feature vector to reduce feature dimensions), upsampling low-dimensional features to increase feature dimensions (this can be achieved through interpolation or duplication), or padding certain dimensions of low-dimensional features with zeros to increase feature dimensions, etc.

[0131] The concatenation operation joins multiple feature vectors along a selected dimension to generate a longer feature vector. This allows for direct concatenation of dimension-aligned embedded features and the first predicted feature, ensuring a regular structure in the concatenated feature vector. For example, if both the dimension-aligned embedded features and the first predicted feature have a dimension of 'd', the concatenated feature vector will have a dimension of 2d. The concatenation operation connects two feature vectors along a chosen dimension to create a longer feature vector. This regular structure helps avoid potential problems when the model handles irregular input dimensions.

[0132] Step 303: Based on the preset decoding network, perform feature prediction on the fused features to obtain the second predicted features.

[0133] A pre-trained decoding network (PSN) processes input features through a multi-layered neural network. Each layer further extracts and transforms the features, generating a higher-level feature representation that can be used to produce the final prediction result. The PSN can be an autoregressive head, consisting of a fully connected layer (FC) and a decoder. The fully connected layer can reduce the dimensionality of the input data or transform its features, such as by learning a weight matrix to map the fused feature data to a lower-dimensional space. The decoder can then predict the next feature based on the dimensionality-reduced data.

[0134] The second predicted feature can be a feature representation obtained by a preset decoding network after multi-layer processing and transformation of the fused features.

[0135] In one implementation, the parameter update process of the preset decoding network includes:

[0136] Obtain sample word segments, and predict the sample word segments based on a preset large language model to obtain the sample predicted word segments and the third prediction features corresponding to the sample predicted word segments;

[0137] The feature extraction layer is called to extract features from the predicted word segmentation of the sample to obtain the sample embedding features. The sample embedding features and the third prediction features are then fused to obtain the sample fusion features.

[0138] The feature prediction layer is invoked to predict the embedded features of the sample, resulting in the fourth predicted feature;

[0139] Based on the preset decoding network, feature prediction is performed on the sample fusion features to obtain the fifth predicted feature, and the parameters of the preset decoding network are updated based on the difference between the fourth and fifth predicted features.

[0140] In this implementation, sample word segmentation refers to the basic unit of text data used to train the model during the model training process, which can be in the form of words, sub-words, or characters. The sample word segmentation input to the preset large language model can be obtained by cleaning and normalizing the original text and then performing word segmentation. Based on the size of the original text, each input sample word segmentation can be one or more words.

[0141] Sample prediction word segmentation can be obtained by processing sample word segmentation through a feature extraction layer, a feature prediction layer, and an inference output layer using a pre-defined large language model. The feature extraction layer extracts features from the sample word segmentation, and the feature prediction layer predicts features from the extracted features to obtain a third predicted feature. In other words, sample prediction word segmentation is obtained by inferring from the third predicted feature by the inference output layer.

[0142] Sample embedding features can be the semantic information of the lexical units of the sample prediction word segmentation captured by the feature extraction layer and mapped to the feature representation in a low-dimensional vector space.

[0143] Sample fusion features can be created by merging features from different sources or of different types to form a new feature representation. In this case, it could be a combination of sample embedding features and third-party predictive features. Different features can capture different information, and sample fusion features can simultaneously include information contained in both sample embedding features and third-party predictive features.

[0144] The fourth predictive feature can be a feature representation generated after processing the input features (such as sample embedding features) through the feature prediction layer. These feature representations can capture the rich semantic information and contextual relationships of the sample embedding features.

[0145] The fifth predictive feature can be a feature representation generated after processing the sample fusion features through a preset decoding network. These feature representations can capture the rich semantic information and contextual relationships of the sample fusion features.

[0146] Understandably, the fourth prediction feature is the prediction result of the preset large language model. In order for the preset decoding network to learn the prediction performance of the preset large language model, the difference between the fourth prediction feature and the fifth prediction feature can be calculated, and the parameters of the preset decoding network can be updated using the backpropagation algorithm, so that the prediction result of the preset decoding network is closer to the prediction result of the preset large language model, thereby improving the output accuracy of the preset decoding network.

[0147] In one implementation, updating the parameters of a preset decoding network based on the difference between a fourth predicted feature and a fifth predicted feature includes:

[0148] Calculate the loss value based on the fourth and fifth prediction features;

[0149] The gradient value is determined based on the loss value, and the parameters of the preset decoding network are updated based on the gradient value.

[0150] In this implementation, the loss value can be a quantitative indicator that measures the difference between the model's predicted result and the actual result. By minimizing the loss value, the model can gradually optimize its predictive ability, thereby better fitting the training data. The loss value can be calculated by selecting an appropriate loss function according to the task type. For example, mean squared error (MSE) can be used for regression tasks, and cross-entropy loss can be used for classification tasks. Then, the difference between the model's predicted result and the actual result is calculated using the selected loss function. Here, it can be based on the selected loss function to calculate the difference between the predicted value of the preset decoding network (the fifth predicted feature) and the predicted value output by the feature prediction layer (the fourth predicted feature).

[0151] The gradient value can be the derivative of the loss function with respect to the model parameters, indicating the direction and rate of change of the loss function in the parameter space. The gradient value guides the update of model parameters, thereby minimizing the loss function. The gradient value is calculated using the backpropagation algorithm, which utilizes the chain rule to calculate the gradient value of the loss function layer by layer, starting from the loss value, until it reaches the input layer of the model. It can be understood that the magnitude of the gradient value represents the rate of change of the loss function in that direction; a larger gradient value indicates that the loss function changes rapidly in that direction, and a smaller gradient value indicates that the change is slower.

[0152] By minimizing the loss value, the model can gradually optimize its predictive ability. The loss value reflects the difference between the model's prediction and the actual result. By calculating the gradient value and updating the parameters, the model can gradually reduce this difference, thereby improving the accuracy and reliability of the prediction.

[0153] In one implementation, after performing feature prediction on the fused features based on a preset decoding network to obtain the second predicted features, the method further includes:

[0154] The cache is updated based on the key value corresponding to the preset decoding network according to the second prediction feature;

[0155] The inference output layer is invoked to infer the second predicted features to obtain predicted word segments, including:

[0156] The inference output layer is invoked to infer the predicted feature sequence in the key-value pair cache to obtain the predicted word segmentation.

[0157] In this implementation, the key-value pair cache can be a data structure used to store and quickly retrieve intermediate computation results. In deep learning models, fast retrieval of the key-value pair cache can accelerate the model's inference efficiency and performance. Specifically, the key-value pair cache can store intermediate feature representations generated during the model's inference process, avoiding redundant computation. Keys and values ​​can be extracted from the second predicted features. The key can be a feature extracted from the second predicted features used to calculate attention weights, and the value can be the actual feature information extracted from the second predicted features. The key-value pair consisting of the key and value can then be synchronized to the key-value pair cache, accelerating subsequent feature determination efficiency.

[0158] The predicted feature sequence can be a sequence of features generated by the model in the key-value pair cache. That is, it can include each predicted feature generated in multiple prediction processes, contain contextual information in the prediction process, provide rich semantic information, and enable the model to generate higher quality output.

[0159] Step 304: Call the inference output layer to infer the second predicted feature to obtain the predicted word segmentation, and verify the predicted word segmentation based on the preset large language model, and generate the inference content according to the verification result.

[0160] Predictive word segmentation refers to the process in natural language processing tasks where a model generates or predicts the next word based on input information. The inference output layer, which can be a linear layer or a fully connected layer, maps the second predicted features to a vocabulary and then determines the predicted word from the vocabulary. It should be noted that the scheme of using the feature extraction layer of a pre-defined large language model to extract features from the target word and using the inference output layer to process the output of the pre-defined decoding network is merely an example. Feature extraction of the target word can also be performed by a model other than the pre-defined large language model, such as a draft model. This draft model can extract features from the target word, predict the extracted features using a pre-defined decoding network, and then perform inference output on the prediction results.

[0161] The reasoning content can be obtained by verifying the generated predictive word segments using a pre-set large language model to ensure their rationality and accuracy. That is, based on the verification results, the successfully verified predictive word segments are combined into complete sentences or paragraphs to generate the final reasoning content.

[0162] The validation process can begin by converting the predicted word segments and their context into a format that the model can process, such as a sequence of word indexes. These sequences are then input into a pre-defined large language model, which determines the validation result based on the reasonableness of the word segments in the output. For example, in a question-answering system, a user asks a question and needs an accurate and relevant answer. First, a series of predicted word segments can be generated based on the question and the predictive features of a pre-defined decoding network. These segments are then validated using a pre-defined large language model. If the validation results show that the predicted word segments are reasonable, the model combines these segments into a complete answer. If the validation results are unsatisfactory, the model can regenerate the predicted word segments or adjust the generation strategy to improve the quality of the answer.

[0163] In one implementation, the predicted word segmentation is verified based on a preset large language model, and inference content is generated based on the verification results, including:

[0164] Obtain the first confidence score corresponding to the predicted word segmentation;

[0165] The predicted word segment, the target word segment, and the content corresponding to the target word segment are generated into prompt words and then concatenated to obtain the concatenated content.

[0166] The pre-defined large language model is invoked to infer the concatenated content, and the second confidence level corresponding to the predicted word segment is determined based on the inference results;

[0167] The verification results of the predicted word segmentation are determined based on the first confidence level and the second confidence level, and the inference content is generated based on the verification results.

[0168] In this embodiment, the first confidence level can refer to the confidence level when generating predicted word segments based on the predicted features output by the preset decoding network. This confidence level can be determined based on the output probability distribution, that is, it represents the degree of confidence in the generated predicted word segments.

[0169] Content generation prompts can refer to the prompts output to the pre-defined large language model during inference and prediction word segmentation, which provide contextual information to help predict and segment words generate more accurate text.

[0170] Concatenating content can refer to combining multiple text fragments or feature sequences into a longer text sequence. Here, by concatenating content-generated prompts to the predicted and target word segments, the pre-defined large language model can be guided to enter the word segmentation verification function. After receiving the concatenated content, the pre-defined large language model can evaluate the rationality and accuracy of the predicted word segmentation based on the entire context information. The pre-defined large language model will generate a confidence score, representing its confidence level in the predicted word segmentation. This confidence score, which can be called the second confidence score, can be used to verify whether the predicted word segmentation conforms to the contextual logic.

[0171] The verification result refers to the conclusion drawn from evaluating the rationality and accuracy of the predicted word segmentation by a pre-defined large language model in a natural language processing task. The first confidence level can be understood as the output confidence of the predicted word segmentation based on the features inferred from the pre-defined decoding network. The second confidence level can be understood as the output confidence of the pre-defined large language model for the predicted word segmentation. The first and second confidence levels can be directly compared to determine the rationality of the predicted word segmentation. If the second confidence level is higher than or equal to the first confidence level, meaning the pre-defined large language model has a higher probability of inferring the predicted word segmentation, then the predicted word segmentation can be considered reasonable. Otherwise, it can be considered that the pre-defined large language model's confidence level for the predicted word segmentation does not reach the confidence level of inferring the predicted word segmentation based on the features of the pre-defined decoding network, and the predicted word segmentation is unreasonable.

[0172] Inference content can refer to the final output of a pre-defined large language model in a natural language processing task. This content can be text, sentences, paragraphs, or any other form of natural language expression, including predicted word segments indicating successful verification. By obtaining confidence scores and determining verification results based on those scores, higher-quality inference content that better conforms to contextual logic can be generated.

[0173] In one implementation, the inference output layer is invoked to infer the second predicted features to obtain predicted word segments, and the predicted word segments are verified based on a preset large language model. Based on the verification results, inferred content is generated, including:

[0174] The inference output layer is invoked to infer multiple predicted word segments from the second predicted features;

[0175] The predicted word segmentation was validated based on a pre-defined large language model, and the validation results were obtained.

[0176] Based on the verification results of each predicted word in multiple predicted word segments, valid word segments are determined from multiple predicted word segments, and inference content is generated based on the valid word segments.

[0177] In this implementation, the multiple predicted word segments can be multiple possible word segments of the next word element output by the inference output layer in a single inference, or word segments included in a word element sequence determined by multiple inferences. The pre-defined large language model can evaluate the reasonableness and accuracy of each predicted word segment by calculating confidence levels to perform word segmentation verification; that is, the verification result can be obtained based on the evaluation results. Specifically, a passing evaluation indicates that the predicted word segment has passed verification, while a failing evaluation indicates that the predicted word segment has failed verification.

[0178] The method for determining the lexical sequence in multiple inferences can be referred to Figure 4 As shown, for example, f1 is the predicted feature of the initial word segmentation by the preset large language model, and t2 is the word segmentation obtained by the preset large language model based on f1. Then t2 and f1 can be combined for the next inference. Figure 4 In the process, t2 and f1 are passed through a feature extraction layer and a preset decoding network for prediction to obtain a new predicted feature f2. Specifically, the feature extraction layer can extract features from t2, fuse the extracted features with f1, and then the preset decoding network can predict the next feature based on the fused features to obtain f2. Then, the inference output layer can be used to infer f2 to obtain the predicted word segment t3. Based on the new predicted feature f2 and the new predicted word segment t3, the next prediction can be performed through the feature extraction layer and the preset decoding network to predict f3. Then, the inference output layer can be used to infer f3 to obtain t4. At this point, the word sequence can be (t3, t4).

[0179] Effective word segmentation can be any predicted word segmentation that has been verified as reasonable and accurate, or it can be referred to as word segmentation accepted by the pre-defined large language model. When multiple predicted word segments are multiple possible word segments for the next lexical unit output by the inference output layer in a single inference, the inferred content is one word segmentation that has passed verification among these multiple predicted word segments, i.e., one is selected from the multiple predicted word segments in a single inference. When multiple predicted word segments are word segments included in the lexical unit sequence determined by multiple inferences of the inference output layer, the inferred content can be a word segmentation sequence based on one or more word segments that have passed verification in each inference by the inference output layer, according to the inference order of multiple inferences. That is, when the inference output layer generates a series of lexical units through multiple inferences, one or more word segments generated in each inference are verified. The final inferred content is composed of these verified word segments combined in the order of inference. By generating multiple possible word segments through multiple inferences and selecting verified word segments, the model can generate diverse text content. This not only enriches the expression of the generated content but also improves the creativity and flexibility of the model.

[0180] In one example, the inference output layer iterates through the second predicted feature to obtain multiple predicted word segments. Each iteration adds a newly sampled token and its corresponding feature to the input sequence for the next feature prediction, until a word segment sequence of a preset length is generated. The length of this word segment sequence can be determined by the number of inference iterations by the inference output layer. When the number of inference iterations is not greater than a preset threshold corresponding to the preset length, the predicted word segments output by the inference output layer are iteratively processed. Using the second predicted feature as the new first predicted feature and the predicted word segment as the new target word segment, features can be extracted again based on the feature extraction layer for the new target word segment. The extracted features are then fused with the new first predicted feature. A preset decoding network predicts new features based on the fused features. The inference output layer then infers new predicted word segments based on these new features until the number of inference iterations by the inference output layer reaches the preset threshold, resulting in a word segment set. This word segment set can include multiple word segments arranged in a tree structure, where sibling nodes of the tree structure include word segments obtained from the same feature inference iteration by the inference output layer.

[0181] The preset large language model can perform token verification on the tokens of sibling nodes at each level in the tree structure of the token set, and then determine the inference content from the token set based on the token verification result. The preset large language model starts from the root node of the tree structure and verifies the tokens of sibling nodes level by level. The verification process evaluates the rationality of each token in the current context based on the internal probability distribution of the model. If the verification result of a certain token meets the preset probability threshold, it will be retained and passed to the next level; conversely, tokens that do not meet the requirements will be pruned, that is, removed from the candidate set. This process of token verification and pruning is carried out layer by layer. As the level deepens, the tree structure gradually narrows, and finally a target token branch is formed. This branch is composed of verified tokens and can form coherent and logical inference content.

[0182] The tree structure of the token set can refer to Figure 5 As shown, for content generation targeting the initial text "how do I", the preset large language model receives the initial text input "how do I", and generates one or more possible next words through its internal prediction mechanism, such as "can". This generated word becomes the first node of the tree structure. Next, the preset decoding network can be used as a component to draft tokens, and predict the next possible word, such as Figure 5 The tokens in the dashed boxes are the tokens in the token set, presented in the form of a tree structure. In the drafting process, the first drafting can be performed for "can" to generate two possible tokens: "make" and "help". These two words respectively represent two different inference paths starting from the initial word, forming two branches of the tree structure. Inference continues along these branches, and each newly generated token becomes the starting point for the next step of inference. For example, the second drafting is performed in parallel on "make" and "help". Starting from "make", "a" and "our" can be predicted; while starting from "help", "process" and "you" can be predicted. Correspondingly, taking the "help" branch as an example, the third drafting is performed on "process" and "you". For "process", the next possible tokens "this" and "your" can be predicted, and for "you", the next possible tokens that can be predicted are "go" and "feel". In this way, each newly generated token expands new branches, making the tree structure grow continuously.

[0183] In one embodiment, invoking the inference output layer to perform inference on the prediction features to obtain predicted tokens includes:

[0184] Invoke the inference output layer to perform inference on the prediction features to obtain a token probability distribution;

[0185] Determine the predicted token according to the token probability distribution.

[0186] In this implementation, the word segmentation probability distribution refers to the probability distribution of each possible word generated by the inference output layer being predicted as the next word. This probability distribution is part of the model output and can be in the form of a vector, where each element represents the prediction probability of a word. To ensure that these probability values ​​have the properties of a probability distribution, i.e., the sum of all probability values ​​is 1 and each probability value is between 0 and 1, the softmax function can be used to transform the output of the inference output layer. Through the processing of the softmax function, the original output of the inference output layer is normalized into a probability distribution, so that the prediction probability of each word is within a reasonable range and the sum of all probability values ​​is exactly 1.

[0187] The word segmentation probability distribution not only provides the most likely next word segment, but also other possible word segments and their probabilities. This enables the model to generate diverse texts, enhancing the model's adaptability and accuracy.

[0188] In one implementation, determining the predicted word segmentation based on the word segmentation probability distribution includes:

[0189] Word segmentation ranking is based on word segmentation probability distribution;

[0190] The first number of word segments that appear first in the order are identified as predicted word segments.

[0191] In this implementation, all possible word segments are sorted according to their probability values ​​in the word segmentation probability distribution. The sorted word segmentation list can be arranged from high to low probability, with the word segmentation with the highest probability at the top. For example, if the probabilities of the words "cat", "dog", and "bird" in the word segmentation probability distribution are 0.6, 0.3, and 0.1 respectively, then the sorting result would be "cat", "dog", and "bird".

[0192] After sorting, the top N words are selected as predicted words from the sorted word segmentation list according to a predetermined number N (the first number). This number N can be set according to the specific task requirements; it can be the word with the highest probability, or multiple predicted words ranked first in probability. When selecting multiple predicted words, for example, if N is set to 3, then in the above example, "cat," "dog," and "bird" can all be considered as candidates for predicted words, instead of just selecting "cat" with the highest probability. This allows the model to explore different expressions when generating text, avoiding generating overly simplistic or repetitive content. Selecting the most probable word as the predicted word based on the probability distribution can be achieved by taking the maximum value of the probability distribution, or by introducing randomness through sampling methods. In text generation tasks, selecting multiple predicted words can generate a variety of possible text variations, thereby improving the diversity and richness of the generated content.

[0193] The content generation method provided in this embodiment includes obtaining target word segments predicted by a preset large language model and obtaining first predicted features corresponding to the target word segments. The preset large language model includes a feature extraction layer, a feature prediction layer, and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. The feature extraction layer is called to extract features from the target word segments to obtain embedded features, and the embedded features and the first predicted features are fused to obtain fused features. The fused features are then used to predict features based on a preset decoding network to obtain second predicted features. The inference output layer is called to infer the second predicted features to obtain predicted word segments, and the predicted word segments are verified based on the preset large language model. Inference content is generated based on the verification results.

[0194] This method fuses the features predicted by the language model with the features of word segmentation inferred based on those features, then uses a pre-defined decoding network to predict the fusion result, and infers new word segments based on the prediction result for verification. Because the fusion of predicted features provides more prior knowledge, it improves the acceptance rate of the predicted candidate word segments, thereby improving the generation efficiency.

[0195] This disclosure provides a detailed description of the embodiments in conjunction with specific application scenarios.

[0196] like Figure 6 The diagram shown is another flowchart illustrating the content generation method provided in this disclosure. The method specifically includes the following steps:

[0197] Step 601: Obtain the content generation request and perform token prediction based on the target model according to the content generation request to obtain the first predicted token.

[0198] Content generation requests can be requests initiated by an object or system, instructing the model to generate specific text content. These requests can be simple text prompts or complex instructions, depending on the task requirements. The content generation request can include the request content and generation prompts. The request content can specify the specific text to be generated; it can be a sentence, a paragraph, or a topic, providing the model with context and direction for content generation. The generation prompts can be initial prompts guiding the target model's inference.

[0199] The target model can be a deep learning model trained on a large amount of data, capable of understanding and generating natural language text. These models can be based on the Transformer architecture, a network structure suitable for processing sequential data. It employs a full attention structure instead of traditional recurrent neural networks (RNNs) or convolutional neural networks (CNNs). Its core idea is to model the relationship between each element in the input sequence and other elements as an attention weight matrix, and to compute the representation of each element through a self-attention mechanism. This structure makes the Transformer highly capable of capturing long-range dependencies in sequences, thus achieving excellent performance in Natural Language Processing (NLP) tasks.

[0200] The first predicted token can be a lexical unit generated by the target model based on the initial inference of the content generation request. The target model can convert the content generation request into an embedded vector representation, and generate a feature representation based on the semantic and contextual information of the content generation request captured by the Transformer layer. Finally, the predicted token inferred based on this feature representation is used as the first predicted token.

[0201] Step 602: Obtain the first predicted feature corresponding to the first predicted token in the token prediction process of the target model.

[0202] The first predicted feature can be the feature representation initially generated by the target model based on the input content generation request. Specifically, when the target model receives an input sequence, it first converts each token in the input sequence into an embedded feature. These embedded features can then be fed into the encoder part of the Transformer architecture. After processing through multiple Transformer layers, a higher-level feature representation is generated. The target model can then infer the first predicted token based on this first predicted feature.

[0203] Step 603: The embedding network layer based on the draft model extracts features from the first predicted token to obtain embedded features, and then fuses the embedded features with the first predicted features to obtain fused features.

[0204] The draft model can be a lightweight model used to quickly generate candidate tokens, which are then validated and adjusted by the target model. The draft model can be a standalone model or an extension of the target model.

[0205] Embedding network layers can convert discrete words or characters into continuous vector representations. These vectors can capture the semantic and contextual information of words. Here, the embedding network layer in the draft model converts the first predicted token into a vector representation, that is, it maps the word index of the input first predicted token into a fixed-dimensional vector space. These vectors are called embedding features, which can capture the semantic information of words.

[0206] The fusion operation can combine semantic information based on embedded features with contextual information based on the first predicted feature to generate a more comprehensive feature representation, resulting in a fused feature. During the generation of the fused feature, if the dimensions of the embedded feature and the first predicted feature are inconsistent, feature alignment can be performed first. Fusion can be achieved through feature concatenation or weighted summation.

[0207] Step 604: The decoding network layer based on the draft model performs feature prediction on the fused features to obtain the second predicted features.

[0208] The decoding network layer can process the input features through multiple layers of neural networks. Each layer further extracts and transforms the features to generate higher-level feature representations. These higher-level feature representations refer to the feature representations obtained after multiple layers of processing and transformation. These feature representations are more complex and abstract than the fused features of the input. They can not only contain the basic meaning of words, but also capture the contextual relationships, syntactic structures, and semantic dependencies between words. Here, these higher-level feature representations can be called second predictive features.

[0209] Step 605: The inference network layer based on the draft model infers the second predicted feature to obtain the second predicted token.

[0210] The inference network layer transforms feature representations into concrete predictions, converting the higher-level feature representations output by the decoding network layer into prediction tokens, often referred to as the second prediction token. The inference network layer can consist of linear layers, softmax function layers, and sampling layers. The linear layers map the second prediction features to the size of the vocabulary, generating a prediction probability distribution for each word. The softmax function layer transforms the output of the linear layers into a probability distribution representing the likelihood of each word being the next word, ensuring that the sum of all probabilities is 1. The sampling layer selects the most probable word as the prediction token based on the probability distribution.

[0211] Step 606: When the number of inferences in the inference network layer is not greater than the preset number, the second prediction token is used as the new first prediction token, the second prediction feature is used as the new first prediction feature, and step 603 is executed.

[0212] The number of inferences can be the number of inference outputs of the inference network layer in the autoregressive generation process, where the model progressively generates text, predicting the next token based on the previously generated tokens at each step, in order to generate coherent and natural text.

[0213] The preset number of iterations refers to the maximum number of inference iterations performed by the inference network layer. This preset number can be set according to the specific task requirements and model performance. For example, in a text generation task, the preset number of iterations could be 5, meaning the model will perform a maximum of 5 inference iterations. The preset number of iterations can limit the length of the generated text, preventing excessively long text from being generated, or preventing the model from experiencing a "fatigue effect" as the number of generation steps increases, where generated tokens gradually deviate from the context, leading to a decrease in acceptance rate.

[0214] By using the second predicted feature as the new first predicted feature and the second predicted token as the new first predicted token, the features of the new first predicted token can be extracted again based on the embedded network layer in step 603. The extracted features are then fused with the new first predicted feature. The decoding network layer predicts new features based on the fused features, and the inference network layer infers new predicted tokens based on the predicted new features to generate coherent and natural text.

[0215] Step 607: When the number of inferences in the inference network layer is greater than the preset number, a set of prediction tokens is obtained based on the multiple prediction results of the inference network layer.

[0216] The predicted token set can be the collection of all possible tokens generated by the inference network layer through multiple inference steps. These possible tokens are generated at each inference step and reflect multiple prediction results for the next word. The draft model typically performs multi-step inference, with the tokens and features generated at each step serving as input for the next step. This process can be repeated until the preset number of inferences is reached. When the preset number of inferences is reached, the draft model can collect all the tokens generated at each inference step to form a token set as the predicted token set.

[0217] Step 608: Perform token verification on the predicted token set based on the target model, and determine the inference content from the predicted token set based on the token verification results.

[0218] The token verification process can evaluate each token in the predicted token set, which can be achieved by calculating the confidence score of each token, reflecting the model's prediction probability for that token. Furthermore, the reasonableness of the tokens can be further evaluated through contextual consistency checks and semantic accuracy verification. Contextual consistency checks ensure that the token matches the contextual information, while semantic accuracy verification ensures that the token is semantically logical. Regarding confidence scores, the confidence score of the predicted token in the target model can be compared with the confidence score of the same predicted token in the draft model. If the confidence score of the predicted token in the target model is greater than or equal to the confidence score of the predicted token in the draft model, the predicted token is considered successfully verified.

[0219] During the verification process, the target model can filter out reasonable tokens according to criteria, select the most suitable token from the predicted token set, and generate the final inference content. If all tokens in the predicted token set fail verification, the target model can choose to generate a default hint or regenerate new predicted tokens from the draft model for verification. The successfully verified predicted tokens can be called the inference content.

[0220] Understandably, the set of predicted tokens can be organized into a tree structure. This structure allows the model to generate multiple possible tokens at each step of inference, which are organized into sibling nodes, each representing a possible token. The draft model can then continue with the next step of inference based on each sibling node, generating child nodes, until a preset number of inferences is reached.

[0221] The verification process of the target model for the predicted token set can start from the root node, verify the tokens on each child node of the root node, remove the branches containing the tokens that failed verification, and continue to verify the grandchild nodes of the branches containing the successfully verified tokens, removing new branches that failed verification, until the final level node of the tree structure is verified. Then, the successfully verified tokens can be arranged in order to form a continuous text, which can be called the inference content.

[0222] In one example, during the validation phase, the target model calculates the probability of each token in the predicted token set within the original distribution. To ensure the quality of the generated text, a multi-round speculative sampling algorithm is used in the validation phase. This algorithm recursively processes each node in the tree-like predicted token set. If a token is accepted (i.e., its probability in the target model's probability distribution is greater than its probability in the draft model), it is retained in the final output; if a token is rejected, the algorithm may adjust the probability distribution (by weighting the probability distribution at the location of the rejected token, decreasing its probability while increasing the probabilities of other candidate tokens), or it may resample the token. This process can be recursively performed until all tokens have been processed. In this way, the validation phase ensures that the distribution of the output text is consistent with the distribution of the target model. By calculating and adjusting the probability of each token during the validation phase, the quality of the generated text is guaranteed not to be affected even during accelerated generation. Due to the small size of the draft model, its predictive power for long sequences is limited. If the average acceptance length in the most recent three validation processes is less than 3, it indicates that the generation quality of the current draft model may be low. At this point, the parameters of the draft model can be reset, and a new key-value cache (kvcache) can be regenerated to ensure the stability and quality of the generation process.

[0223] In one example, the architecture diagram for drafting tokens in the draft model can be referenced. Figure 7As shown, the target model 71 performs preliminary prediction on the initial text to obtain predicted tokens, and then the draft model processes possible next tokens of the predicted tokens. Specifically, the target model 71 may include a first embedding layer 711, a transformer layer 712, and a first language model head 713, and the draft model 72 includes a second embedding layer 721, an autoregressive head 722, and a second language model head 723. The first embedding layer 711 may perform feature extraction on the two tokens "wo" and "zenme" based on the initial text "wo zenme" respectively, to obtain vector 1 corresponding to the token "wo" and vector 2 corresponding to the token "zenme". The transformer layer 712 may predict the next feature, such as feature 1, based on vector 1, and predict the next feature, such as feature 2, based on vector 2. The first language model head 713 may calculate the probability distribution of tokens from the features, and may sample from the probability distribution to obtain the next token, for example, the token "zenme" may be calculated and sampled based on feature 1, and the token "neng" may be calculated and sampled based on feature 2. Then the draft model may perform a drafting operation on the token "zenme" and the token "neng".

[0224] The second embedding layer 721 performs feature extraction on the token "zenme" to obtain vector 3, and performs feature extraction on the token "neng" to obtain vector 4. Then, feature 1 output by the transformer layer 712 and vector 3 may be fused and input into the autoregressive head 722 to predict the next feature, and feature 2 and vector 4 may be fused and input into the autoregressive head 722 to predict the next feature. It can be understood that the next token of the token "zenme" is the known "neng", so the next feature predicted by inputting the fused result of feature 1 and vector 3 into the autoregressive head 722 may not be presented. The autoregressive head 722 outputs feature 3 corresponding to the fused feature of feature 2 and vector 4, where feature 3 is a higher-level feature representation predicted based on the fused feature of feature 2 and vector 4. The second language model head 723 may perform token inference based on this feature 3, that is, calculate the token probability distribution and perform multiple samplings to obtain the next tokens, such as the token "zhizuo" and the token "bangzhu". Then the draft model 72 may perform the next drafting based on the inferred tokens "zhizuo" and "bangzhu".

[0225] The second embedding layer 721 extracts features from the token "make" to obtain vector 5, and extracts features from the token "help" to obtain vector 6. Then, it fuses feature 3, previously output by the autoregressive head 722, with vector 5 and inputs this fusion into the autoregressive head 722 to predict the next feature. It also fuses feature 3 with vector 6 and inputs this fusion into the autoregressive head 722 to predict the next feature. The autoregressive head 722 outputs feature 4, the fused feature of feature 3 and vector 5, and in parallel outputs feature 5, the fused feature of feature 3 and vector 6. The second language model head 723 can perform token inference based on features 4 and 5 respectively, that is, calculate the token probability distribution and perform multiple samplings to obtain possible next tokens, such as the tokens "one" and "our" corresponding to feature 4, and the tokens "process" and "you" corresponding to feature 5. The draft model 72 can then use the inferred tokens "one," "our," "process," and "you" to draft the next token. The following example uses the next draft of tokens "process" and "you."

[0226] The second embedding layer 721 extracts features from the token "processing" to obtain vector 7, and extracts features from the token "you" to obtain vector 8. Then, it fuses feature 5, previously output by the autoregressive head 722, with vector 7 and inputs this fusion into the autoregressive head 722 to predict the next feature. It also fuses feature 5 with vector 8 and inputs this fusion into the autoregressive head 722 to predict the next feature. The autoregressive head 722 outputs feature 6, corresponding to the fused feature of feature 5 and vector 7, and outputs feature 7, corresponding to the fused feature of feature 5 and vector 8, in parallel. The second language model head 723 can perform token inference based on features 6 and 7 respectively, that is, calculate the token probability distribution and perform multiple samplings to obtain possible next tokens, such as the tokens "this" and "your" corresponding to feature 6, and the tokens "go" and "feel" corresponding to feature 7.

[0227] Description of apparatus and devices according to embodiments of this disclosure

[0228] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0229] It should be noted that in the various specific embodiments of this disclosure, when processing is required based on data related to the characteristics of the target object, such as target object attribute information or a set of attribute information, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with the relevant laws, regulations, and standards of the relevant regions. In addition, when this application embodiment needs to obtain target object attribute information, separate permission or consent from the target object will be obtained through pop-up windows or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of this application embodiment be obtained.

[0230] Figure 8 A schematic diagram of the structure of the content generation apparatus 800 provided in this embodiment of the disclosure. The apparatus 800 includes:

[0231] The acquisition unit 801 is used to acquire the target word predicted by the preset large language model during the content generation process and to acquire the first predicted feature corresponding to the target word. The preset large language model includes a feature extraction layer, a feature prediction layer and an inference output layer. The first predicted feature is the feature output by the feature prediction layer.

[0232] The fusion unit 802 is used to call the feature extraction layer to extract features from the target word segmentation, obtain embedded features, and fuse the embedded features and the first predicted features to obtain fused features.

[0233] Prediction unit 803 is used to perform feature prediction on the fused features based on a preset decoding network to obtain a second predicted feature;

[0234] The generation unit 804 is used to call the inference output layer to infer the second prediction feature to obtain the predicted word segmentation, and to verify the predicted word segmentation based on the preset large language model, and generate the inference content based on the verification result.

[0235] Optionally, in one feasible implementation, the fusion unit 802 includes:

[0236] The first acquisition subunit is used to acquire the fusion weights corresponding to the embedded features and the first predicted features;

[0237] The fusion subunit is used to fuse the embedded features and the first predicted features according to the fusion weights to obtain the fused features.

[0238] Optionally, in one feasible implementation, the first acquisition subunit includes:

[0239] The acquisition module is used to acquire the content generation scenario corresponding to the target word segmentation predicted by the preset large language model, and to acquire the mapping relationship table between the scenario and the weight.

[0240] The first determining module is used to determine the fusion weights corresponding to the embedded features and the first predicted features from the mapping relationship table based on the content generation scenario.

[0241] Optionally, in one feasible implementation, the fusion unit 802 includes:

[0242] The dimension alignment subunit is used to perform dimension alignment between the embedded features and the first predicted features;

[0243] The first splicing subunit is used to splice the dimension-aligned embedded features and the first predicted features to obtain the fused features.

[0244] Optionally, in one feasible implementation, the prediction unit 803 further includes:

[0245] The update subunit is used to update the cache based on the key value corresponding to the preset decoding network according to the second prediction feature;

[0246] The generation unit 804 includes:

[0247] The first inference subunit is used to call the inference output layer to infer the predicted feature sequence in the key-value pair cache to obtain the predicted word segmentation.

[0248] Optionally, in one feasible implementation, the generation unit 804 includes:

[0249] The second acquisition subunit is used to acquire the first confidence level corresponding to the predicted word segmentation;

[0250] The second splicing subunit is used to splice together the predicted word segmentation, the target word segmentation, and the content corresponding to the target word segmentation to generate prompt words, thus obtaining the spliced ​​content.

[0251] The first determining subunit is used to call a preset large language model to reason about the concatenated content and determine the second confidence level corresponding to the predicted word segmentation based on the reasoning results;

[0252] The first generation subunit is used to determine the verification result of the predicted word segmentation based on the first confidence level and the second confidence level, and to generate inference content based on the verification result.

[0253] Optionally, in one feasible implementation, the generation unit 804 includes:

[0254] The second inference subunit is used to call the inference output layer to infer the second prediction feature and obtain multiple predicted word segments;

[0255] The verification subunit is used to verify multiple predicted word segments based on a pre-defined large language model and obtain the verification results;

[0256] The second generation subunit is used to determine the effective word segment from multiple predicted word segments based on the verification result of each predicted word segment in multiple predicted word segments, and generate inference content based on the effective word segments.

[0257] Optionally, in one feasible implementation, the generation unit 804 includes:

[0258] The third inference subunit is used to call the inference output layer to infer the predicted features and obtain the word segmentation probability distribution.

[0259] The second determining subunit is used to determine the predicted word segmentation based on the word segmentation probability distribution.

[0260] Optionally, in one feasible implementation, the second determining subunit includes:

[0261] The sorting module is used to sort words based on their probability distribution.

[0262] The second determination module is used to determine the first number of word segments that are ranked first as predicted word segments.

[0263] This disclosure also provides a parameter update device for a preset decoding network, including:

[0264] The acquisition unit is used to acquire sample word segments and predict the sample word segments based on a preset large language model to obtain the sample predicted word segments and the third prediction feature corresponding to the sample predicted word segments.

[0265] The fusion unit is used to call the feature extraction layer to extract features from the sample prediction word segmentation, obtain sample embedding features, and fuse the sample embedding features with the third prediction features to obtain sample fusion features.

[0266] The prediction unit is used to call the feature prediction layer to predict the embedded features of the sample and obtain the fourth predicted feature.

[0267] The update unit is used to perform feature prediction on the sample fusion features based on the preset decoding network to obtain the fifth predicted feature, and to update the parameters of the preset decoding network based on the difference between the fourth and fifth predicted features.

[0268] Optionally, in one feasible implementation, the updating unit includes:

[0269] The calculation subunit is used to calculate the loss value based on the fourth and fifth prediction features;

[0270] The update sub-unit is used to determine the gradient value based on the loss value and update the parameters of the preset decoding network based on the gradient value.

[0271] Reference Figure 9 , Figure 9To illustrate the structural block diagram of a portion of the terminal 140 implementing the content generation method of this embodiment, the terminal 140 includes: a radio frequency (RF) circuit 910, a memory 915, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, a processor 980, and a power supply 990, among other components. Those skilled in the art will understand that... Figure 9 The terminal 140 structure shown does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0272] The RF circuit 910 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 980; in addition, it transmits uplink data to the base station.

[0273] The memory 915 can be used to store software programs and modules. The processor 980 executes various terminal functions and document editing by running the software programs and modules stored in the memory 915.

[0274] The input unit 930 can be used to receive input numeric or character information, and to generate key signal inputs related to the terminal's settings and function control. Specifically, the input unit 930 may include a touch panel 931 and other input devices 932.

[0275] The display unit 940 can be used to display input or provided information, as well as various menus of the terminal. The display unit 940 may include a display panel 941.

[0276] Audio circuitry 960, speaker 961, and microphone 962 provide an audio interface.

[0277] In this embodiment, the processor 980 included in the terminal 140 can execute the content generation method of the previous embodiment.

[0278] The terminal 140 in this disclosure includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0279] Figure 10This is a partial structural block diagram of a server 110 for implementing the content generation method of this disclosure. The server 110 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1022 (e.g., one or more processors) and storage devices 1032, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1042 or data 1044. The storage devices 1032 and storage media 1030 can be temporary or persistent storage. The program stored in the storage media 1030 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server 110. Furthermore, the CPU 1022 may be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server 110.

[0280] Server 110 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0281] The central processing unit 1022 in server 110 can be used to execute the content generation method of the present disclosure embodiments.

[0282] This disclosure also provides a storage medium for storing program code, which is used to execute the content generation methods of the foregoing embodiments.

[0283] This disclosure also provides a computer program product comprising a computer program. An electronic device's processor reads and executes the computer program, causing the electronic device to perform the content generation method described above.

[0284] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0285] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0286] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0287] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0288] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0289] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0290] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0291] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0292] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A content generation method, characterized in that, include: The target word predicted by the preset large language model during the content generation process is obtained, and the first predicted feature corresponding to the target word is obtained. The preset large language model includes a feature extraction layer, a feature prediction layer and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. The feature extraction layer is invoked to extract features from the target word segmentation to obtain embedded features, and the embedded features and the first predicted features are fused to obtain fused features. Based on a preset decoding network, feature prediction is performed on the fused features to obtain a second predicted feature; The inference output layer is invoked to infer the second predicted feature to obtain predicted word segmentation, and the predicted word segmentation is verified based on the preset large language model. Based on the verification result, inference content is generated.

2. The method according to claim 1, characterized in that, The parameter update process of the preset decoding network includes: Obtain sample word segments, and predict the sample word segments based on the preset large language model to obtain the sample predicted word segments and the third prediction feature corresponding to the sample predicted word segments; The feature extraction layer is invoked to extract features from the predicted word segmentation of the sample to obtain sample embedding features, and the sample embedding features and the third prediction features are fused to obtain sample fusion features; The feature prediction layer is invoked to predict the embedded features of the sample, resulting in a fourth predicted feature. Based on the preset decoding network, feature prediction is performed on the sample fusion features to obtain a fifth predicted feature, and the parameters of the preset decoding network are updated based on the difference between the fourth predicted feature and the fifth predicted feature.

3. The method according to claim 2, characterized in that, The step of updating the parameters of the preset decoding network based on the difference between the fourth and fifth prediction features includes: Calculate the loss value based on the fourth prediction feature and the fifth prediction feature; The gradient value is determined based on the loss value, and the parameters of the preset decoding network are updated based on the gradient value.

4. The method according to claim 1, characterized in that, The step of fusing the embedded features and the first predicted features to obtain fused features includes: Obtain the fusion weights corresponding to the embedded features and the first predicted features; The embedded features and the first predicted features are fused according to the fusion weights to obtain fused features.

5. The method according to claim 4, characterized in that, The step of obtaining the fusion weights corresponding to the embedded features and the first predicted features includes: Obtain the content generation scenario corresponding to the prediction of the target word by the preset large language model, and obtain the mapping relationship table between the scenario and the weight; Based on the content-generated scenario, the fusion weights corresponding to the embedded features and the first predicted features are determined from the mapping relationship table.

6. The method according to claim 1, characterized in that, The step of fusing the embedded features and the first predicted features to obtain fused features includes: Perform dimensional alignment between the embedded features and the first predicted features; The dimension-aligned embedded features and the first predicted features are concatenated to obtain the fused features.

7. The method according to claim 1, characterized in that, After performing feature prediction on the fused features based on a preset decoding network to obtain the second predicted feature, the method further includes: The cache is updated based on the key value corresponding to the preset decoding network according to the second predicted feature; The step of calling the inference output layer to infer the second predicted features to obtain predicted word segmentation includes: The inference output layer is invoked to infer the predicted feature sequence in the key-value pair cache to obtain the predicted word segmentation.

8. The method according to claim 1, characterized in that, The step of verifying the predicted word segmentation based on the preset large language model and generating inference content based on the verification results includes: Obtain the first confidence level corresponding to the predicted word segmentation; The predicted word segment, the target word segment, and the content corresponding to the target word segment are concatenated to generate prompt words to obtain the concatenated content. The preset large language model is invoked to infer the concatenated content, and the second confidence level corresponding to the predicted word segment is determined based on the inference result; The verification result of the predicted word segmentation is determined based on the first confidence level and the second confidence level, and the inference content is generated based on the verification result.

9. The method according to claim 1, characterized in that, The process involves calling the inference output layer to infer the second predicted feature to obtain predicted word segmentation, validating the predicted word segmentation based on the preset large language model, and generating inference content based on the validation results, including: The inference output layer is invoked to infer multiple predicted word segments from the second predicted features; The multiple predicted word segments are verified based on the preset large language model to obtain the verification results; Based on the verification results of each of the multiple predicted word segments, valid word segments are determined from the multiple predicted word segments, and inference content is generated based on the valid word segments.

10. The method according to claim 1, characterized in that, The step of calling the inference output layer to infer the predicted features to obtain predicted word segmentation includes: The inference output layer is invoked to infer the predicted features, thereby obtaining the word segmentation probability distribution; The predicted word segmentation is determined based on the word segmentation probability distribution.

11. The method according to claim 10, characterized in that, The step of determining the predicted word segmentation based on the word segmentation probability distribution includes: Segmentation and ranking are performed based on the aforementioned segmentation probability distribution; The first number of word segments that appear first in the order are identified as predicted word segments.

12. A content generation apparatus, characterized in that, include: The acquisition unit is used to acquire the target word predicted by the preset large language model during the content generation process and to acquire the first predicted feature corresponding to the target word. The preset large language model includes a feature extraction layer, a feature prediction layer and an inference output layer. The first predicted feature is the feature output by the feature prediction layer. The fusion unit is used to call the feature extraction layer to extract features from the target word segmentation to obtain embedded features, and to fuse the embedded features and the first predicted features to obtain fused features. The prediction unit is used to perform feature prediction on the fused features based on a preset decoding network to obtain a second predicted feature; The generation unit is used to call the inference output layer to infer the second prediction feature to obtain the predicted word segmentation, and to verify the predicted word segmentation based on the preset large language model, and generate inference content based on the verification result.

13. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the content generation method according to any one of claims 1 to 11.

14. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the content generation method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program that is read and executed by a processor of an electronic device, causing the electronic device to perform the content generation method according to any one of claims 1 to 11.