Sampling method and device

Abstract

The invention discloses a sampling method and device. The method comprises the following steps: acquiring a first element sequence of to-be-sampled data; acquiring a second element sequence of the to-be-sampled data; determining a repeated element sequence based on elements included in the first element sequence and elements included in the second element sequence; and sampling based on the repeated element sequence to obtain an output element sequence of the to-be-sampled data.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a sampling method and device. Background Art

[0002] Currently, in the sampling process of large autoregressive (AR) models, elements are the basic units of AR reasoning and sampling. AR converts the data to be sampled into a sequence of elements with known conditions. Based on this sequence of elements and the sequence of elements with partial sampling results, the sampling result corresponding to the next index identifier is obtained. However, this existing sampling method may produce multiple duplicate elements during sampling, resulting in the erroneous sampling of duplicate and unexpected results, reducing sampling accuracy. Summary of the Invention

[0003] The embodiments of the present application provide a sampling method and device, which improve the accuracy of sampling by performing element duplication judgment during sampling, thereby improving the accuracy of content generated by the autoregressive model, making the generated content meet user expectations and improving user experience.

[0004] In a first aspect, an embodiment of the present application provides a sampling method, the method comprising:

[0005] Obtaining a first element sequence of data to be sampled, wherein the first element sequence includes an element corresponding to a first index identifier, an element corresponding to an index identifier between the first index identifier and a current index identifier, and an element corresponding to the current index identifier, and the first index identifier is smaller than the current index identifier;

[0006] Obtaining a second element sequence of the data to be sampled, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, wherein the third index identifier is smaller than the first index identifier, and the second index identifier is smaller than the third index identifier;

[0007] determining a repeating element sequence based on elements included in the first element sequence and elements included in the second element sequence;

[0008] Sampling is performed based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

[0009] In a second aspect, an embodiment of the present application provides a sampling device, the device comprising: a transceiver unit and a processing unit;

[0010] The transceiver unit is configured to obtain a first element sequence of data to be sampled, wherein the first element sequence includes an element corresponding to a first index identifier, an element corresponding to an index identifier between the first index identifier and a current index identifier, and an element corresponding to the current index identifier, and the first index identifier is smaller than the current index identifier;

[0011] the transceiver unit being configured to obtain a second element sequence of the data to be sampled, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, the third index identifier being smaller than the first index identifier, and the second index identifier being smaller than the third index identifier;

[0012] The processing unit is configured to determine a repeating element sequence based on elements included in the first element sequence and elements included in the second element sequence;

[0013] The processing unit is configured to perform sampling based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the processor being connected to the memory, the memory being used to store computer programs, and the processor being used to execute the computer programs stored in the memory, so that the electronic device performs the method of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program enables a computer to execute the method of the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and is computer-operable to enable the computer to execute the method of the first aspect.

[0017] The implementation of the embodiments of the present application has the following beneficial effects:

[0018] It can be seen that in an embodiment of the present application, after sampling and obtaining the element corresponding to the current index identifier, the element corresponding to the current index identifier and the first element sequence that is before the element corresponding to the current index identifier and is adjacent to the element corresponding to the current index identifier will be repeatedly judged. Specifically, a first element sequence of the data to be sampled is obtained, wherein the first element sequence includes the element corresponding to the first index identifier, the element corresponding to the index identifier located between the first index identifier and the current index identifier, and the element corresponding to the current index identifier. Further, a second element sequence of the data to be sampled is obtained, wherein the second element sequence includes the element corresponding to the second index identifier, the element corresponding to the index identifier located between the second index identifier and the third index identifier, and the element corresponding to the third index identifier. The second element sequence is an element sequence that has been generated before the first element sequence, wherein the first element sequence can be understood as an element sequence to be judged. Based on the elements included in the first element sequence and the elements included in the second element sequence, a repeated element sequence is determined, that is, based on the already generated element sequence (second element sequence), a repeated element sequence in the element sequence to be judged (first element sequence) and the already generated element sequence (second element sequence) is determined. The repeated element sequence is an unexpected result incorrectly sampled by the large autoregressive model, and the repeated element sequence cannot be used as part of the output element sequence obtained by the final sampling.

[0019] Therefore, sampling is performed based on the repeating element sequence to obtain the output element sequence of the data to be sampled. Sampling again based on the repeating element sequence can sample new non-repeating elements, avoiding using the repeating element sequence as part of the output element sequence obtained by sampling the data to be sampled. This allows the model to sample a non-repeating output element sequence that meets the expectations, thereby improving sampling accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a flow chart of an initial sampling method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a single element repeat determination provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of another initial sampling method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a sampling method according to an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of element segment duplication determination provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of another method for determining element segment duplication provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of another sampling method provided in an embodiment of the present application;

[0029] Figure 9 A schematic flow chart of another sampling method provided in an embodiment of the present application;

[0030] Figure 10 A block diagram of the functional units of a sampling device provided in an embodiment of the present application;

[0031] Figure 11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] The terms "first," "second," "third," and "fourth," etc., in the specification, claims, and drawings of this application are used to distinguish between different objects, not to describe a particular order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0034] References herein to "embodiments" mean that a particular feature, result, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] In order to facilitate understanding of the technical solution of this application, the relevant technical terms involved in this application are first explained.

[0036] Element: Also known as Token, it is the smallest semantic unit for processing and generating text. These units can be words, subwords, or characters. Token is the smallest semantic unit used to represent a word in a language model.

[0037] Autoregressive (AR) models are a type of generative neural network model. Their characteristic is that they employ iterative step-by-step reasoning during task reasoning. The result of each step not only forms part of the final inference result but also serves as input for the next step. In other words, during inference, the independent and dependent variables are not separate variables, but rather the same variable at different times. The current state of the resulting "variable" is derived from the state value at the previous time step.

[0038] Autoregressive large model inference input and output: The inference input of the autoregressive large model is a known conditional sequence. This known conditional sequence is composed of the sequence of elements in the known conditional sequence and the sequence of elements in the partially sampled result. The output of the autoregressive large model is the prediction of the next element. At each inference step, the main goal is to obtain the probability distribution of the next element to be generated, given the known inference input. This probability distribution is calculated by converting the logical value directly output by the large model into a probability value through a function similar to softmax. The length of this logical value and the probability value vector are both equal to the size of the output element target space. For example, a Chinese sentence consists of 6,500 different Chinese characters and 100 different punctuation elements. Therefore, the size of the output element target space is 6,600. Therefore, the probability value is the probability distribution of the next element being one of these 6,600 different elements.

[0039] Inference: In the inference process of an autoregressive model, this is a method of generating elements one by one, used to produce a continuous sequence of elements. For example, when generating text, an autoregressive model will predict the most likely multiple elements corresponding to the next index token. This process is called inference.

[0040] Furthermore, sampling is performed from multiple elements corresponding to the index identifier obtained by inference prediction to obtain the element corresponding to the index identifier. This process is called sampling.

[0041] Top-K sampling: Given a known model's predicted probability distribution for the next element, sort these probability values ​​from large to small, and take the top K possible results with the highest probability and sample from them. This is called Top-K sampling.

[0042] Top-P sampling: Top-P sampling, also known as kernel sampling, is based on the model's predicted probability distribution for the next element. These probability values ​​are sorted from largest to smallest, and then the sorted probabilities are summed until the sum is greater than or equal to a threshold, P. All elements that have participated in the summation are considered possible outcomes, from which samples are taken. This is called top-P sampling.

[0043] Longest Substring Matching Algorithm: For two homogeneous strings, an algorithm is used to find the longest common substring between them. For example, one string is ABBCAD and the other is ECBCA. The human eye can observe that their largest common substring is the substring "BCA" of length 3. Computers can achieve this goal through a variety of different algorithms, such as direct brute force search, Rabin-Karp algorithm, dynamic programming, and other algorithms. This application refers to this algorithm as the longest substring matching algorithm and does not limit the specific algorithm used.

[0044] Prompt: Prompt text, representing a command or instruction, used to indicate that the large language model needs to perform an action or generate output.

[0045] The sequence end element is a special marker used to indicate the end of a sequence. In this application, it is called a preset type element.

[0046] First, let's explain the problems with existing technologies. Currently, during the sampling process, large autoregressive models use a sequence of elements with known conditions and a sequence of elements with already obtained partial sampling results to determine the next element's sampling result. This existing sampling method can produce repeated sequences of elements, leading to the erroneous sampling of unintended, repeated results. Because autoregressive models output one element at a time, each element prediction is influenced by the known original input conditions and the results of the already predicted partial elements. Therefore, this problem of repeated sequences of elements is an inherent flaw of large autoregressive models, especially under certain input contexts.

[0047] For example, consider a large autoregressive model for a story generation task. The input story begins with the sentence: "Once upon a time, there was a mountain. On the mountain, there was a temple." Ideally, the model would be expected to generate a coherent and engaging story based on this introduction. However, due to the problem of repeated sequences of elements, the model might produce the following output: "Once upon a time, there was a mountain. On the mountain, there was a temple. In the temple lived an old monk who guarded a secret. Hidden within the temple was an ancient painting that was said to lead to a hidden treasure. One day, a young monk accidentally touched a mechanism, and the mural slowly rose, revealing a treasure map. The master and his apprentice decided to search for the treasure. After enduring many hardships, they finally found the treasure and decided to donate it to the village at the foot of the mountain to help the villagers overcome their difficulties. Overcome difficulties, overcome difficulties, overcome difficulties."

[0048] In this example, the model generated multiple repetitions of "get through the hard times" at the end. This is because a problem with the sequence of repeated elements occurred during sampling. As a result, the final generated repetitive content of "get through the hard times" did not meet the user's expectations, reducing the accuracy of the generated content and the user experience.

[0049] It should be noted that the sampling method provided by the present invention is applicable to various autoregressive large models that use discrete element language representation, and is not limited to specific application fields, including but not limited to natural language processing (NLP), text-to-speech (TTS), and text-based graphs, etc., which are not limited in this application. In these models, due to the problem of repeated sampling, the method of the present invention can be applied to improve this problem. In the autoregressive generative large model, sampling is the process of generating target words or data points one by one. This mechanism has applications in many fields, such as text generation in NLP, speech synthesis in TTS, and creating new images in image generation. These models usually rely on the previously generated output to calculate the next output. This sequential feature makes them particularly susceptible to repeated sampling defects, that is, the model may repeatedly generate the same output, resulting in a lack of diversity and novelty in the output results.

[0050] See Figure 1 , Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application.

[0051] It should be noted that the sampling method in this application can be applied to a variety of scenarios, including but not limited to the following scenarios:

[0052] Application scenario 1: NLP field.

[0053] This application provides a natural language processing method, including: obtaining a first prompt word input by a target object (wherein the first prompt word is the data to be sampled in the scenario), sampling the first prompt word based on the sampling method provided by this application, and obtaining the first target generated content. It should be noted that the specific method of sampling the first prompt word is similar to the sampling method described below, and this application will not elaborate on it here. Figure 1 As shown, the system includes a client, a first user interface, a second user interface, a server, and an autoregressive large model, wherein the autoregressive large model performs sampling using the sampling method described in this application. A target user can input a first prompt word in the first user interface. The first prompt word can be: Generate a story of less than 100 words starting with "Once upon a time there was a mountain, and on the mountain there was a temple." The client sends the received first prompt word to the server, which samples the first prompt word using the autoregressive large model to obtain first target generated content. The first target generated content is then sent to the client, and the client displays the first target generated content as "XXXXXX..." on the second user interface. The specific content is not limited here.

[0054] It can be seen that in the embodiment of the present application, there are no multiple repeated text contents in the generated first target generated content. This is because in the process of the sampling method of the present application, the repeated element sequence is avoided as part of the target sequence obtained by sampling. Therefore, the model can obtain sampling results that meet expectations. By improving the accuracy of sampling, the accuracy of the model-generated content is improved, so that the generated content meets user expectations and improves user experience.

[0055] Application scenario 2: TTS field.

[0056] The present application provides a text-to-speech method, comprising: obtaining a second prompt word input by a target object (wherein the second prompt word is the data to be sampled in the scenario), sampling the second prompt word based on the sampling method provided by the present application, and finally obtaining the second target generated content. It should be noted that the specific method for sampling the second prompt word is similar to the sampling method described below, and the present application will not elaborate on it here. Figure 1 As shown, it also includes a third user interface and a fourth user interface. The target object can input a second prompt word in the third user interface. The second prompt word can be: convert "study hard and make progress every day" into voice, and the timbre is that of a girl; the client sends the received second prompt word to the server, and the server samples the second prompt word through the autoregressive large model to obtain the second target generated content, and then sends the second target generated content to the client. The client displays the second target generated content in the fourth user interface. The target object can click on the second target generated content to play the corresponding voice information.

[0057] Furthermore, the second prompt word may also include the data to be sampled in the voice mode, converting the time-continuous voice signal into a discrete expression similar to a text element, and then sampling the second prompt word based on the sampling method provided in this application to finally obtain the second target generated content.

[0058] During the text-to-speech process, a trained autoregressive large model can be used for text-to-speech conversion. Specifically, during the training process, training samples are constructed. The training samples include text element samples and speech element samples. These samples can be trained using the AR large model training architecture to obtain a trained autoregressive large model for text-to-speech conversion. Compared with ordinary automatic language processing, the main characteristics of text-to-speech are: first, text-to-speech is cross-modal, with the input mainly being text modal elements and the output being speech modal elements. Second, an additional speech elementization module is required to assist in training. This module mainly converts time-continuous speech signals into discrete expressions similar to text elements to facilitate large model modeling. Among them, speech elements can be regarded as a discrete, compressed representation of speech signals.

[0059] It can be seen that in the embodiment of the present application, there are no multiple repeated voice contents in the generated second target generated content. This is because in the process of the sampling method of the present application, the repeated element sequence is avoided as part of the target sequence obtained by sampling. Therefore, the model can obtain sampling results that meet expectations. By improving the accuracy of sampling, the accuracy of the model-generated content is improved, so that the generated content meets user expectations and improves user experience.

[0060] Application scenario three: cultural image field.

[0061] This application provides a text generation method, including: obtaining a third prompt word input by a target object (wherein the third prompt word is the data to be sampled in the scene), sampling the third prompt word based on the sampling method provided by this application, and finally obtaining the third target generated content. It should be noted that the specific method of sampling the third prompt word is similar to the sampling method described below, and this application will not elaborate on it here. Figure 1 As shown, the system also includes a fifth user interface and a sixth user interface. In the fifth user interface, the target user can enter a third prompt word. The third prompt word can be: "Please help me generate an image of a puppy running on the grass." The client sends the received third prompt word to the server. The server samples the third prompt word using an autoregressive large model to generate third target generated content. The server then sends the third target generated content to the client. The client displays the third target generated content in the sixth user interface. The target user can click on the third target generated content to play the corresponding voice message.

[0062] In the process of text generation diagram, the text generation diagram can be performed through the trained autoregressive large model. Specifically, training samples are constructed during the training process. The training samples include text element samples and image element samples. The training can be performed through the AR large model training architecture to obtain the trained autoregressive large model for text generation diagram.

[0063] It can be seen that in the embodiment of the present application, there are no multiple repeated image contents in the generated third target generated content. This is because in the process of the sampling method of the present application, the repeated element sequence is avoided as part of the target sequence obtained by sampling. Therefore, the model can obtain sampling results that meet expectations. By improving the accuracy of sampling, the accuracy of the model-generated content is improved, so that the generated content meets user expectations and improves the user experience.

[0064] It should be noted that the sampling method provided in this application includes both a single element duplication determination method and a multiple element duplication determination method, that is, an element segment duplication determination method. For ease of description, this application first describes a single element duplication determination method.

[0065] See Figure 2 , Figure 2 This is a flow chart of an initial sampling method provided in an embodiment of the present application. The initial sampling method provided in the present application includes repeated judgment of a single element, and the method includes but is not limited to steps 201-203:

[0066] 201: Perform initial sampling on multiple third candidate elements corresponding to the current index identifier to obtain a first pre-sampled element.

[0067] It should be noted that during inference, the autoregressive model infers multiple candidate elements corresponding to each index identifier, obtains the probability distribution of the predicted candidate elements corresponding to each index identifier, and then performs sampling based on the probability distribution of the predicted candidate elements corresponding to each index identifier. The index identifier represents the position of the element in the sequence. Each time a new element is sampled and added to the sequence, the index identifier of the new element corresponds to the sampling position.

[0068] The inference sampling of the current index identifier is explained using a large autoregressive model. Specifically, multiple elements with the largest probability are taken from the probability distribution of the predicted candidate elements corresponding to the current index identifier as the multiple third candidate elements corresponding to the current index identifier. Then, sampling is performed based on the probability distribution of the multiple third candidate elements corresponding to the current index identifier, that is, the range of sampled elements is multiple third candidate elements. Among them, the sampling method can be Top-P sampling, Top-k sampling, or a combination of these two sampling methods, and this application does not limit this. In this application, the sampling method is Top-P sampling as an example for explanation, where P is the first probability cumulative value, which can also be called a kernel parameter. Finally, the multiple third candidate elements are initially sampled to obtain the first pre-sampling element corresponding to the current index identifier.

[0069] It should be noted that the number of multiple third candidate elements can be set according to different situations. If the prediction accuracy of the autoregressive model itself is high, the number of third candidate elements can be smaller. If the autoregressive model itself has a small amount of training data, or other reasons, the accuracy is poor, the number of third candidate elements can be greater.

[0070] 202: If the first pre-sampled element is a valid element, use the first pre-sampled element as the element corresponding to the current index identifier.

[0071] It should be noted that before step 202, the process further includes: determining whether the first pre-sampled element is a valid element. It should be noted that when sampling the position corresponding to each index identifier, the pre-sampled element corresponding to the position is first obtained, and then a single element repeat judgment is performed on the pre-sampled element. For example, the first pre-sampled element is the pre-sampled element corresponding to the current index identifier, such as Figure 3 The n shown is the current index identifier. Determining whether the first pre-sampled element is a valid element can be understood as performing a single element repetition judgment on the first pre-sampled element. If the first pre-sample does not belong to the case of single element repetition, the first pre-sampled element is a valid element. If the first pre-sample belongs to the case of single element repetition, the first pre-sampled element is not a valid element.

[0072] Exemplarily, determining whether the first pre-sampled element is a valid element specifically includes: determining whether the first pre-sampled element is a preset type element, wherein the preset type element is a sequence end element. If so, determining that the first pre-sampled element is a valid element, and using the first pre-sampled element as the element corresponding to the current index identifier. If not, obtaining a third element sequence of the data to be sampled, wherein the third element sequence includes an element corresponding to the previous index identifier of the current index identifier, elements corresponding to the first j index identifiers of the current index identifier, and elements corresponding to index identifiers located between the previous index identifier and the first j index identifiers, wherein the first j index identifiers are smaller than the previous index identifier. Figure 3 As shown, each index identifier corresponds to an element, such as Figure 3 The index identifiers in are 0, 1, 2, nj, n-1, n, etc. n represents the current index identifier, n-1 represents the previous index identifier, and nj represents the previous j index identifiers. The third element sequence is composed of the element corresponding to n-1, the element corresponding to nj, and the elements between n-1 and nj. The length of j-1 is the length of the single element repetition judgment window, that is, w1; based on the value of the first pre-sampled element and the value of each element in the third element sequence, the second repetition degree of the first pre-sampled element is determined; if the second repetition degree is less than the second threshold, the first pre-sampled element is a valid element; if the second repetition degree is greater than or equal to the second threshold, the first pre-sampled element is not a valid element.

[0073] Specifically, based on the value of the first pre-sampled element and the value of each element in the third element sequence, the second repetition degree of the first pre-sampled element is determined, and the value of each element in the third element sequence is compared with the value of the first pre-sampled element one by one. If they are equal, the repetition value is increased by 1 until the first pre-sampled element is compared with all elements in the third element sequence to obtain the final total repetition value. Based on the total repetition value and the length of the single element repetition judgment window, the second repetition degree of the first pre-sampled element is obtained, wherein the ratio of the total repetition value to the length of the single element repetition judgment window can be directly used as the second repetition degree, or the ratio can be obtained and then multiplied by the corresponding weight to obtain the second repetition degree. This is not limited to the comparison in this application. Furthermore, if the second degree of repetition is less than the second threshold, the first pre-sampled element is a valid element. A valid element indicates that the element is not a repeated sample and does not belong to the case of single element repetition. It can also be understood that the first pre-sampled element has passed the single element repetition judgment. The first pre-sampled element can be directly used as the element corresponding to the current index identifier, and the first pre-sampled element can be added to the end of the existing inference result element sequence. It should be noted that the sampling result corresponding to the current index identifier refers to the target element finally corresponding to the current index identifier. The existing inference result element sequence is an element sequence composed of elements corresponding to index identifier n-1 and all index identifiers before index identifier n-1; if the second degree of repetition is greater than or equal to the second threshold, the first pre-sampled element is not a valid element and belongs to the case of single element repetition. It can also be understood that the first pre-sampled element has not passed the single element repetition judgment, that is, it indicates that the first pre-sampled element is a repeated sample and is invalid. The first pre-sampled element cannot be directly used as the element corresponding to the current index identifier, and the element corresponding to the current index identifier needs to be re-initialized. The specific processing methods corresponding to when the first pre-sampled element is a valid element and when the first pre-sampled element is not a valid element will be described in detail below, and this application will not go into details here.

[0074] 203: If the first pre-sampled element is not a valid element, adjust multiple third candidate elements corresponding to the current index identifier based on a first probability accumulation threshold and a scaling factor to obtain an element corresponding to the current index identifier.

[0075] For example, if the first pre-sampled element is not a valid element and is a case of single element repetition, it can also be understood that the first pre-sampled element fails the single element repetition judgment, which means that the first pre-sampled element is a repeated sample and is invalid. The first pre-sampled element cannot be directly used as the element corresponding to the current index identifier, and the element corresponding to the current index identifier needs to be re-initialized.

[0076] Specifically, the first probability accumulation threshold is adjusted based on the scaling factor to obtain a second probability accumulation threshold; based on the second probability accumulation threshold, a plurality of fourth candidate elements corresponding to the current index identifier are obtained, wherein the number of the plurality of fourth candidate elements is greater than the number of the plurality of third candidate elements; random sampling is performed from the plurality of fourth candidate elements to obtain a second pre-sampled element; if the second pre-sampled element is different from the first pre-sampled element, the second pre-sampled element is used as the element corresponding to the current index identifier. Furthermore, if the second pre-sampled element is the same as the first pre-sampled element, the second probability accumulation threshold is continuously adjusted based on the scaling factor and then sampled again until the sampled element is different from the first pre-sampled element, and the probability accumulation threshold is then adjusted to the first probability accumulation threshold.

[0077] Specifically, the first probability accumulation threshold P1 is adjusted based on the scaling factor to obtain the second probability accumulation threshold P2, which can be directly obtained by summing the scaling factor and the first probability accumulation threshold. The second probability accumulation threshold P2 can also be obtained by formula (1):

[0078] P2=P1*(1+γ) Formula (1)

[0079] Wherein, is the γ scaling factor, and P1 is the first probability accumulation threshold.

[0080] It can be seen that in an embodiment of the present application, if the first pre-sampled element is not a valid element, the first probability accumulation threshold is adjusted based on the scaling factor to obtain a second probability accumulation threshold; based on the second probability accumulation threshold, multiple fourth candidate elements corresponding to the current index identifier are obtained, wherein the number of the multiple fourth candidate elements is greater than the number of the multiple third candidate elements; random sampling is performed from the multiple fourth candidate elements to obtain a second pre-sampled element; if the second pre-sampled element is different from the first pre-sampled element, the second pre-sampled element is used as the element corresponding to the current index identifier. First, by adjusting the first probability accumulation threshold to the second probability accumulation threshold, the range of candidate elements is expanded to obtain multiple fourth candidate elements, and then random sampling is performed from the multiple fourth candidate elements to reduce the probability of re-sampling the first pre-sampled element and thus repeating the sampling. Further, the generated second pre-sampled element is judged. If the second pre-sampled element is different from the first pre-sampled element, the second pre-sampled element is re-used as the first pre-sampled element, ensuring that the re-sampling result does not repeat a single element, thereby improving the accuracy of sampling at each position.

[0081] It should be noted that in addition to performing a single element repetition judgment on the pre-sampling results of each position during sampling, the present application will also perform a multiple element repetition judgment after obtaining the sampling results of each position, that is, an element fragment repetition judgment. In the present application, a single element repetition judgment and a multiple element repetition judgment can be performed at each index identifier. In order to improve the judgment efficiency, the element fragment repetition judgment can also be performed periodically on the sampling results. Specifically, the number of index identifiers between each element fragment repetition judgment cycle is v. For example, if the element fragment repetition judgment is performed at index identifier j, the periodic point of the next fragment repetition judgment is at index identifier j+v. Therefore, determining whether the current index identifier n meets the preset conditions is to determine whether the current index identifier n is the periodic point position of the fragment repetition judgment. For example, if the periodic point position of the previous fragment repetition judgment is at index identifier nv, then the current index identifier meets the preset conditions, and it is necessary not only to perform a single element repetition judgment at the current index identifier, but also to perform an element fragment repetition judgment at the current index identifier.

[0082] Optionally, if the first pre-sampled element is a valid element, it is determined whether the current index identifier meets a preset condition.

[0083] If the preset conditions are met, the element fragment duplication judgment is started at the current index marker. The specific element fragment duplication judgment method will be described in detail below and will not be elaborated in this application.

[0084] If the preset conditions are not met, sampling continues until the sampled element is a valid element, and the index identifier of the valid element meets the preset conditions, then the element fragment repetition judgment for the valid element begins. Exemplarily, if the preset conditions are not met, it means that the index identifier is not the periodic point position for fragment repetition judgment, and sampling needs to be continued until the sampled element is a valid element, and the index identifier of the valid element meets the preset conditions. The method for judging whether the sampled element is a valid element and whether the position of the valid element meets the preset conditions has been explained in the previous article, and this application will not repeat it here. For example, if the preset conditions are not met, sampling continues, and the index identifier of the next sampled element is n+1. First, a single element repetition judgment is performed on the element with the index identifier n+1. If the pre-sampled element is a valid element, and the periodic point position of the previous fragment repetition judgment is the n+1-vth position, an element fragment repetition judgment is performed.

[0085] As can be seen, in the embodiment of the present application, during the inference and adoption process, a single element duplication check is performed on the element corresponding to each index identifier. This ensures that no single element duplication occurs when an element is sampled by each index identifier, alleviating the pressure of subsequent element fragment duplication checks and improving sampling efficiency. Furthermore, an element fragment duplication check is performed every v index identifiers. By combining single element duplication checks with element fragment duplication checks, both sampling accuracy and sampling efficiency are achieved.

[0086] In one embodiment of the present application, another initial sampling method is provided, see Figure 4 , Figure 4 This is a flow chart of another initial sampling method provided in an embodiment of the present application. Another initial sampling method provided in the present application also includes single element repetition judgment, and the method includes but is not limited to steps 401-407:

[0087] 401: Perform Top-P initial sampling on multiple third candidate elements corresponding to the current index identifier to obtain a first pre-sampled element.

[0088] It should be noted that the method of performing Top-P initial sampling on multiple third candidate elements corresponding to the current index identifier to obtain the first pre-sampling element is similar to the method of performing initial sampling on multiple third candidate elements corresponding to the current index identifier to obtain the first pre-sampling element, and this application will not go into details here.

[0089] 402: Determine whether the first pre-sampled element is an element of a preset type.

[0090] Among them, the preset type element is the sequence end element.

[0091] If yes, go directly to step 406.

[0092] 403: If not, obtain the third element sequence of the data to be sampled.

[0093] It should be noted that the method for obtaining the third element sequence of the data to be sampled has been specifically described in step 202, and this application will not elaborate on it here.

[0094] 404: Determine a second repetition degree of the first pre-sampled element based on the value of the first pre-sampled element and the value of each element in the third element sequence.

[0095] It should be noted that the method for determining the second repetition degree of the first pre-sampled element based on the value of the first pre-sampled element and the value of each element in the third element sequence has been specifically described in step 202 and will not be repeated herein.

[0096] 405: Determine whether the second repetition degree is greater than a second threshold.

[0097] It should be noted that the method for determining whether the second repetition degree is greater than the second threshold has been specifically described in step 202 and will not be elaborated herein.

[0098] 406: If yes, determine that the first pre-sampled element is a valid element, and use the first pre-sampled element as the element corresponding to the current index identifier.

[0099] It should be noted that the method of determining whether the first pre-sampled element is a valid element and using the first pre-sampled element as the element corresponding to the current index identifier has been specifically described in step 202, and will not be repeated here in this application.

[0100] 407: If not, adjust P1 to P1*(1+γ).

[0101] It should be noted that the method of adjusting P1 to P1*(1+γ) has been specifically described in step 203 and will not be repeated in this application.

[0102] See Figure 5 , Figure 5 This is a flowchart of a sampling method provided in an embodiment of the present application. The sampling method provided in the present application includes an element segment repetition judgment process, which includes but is not limited to steps 501-504.

[0103] 501: Obtain the first element sequence of the data to be sampled.

[0104] Exemplarily, through the method described in steps 201-203, it has been determined that the element corresponding to the current index identifier is a valid element and the current index identifier meets the preset conditions. Then the current index identifier is the periodic point position for fragment repetition judgment, and element fragment repetition judgment needs to be performed at the current index identifier.

[0105] Specifically, a first element sequence of the data to be sampled is obtained, wherein the first element sequence includes elements corresponding to a first index identifier, elements corresponding to an index identifier between the first index identifier and the current index identifier, and elements corresponding to the current index identifier, and the first index identifier is smaller than the current index identifier. It should be noted that the data to be sampled can be data in text mode, data in speech mode, or data in image mode, and this application does not make specific limitations on this. In the process of sampling based on the data to be sampled through the large autoregressive model, elements will be output one by one. After obtaining the element corresponding to the current index identifier, the element and the multiple elements before and immediately adjacent to the element (i.e., the first element sequence) will be subjected to element segment repetition judgment. As Figure 6 As shown, each index identifier corresponds to an element, such as Figure 6The index identifiers in are 0, 1, 2, k, i-1, i, n, etc. The first index identifier is i, and the first element sequence (such as Figure 6 The first element sequence shown in FIG. 1 is understood as the element sequence to be judged, wherein the sequence length of the first element sequence is the window length w2 to be judged.

[0106] 502: Obtain a second element sequence of the data to be sampled.

[0107] Exemplarily, a second element sequence of the data to be sampled is obtained, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, wherein the third index identifier is smaller than the first index identifier, and the second index identifier is smaller than the third index identifier. Figure 6 As shown, the second index is identified as k, the third index is identified as i-1, and the second element sequence (such as Figure 6 The sequence length of the second element sequence shown is understood to be the element segment repetition judgment window length. The second element sequence is an element sequence generated before the first element sequence. It should be noted that the second element sequence can be the entire element sequence generated before the first element sequence, or it can be a partial element sequence generated, and this application does not limit this.

[0108] 503: Determine a repeated element sequence based on the elements included in the first element sequence and the elements included in the second element sequence.

[0109] Exemplarily, determining a repeating element sequence based on the elements included in the first element sequence and the elements included in the second element sequence includes: judging the overlap of the first element sequence and the second element sequence to determine multiple overlapping consecutive elements; determining a first degree of repetition of the first element sequence based on the number of the multiple consecutive elements and the number of the first element sequence; if the first degree of repetition is greater than or equal to a first threshold, treating the multiple consecutive elements as a repeating element sequence.

[0110] Optionally, the first element sequence and the second element sequence are overlapped to determine a plurality of overlapping consecutive elements, specifically, as follows: Figure 7As shown, the first element sequence and the second element sequence are subjected to a longest substring matching algorithm to determine overlap, thereby obtaining a repeated element sequence. This means determining the maximum common substring of the first element sequence and the second element sequence, i.e., the overlapping multiple consecutive elements. A first degree of repetition of the first element sequence is determined based on the number of the multiple consecutive elements and the number of the first element sequence. The first degree of repetition of the first element sequence can be determined based on a ratio of the number of the multiple consecutive elements to the number of the first element sequence and preset weights corresponding to the multiple consecutive elements, or the ratio of the number of the multiple consecutive elements to the number of the first element sequence can be directly used as the first degree of repetition.

[0111] The first repeatability r can be calculated using formula (2):

[0112]

[0113] Wherein, w2 is the number of the plurality of consecutive elements, and w3 is the number of the first element sequence.

[0114] If the first repetition degree is greater than or equal to the first threshold, multiple consecutive elements are treated as a repeated element sequence, such as Figure 7 If the first degree of repetition is greater than or equal to the first threshold, it indicates that multiple consecutive elements in the first element sequence have repeated element fragments, and resampling is required based on the repeated element sequence. If the first degree of repetition is less than the first threshold, it indicates that multiple consecutive elements in the first element sequence do not have repeated element fragments, and sampling continues with the element corresponding to the next index identifier.

[0115] 504: Perform sampling based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

[0116] Optionally, the repeated element sequence may be directly resampled based on the method described in steps 201 - 203 or steps 401 - 407 to obtain the output element sequence of the data to be sampled.

[0117] Optionally, when determining the occurrence of element segment duplication, the cause of the element segment duplication is determined by a left and right detection method based on the starting point of the repeated element sequence. The occurrence of element segment duplication in the autoregressive large model may be caused by two reasons: the first reason is that there is actually no deviation in the iterative sampling, and the entire target sequence generation has been completed, but because the random sampling strategy does not sample the sequence end element in time, the autoregressive large model continues to generate randomly; the second reason is that the elements corresponding to a certain index identifier or multiple index identifiers deviate during the iterative sampling process, resulting in an error in the direction of the generated sequence, that is, an iterative sampling deviation occurs. Therefore, the present application needs to determine the cause of the element segment duplication, and then determine different methods for re-sampling based on the repeated element sequence for different reasons. The specific methods are as follows.

[0118] Illustratively, sampling based on the repeating element sequence to obtain the output element sequence of the to-be-sampled data includes: obtaining a fourth index identifier corresponding to a first element in the repeating element sequence, wherein the fourth index identifier is a starting position of the repeating element sequence; and determining, based on a difference between index identifiers and a first threshold, X index identifiers and Y index identifiers that match the fourth index identifier; wherein the difference between the fourth index identifier and the smallest of the X index identifiers is less than the first threshold, the X index identifiers are continuous, the difference between the largest of the Y index identifiers and the fourth index identifier is less than the first threshold, and the Y index identifiers are continuous.

[0119] Specifically, give examples, such as Figure 7 As shown, the fourth index identifier is m. The first threshold may be s, the X index identifiers that match the fourth index identifier are index identifiers between m and m-s1, and the Y index identifiers that match the fourth index identifier are m+s2, where s1 and s2 are both less than the first threshold, and s1 and s2 may be the same or different, which is not specifically limited in this application.

[0120] Specifically, the fourth index identifier corresponding to the first element in the repeating element sequence is obtained. Left and right detection is initiated based on the fourth index identifier corresponding to the first element in the repeating element sequence, with X index identifiers and Y index identifiers detected to the left and right, respectively, to obtain X index identifiers and Y index identifiers, respectively. This can also be understood as forward and backward detection, with X index identifiers detected forward and Y index identifiers detected backward, respectively, which is not distinguished in this application.

[0121] Furthermore, the first candidate elements corresponding to the X index identifiers and the second candidate elements corresponding to the Y index identifiers are determined. Because during sampling, iterative sampling is performed at each index identifier to obtain a predicted element probability distribution, and then multiple elements with the highest probability are selected as multiple first candidate elements corresponding to the current index identifier, the first candidate elements corresponding to the X index identifiers and the second candidate elements corresponding to the Y index identifiers can be directly obtained.

[0122] Exemplarily, sampling is performed based on the first candidate element, the second candidate element, and the repeated element sequence to obtain the output element sequence. Based on the first candidate element and the second candidate element, the cause of the element segment duplication can be determined, and then different methods for obtaining the output element sequence are determined for different causes. Specifically, if a preset type element exists in the first candidate element, or if a preset type element does not exist in the first candidate element and a preset type element exists in the second candidate element, then the cause of the element segment duplication is the first cause mentioned above; if the preset type element does not exist in either the first candidate element or the second candidate element, then the cause of the element segment duplication is the second cause mentioned above.

[0123] As can be seen, in the embodiment of the present application, sampling is performed based on the first candidate element, the second candidate element, and the repeated element sequence to obtain the output element sequence. Based on each of the first candidate element and the second candidate element, the cause of the element segment duplication can be determined. Based on different causes, different methods for resampling based on the repeated element sequence to obtain the output element sequence can be determined. This allows for different case-by-case handling of element segment duplication issues, making the sampling method more scientific and efficient.

[0124] Optionally, if there is a preset type element in the first candidate element, the output element sequence is obtained based on the index identifier corresponding to the preset type element. The preset type element is a sequence end element. If there is a preset type element in the first candidate element, there is a high probability of a sequence end element prediction result in the first candidate element to the left of the starting point of the repeated element sequence, indicating that the entire target sequence has been generated before the fourth index identifier, but because the random sampling strategy did not sample the sequence end element in time, the autoregressive large model continued to generate randomly, which means that the reason for the repetition of element fragments is the first reason mentioned above. At this time, the output element sequence should be obtained based on the index identifier corresponding to the preset type element.

[0125] Furthermore, based on the index identifier corresponding to the preset type element, the output element sequence is obtained, including: determining the minimum index identifier among the index identifiers corresponding to the preset type element; deleting the target element corresponding to the data to be sampled, and obtaining the output element sequence, wherein the target element includes the element corresponding to the minimum index identifier and the element corresponding to the index identifier located after the minimum index identifier. First, obtain the index identifier corresponding to the preset type element, determine the minimum index identifier (the index identifier at the front position), that is, the index identifier of the sequence end element with a high probability of appearing first, determine the minimum index identifier at the front position as the correct end position, and end the model inference loop at the correct end position to directly obtain the output element sequence, that is, delete the target element corresponding to the data to be sampled, and obtain the output element sequence, wherein the target element includes the element corresponding to the minimum index identifier and the element corresponding to the index identifier located after the minimum index identifier.

[0126] It can be seen that in an embodiment of the present application, if there is a preset type element in the first candidate element, the output element sequence is obtained based on the index identifier corresponding to the preset type element; among the index identifiers corresponding to the preset type elements, the minimum index identifier is determined; the target element corresponding to the data to be sampled is deleted to obtain the output element sequence, and the target element includes the element corresponding to the minimum index identifier and the element corresponding to the index identifier located after the minimum index identifier, that is, in the first candidate element to the left of the starting point of the repeated element sequence, there is a high probability of a sequence end element prediction result, indicating that the entire generated sequence has been generated before the fourth index identifier, but because the random sampling strategy did not sample the sequence end element in time, the autoregressive large model continued to generate randomly, which means that the reason for the duplication of element fragments is the first reason mentioned above. First, obtain the minimum index identifier among the index identifiers corresponding to the preset type elements, that is, the index identifier of the sequence end element that appears first with a high probability, and determine the minimum index identifier at the front as the correct end position. The model reasoning loop should be ended at the correct end position to directly obtain the output element sequence, that is, the target element corresponding to the data to be sampled is deleted to obtain the output element sequence. In this embodiment, the cause of the repeated element fragment is first determined, and then the correct end position is determined, so that the model ends the reasoning loop at the correct end position and directly obtains the output element sequence. The situation in which the autoregressive large model continues to generate randomly due to the random sampling strategy not sampling the sequence end element in time is stopped, and the correct output element sequence is obtained, thereby improving the sampling accuracy, thereby improving the accuracy of the model-generated content, making the generated content meet user expectations, and improving the user experience.

[0127] Optionally, if there is no preset type element in the first candidate elements and there is a preset type element in the second candidate elements, the repeated element sequence is sampled starting from the fourth index identifier to obtain the output element sequence.

[0128] If there is no preset type element in the first candidate element, and there is a preset type element in the second candidate element, then it means that the position of the predicted result of the sequence end element with a high probability is not before the fourth index identifier, but after the fourth index identifier. At this time, the index identifier corresponding to the preset type element in the second candidate element is not obtained to determine the minimum index identifier, and the element corresponding to the minimum index identifier and the element corresponding to the index identifier after the minimum index identifier are deleted. Instead, starting from the fourth index identifier, the repeated element sequence is sampled to obtain the output element sequence. Because directly deleting the element corresponding to the minimum index identifier and the element corresponding to the index identifier after the minimum index identifier cannot solve the problem of repeated element segments from the fourth index identifier to the minimum index identifier, the repeated element sequence should be sampled starting from the fourth index identifier to obtain the output element sequence.

[0129] Further, starting from the fourth index identifier, the repeated element sequence is sampled to obtain the output element sequence, including: deleting the element corresponding to the fourth index identifier and deleting the element corresponding to the index identifier located after the fourth index identifier, resampling the element corresponding to the fourth index identifier, and obtaining a new element corresponding to the fourth index identifier; obtaining a first new element sequence of the data to be sampled, wherein the first new element sequence includes the element corresponding to the fifth index identifier, the element corresponding to the index identifier located between the fifth index identifier and the fourth index identifier, and the new element corresponding to the fourth index identifier, wherein the fifth index identifier is smaller than the fourth index identifier; obtaining a second new element sequence of the data to be sampled, wherein the second new element sequence The prime sequence includes the element corresponding to the sixth index identifier, the element corresponding to the index identifier between the sixth index identifier and the seventh index identifier, and the element corresponding to the seventh index identifier, the sixth index identifier is smaller than the seventh index identifier, and the seventh index identifier is smaller than the fifth index identifier; if there is no repeated element sequence between the first new element sequence and the second new element sequence, the element corresponding to the next index identifier of the fourth index identifier is sampled until the preset type element is sampled, the sampling process is terminated, and the output element sequence is obtained; if there is a repeated element sequence between the first new element sequence and the second new element sequence, sampling is performed based on the repeated element sequence until the preset type element is sampled, the sampling process is terminated, and the output element sequence is obtained. It should be noted that the method of resampling the element corresponding to the fourth index identifier to obtain the new element corresponding to the fourth index identifier is similar to the method of sampling the element corresponding to the current index identifier, and this application will not elaborate on it here.

[0130] It can be seen that in an embodiment of the present application, if there is no preset type element in the first candidate element, and there is a preset type element in the second candidate element, it means that the index identifier corresponding to the sequence end element prediction result with a high probability is after the fourth index identifier. At this time, the index identifier corresponding to the preset type element in the second candidate element is not obtained to determine the minimum index identifier, and the element corresponding to the minimum index identifier and the element corresponding to the index identifier after the minimum index identifier are deleted. Instead, starting from the fourth index identifier, the repeated element sequence is sampled to obtain the output element sequence. Because directly deleting the element corresponding to the minimum index identifier and the element corresponding to the index identifier after the minimum index identifier cannot solve the problem of repeated element fragments from the fourth index identifier to the minimum index identifier, the repeated element sequence should be sampled starting from the fourth index identifier to obtain the output element sequence. In this way, the problem of repeated element fragments from the fourth index identifier to the minimum index identifier is solved, the accuracy of sampling is improved, and the accuracy of model-generated content is improved, so that the generated content meets user expectations and improves user experience.

[0131] Optionally, if there is no preset type element in either the first candidate element or the second candidate element, then based on the fourth index identifier and the second threshold, determine a target index identifier that is spaced from the fourth index identifier by the second threshold; starting from the target index identifier, perform sampling to obtain the output element sequence, wherein the target index identifier is smaller than the fourth index identifier. If there is no preset type element in either the first candidate element or the second candidate element, it means that no highly probable sequence end element prediction result is found near the element corresponding to the fourth index identifier, that is, the reason for the element segment repetition at this time is not the first reason, but the second reason, that is, the sampling result of iterative sampling of a certain index identifier or multiple index identifiers deviates, resulting in an error in the direction of the generated sequence, that is, an iterative sampling deviation occurs.

[0132] It should be noted that sampling is performed based on the fourth index identifier to obtain the output element sequence, and based on the fourth index identifier and the second threshold, a target index identifier that is separated from the fourth index identifier by the second threshold can be determined; starting from the target index identifier, sampling is performed to obtain the output element sequence.

[0133] Specifically, starting from the target index identifier, sampling is performed to obtain the output element sequence, including: deleting the element corresponding to the target index identifier, and deleting the element corresponding to the index identifier located after the target index identifier, and resampling the element corresponding to the target index identifier to obtain a new element corresponding to the target index identifier; obtaining a third new element sequence of the data to be sampled, wherein the third new element sequence includes the element corresponding to the eighth index identifier, the element corresponding to the index identifier located between the eighth index identifier and the target index identifier, and the new element corresponding to the target index identifier, wherein the eighth index identifier is smaller than the target index identifier; obtaining a fourth new element sequence of the data to be sampled, wherein the fourth new element sequence includes Including the elements corresponding to the ninth index identifier, the elements corresponding to the index identifiers between the ninth index identifier and the tenth index identifier, and the elements corresponding to the tenth index identifier, the ninth index identifier is smaller than the tenth index identifier, and the tenth index identifier is smaller than the eighth index identifier; if there is no repeated element sequence between the third new element sequence and the fourth new element sequence, then the elements corresponding to the next index identifier of the target index identifier are sampled until the preset type element is sampled, the sampling process is terminated, and the output element sequence is obtained; if there is a repeated element sequence between the third new element sequence and the fourth new element sequence, then sampling is performed based on the repeated element sequence until the preset type element is sampled, the sampling process is terminated, and the output element sequence is obtained. It should be noted that the method of resampling the elements corresponding to the target index identifier to obtain the new elements corresponding to the target index identifier is similar to the method of sampling the elements corresponding to the current index identifier, and this application will not elaborate on them here.

[0134] It can be seen that in an embodiment of the present application, if there is no preset type element in either the first candidate element or the second candidate element, it means that the element fragment duplication due to the second reason has occurred, indicating that the overall reasoning sequence direction of the model has deviated, and the deviation must have occurred before the starting position of the repeated element sequence (the fourth index identifier). Therefore, if it is directly cut off and discarded at the fourth index identifier, although it may seem that a few steps of reasoning may be reduced subsequently, the turning point where the reasoning direction is wrong is most likely before the fourth index identifier. If it is directly cut off at the fourth index identifier, there is a high probability that element fragment duplication may still occur in subsequent reasoning. Therefore, the present application starts from the target index identifier and re-samples, which can locate the exact inflection point where the reasoning direction is wrong (i.e., the target index identifier). Re-sampling from this inflection point can prevent element fragment duplication from occurring here, improve the accuracy of sampling, and thus improve the accuracy of model-generated content, so that the generated content meets user expectations and improves user experience.

[0135] It can be seen that in the embodiment of the present application, if there is a preset type element in the first candidate element, then based on the index identifier corresponding to the preset type element, the output element sequence is obtained; if there is no preset type element in the first candidate element, and there is a preset type element in the second candidate element, then starting from the fourth index identifier, the repeated element sequence is sampled to obtain the output element sequence; if there is no preset type element in both the first candidate element and the second candidate element, then based on the fourth index identifier and the second threshold, the target index identifier that is spaced from the fourth index identifier by the second threshold is determined; starting from the target index identifier, sampling is performed to obtain the output element sequence, wherein the target index identifier is less than the fourth index identifier. When there is repeated element fragments, processing is performed according to the situation, making the sampling method more scientific and efficient, and avoiding the recurrence of repeated element fragments, improving the accuracy of sampling, thereby improving the accuracy of model-generated content, making the generated content meet user expectations, and improving user experience.

[0136] It can be seen that in an embodiment of the present application, after sampling and obtaining the element corresponding to the current index identifier, the element corresponding to the current index identifier and the first element sequence that is before the element corresponding to the current index identifier and is adjacent to the element corresponding to the current index identifier will be repeatedly judged. Specifically, a first element sequence of the data to be sampled is obtained, wherein the first element sequence includes the element corresponding to the first index identifier, the element corresponding to the index identifier located between the first index identifier and the current index identifier, and the element corresponding to the current index identifier. Further, a second element sequence of the data to be sampled is obtained, wherein the second element sequence includes the element corresponding to the second index identifier, the element corresponding to the index identifier located between the second index identifier and the third index identifier, and the element corresponding to the third index identifier. The second element sequence is an element sequence that has been generated before the first element sequence, wherein the first element sequence can be understood as an element sequence to be judged. Based on the elements included in the first element sequence and the elements included in the second element sequence, a repeated element sequence is determined, that is, based on the already generated element sequence (second element sequence), a repeated element sequence in the element sequence to be judged (first element sequence) and the already generated element sequence (second element sequence) is determined. The repeated element sequence is an unexpected result sampled by the autoregressive large model in error, and the repeated element sequence cannot be used as part of the output element sequence obtained by the final sampling. Therefore, sampling is performed based on the repeated element sequence to obtain the output element sequence of the data to be sampled. Sampling is performed again based on the repeated element sequence to sample new non-repeating elements, avoiding using the repeated element sequence as part of the output element sequence obtained by sampling the data to be sampled, so that the model can sample a non-repeating output element sequence that meets expectations, improve the accuracy of sampling, and thus improve the accuracy of the content generated by the model, so that the generated content meets user expectations and improves the user experience.

[0137] See Figure 8 , Figure 8 A flow chart of another sampling method provided in an embodiment of the present application, which includes but is not limited to steps 801-805, 8051-8052, and 8051A-8051C:

[0138] 801: Obtain the first element sequence of the data to be sampled.

[0139] It should be noted that the specific method for obtaining the first element sequence of the data to be sampled has been specifically described in step 501, and this application will not repeat it here.

[0140] 802: Obtain a second element sequence of the data to be sampled.

[0141] It should be noted that the method for obtaining the second element sequence of the data to be sampled has been specifically described in step 502, and this application will not repeat it here.

[0142] 803: Determine a repeated element sequence based on the elements included in the first element sequence and the elements included in the second element sequence.

[0143] It should be noted that the method for determining a repeated element sequence based on the elements included in the first element sequence and the elements included in the second element sequence has been specifically described in step 503, and this application will not repeat it here.

[0144] 804: Determine a first repetitiveness of the first element sequence based on the number of the plurality of consecutive elements and the number of the first element sequence.

[0145] It should be noted that the method for determining the first repetition degree of the first element sequence based on the number of the plurality of consecutive elements and the number of the first element sequence has been specifically described in step 503 and will not be repeated herein.

[0146] 805: Determine whether the first repetition degree is greater than a first threshold.

[0147] It should be noted that the method for determining whether the first repetition degree is greater than the first threshold has been specifically described in step 503 and will not be repeated herein.

[0148] 8051: If yes, obtain a fourth index identifier corresponding to the first element in the repeated element sequence, and determine X index identifiers and Y index identifiers that match the fourth index identifier based on a difference between index identifiers and a first threshold.

[0149] It should be noted that, if so, the method of obtaining the fourth index identifier corresponding to the first element in the repeated element sequence and determining the X index identifiers and Y index identifiers that match the fourth index identifier based on the difference between the index identifiers and the first threshold has been specifically described in step 504 and will not be repeated here in this application.

[0150] 8051A: If there is a preset type element among the first candidate elements, determine the minimum index identifier among the index identifiers corresponding to the preset type elements; delete the target element corresponding to the data to be sampled to obtain the output element sequence.

[0151] It should be noted that if there is a preset type element in the first candidate element, the minimum index identifier is determined among the index identifiers corresponding to the preset type elements; the target element corresponding to the data to be sampled is deleted, and the method for obtaining the output element sequence has been specifically described in step 504, and this application will not repeat it here.

[0152] 8051B: If no element of the preset type exists in the first candidate elements, and an element of the preset type exists in the second candidate elements, sampling the repeated element sequence starting from the fourth index identifier to obtain the output element sequence;

[0153] It should be noted that if there is no preset type element in the first candidate element and there is a preset type element in the second candidate element, then starting from the fourth index identifier, the repeated element sequence is sampled to obtain the output element sequence. The method has been specifically described in step 504 and will not be repeated here in this application.

[0154] 8051C: If there is no preset type element in the first candidate element and the second candidate element, based on the fourth index identifier and the second threshold, determine the target index identifier that is separated from the fourth index identifier by the second threshold; starting from the target index identifier, sample to obtain the output element sequence.

[0155] It should be noted that if there is no preset type element in either the first candidate element or the second candidate element, then based on the fourth index identifier and the second threshold, a target index identifier that is separated from the fourth index identifier by the second threshold is determined; starting from the target index identifier, sampling is performed to obtain the output element sequence. The method has been specifically described in step 504 and will not be repeated here in this application.

[0156] 8052: If not, no element segment duplication occurs, sampling the element corresponding to the next index identifier of the fourth index identifier.

[0157] It should be noted that the method for sampling the element corresponding to the next index identifier of the fourth index identifier is similar to the method for sampling the element corresponding to the current index identifier, and this application will not go into details here.

[0158] See Figure 9 , Figure 9 A flow chart of another sampling method provided in an embodiment of the present application, which includes but is not limited to steps 901-904 and 9041-9042:

[0159] 901: Perform initial sampling on multiple third candidate elements corresponding to the current index identifier to obtain a first pre-sampled element.

[0160] It should be noted that the method of performing initial sampling on the multiple third candidate elements corresponding to the current index identifier to obtain the first pre-sampled element has been described in step 201 and will not be repeated in this application.

[0161] 902: Perform single element duplication determination on the first pre-sampled element.

[0162] It should be noted that the method for performing a single element repetition judgment on the first pre-sampled element has been described in steps 201-203, and this application will not elaborate on it here.

[0163] 903: If the current index identifier is a periodic point position for segment duplication determination, perform element segment duplication determination based on the current index identifier.

[0164] It should be noted that the method for determining element segment duplication based on the current index identifier has been described in steps 501-504, and this application will not elaborate on it here.

[0165] 904: Determine whether a preset type element is sampled.

[0166] 9041: If yes, sampling ends.

[0167] 9042: If not, sample the element corresponding to the next index identifier of the current index identifier.

[0168] It should be noted that this application describes the settings for the single-element repetition judgment window length w1 and the pending judgment window length w2. The selection of w1 and w2 will vary depending on the model's capabilities and application domain. For example, in the NLP field, where a large model generates a sentence, the single-element repetition judgment window length w1 and the pending judgment window length w2 tend to be longer. For example, suppose the final inference result translates to the text sequence "The Taklamakan Desert has the characteristics of the Taklamakan Desert." If the w2 window is too short, it can easily be misidentified as a repeated element segment. Conversely, if a large TTS model generates speech elements, each element may represent 4-6 frames. If a frame is 20 milliseconds, then an element has a pronunciation duration of nearly 100 milliseconds. Since speech information is richer than text, theoretically, even if the sentence "The Taklamakan Desert has the characteristics of the Taklamakan Desert..." is translated into speech, the resulting elements should be different due to differences in pronunciation pitch, intonation, and speaking speed, even for the same text. At this time, we can set the single element repeated judgment window length w1 and the to-be-judged window length w2 to be shorter, so that the standards for single element repeated judgment and element fragment repeated judgment are more stringent, preventing repeated sampling and improving sampling accuracy, thereby improving the accuracy of model-generated content, making the generated content meet user expectations, and improving user experience.

[0169] Furthermore, the second threshold can be set slightly smaller, making the criteria for determining single element duplication more stringent. This prevents the occurrence of single element duplication in the generated target sequence, improves sampling accuracy, and thus improves the precision of the model-generated content, ensuring that the generated content meets user expectations and enhances the user experience. Of course, this can also be set based on the application domain of the model. The first threshold tends to be slightly larger to prevent false positives. The size of the first and second thresholds is also related to the corresponding single element duplication determination window length w1 and the length of the window to be determined w2. The smaller the window length, the larger the threshold should be; the larger the window length, the smaller R should be. It is important to prevent both missed judgments and false positives.

[0170] See Figure 10 , Figure 10 This is a block diagram of the functional units of a sampling device provided in an embodiment of the present application. The sampling device 1000 includes: a transceiver unit 1001 and a processing unit 1002;

[0171] The transceiver unit 1001 is configured to obtain a first element sequence of data to be sampled, wherein the first element sequence includes an element corresponding to a first index identifier, an element corresponding to an index identifier between the first index identifier and a current index identifier, and an element corresponding to the current index identifier, wherein the first index identifier is smaller than the current index identifier;

[0172] Processing unit 1002 is configured to obtain a second element sequence of the data to be sampled, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, wherein the third index identifier is smaller than the first index identifier, and the second index identifier is smaller than the third index identifier;

[0173] determining a repeating element sequence based on elements included in the first element sequence and elements included in the second element sequence;

[0174] Sampling is performed based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

[0175] In one embodiment of the present application, in performing sampling based on the repeated element sequence to obtain the output element sequence of the to-be-sampled data, the processing unit 1002 is specifically configured to:

[0176] Obtaining the fourth index identifier corresponding to the first element in the repeated element sequence;

[0177] Determining, based on the difference between the index identifiers and a first threshold, X index identifiers and Y index identifiers that match the fourth index identifier; wherein the difference between the fourth index identifier and the smallest of the X index identifiers is less than the first threshold, and the X index identifiers are consecutive; and the difference between the largest of the Y index identifiers and the fourth index identifier is less than the first threshold, and the Y index identifiers are consecutive;

[0178] Determine a first candidate element corresponding to the X index identifiers and a second candidate element corresponding to the Y index identifiers;

[0179] Sampling is performed based on the first candidate element, the second candidate element, and the repeated element sequence to obtain the output element sequence.

[0180] In one embodiment of the present application, in obtaining the output element sequence by sampling based on the first candidate element, the second candidate element, and the repeated element sequence, the processing unit 1002 is specifically configured to:

[0181] If there is a preset type element in the first candidate elements, obtaining the output element sequence based on the index identifier corresponding to the preset type element;

[0182] If no element of the preset type exists in the first candidate elements and an element of the preset type exists in the second candidate elements, sampling the repeated element sequence starting from the fourth index identifier to obtain the output element sequence;

[0183] If there is no preset type element in either the first candidate element or the second candidate element, then based on the fourth index identifier and the second threshold, determine the target index identifier that is separated from the fourth index identifier by the second threshold; starting from the target index identifier, sample to obtain the output element sequence, wherein the target index identifier is smaller than the fourth index identifier.

[0184] In one embodiment of the present application, in terms of obtaining the output element sequence based on the index identifier corresponding to the preset type element, the processing unit 1002 is specifically configured to:

[0185] Determine the minimum index identifier among the index identifiers corresponding to the preset type elements;

[0186] The target element corresponding to the data to be sampled is deleted to obtain the output element sequence, where the target element includes the element corresponding to the minimum index identifier and the elements corresponding to the index identifiers located after the minimum index identifier.

[0187] In one embodiment of the present application, in terms of sampling the repeated element sequence starting from the fourth index marker to obtain the output element sequence, the processing unit 1002 is specifically configured to:

[0188] Deleting the element corresponding to the fourth index identifier and the elements corresponding to the index identifiers after the fourth index identifier, resampling the element corresponding to the fourth index identifier, and obtaining a new element corresponding to the fourth index identifier;

[0189] Obtaining a first new element sequence of the data to be sampled, wherein the first new element sequence includes an element corresponding to a fifth index identifier, an element corresponding to an index identifier between the fifth index identifier and the fourth index identifier, and a new element corresponding to the fourth index identifier, wherein the fifth index identifier is smaller than the fourth index identifier;

[0190] Obtaining a second new element sequence of the data to be sampled, wherein the second new element sequence includes an element corresponding to a sixth index identifier, an element corresponding to an index identifier between the sixth index identifier and a seventh index identifier, and an element corresponding to the seventh index identifier, the sixth index identifier is smaller than the seventh index identifier, and the seventh index identifier is smaller than the fifth index identifier;

[0191] If there is no repeated element sequence between the first new element sequence and the second new element sequence, sampling the element corresponding to the next index identifier of the fourth index identifier until the preset type element is sampled, ending the sampling process, and obtaining the output element sequence;

[0192] If there is a repeated element sequence between the first new element sequence and the second new element sequence, sampling is performed based on the repeated element sequence until the preset type element is sampled, and the sampling process is terminated to obtain the output element sequence.

[0193] In one embodiment of the present application, in terms of sampling from the target index identifier to obtain the output element sequence, the processing unit 1002 is specifically configured to:

[0194] Delete the element corresponding to the target index identifier, delete the elements corresponding to the index identifiers located after the target index identifier, and resample the elements corresponding to the target index identifier to obtain new elements corresponding to the target index identifier;

[0195] Obtaining a third new element sequence of the data to be sampled, wherein the third new element sequence includes an element corresponding to an eighth index identifier, an element corresponding to an index identifier located between the eighth index identifier and the target index identifier, and a new element corresponding to the target index identifier, wherein the eighth index identifier is smaller than the target index identifier;

[0196] Obtaining a fourth new element sequence of the data to be sampled, wherein the fourth new element sequence includes an element corresponding to a ninth index identifier, elements corresponding to index identifiers between the ninth index identifier and the tenth index identifier, and an element corresponding to the tenth index identifier, the ninth index identifier being smaller than the tenth index identifier, and the tenth index identifier being smaller than the eighth index identifier;

[0197] If there is no repeated element sequence between the third new element sequence and the fourth new element sequence, sampling the element corresponding to the next index identifier of the target index identifier until the preset type element is sampled, ending the sampling process, and obtaining the output element sequence;

[0198] If there is a repeated element sequence between the third new element sequence and the fourth new element sequence, sampling is performed based on the new repeated element sequence until the preset type element is sampled, and the sampling process is terminated to obtain the output element sequence.

[0199] In one embodiment of the present application, in determining a repeating element sequence based on the first element sequence and the second element sequence, the processing unit 1002 is specifically configured to:

[0200] Performing an overlap determination on the first element sequence and the second element sequence to determine a plurality of overlapping consecutive elements;

[0201] determining a first degree of repetitiveness of the first sequence of elements based on the number of the plurality of consecutive elements and the number of the first sequence of elements;

[0202] If the first degree of repetition is greater than or equal to a fifth threshold, the plurality of consecutive elements are used as the repetitive element sequence.

[0203] In one embodiment of the present application, the processing unit 1002 is specifically configured to:

[0204] Performing initial sampling on a plurality of third candidate elements corresponding to the current index identifier to obtain a first pre-sampled element;

[0205] If the first pre-sampled element is a valid element, use the first pre-sampled element as the element corresponding to the current index identifier;

[0206] If the first pre-sampled element is not a valid element, multiple third candidate elements corresponding to the current index identifier are adjusted based on a first probability accumulation threshold and a scaling factor to obtain an element corresponding to the current index identifier.

[0207] In one embodiment of the present application, in terms of adjusting the multiple third candidate elements corresponding to the current index identifier based on the sampled first probability accumulation threshold and the scaling factor to obtain the element corresponding to the current index identifier, the processing unit 1002 is specifically configured to:

[0208] Adjusting the first probability accumulation threshold based on the scaling factor to obtain a second probability accumulation threshold;

[0209] Based on the second probability accumulation threshold, obtaining a plurality of fourth candidate elements corresponding to the current index identifier, wherein the number of the plurality of fourth candidate elements is greater than the number of the plurality of third candidate elements;

[0210] Randomly sampling from the plurality of fourth candidate elements to obtain a second pre-sampled element;

[0211] If the second pre-sampled element is different from the first pre-sampled element, the second pre-sampled element is used as the element corresponding to the current index identifier.

[0212] See Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 11 As shown, the electronic device 1100 includes a transceiver 1101, a processor 1102, and a memory 1103. These are connected via a bus 1104. The memory 1103 is used to store computer programs and data, and can transmit the data stored in the memory 1103 to the processor 1102.

[0213] The processor 1102 is configured to read the computer program in the memory 1103 and perform the following operations:

[0214] Controlling the transceiver 1101 to obtain a first element sequence of data to be sampled, wherein the first element sequence includes an element corresponding to a first index identifier, an element corresponding to an index identifier between the first index identifier and a current index identifier, and an element corresponding to the current index identifier, and the first index identifier is smaller than the current index identifier;

[0215] Obtaining a second element sequence of the data to be sampled, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, wherein the third index identifier is smaller than the first index identifier, and the second index identifier is smaller than the third index identifier;

[0216] determining a repeating element sequence based on elements included in the first element sequence and elements included in the second element sequence;

[0217] Sampling is performed based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

[0218] Specifically, the transceiver 1101 may be Figure 10 The transceiver unit 1001 of the sampling device 1000 of the embodiment, the processor 1102 may be Figure 10 The processing unit 1002 of the sampling device 1000 of the embodiment.

[0219] Specifically, the transceiver 1101 may be Figure 10 The transceiver unit 1001 of the sampling device 1000 of the embodiment, the processor 1102 may be Figure 10 Therefore, the specific functions of the processor 1102 can refer to the specific functions of the processing unit 1002, and the specific functions of the transceiver 1101 can refer to the specific functions of the transceiver 1001.

[0220] It should be understood that the electronic devices in this application may include smartphones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices (MIDs) or wearable devices. The above electronic devices are only examples and are not exhaustive, including but not limited to the above electronic devices. In actual applications, the above electronic devices may also include: smart car terminals, computer equipment, etc.

[0221] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all of the steps of any sampling method described in the above method embodiments.

[0222] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any sampling method described in the above method embodiments.

[0223] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of actions, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0224] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0226] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0227] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0228] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0229] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0230] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A sampling method, characterized in that: The method comprises: Obtaining a first element sequence of data to be sampled, wherein the first element sequence includes an element corresponding to a first index identifier, an element corresponding to an index identifier between the first index identifier and a current index identifier, and an element corresponding to the current index identifier, and the first index identifier is smaller than the current index identifier; Obtaining a second element sequence of the data to be sampled, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, wherein the third index identifier is smaller than the first index identifier, and the second index identifier is smaller than the third index identifier; determining a repeating element sequence based on elements included in the first element sequence and elements included in the second element sequence; Sampling is performed based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

2. The method according to claim 1, characterized in that The sampling based on the repeated element sequence to obtain the output element sequence of the data to be sampled includes: Obtaining the fourth index identifier corresponding to the first element in the repeated element sequence; Determining, based on the difference between the index identifiers and a first threshold, X index identifiers and Y index identifiers that match the fourth index identifier; wherein the difference between the fourth index identifier and the smallest of the X index identifiers is less than the first threshold, and the X index identifiers are consecutive; and the difference between the largest of the Y index identifiers and the fourth index identifier is less than the first threshold, and the Y index identifiers are consecutive; Determine a first candidate element corresponding to the X index identifiers and a second candidate element corresponding to the Y index identifiers; Sampling is performed based on the first candidate element, the second candidate element, and the repeated element sequence to obtain the output element sequence.

3. The method according to claim 2, characterized in that The sampling based on the first candidate element, the second candidate element, and the repeated element sequence to obtain the output element sequence includes: If there is a preset type element in the first candidate elements, obtaining X index identifiers of the output element sequence based on the index identifier corresponding to the preset type element; If no element of the preset type exists in the first candidate elements and an element of the preset type exists in the second candidate elements, sampling the repeated element sequence starting from the fourth index identifier to obtain the output element sequence; If there is no preset type element in either the first candidate element or the second candidate element, then based on the fourth index identifier and the second threshold, determine the target index identifier that is separated from the fourth index identifier by the second threshold; starting from the target index identifier, sample to obtain the output element sequence, wherein the target index identifier is smaller than the fourth index identifier.

4. The method according to claim 3, characterized in that The obtaining the output element sequence based on the index identifier corresponding to the preset type element includes: Determine the minimum index identifier among the index identifiers corresponding to the preset type elements; The target element corresponding to the data to be sampled is deleted to obtain the output element sequence, where the target element includes the element corresponding to the minimum index identifier and the elements corresponding to the index identifiers located after the minimum index identifier.

5. The method according to claim 3 or 4, characterized in that The step of sampling the repeated element sequence starting from the fourth index marker to obtain the output element sequence includes: Deleting the element corresponding to the fourth index identifier and the elements corresponding to the index identifiers after the fourth index identifier, resampling the element corresponding to the fourth index identifier, and obtaining a new element corresponding to the fourth index identifier; Obtaining a first new element sequence of the data to be sampled, wherein the first new element sequence includes an element corresponding to a fifth index identifier, an element corresponding to an index identifier between the fifth index identifier and the fourth index identifier, and a new element corresponding to the fourth index identifier, wherein the fifth index identifier is smaller than the fourth index identifier; Obtaining a second new element sequence of the data to be sampled, wherein the second new element sequence includes an element corresponding to a sixth index identifier, an element corresponding to an index identifier between the sixth index identifier and a seventh index identifier, and an element corresponding to the seventh index identifier, the sixth index identifier is smaller than the seventh index identifier, and the seventh index identifier is smaller than the fifth index identifier; If there is no repeated element sequence between the first new element sequence and the second new element sequence, sampling the element corresponding to the next index identifier of the fourth index identifier until the preset type element is sampled, ending the sampling process, and obtaining the output element sequence; If there is a repeated element sequence between the first new element sequence and the second new element sequence, sampling is performed based on the repeated element sequence until the preset type element is sampled, and the sampling process is terminated to obtain the output element sequence.

6. The method according to any one of claims 3 to 5, characterized in that: The step of sampling from the target index identifier to obtain the output element sequence includes: Delete the element corresponding to the target index identifier, delete the elements corresponding to the index identifiers located after the target index identifier, and resample the elements corresponding to the target index identifier to obtain new elements corresponding to the target index identifier; Obtaining a third new element sequence of the data to be sampled, wherein the third new element sequence includes an element corresponding to an eighth index identifier, an element corresponding to an index identifier located between the eighth index identifier and the target index identifier, and a new element corresponding to the target index identifier, wherein the eighth index identifier is smaller than the target index identifier; Obtaining a fourth new element sequence of the data to be sampled, wherein the fourth new element sequence includes an element corresponding to a ninth index identifier, elements corresponding to index identifiers between the ninth index identifier and the tenth index identifier, and an element corresponding to the tenth index identifier, the ninth index identifier being smaller than the tenth index identifier, and the tenth index identifier being smaller than the eighth index identifier; If there is no repeated element sequence between the third new element sequence and the fourth new element sequence, sampling the element corresponding to the next index identifier of the target index identifier until the preset type element is sampled, ending the sampling process, and obtaining the output element sequence; If there is a repeated element sequence between the third new element sequence and the fourth new element sequence, sampling is performed based on the repeated element sequence until the preset type element is sampled, and the sampling process is terminated to obtain the output element sequence.

7. The method according to any one of claims 1 to 6, characterized in that The determining of a repeating element sequence based on the first element sequence and the second element sequence includes: Performing an overlap determination on the first element sequence and the second element sequence to determine a plurality of overlapping consecutive elements; determining a first degree of repetitiveness of the first sequence of elements based on the number of the plurality of consecutive elements and the number of the first sequence of elements; If the first degree of repetition is greater than or equal to a fifth threshold, the plurality of consecutive elements are used as the repetitive element sequence.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Performing initial sampling on a plurality of third candidate elements corresponding to the current index identifier to obtain a first pre-sampled element; If the first pre-sampled element is a valid element, use the first pre-sampled element as the element corresponding to the current index identifier; If the first pre-sampled element is not a valid element, multiple third candidate elements corresponding to the current index identifier are adjusted based on a first probability accumulation threshold and a scaling factor to obtain an element corresponding to the current index identifier.

9. The method according to claim 7 or 8, characterized in that The step of adjusting the plurality of third candidate elements corresponding to the current index identifier based on the first probability accumulation threshold and the scaling factor to obtain the element corresponding to the current index identifier includes: Adjusting the first probability accumulation threshold based on the scaling factor to obtain a second probability accumulation threshold; Based on the second probability accumulation threshold, obtaining a plurality of fourth candidate elements corresponding to the current index identifier, wherein the number of the plurality of fourth candidate elements is greater than the number of the plurality of third candidate elements; Randomly sampling from the plurality of fourth candidate elements to obtain a second pre-sampled element; If the second pre-sampled element is different from the first pre-sampled element, the second pre-sampled element is used as the element corresponding to the current index identifier.

10. A sampling device, characterized in that: The device includes: transceiver unit and processing unit; The transceiver unit is configured to obtain a first element sequence of data to be sampled, wherein the first element sequence includes an element corresponding to a first index identifier, an element corresponding to an index identifier between the first index identifier and a current index identifier, and an element corresponding to the current index identifier, and the first index identifier is smaller than the current index identifier; the transceiver unit being configured to obtain a second element sequence of the data to be sampled, wherein the second element sequence includes an element corresponding to a second index identifier, an element corresponding to an index identifier between the second index identifier and a third index identifier, and an element corresponding to the third index identifier, the third index identifier being smaller than the first index identifier, and the second index identifier being smaller than the third index identifier; The processing unit is configured to determine a repeating element sequence based on elements included in the first element sequence and elements included in the second element sequence; The processing unit is configured to perform sampling based on the repeated element sequence to obtain an output element sequence of the data to be sampled.

Images