Digital sequence-based model training method and model prediction method

Abstract

The invention discloses a model training method and a model prediction method based on a digital sequence, and relates to the technical field of large models, and the method comprises the steps: obtaining a training sample and a target digital sequence corresponding to the training sample; inputting the training sample into the initial model to obtain an initial prediction digital sequence; obtaining first digits located on different digits in the initial prediction digit sequence and second digits corresponding to the first digits on digits in the target digit sequence; adjusting the first number based on the difference distance between the first number and the second number to obtain an intermediate prediction number sequence; and adjusting model parameters of the initial model based on a comparison result of the intermediate prediction digital sequence and the target digital sequence to obtain a trained target model. According to the obtained target model, the sequence relation and the digital difference of the digital sequence are enhanced, the target model is helped to better understand the relative relation between the numerical values in the digital sequence, and therefore the prediction precision of the target model on the digits can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of large model technology, and in particular to a model training method and a model prediction method based on digital sequences. Background Art

[0002] In recent years, the rapid development of artificial intelligence has promoted the rapid development of large-scale language models. Many of these large-scale language models use autoregressive methods for text generation. Autoregressive methods are widely used in many sequence generation tasks, such as language modeling in natural language processing, image description generation, and automatic speech recognition.

[0003] In an autoregressive approach, a sequence of numbers is generated by gradually predicting each token. The cross-entropy loss function is the most common choice for autoregressive training. It trains large language models by minimizing the difference between the predicted probability distribution and the true label. However, the cross-entropy loss function treats each digit as an independent token and does not consider numerical relationships. This means that large language models can only adjust parameters based on the output of "correct prediction" and "incorrect prediction" during model training.

[0004] Therefore, in application scenarios involving digital sequence prediction (such as "visual image positioning") and calculation (such as "mathematical arithmetic"), due to the algorithmic limitations of the autoregressive cross-entropy method within large language models, the final model output results are inaccurate, affecting the user experience. Summary of the Invention

[0005] In view of this, the present disclosure provides a model training method and a model prediction method based on digital sequences to solve the problem that the algorithm limitations of the cross-entropy method of autoregression in existing large-scale language models result in inaccurate model output results, affecting the user experience.

[0006] In a first aspect, the present disclosure provides a model training method based on digital sequences, the method comprising:

[0007] Obtaining training samples and target digital sequences corresponding to the training samples, wherein the training samples include image samples to be predicted and initial model prompt words, the initial model prompt words are used to describe the numerical form to be output after the initial model processes the image samples, and the target digital sequence includes at least one number, and the target digital sequence is used to represent the final answer to be output after the initial model processes the image samples;

[0008] Input the training samples into the initial model to obtain the initial predicted digital sequence;

[0009] Obtaining first numbers located at different digits in the initial predicted digit sequence and second numbers corresponding to the first numbers in the target digit sequence;

[0010] Adjusting the first number based on the difference between the first number and the second number to obtain an intermediate predicted number sequence;

[0011] Based on the comparison results between the intermediate predicted digital sequence and the target digital sequence, the model parameters of the initial model are adjusted to obtain the trained target model.

[0012] In a second aspect, the present disclosure provides a model prediction method based on a digital sequence, the method comprising:

[0013] Obtaining a target image to be predicted and a target prompt word, wherein the target prompt word is used to describe the numerical form to be output after processing the target image;

[0014] The target image and target prompt word are input into the target model, and the target answer is output, wherein the target model is used to process the target image based on the target prompt word, the target model is obtained by adjusting the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, the intermediate predicted digital sequence is based on the difference distance between the first number and the second number, and is obtained by adjusting the first number based on the difference distance, the first number is a number located at a different digit in the initial predicted digital sequence, the second number is a number in the target digital sequence that corresponds to the first number in digits, the initial predicted digital sequence is the image sample to be predicted and the initial model prompt word obtained after inputting the initial model, and the target digital sequence is the final answer to be output after the initial model processes the image sample.

[0015] In a third aspect, the present disclosure provides a model training device based on a digital sequence, the device comprising:

[0016] A first acquisition module is configured to acquire training samples and target numerical sequences corresponding to the training samples, wherein the training samples include image samples to be predicted and initial model prompt words, the initial model prompt words are used to describe the numerical form to be output by the initial model after processing the image samples, and the target numerical sequence includes at least one number, and the target numerical sequence is used to represent the final answer to be output by the initial model after processing the image samples;

[0017] The first obtaining module is used to input the training sample into the initial model to obtain the initial predicted digital sequence;

[0018] A second acquisition module is used to acquire first numbers located at different digits in the initial predicted digit sequence, and second numbers corresponding to the first numbers in the target digit sequence;

[0019] A second obtaining module is configured to adjust the first number based on the difference between the first number and the second number to obtain an intermediate predicted number sequence;

[0020] The third obtaining module is used to adjust the model parameters of the initial model based on the comparison results of the intermediate predicted digital sequence and the target digital sequence to obtain the trained target model.

[0021] In a fourth aspect, the present disclosure provides a model prediction device based on a digital sequence, the device comprising:

[0022] The third acquisition module is used to obtain the target image to be predicted and the target prompt word, wherein the target prompt word is used to describe the numerical form to be output after processing the target image;

[0023] The output module is used to input the target image and the target prompt word into the target model and output the target answer, wherein the target model is used to process the target image based on the target prompt word, the target model is obtained by adjusting the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, the intermediate predicted digital sequence is based on the difference distance between the first number and the second number, and is obtained by adjusting the first number based on the difference distance, the first number is a number located at a different digit in the initial predicted digital sequence, the second number is a number in the target digital sequence that corresponds to the first number in digits, the initial predicted digital sequence is the image sample to be predicted and the initial model prompt word is input into the initial model, and the target digital sequence is the final answer to be output after the initial model processes the image sample.

[0024] In a fifth aspect, the present disclosure provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions stored in the memory, and the processor executing the computer instructions to execute the model training method based on digital sequence of the above-mentioned first aspect or any corresponding embodiment thereof or the model prediction method based on digital sequence of the second aspect.

[0025] In a sixth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the digital sequence-based model training method of the above-mentioned first aspect or any corresponding embodiment thereof or the digital sequence-based model prediction method of the second aspect.

[0026] In a seventh aspect, the present disclosure provides a computer program product comprising computer instructions, the computer instructions being used to enable a computer to execute the digital sequence-based model training method of the first aspect or any corresponding embodiment thereof, or the digital sequence-based model prediction method of the second aspect.

[0027] The model training method based on digital sequence provided by the embodiment of the present disclosure obtains training samples and target digital sequence corresponding to the training samples, inputs the training samples into the initial model, obtains the initial predicted digital sequence, then obtains the first digit located at different digits in the initial predicted digital sequence and the second digit corresponding to the first digit in the target digital sequence, adjusts the first digit based on the difference distance between the first digit and the second digit, obtains the intermediate predicted digital sequence; based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, adjusts the model parameters of the initial model to obtain the trained target model. In this way, the embodiment of the present disclosure performs single-digit deviation measurement based on the first digit at different digits in the initial predicted digital sequence and the corresponding second digit in the target digital sequence, and then compares the generated intermediate predicted digital sequence with the target digital sequence to achieve digital overall deviation measurement, thereby adjusting the model parameters according to the comparison result of the digital overall deviation measurement, so that the target model obtained in this way enhances the order relationship and digital difference of the digital sequence, helps the target model better understand the relative relationship between each numerical value in the digital sequence, thereby improving the target model's prediction accuracy for the number, improving the accuracy of the digital sequence as a whole, and solving the problem that the related art ignores the relationship between numerical values. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure;

[0030] Figure 2 is a flowchart of a model training method based on digital sequences according to an embodiment of the present disclosure;

[0031] Figure 3 is a flowchart of another model training method based on digital sequences according to an embodiment of the present disclosure;

[0032] Figure 4 is a flowchart of another model training method based on digital sequences according to an embodiment of the present disclosure;

[0033] Figure 5 is a flow chart of a model prediction method based on a digital sequence according to an embodiment of the present disclosure;

[0034] Figure 6is a structural block diagram of a model training device based on digital sequences according to an embodiment of the present disclosure;

[0035] Figure 7 is a structural block diagram of a model prediction device based on a digital sequence according to an embodiment of the present disclosure;

[0036] Figure 8 Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present disclosure.

[0038] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0039] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0040] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0041] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0042] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0043] Many large language models use autoregressive methods for text generation, and autoregressive methods are widely used in many sequence generation tasks. However, for tasks such as numerical sequence prediction (such as "visual image localization") and calculation (such as "mathematical arithmetic"), autoregressive methods are limited due to the inherent limitations of the cross-entropy algorithm in autoregressive methods. It assumes that the distances between numbers are the same, and can only produce two results: "correct prediction" and "incorrect prediction." This ignores the distance relationship between numbers and fails to understand the relationship between different values. As a result, large models based on the cross-entropy loss function have limited performance on numerical sequence prediction tasks, affecting user experience.

[0044] As an optional application scenario of the embodiment of the present disclosure, Figure 1 As shown, Figure 1 3 is a schematic diagram of an application scenario according to an embodiment of the present disclosure, which includes a mobile terminal, a server, and the interaction between the mobile terminal and the server.

[0045] According to an embodiment of the present disclosure, an embodiment of a model training method based on a digital sequence is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] In this embodiment, a model training method based on digital sequence is provided, which can be used for the above-mentioned server. Figure 2 is a flow chart of a model training method based on digital sequences according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:

[0047] Step S201: Obtain a training sample and a target digital sequence corresponding to the training sample, wherein the training sample includes an image sample to be predicted and an initial model prompt word, the initial model prompt word is used to describe the numerical form to be output after the initial model processes the image sample, and the target digital sequence contains at least one number, and the target digital sequence is used to represent the final answer to be output after the initial model processes the image sample.

[0048] Optionally, embodiments of the present disclosure may use image-text scenarios such as "visual image positioning," "scene text detection," and "clock understanding" as model training scenarios. First, the server obtains a training sample input by a mobile terminal (including an image sample to be predicted (e.g., an image) and an initial model prompt (e.g., "Where is the car in the picture?"; the model is required to output the result in the form of coordinates)).

[0049] It should be noted that the server here should be a computing device included in the initial model and used to process the input image samples to be predicted; the initial model can be a large language model or a multimodal large language model, etc. The initial model prompt word refers to the text information to be entered into the initial model, which is used to guide the initial model to generate specific content or complete a specific task.

[0050] From the above, it can be seen that in the embodiment of the present disclosure, the initial model prompt word contains information related to numerical values, such as "coordinates". At the same time, "coordinates" are also used to describe the numerical form that the initial model needs to output after processing the image sample. Here, the numerical form that needs to be output is "coordinate form".

[0051] Furthermore, during the initial model training phase, the server also needs to obtain the target numerical sequence corresponding to the training sample. This target numerical sequence is the final answer that the initial model should output after processing the input image sample. Specifically, during initial model training, the target numerical sequence is used as the target value. By continuously optimizing the initial model parameters, the initial model is trained to output the target numerical sequence, or a numerical sequence that is as close to the target numerical sequence as possible, based on the image sample and the initial model prompt.

[0052] It should be noted that a digital sequence is a set consisting of at least one number, so the target digital sequence contains at least one number. For example, if the target digital sequence is 73, the target digital sequence contains the number "7" and the number "3".

[0053] Step S202: input the training sample into the initial model to obtain an initial predicted digital sequence.

[0054] Optionally, an image sample and an initial model prompt are input into the initial model. For example, in the "visual image localization" scenario, the input is a poster image and a question, such as "Where is the car in the image? I want the model to output the result in the form of coordinates." The initial model then outputs the initial predicted number sequence: [323, 400]. Here, [323, 400] indicates that the car is located at a horizontal position of 323 pixels and a vertical position of 400 pixels in the poster image. Another example is the "scene text detection" scenario. The input is a promotional image and a question, such as "I want the model to output all the text information in the image in the form of coordinates." The initial model then outputs the initial predicted number sequence: [11, 63][56, 97]. Here, [11, 63][56, 97] indicates that there is a piece of text at the horizontal position 11 pixels and the vertical position 63 pixels, and at the horizontal position 56 pixels and the vertical position 97 pixels, respectively. For example, in the "clock understanding" scenario, the input is a clock image and a question, such as "Ask the model to output the time information represented by the clock." In this case, the initial model outputs the initial predicted number sequence: 9:33. Here, 9:33 means that the input clock image represents 9:33.

[0055] It can be understood that the above-mentioned initial predicted digital sequence is the number output immediately after the training sample is input into the initial model (that is, the initial answer). Generally speaking, the initial predicted digital sequence is not consistent with the final desired target digital sequence. Therefore, it is necessary to analyze the initial predicted digital sequence and adjust the model parameters of the initial model based on the analysis results.

[0056] Step S203: obtaining first numbers located at different digits in the initial predicted digit sequence and second numbers corresponding to the first numbers in the target digit sequence.

[0057] Alternatively, since the initial predicted digital sequence is composed of numbers that are composed of different digits, the digits here are tens, hundreds, thousands, etc., for example, "73" is composed of "7" in the tens place and "3" in the units place. Similarly, the target digital sequence is also composed of numbers in different digits.

[0058] At this point, it is necessary to obtain the first digits in different digit positions in the initial predicted number sequence, such as the "7" in the tens place and the "3" in the ones place. Here, "7" and "3" are both called the first digits. At the same time, it is also necessary to obtain the second digit corresponding to the first digit in the target number sequence. For example, if the target number sequence is "89", the "8" in the tens place and the "9" in the ones place are both called the second digits. At the same time, based on the digit position, the "7" and "8" in the tens place correspond to each other, and the "3" and "9" in the ones place correspond to each other.

[0059] Step S204: adjusting the first number based on the difference between the first number and the second number to obtain an intermediate predicted number sequence.

[0060] Optionally, in the disclosed embodiment, the Earth Mover's Distance (EMD) is introduced into the model training task to achieve an understanding of the relative relationship between the numerical values of digital tokens (i.e., tags, representing the smallest unit after text segmentation). Among them, the Earth Mover's Distance is a method for measuring the difference between two probability distributions. It originates from the transportation problem and aims to find the optimal transportation plan to transform one distribution into another distribution, which is achieved by calculating the minimized "transportation cost".

[0061] Since the model training of the initial model is a process of continuous updating and iteration, each time a training sample is input into the initial model, an initial predicted digital sequence will be output, and then each time the first digit in a different digit can be obtained, as well as the second digit in the target predicted digital sequence that corresponds to the first digit in digit, and then each update iteration can obtain the difference distance between the first digit and the second digit, and then based on the EMD algorithm, the first digit is used as the distribution to be converted, and the second digit is used as the final distribution to be converted, and the first digit is continuously adjusted based on the difference distance obtained each time, and then an intermediate predicted digital sequence composed of the adjusted first digit is obtained.

[0062] Step S205 , based on the comparison result between the intermediate predicted digital sequence and the target digital sequence, the model parameters of the initial model are adjusted to obtain a trained target model.

[0063] Optionally, in order to enable the initial model to improve the overall accuracy of the digital sequence at each prediction step, the overall digits of the intermediate predicted digital sequence are compared with the target digital sequence to obtain the overall difference value between the intermediate predicted digital sequence and the target digital sequence, and then the model parameters of the initial model are adjusted based on the overall difference value to obtain the trained target model.

[0064] The model training method based on digital sequence provided by the embodiment of the present disclosure obtains training samples and target digital sequence corresponding to the training samples, inputs the training samples into the initial model, obtains the initial predicted digital sequence, then obtains the first digit located at different digits in the initial predicted digital sequence and the second digit corresponding to the first digit in the target digital sequence, adjusts the first digit based on the difference distance between the first digit and the second digit, obtains the intermediate predicted digital sequence; based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, adjusts the model parameters of the initial model to obtain the trained target model. In this way, the embodiment of the present disclosure performs single-digit deviation measurement based on the first digit at different digits in the initial predicted digital sequence and the corresponding second digit in the target digital sequence, and then compares the generated intermediate predicted digital sequence with the target digital sequence to achieve digital overall deviation measurement, thereby adjusting the model parameters according to the comparison result of the digital overall deviation measurement, so that the target model obtained in this way enhances the order relationship and digital difference of the digital sequence, helps the target model better understand the relative relationship between each numerical value in the digital sequence, thereby improving the target model's prediction accuracy for the number, improving the accuracy of the digital sequence as a whole, and solving the problem that the related art ignores the relationship between numerical values.

[0065] In this embodiment, a model training method based on digital sequence is provided, which can be used for the above-mentioned server. Figure 3 is a flow chart of another model training method based on digital sequence according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:

[0066] Step S301: Obtain training samples and target numerical sequences corresponding to the training samples. The training samples include image samples to be predicted and initial model prompt words. The initial model prompt words are used to describe the numerical form that the initial model needs to output after processing the image samples. The target numerical sequence contains at least one number and is used to represent the final answer that the initial model needs to output after processing the image samples. For details, please refer to Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0067] Step S302: Input the training sample into the initial model to obtain the initial predicted digital sequence. Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0068] Step S303: Obtain the first digits at different positions in the initial predicted digit sequence and the second digits in the target digit sequence that correspond to the first digits. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0069] Step S304: Based on the difference between the first number and the second number, adjust the first number to obtain an intermediate predicted number sequence.

[0070] Specifically, the above step S304 includes:

[0071] Step S3041: Obtain the difference distance between the first number and the second number.

[0072] Optionally, in the embodiments of the present disclosure, a direct comparison method can be used: for any two numbers, a direct size comparison can be performed to obtain the difference distance between the two numbers. For example: comparing 3 and 5, it can be known through numerical calculation that 3 is less than 5, and the difference distance is 2. It can also be based on the definition of the number axis: the position of the numbers on the number axis determines their order, and the numbers on the right of the number axis are always greater than the numbers on the left. For example, on the number axis, -2 is to the right of -3, so -2 is greater than –3, and the difference distance is 1. The difference distance between two numbers can also be obtained by using a vocabulary, which will be described in subsequent embodiments.

[0073] Step S3042, obtaining the influence weight corresponding to the current digit where the first digit is located.

[0074] Optionally, in a numerical sequence, high-order digits (such as the "1" in "12") contribute more to the entire numerical value, while low-order digits (such as "2") contribute less. Different weights are assigned to the digits where different digits are located, so that the initial model pays more attention to high-order digits (such as the position of the tens digit "1" in the overall number "12") during training. This weighting method helps the model better understand the numerical relationship between multiple digits. Therefore, it is currently necessary to obtain the corresponding influence weight of the current digit (tens or units) where the first digit is located.

[0075] Step S3043: Adjust the first number based on the difference distance and the influence weight to obtain a third number.

[0076] Optionally, when adjusting the first digit, the initial model needs to pay more attention to the prediction accuracy of the higher-order digits. Therefore, in addition to considering the difference distance between the first digit and the second digit in each digit, the embodiment of the present disclosure also needs to give priority to the influence weight of the higher-order digits, and combine the difference distance and the influence weight to adjust the first digit to obtain the third digit.

[0077] Step S3044: merge the third digits to obtain an intermediate predicted digit sequence.

[0078] Alternatively, the disclosed embodiment may employ a Gumbel-softmax approximation method to combine these third digits into a complete digital representation, thereby obtaining an intermediate predicted digital sequence. The obtained combined digit (e.g., 34) still retains the differentiable property and can thus be used for initial model training.

[0079] Step S305: Based on the comparison results of the intermediate predicted digital sequence and the target digital sequence, the model parameters of the initial model are adjusted to obtain the trained target model. Figure 2 Step S205 of the illustrated embodiment will not be described in detail here.

[0080] The disclosed embodiment measures the distance between the digital distribution of the first and second numbers and adjusts the first number in combination with the influence weight of the first number, thereby helping the model understand the positional relationship between multiple numbers and improving the accuracy and efficiency of the model in digital prediction tasks.

[0081] In some optional implementations, step S3041 includes:

[0082] Step a1: Obtain first position information of a first number in a preset table and second position information of a second number in the preset table, wherein the preset table is used to arrange numbers.

[0083] Step a2: Obtain a first difference between the first position information and the second position information.

[0084] Step a3: determining a difference distance based on the first difference value.

[0085] Optionally, in an embodiment of the present disclosure, the order of arrangement of the initial predicted digital sequence and the individual numbers in the target digital sequence is determined by obtaining a preset table, such as a vocabulary. The vocabulary is a set of all possible words (identifiers). The digital identifiers in the vocabulary are arranged in a certain order, and this order reflects the numerical size relationship of the numbers. For example, if the order of the digital identifiers in the vocabulary is "1", "2", "3", "4", etc., then through this order, the model can understand the relative size and difference distance between the numbers.

[0086] Specifically, the initial model outputs a series of tokens when making predictions. When these tokens contain numbers, EMD identifies these numeric tokens from the output. Then, based on the aforementioned order of numbers within the vocabulary, EMD analyzes the position of these numeric tokens within the vocabulary and understands their relative relationships.

[0087] Furthermore, EMD checks the first position information of the first digit and the second position information of the second digit in the vocabulary at the same digit position. For example, it obtains the first digit and the second digit in the tens digit, and obtains their first position information and second position information in the vocabulary respectively (assuming that the first digit "3" is positioned before the second digit "4"). It can then be inferred that there is a difference of one digit between the first digit 3 and the second digit 4, and the corresponding first difference is 1, and the difference distance is then obtained as 1. At this time, it is considered that the first digit 3 is closer to the second digit 4.

[0088] Of course, if the first digit in the tens place is 8, and the second digit in the tens place is 4, since there are 4 digits between 8 and 4 in the vocabulary, and then the EMD is used to analyze the vocabulary, the difference between 8 and 4 is 4. At this time, it is considered that the first digit 8 and the second digit 4 are quite different. The initial model needs to be informed that the first digit in the tens place of the next output needs to be closer to the number 4 than 8, such as outputting the numbers 6 and 5. In this way, the result deviation evaluation achieved with the help of EMD enables the model to have the ability to measure the deviation of a single number.

[0089] The disclosed embodiment can use EMD to understand the distribution differences between numbers through vocabulary matching, and then evaluate the difference between the predicted value (i.e., the first number) and the true value (i.e., the second number), thereby optimizing the prediction performance of the initial model during the training process and improving the accuracy of the prediction results.

[0090] In some optional implementations, step S3042 includes:

[0091] Step b1, obtaining the total length of the initial predicted digital sequence.

[0092] Step b2: determining the digits included in the initial predicted digital sequence based on the total length.

[0093] Step b3: Determine the influence weights of different digits based on a weighted algorithm to obtain the influence weight of the first digit on the current digit.

[0094] Optionally, in order to better handle the positional relationship between numbers, exponential position weighting is introduced on the basis of EMD, and different influence weights are assigned to different digits, so that the initial model pays more attention to high-order digits during training, such as giving priority to adjusting the values of high-order digits to the target value (here refers to the value corresponding to the second digit).

[0095] Specifically, the server obtains the total length of the initial predicted digital sequence, such as 2 digits. In this case, it is considered that the digits contained in the initial predicted digital sequence are tens and units.

[0096] The influence weights of different digits are obtained based on the set weighting algorithm. The specific weighting calculation formula is:

[0097]

[0098] Among them, σ is a parameter that controls the increment, which is used to adjust the magnitude of the position weighting; n is the total length of the initial predicted digit sequence, and i is the position of the current digit. For example, if for the number "345", we want to increase the initial model's attention to the hundreds digit 3, while keeping the normal attention to the units digit 5. For example, if σ is set to 0.2, the influence weight w for the units digit 3 is exp =1.0, the influence weight w for the tens digit 4 exp =1.2, and the influence weight of the hundredth digit 5 is w exp = 1.44. By assigning increasing weights to higher-order digits, the initial model can focus more on the prediction accuracy of higher-order digits.

[0099] Therefore, no matter which digit the first digit is located in, its corresponding influence weight can be obtained.

[0100] The disclosed embodiments can help the initial model pay more attention to the prediction accuracy of high-digit numbers by assigning influence weights to numbers in different digits, thereby enhancing the ability of the initial model in predicting multi-digit numbers. In particular, when processing tasks involving larger numerical values, the loss function of the initial model can be more effectively optimized to reduce the noise impact of low-digit numbers.

[0101] In some optional implementations, step S3043 includes:

[0102] Step c1: Obtain the difference distances in sequence based on the influence weights to obtain the target sequence.

[0103] Step c2: Based on the order of the difference distances in the target sequence and the difference distances, the first digits are adjusted in sequence to obtain intermediate digits.

[0104] Step c3, obtaining a second difference between the value of the middle digit and the value of the second digit.

[0105] Step c4: when the second difference is less than the first difference threshold, obtain a third number.

[0106] Optionally, since the embodiment of the present disclosure requires the initial model to better understand the numerical relationship between multi-digit numbers, thereby improving the initial model's understanding and prediction capabilities of complex digital sequences, it is necessary to let the initial model pay more attention to the prediction accuracy of high-digit numbers, so it is currently necessary to sort the adjustment priority of the numbers on each digit.

[0107] Furthermore, the influence weight of each digit is obtained, and the difference distance between the first digit and the second digit is obtained in sequence based on the influence weight. For example, the influence weight of the digit increases from the ones digit, tens digit, and hundreds digit in sequence, so the difference distance can be obtained in sequence from front to back in positive order, and then the target sequence is obtained after sorting the difference distance in positive order.

[0108] Then, based on the order and value of each difference distance in the target sequence, the first digit is adjusted in sequence (i.e., the high-order digit with the highest influence weight value is adjusted first, and then the digit with the second highest influence weight value is adjusted), so that the value of the adjusted first digit is infinitely close to the value of the second digit, and the digit obtained after each adjustment is called the intermediate digit.

[0109] Then, a second difference between the values of the middle digit and the second digit is obtained, and each time the second difference is less than a first difference threshold (e.g., 0.1), the currently adjusted digit is used as the third digit. It should be noted that the first difference threshold here is the maximum value that can differ between the values of the middle digit and the second digit, and is used to measure the difference between the values of the middle digit and the second digit.

[0110] According to the above method, the embodiment of the present disclosure can give priority to adjusting the high-order digits with greater influence, more effectively optimize the loss function of the initial model, enhance the influence of the high-order digits on the entire digital sequence, and reduce the noise impact of the low-order digits.

[0111] In this embodiment, a model training method based on digital sequence is provided, which can be used for the above-mentioned server. Figure 4 is a flow chart of another model training method based on digital sequence according to an embodiment of the present disclosure, such as Figure 4 As shown, the process includes the following steps:

[0112] Step S401: Obtain training samples and target numerical sequences corresponding to the training samples. The training samples include image samples to be predicted and initial model prompt words. The initial model prompt words are used to describe the numerical form that the initial model needs to output after processing the image samples. The target numerical sequence contains at least one number and is used to represent the final answer that the initial model needs to output after processing the image samples. For details, please refer to Figure 3 Step S301 of the illustrated embodiment will not be described in detail here.

[0113] Step S402: Input the training sample into the initial model to obtain the initial predicted digital sequence. Figure 3 Step S302 of the illustrated embodiment will not be described in detail here.

[0114] Step S403: Obtain the first digits at different positions in the initial predicted digit sequence and the second digits in the target digit sequence that correspond to the first digits. Figure 3 Step S303 of the illustrated embodiment will not be described in detail here.

[0115] Step S404: Based on the difference between the first and second digits, adjust the first digit to obtain an intermediate predicted digit sequence. Figure 3 Step S304 of the illustrated embodiment will not be described in detail here.

[0116] Step S405 , based on the comparison result between the intermediate predicted digital sequence and the target digital sequence, the model parameters of the initial model are adjusted to obtain a trained target model.

[0117] Specifically, the above step S405 includes:

[0118] Step S4051: compare the intermediate predicted digital sequence with the target digital sequence to obtain a loss value.

[0119] Optionally, for a number composed of multiple characters, the overall deviation between the predicted value and the true value is measured. Specifically, the intermediate predicted number sequence is compared with the target number sequence. The difference and amplitude between the two can be compared separately, or the obtained difference and amplitude can be combined to obtain the loss value.

[0120] Step S4052: compare the loss value with the second difference threshold. If the loss value is greater than the second difference threshold, adjust the model parameters until the loss value is less than or equal to the second difference threshold. Stop adjusting the model parameters and obtain the target model.

[0121] Optionally, the obtained loss value is compared with a set second difference threshold. It should be understood that the second difference threshold here is a critical value at which the intermediate predicted digital sequence is different from the target digital sequence, and is used to measure the difference between the intermediate predicted digital sequence and the target digital sequence.

[0122] When the loss value is greater than the second difference threshold, it is considered that the intermediate predicted digital sequence is significantly different from the target digital sequence, and the model parameters need to be adjusted until the loss value is less than or equal to the second difference threshold. Then, the adjustment of the model parameters is stopped to obtain the target model.

[0123] The disclosed embodiment takes into account the overall numerical differences of the entire digital sequence, which can help the model optimize the accuracy of digital sequence prediction from multiple perspectives, thereby improving the initial model's prediction accuracy for numbers.

[0124] In some optional implementations, step S3022 includes:

[0125] Step d1, obtaining a third difference between the values of the intermediate predicted digital sequence and the values of the target digital sequence.

[0126] Step d2: obtaining the amplitude between the values of the intermediate predicted digital sequence and the values of the target digital sequence.

[0127] Step d3: merge the third difference value and the amplitude value to obtain the loss value.

[0128] Optionally, the absolute value is used to calculate the difference between the numerical value of the intermediate predicted digital sequence and the numerical value of the target digital sequence to obtain a third difference, which is calculated using the formula: V abs =|XY|.

[0129] Use the amplitude to calculate the ratio between the values of the intermediate predicted digital sequence and the values of the target digital sequence. The calculation formula is: V mag =log(X / Y).

[0130] Where X represents the value of the intermediate predicted digital sequence, Y represents the value of the target digital sequence; V abs Represents the third difference, V mag These two results together form a measure of the overall deviation of the number.

[0131] Then V abs and V mag Combine them, for example, find the sum of the two and then get the loss value.

[0132] The disclosed embodiment uses the difference in absolute values and amplitude to implement the deviation measurement of the overall number, helping the initial model to better understand the differences between numbers, thereby improving the initial model's prediction accuracy for numbers.

[0133] In this embodiment, a model prediction method based on digital sequences is provided, which can be used on the server side of the trained target model. Figure 5 is a flow chart of a model prediction method based on a digital sequence according to an embodiment of the present disclosure, such as Figure 5 As shown, the process includes the following steps:

[0134] Step S501 : obtaining a target image to be predicted and a target prompt word, wherein the target prompt word is used to describe the numerical form to be output after processing the target image.

[0135] Optionally, the disclosed embodiments describe the application phase of digital prediction based on a trained target model. Specifically, based on the current application scenario, a target image and a target prompt word to be predicted are obtained. For example, for a "visual image localization" task, the input is a target image and a target prompt word, such as "Where is the car in the picture? The model is required to output the result in the form of coordinates."

[0136] It is understandable that the above target prompt words describe the numerical form that needs to be output after the target image is processed, such as the "coordinate" form.

[0137] Step S502: input the target image and target prompt word into the target model, and output the target answer, wherein the target model is used to process the target image based on the target prompt word, the target model is obtained by adjusting the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, the intermediate predicted digital sequence is based on the difference distance between the first number and the second number, and is obtained by adjusting the first number based on the difference distance, the first number is a number located at a different digit in the initial predicted digital sequence, the second number is a number in the target digital sequence that corresponds to the first number in digits, the initial predicted digital sequence is the image sample to be predicted and the initial model prompt word obtained after inputting the initial model, and the target digital sequence is the final answer to be output after the initial model processes the image sample.

[0138] Optionally, the target image and target prompt word are input into a trained target model. The target model processes the target image based on the target prompt word and then outputs a target answer. For example, the output "[325,600]" indicates that the car is located at a horizontal position of 325 pixels and a vertical position of 600 pixels in the target image.

[0139] In addition, the target model is obtained by adjusting the model parameters of the initial model based on the comparison results of the intermediate predicted digital sequence and the target digital sequence. The intermediate predicted digital sequence is obtained by adjusting the first digital based on the difference distance between the first and second digitals. The first digital is a digital located at a different digit in the initial predicted digital sequence, and the second digital is a digital corresponding to the first digital in the target digital sequence. The initial predicted digital sequence is obtained by inputting the image sample to be predicted and the initial model prompt word into the initial model. The target digital sequence is the final answer to be output by the initial model after processing the image sample. The above implementation process refers to the above embodiments and will not be repeated here.

[0140] The disclosed embodiments can help the initial model understand the distance between numbers during training: by understanding the size relationship between individual numeric characters while considering the numerical differences of the entire numeric sequence, and then obtaining the relative relationship between the numbers in size and position, it can help the target model achieve better prediction results.

[0141] As a specific embodiment of the present disclosure, the following description of specific model training and model application is carried out using visual image positioning as the current application scenario:

[0142] Model training phase:

[0143] The model is fed with a poster image and a prompt, such as "Where is the car in the picture?" and asked to output the result in the form of coordinates.

[0144] After model analysis, the first digital sequence (i.e., the prediction result, such as [38,8]) is obtained.

[0145] The first digital sequence is split into single characters, and the target digital sequence that the model currently needs to output (i.e., the true value, such as [22,2]) is split into single characters. Then, the single characters in the obtained first digital sequence are compared with the single characters in the target digital sequence in turn (by obtaining the position of these numbers in the vocabulary). The EMD is used to measure the "transportation cost" of each number to evaluate the difference between the predicted result and the true value, and the difference distance between the first number located at a different digit in the first digital sequence and the second number at the digit corresponding to the first number in the target digital sequence is obtained.

[0146] For example, the first digit "3" at the tens place in the first digit sequence and the second digit "2" at the tens place in the target digit sequence, in the vocabulary, "3" is located after "2", and the two are one difference distance apart. Therefore, when the first digit "3" is subsequently adjusted, it can be quickly adjusted to the second digit "2". The first digit "8" at the units place in the first digit sequence and the second digit "2" at the tens place in the target digit sequence, in the vocabulary, "8" is located after "2", and the two are six difference distance apart. Therefore, when the first digit "8" is subsequently adjusted, the first digit can be quickly adjusted based on this difference distance "6" and a set first difference threshold (for example, 4) to make it infinitely close to the second digit "2", for example, adjusting the tens place "3" to "2" and the units place "8" to "4".

[0147] Based on EMD, exponential position weighting is introduced to assign different influence weights to numbers in different digits, so that the model pays more attention to high-digit numbers (such as the position of the tens digit "3" in the overall number "38") during training. In this way, during training, high-digit numbers are adjusted first each time.

[0148] The adjusted first digits on each digit are then merged using the Gumbel-softmax approximation method to obtain a merged number, such as the "24" mentioned above. The obtained "24" still maintains the differentiable property and can be used for model training.

[0149] Then, the numerical difference and ratio between the merged "24" and the true value "22" are calculated, and then the obtained numerical difference and ratio are fused, such as finding the sum of the two, to obtain the loss value. Then, the prediction result is adjusted based on the loss value so that the final loss value meets the model stopping condition (for example, the loss value is less than or equal to the set second difference threshold, and the model parameters of the model are stopped to obtain the trained target model.

[0150] Model application phase:

[0151] The target model is put into practical use. Specifically, an image of a clock and the model prompt "Require the model to output the time information represented by the clock" are input into the target model, and the target model outputs "9:37", indicating that the input clock image represents 9:37.

[0152] In this embodiment, a model training device based on a digital sequence is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0153] This embodiment provides a model training device based on digital sequence, such as Figure 6 , the device comprises:

[0154] A first acquisition module 601 is configured to acquire training samples and target numerical sequences corresponding to the training samples, wherein the training samples include image samples to be predicted and initial model prompt words, the initial model prompt words being used to describe the numerical form to be output by the initial model after processing the image samples, and the target numerical sequence including at least one number representing the final answer to be output by the initial model after processing the image samples;

[0155] The first obtaining module 602 is used to input the training sample into the initial model to obtain an initial predicted digital sequence;

[0156] A second acquisition module 603 is configured to acquire first digits located at different digit positions in the initial predicted digit sequence, and second digits corresponding to the first digits in the target digit sequence;

[0157] A second obtaining module 604 is configured to adjust the first number based on the difference between the first number and the second number to obtain an intermediate predicted number sequence;

[0158] The third obtaining module 605 is used to adjust the model parameters of the initial model based on the comparison result between the intermediate predicted digital sequence and the target digital sequence to obtain the trained target model.

[0159] In some optional implementations, the second obtaining module 604 includes:

[0160] A first acquiring unit, configured to acquire a difference distance between the first number and the second number;

[0161] A second obtaining unit is used to obtain the influence weight corresponding to the current digit where the first digit is located;

[0162] an adjusting unit, configured to adjust the first number based on the difference distance and the influence weight to obtain a third number;

[0163] The merging unit is used to merge the third digits to obtain an intermediate predicted digit sequence.

[0164] In some optional implementations, the first acquiring unit includes:

[0165] A first acquiring subunit is configured to acquire first position information of a first number in a preset table and second position information of a second number in the preset table, wherein the preset table is used to arrange numbers;

[0166] A second acquiring subunit, configured to acquire a first difference between the first position information and the second position information;

[0167] The first determining subunit is configured to determine a difference distance based on the first difference value.

[0168] In some optional implementations, the second acquiring unit includes:

[0169] The third acquisition subunit is used to obtain the total length of the initial predicted digital sequence;

[0170] A second determining subunit, configured to determine the digits included in the initial predicted digital sequence based on the total length;

[0171] The first obtaining subunit is used to determine the influence weights of different digits based on a weighted algorithm, and obtain the influence weight of the first digit on the current digit.

[0172] In some optional embodiments, the adjusting unit includes:

[0173] A fourth acquisition subunit is used to sequentially acquire difference distances based on the influence weights to obtain a target sequence;

[0174] an adjusting subunit, configured to adjust the first digits in sequence based on the order of the difference distances and the difference distances within the target sequence to obtain intermediate digits;

[0175] a fifth obtaining subunit, configured to obtain a second difference between the value of the middle digit and the value of the second digit;

[0176] The second obtaining subunit is configured to obtain a third number when the second difference is less than the first difference threshold.

[0177] In some optional implementations, the third obtaining module 605 includes:

[0178] A first comparison unit is used to compare the intermediate predicted digital sequence with the target digital sequence to obtain a loss value;

[0179] The second comparison unit is used to compare the loss value with the second difference threshold. When the loss value is greater than the second difference threshold, the model parameters are adjusted until the loss value is less than or equal to the second difference threshold, and the adjustment of the model parameters is stopped to obtain the target model.

[0180] In some optional implementations, the first comparing unit includes:

[0181] a sixth obtaining subunit, configured to obtain a third difference between the values of the intermediate predicted digital sequence and the values of the target digital sequence;

[0182] a seventh acquisition subunit, configured to acquire an amplitude between the values of the intermediate predicted digital sequence and the values of the target digital sequence;

[0183] The third difference and the amplitude are combined to obtain the loss value.

[0184] The model training device based on digital sequences provided by the embodiments of the present disclosure can execute the model training method based on digital sequences provided by any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method. That is, based on the first digits on different digits in the initial predicted digital sequence and the corresponding second digits in the target digital sequence, a single-digit deviation measurement is performed, and then the generated intermediate predicted digital sequence is compared with the target digital sequence to achieve digital overall deviation measurement, thereby adjusting the model parameters according to the comparison results of the digital overall deviation measurement, so that the target model obtained enhances the sequential relationship and digital difference of the digital sequence, helps the target model to better understand the relative relationship between each numerical value in the digital sequence, thereby improving the target model's prediction accuracy for numbers. The further functional description of each of the above modules and units is the same as that of the corresponding embodiment above, and will not be repeated here.

[0185] This embodiment provides a model prediction device based on digital sequence, such as Figure 7 , the device comprises:

[0186] The third acquisition module 701 is used to acquire a target image to be predicted and a target prompt word, wherein the target prompt word is used to describe the numerical form to be output after processing the target image;

[0187] Output module 702 is used to input the target image and target prompt word into the target model and output the target answer, wherein the target model is used to process the target image based on the target prompt word, the target model is obtained by adjusting the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, the intermediate predicted digital sequence is based on the difference distance between the first number and the second number, and is obtained by adjusting the first number based on the difference distance, the first number is a number located at a different digit in the initial predicted digital sequence, the second number is a number in the target digital sequence that corresponds to the first number in digits, the initial predicted digital sequence is the image sample to be predicted and the initial model prompt word are input into the initial model, and the target digital sequence is the final answer to be output after the initial model processes the image sample.

[0188] The model prediction device based on digital sequences provided by the embodiments of the present disclosure can execute the model prediction method based on digital sequences provided by any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method. That is, it helps the initial model understand the distance between numbers during training: by understanding the size relationship between individual digital characters, while considering the numerical differences of the entire digital sequence, and then obtaining the relative relationship between the numbers in size and position, it can help the target model achieve better prediction results. The further functional description of each of the above modules and units is the same as that of the corresponding embodiments above, and will not be repeated here.

[0189] Figure 8 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

[0190] The following specific reference Figure 8 , which shows a schematic diagram of the structure of the electronic device suitable for implementing the embodiment of the present disclosure. The electronic device may include a processor (such as a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the memory 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device are also stored. The processor 801, ROM 802 and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0191] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and more or fewer devices may be implemented or possessed instead.

[0192] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the memory 808, or installed from the ROM 802. When the computer program is executed by the processor 801, the above-mentioned functions defined in the model training method based on the digital sequence of the embodiment of the present disclosure are performed.

[0193] Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0194] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the model training method based on a digital sequence shown in the above embodiment is implemented.

[0195] A portion of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes but is not limited to a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0196] Although the embodiments of the present disclosure have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A model training method based on digital sequence, characterized in that: The method comprises: Obtaining a training sample and a target numerical sequence corresponding to the training sample, wherein the training sample includes an image sample to be predicted and an initial model prompt word, the initial model prompt word is used to describe the numerical form to be output by the initial model after processing the image sample, and the target numerical sequence includes at least one number, and the target numerical sequence is used to represent the final answer to be output by the initial model after processing the image sample; Inputting the training samples into the initial model to obtain an initial predicted digital sequence; Obtaining first numbers located at different digits in the initial predicted digit sequence, and second numbers in the target digit sequence corresponding to the first numbers in digits; Adjusting the first number based on the difference between the first number and the second number to obtain an intermediate predicted number sequence; Based on the comparison result between the intermediate predicted digital sequence and the target digital sequence, the model parameters of the initial model are adjusted to obtain a trained target model.

2. The method according to claim 1, characterized in that The adjusting the first number based on the difference distance between the first number and the second number to obtain an intermediate predicted number sequence includes: Obtaining a difference between the first number and the second number; Obtaining the influence weight corresponding to the current digit where the first digit is located; Adjusting the first number based on the difference distance and the influence weight to obtain a third number; The third numbers are combined to obtain the intermediate predicted number sequence.

3. The method according to claim 2, characterized in that The obtaining of the difference between the first number and the second number includes: Obtaining first position information of the first number in a preset table and second position information of the second number in a preset table, wherein the preset table is used to arrange numbers; Obtaining a first difference between the first position information and the second position information; The difference distance is determined based on the first difference value.

4. The method according to claim 2, characterized in that The obtaining of the influence weight corresponding to the current digit where the first digit is located includes: Obtaining the total length of the initial predicted digital sequence; Determining the digits included in the initial predicted digital sequence based on the total length; The influence weights of different digits are determined based on a weighted algorithm to obtain the influence weight of the first digit on the current digit.

5. The method according to claim 2, characterized in that The adjusting the first number based on the difference distance and the influence weight to obtain a third number includes: Obtain the difference distances in sequence based on the influence weights to obtain a target sequence; Based on the order of the difference distances in the target sequence and the difference distances, sequentially adjusting the first numbers to obtain intermediate numbers; Obtaining a second difference between the value of the middle digit and the value of the second digit; When the second difference is less than the first difference threshold, the third number is obtained.

6. The method according to claim 1, characterized in that The adjusting the model parameters of the initial model based on the comparison result between the intermediate predicted digital sequence and the target digital sequence to obtain the trained target model includes: Comparing the intermediate predicted digital sequence with the target digital sequence to obtain a loss value; Compare the loss value with a second difference threshold. If the loss value is greater than the second difference threshold, adjust the model parameters until the loss value is less than or equal to the second difference threshold, stop adjusting the model parameters, and obtain the target model.

7. The method according to claim 6, characterized in that The step of comparing the intermediate predicted digital sequence with the target digital sequence to obtain a loss value includes: Obtaining a third difference between the values of the intermediate predicted digital sequence and the values of the target digital sequence; Obtaining an amplitude between a value of the intermediate predicted digital sequence and a value of the target digital sequence; The third difference value and the amplitude value are combined to obtain the loss value.

8. A model prediction method based on digital sequence, characterized in that: The method comprises: Obtaining a target image to be predicted and a target prompt word, wherein the target prompt word is used to describe a numerical form to be output after processing the target image; The target image and the target prompt word are input into a target model, and a target answer is output, wherein the target model is used to process the target image based on the target prompt word, and the target model is obtained by adjusting the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, and the intermediate predicted digital sequence is based on the difference distance between the first number and the second number, and is obtained by adjusting the first number based on the difference distance, the first number is a number located at a different digit in the initial predicted digital sequence, and the second number is a number in the target digital sequence that corresponds to the first number in digits, the initial predicted digital sequence is obtained after the image sample to be predicted and the initial model prompt word are input into the initial model, and the target digital sequence is the final answer to be output after the initial model processes the image sample.

9. A model training device based on digital sequence, characterized in that: The device comprises: A first acquisition module is configured to acquire a training sample and a target numerical sequence corresponding to the training sample, wherein the training sample includes an image sample to be predicted and an initial model prompt word, the initial model prompt word is used to describe the numerical form to be output by the initial model after processing the image sample, and the target numerical sequence includes at least one number, and the target numerical sequence is used to represent the final answer to be output by the initial model after processing the image sample; A first obtaining module is used to input the training sample into the initial model to obtain an initial predicted digital sequence; A second acquisition module is configured to acquire first numbers located at different digits in the initial predicted digit sequence, and second numbers corresponding to the first numbers in the target digit sequence; a second obtaining module, configured to adjust the first number based on a difference distance between the first number and the second number to obtain an intermediate predicted number sequence; The third obtaining module is used to adjust the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence to obtain the trained target model.

10. A model prediction device based on digital sequence, characterized in that: The device comprises: a third acquisition module, configured to acquire a target image to be predicted and a target prompt word, wherein the target prompt word is used to describe a numerical value to be output after processing the target image; An output module is used to input the target image and the target prompt word into a target model and output a target answer, wherein the target model is used to process the target image based on the target prompt word, the target model is obtained by adjusting the model parameters of the initial model based on the comparison result of the intermediate predicted digital sequence and the target digital sequence, the intermediate predicted digital sequence is based on the difference distance between the first number and the second number, and is obtained by adjusting the first number based on the difference distance, the first number is a number located at a different digit in the initial predicted digital sequence, the second number is a number in the target digital sequence that corresponds to the first number in digits, the initial predicted digital sequence is obtained after the image sample to be predicted and the initial model prompt word are input into the initial model, and the target digital sequence is the final answer to be output after the initial model processes the image sample.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the digital sequence-based model training method according to any one of claims 1 to 7 or the digital sequence-based model prediction method according to claim 8 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the model training method based on digital sequences described in any one of claims 1 to 7 or the model prediction method based on digital sequences described in claim 8.

13. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the model training method based on a digital sequence according to any one of claims 1 to 7 or the model prediction method based on a digital sequence according to claim 8.

Images