Inference calculation method and related equipment

Abstract

The invention discloses a reasoning calculation method and related equipment, and relates to the technical field of artificial intelligence, and the reasoning calculation method comprises the steps of obtaining document data to be subjected to reasoning calculation, segmenting the document data to obtain multiple segments of text data, reasoning and calculating the multiple segments of text data based on a preset large language model to obtain multiple segments of target text data, the preset large language model is a trained current large language model, determining relevance among the multiple segments of target text data based on a preset paragraph relation determination algorithm, and splicing the multiple segments of target text data based on the relevance to obtain document data after reasoning calculation. According to the method, the document data to be subjected to reasoning calculation are segmented, reasoning calculation is performed on each segment of text data, and then each segment of text data after reasoning calculation is spliced to obtain the document data after reasoning calculation, so that the reasoning calculation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to reasoning calculation methods and related equipment. Background Art

[0002] With the rapid development of science and technology, large language models (LLMs) have shown excellent performance and demonstrated outstanding capabilities in many tasks. When using large language models to perform various tasks, inference calculations are performed on the data to be processed to obtain the processing results of the large language model on the data to be processed.

[0003] In related inference computing technologies, after an inference computing device receives an inference computing instruction, the inference computing device directly performs inference computing on the data to be processed to obtain an inference computing result.

[0004] In the actual inference calculation process, due to the large scale and huge computing cost of LLMs, the computational complexity of the inference calculation is very high. As a result, when the inference calculation device uses a large language model to infer and calculate longer data, it takes a long time and the computational efficiency is very low.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide an inference calculation method and related equipment, aiming to solve the technical problem that when the inference calculation equipment uses a large language model to infer and calculate longer data, it takes a long time and the calculation efficiency is very low.

[0007] To achieve the above purpose, the present application proposes an inference calculation method, which includes:

[0008] Obtaining document data to be inferred and calculated, and segmenting the document data to obtain multiple segments of text data;

[0009] Based on a preset large language model, reasoning and calculating the multiple sections of text data to obtain multiple sections of target text data, wherein the preset large language model is a current large language model after training;

[0010] Based on a preset paragraph relationship determination algorithm, the correlation between the multiple segments of target text data is determined, and based on the correlation, the multiple segments of target text data are spliced ​​to obtain document data after inference calculation.

[0011] In one embodiment, the step of determining the relevance between the plurality of target text data segments based on a preset paragraph relationship determination algorithm further includes:

[0012] Based on any two segments of the target text data among the multiple segments of the target text data, they are all spliced ​​to obtain multiple segments of spliced ​​data;

[0013] Respectively determining a first length, a second length and a third length corresponding to the spliced ​​data and two segments of target text data corresponding to the spliced ​​data;

[0014] Determine a normalized compression distance between two segments of target text data corresponding to the spliced ​​data based on a preset paragraph relationship determination algorithm, a first length, a second length, and a third length;

[0015] Determine the size of the compression distance and a preset compression distance threshold;

[0016] If the compression distance is less than a preset compression distance threshold, it is determined that there is a correlation between the two segments of target text data corresponding to the spliced ​​data.

[0017] In one embodiment, the step of determining the relevance between the multiple segments of target text data based on the relevance further includes:

[0018] Based on any two segments of target text data corresponding to the correlation, respectively determine the first semantics and the second semantics corresponding to any two segments of target text data corresponding to the correlation;

[0019] Using a word vector embedding module to determine whether the first semantics is similar to the second semantics;

[0020] If they are similar, any two segments of target text data corresponding to the correlation are concatenated to obtain document data after inference calculation.

[0021] In one embodiment, the step of using a word vector embedding module to determine whether the first semantics is similar to the second semantics further includes:

[0022] Use the word vector Embedding module in the preset large language model to vectorize the first semantics and the second semantics to obtain a first feature vector and a second feature vector;

[0023] The distance between the first feature vector and the second feature vector is calculated, and based on the distance and a preset distance threshold, it is determined whether the first semantics is similar to the second semantics.

[0024] In one embodiment, the step of concatenating any two segments of target text data corresponding to the relevance to obtain document data after inference calculation further includes:

[0025] Input any two segments of target text data corresponding to the correlation into a preset prompt word module to obtain a first prompt word and a second prompt word respectively;

[0026] Based on the first prompt word and the second prompt word, any two segments of target text data corresponding to the relevance are concatenated to obtain document data after inference calculation.

[0027] In one embodiment, before the step of inferring and calculating the multiple short data segments based on the preset large language model to obtain the multiple short data segments after inference and calculation, the step further includes:

[0028] Determine the current large language model;

[0029] Acquire sample data, wherein the inference calculation result of the sample data is a first inference result;

[0030] Based on the current large language model, performing reasoning calculation on the sample data to obtain a second reasoning result between the sample data;

[0031] Based on the first reasoning result and the second reasoning result, obtaining a reasoning result quantization difference value, and judging whether the reasoning result quantization difference value is greater than a preset reasoning result quantization difference value;

[0032] If the quantized difference of the inference result is greater than the preset quantized difference of the inference result, adjusting the parameters of the current large language model and recording the number of parameter adjustments;

[0033] Based on the current large language model after adjusting the parameters, return to the step of performing inference calculation on the sample data based on the current large language model to obtain a second inference result between the sample data, until the quantized difference of the inference result is less than or equal to the preset quantized difference of the inference result, or the number of parameter adjustments is equal to the preset number of adjustments, and select the current large language model corresponding to the minimum value of the quantized difference of the inference result as the preset large language model.

[0034] In addition, to achieve the above purpose, the present application also proposes an inference computing device, the inference computing device comprising:

[0035] An acquisition module, the acquisition module is used to acquire document data to be inferred and calculated, and segment the document data to obtain multiple segments of text data;

[0036] An inference calculation module, the inference calculation module is used to infer and calculate the multiple text data segments based on a preset large language model to obtain multiple target text data segments, wherein the preset large language model is a current large language model after training;

[0037] A determination module is used to determine the correlation between the multiple segments of target text data based on a preset paragraph relationship determination algorithm, and based on the correlation, splice the multiple segments of target text data to obtain document data after inference calculation.

[0038] In one embodiment, the determining module includes:

[0039] A first splicing unit, configured to splice any two segments of target text data among the multiple segments of target text data to obtain multiple segments of spliced ​​data;

[0040] A first determining unit, used to respectively determine a first length, a second length and a third length corresponding to the spliced ​​data and two segments of target text data corresponding to the spliced ​​data;

[0041] A second determination unit is used to determine a normalized compression distance between two segments of target text data corresponding to the spliced ​​data based on a preset paragraph relationship determination algorithm, the first length, the second length, and the third length;

[0042] A first judging unit, configured to judge the size of the compression distance and a preset compression distance threshold;

[0043] The third determining unit determines that there is a correlation between the two segments of target text data corresponding to the spliced ​​data if the compression distance is less than a preset compression distance threshold.

[0044] In one embodiment, the determining module further includes:

[0045] A fourth determining unit, configured to respectively determine the first semantics and the second semantics corresponding to the arbitrary two segments of target text data corresponding to the correlation based on the arbitrary two segments of target text data corresponding to the correlation;

[0046] A fifth determining unit, configured to determine whether the first semantics is similar to the second semantics by using a word vector embedding module;

[0047] The second concatenation unit concatenates any two segments of target text data corresponding to the relevance if they are similar, to obtain document data after inference calculation.

[0048] In one embodiment, the determining module further includes:

[0049] A vectorization unit, used to vectorize the first semantics and the second semantics using a word vector Embedding module in a preset large language model to obtain a first feature vector and a second feature vector;

[0050] The calculation unit is used to calculate the distance between the first feature vector and the second feature vector, and determine whether the first semantics is similar to the second semantics based on the distance and a preset distance threshold.

[0051] In one embodiment, the determining module further includes:

[0052] An input unit, used for inputting any two segments of target text data corresponding to the correlation into a preset prompt word module to obtain a first prompt word and a second prompt word respectively;

[0053] The obtaining unit is used to concatenate any two sections of target text data corresponding to the relevance based on the first prompt word and the second prompt word to obtain document data after inference calculation.

[0054] In one embodiment, the inference calculation module includes:

[0055] A sixth determining unit, configured to determine a current large language model;

[0056] An acquisition unit, configured to acquire sample data, wherein the inference calculation result of the sample data is a first inference result;

[0057] An inference calculation unit, configured to perform inference calculation on the sample data based on the current large language model to obtain a second inference result between the sample data;

[0058] A second judgment unit is used to obtain a quantitative difference of the reasoning result based on the first reasoning result and the second reasoning result, and judge whether the quantitative difference of the reasoning result is greater than a preset quantitative difference of the reasoning result;

[0059] an adjustment unit, which adjusts the parameters of the current large language model if the quantization difference of the inference result is greater than a preset quantization difference of the inference result, and records the number of parameter adjustments;

[0060] A training unit is used to return to the step of performing inference calculation on the sample data based on the current large language model after adjusting the parameters to obtain a second inference result between the sample data, until the quantized difference of the inference result is less than or equal to the preset quantized difference of the inference result, or the number of parameter adjustments is equal to the preset number of adjustments, and select the current large language model corresponding to the minimum value of the quantized difference of the inference result as the preset large language model.

[0061] In addition, to achieve the above-mentioned purpose, the present application also proposes an inference computing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the inference computing method described above.

[0062] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the inference calculation method described above are implemented.

[0063] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the inference calculation method described above are implemented.

[0064] One or more technical solutions proposed in this application have at least the following technical effects:

[0065] The present application proposes an inference calculation method and related equipment. In the related technology, in the actual inference calculation process, due to the huge scale and huge computing cost of LLMs, the computational complexity of the inference calculation is very high. Therefore, when the inference calculation equipment uses a large language model to infer and calculate longer data, it takes a long time and the computing efficiency is very low. In the present application, first, the document data to be inferred and calculated is obtained, and the document data is segmented to obtain multiple segments of text data. Then, based on a preset large language model, the multiple segments of text data are inferred and calculated to obtain multiple segments of target text data, wherein the preset large language model is the current large language model after training. Finally, based on a preset paragraph relationship determination algorithm, the correlation between the multiple segments of target text data is determined. Based on the correlation, the multiple segments of target text data are spliced ​​to obtain the document data after inference calculation.

[0066] It is understandable that in the related art, the reasoning and computing device uses a large language model to directly reason and calculate the document data to be reasoned and calculated, and obtains the document data after the reasoning is completed, which takes a lot of time and is very inefficient. In the present application, the document data is divided into multiple segments of text data, and the reasoning and computing device uses a large language model to reason and calculate multiple segments of text data, which can shorten the time of reasoning and improve the computing efficiency. After the reasoning and computing is completed, based on the normalized compression distance between the target text data after the reasoning and computing, the correlation between the target text data after the reasoning and computing is determined, and the associated target text data is spliced ​​to obtain the document data after the reasoning and computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0069] Figure 1 A flowchart of the first embodiment of the inference calculation method of the present application is provided;

[0070] Figure 2 A brief flowchart of the inference calculation method provided in Example 1 of the present application;

[0071] Figure 3 A flowchart diagram of the second embodiment of the inference calculation method of the present application is provided;

[0072] Figure 4 This is a schematic diagram of the module structure of the reasoning computing device according to an embodiment of the present application;

[0073] Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the inference calculation method in the embodiment of the present application.

[0074] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0075] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0076] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0077] The main solutions of the embodiments of this application are:

[0078] In this embodiment, for ease of description, the following description is made with the inference computing device as the execution subject.

[0079] Because the prior art requires a long time to use a large language model to infer and calculate a long piece of data, the calculation efficiency is very low.

[0080] The present application provides a solution, which enables: first, obtaining document data to be inferred and calculated, and segmenting the document data to obtain multiple segments of text data; then, based on a preset large language model, inferring and calculating the multiple segments of text data to obtain multiple segments of target text data, wherein the preset large language model is the current large language model after training; finally, based on a preset paragraph relationship determination algorithm, determining the correlation between the multiple segments of target text data; based on the correlation, splicing the multiple segments of target text data to obtain the document data after inference and calculation.

[0081] It is understandable that in the related art, the reasoning and computing device uses a large language model to directly reason and calculate the document data to be reasoned and calculated, and obtains the document data after the reasoning is completed, which takes a lot of time and is very inefficient. In the present application, the document data is divided into multiple segments of text data, and the reasoning and computing device uses a large language model to reason and calculate multiple segments of text data, which can shorten the time of reasoning and improve the computing efficiency. After the reasoning and computing is completed, based on the normalized compression distance between the target text data after the reasoning and computing, the correlation between the target text data after the reasoning and computing is determined, and the associated target text data is spliced ​​to obtain the document data after the reasoning and computing.

[0082] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an inference computing device, etc. The following takes the inference computing device as an example to illustrate this embodiment and the following embodiments.

[0083] Based on this, the present application embodiment provides an inference calculation method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the inference calculation method of the present application.

[0084] In this embodiment, the inference calculation method includes steps S100 to S300:

[0085] Step S100, obtaining document data to be inferred and calculated, and segmenting the document data to obtain multiple segments of text data;

[0086] In this embodiment, the execution subject is an inference computing device, which can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions.

[0087] In this embodiment, the specific application scenario may be:

[0088] For a document data to be inferred, the staff needs to perform inference calculation on the long text document, and then the staff clicks the "inference calculation" button. After the inference calculation device detects that the staff clicks the "inference calculation" button, the inference calculation device reads the document data to be inferred from the memory.

[0089] Step S200, based on a preset large language model, inferring and calculating the multiple text data segments to obtain multiple target text data segments, wherein the preset large language model is a current large language model after training;

[0090] It should be noted that the preset large language model is a trained large language model, which is a model that outputs the inference calculation results of the input data based on the input data.

[0091] It can be understood that the inference calculation device performs inference calculation on the multiple sections of text data to obtain multiple sections of target text data.

[0092] In this embodiment, the inference computing device uses a large language model to process the input data, which takes a lot of time. The time required for the inference computing is related to the data length of the input data. The time required for the inference computing increases exponentially with the increase in the data length of the input data.

[0093] Step S300, based on a preset paragraph relationship determination algorithm, determine the correlation between the multiple segments of target text data, and based on the correlation, concatenate the multiple segments of target text data to obtain document data after inference calculation.

[0094] It can be understood that after the document data is segmented, the segmented text data is inferred and calculated to obtain multiple segments of target text data. Before the multiple segments of target text data are spliced ​​together, it is necessary to first determine the correlation between each segment of target text data. Based on the correlation, the multiple segments of target text data are spliced ​​together to obtain the document data after inference calculation.

[0095] Specifically, please refer to Figure 2 The step of concatenating the multiple target text data segments based on the correlation to obtain the document data after inference calculation also includes steps S310 to S350:

[0096] Step S310, based on any two segments of the target text data in the multiple segments, splicing is performed to obtain multiple segments of spliced ​​data;

[0097] It should be noted that when two target text data segments are concatenated, the two target text data segments may contain the same text data, and thus, the data after the two target text data segments are concatenated is not necessarily equal to the sum of the lengths of the two target text data segments.

[0098] Step S320, respectively determining a first length, a second length, and a third length corresponding to the concatenated data and two segments of target text data corresponding to the concatenated data;

[0099] It should be noted that the first length, the second length and the third length all represent data lengths.

[0100] Step S330, determining a normalized compression distance between two segments of target text data corresponding to the spliced ​​data based on a preset paragraph relationship determination algorithm, the first length, the second length, and the third length;

[0101] Specifically, the preset paragraph relationship determination algorithm is as follows:

[0102]

[0103] Among them, X and Y represent two segments of target text data corresponding to the spliced ​​data, C(X) and C(Y) represent the first length and the second length respectively, NCD(X,Y) represents the normalized compression distance, min(C(X), C(Y)) represents the smaller length value between the first length and the second length, and max(C(X), C(Y)) represents the larger length value between the first length and the second length.

[0104] Step S340, determining the size of the compression distance and a preset compression distance threshold;

[0105] It can be understood that if the value of the normalized compression distance is smaller, it means that there are more overlapping data in the two sections of target text data corresponding to the spliced ​​data, that is, the degree of association between the two sections of target text data corresponding to the spliced ​​data is higher.

[0106] Step S350: If the compression distance is less than a preset compression distance threshold, it is determined that there is a correlation between the two segments of target text data corresponding to the spliced ​​data, and based on the correlation, the correlation between the multiple segments of target text data is determined.

[0107] It should be noted that in this embodiment, there may be a segment of target text data among the multiple segments of target text data whose normalized compression distance values ​​with other segments of target text data are all smaller than a preset compression distance threshold. At this time, it is determined that there is a correlation between the two segments of target text data corresponding to the minimum value of the normalized compression distance.

[0108] In this embodiment, a preset paragraph relationship determination algorithm is used to determine whether there is a correlation between two target text data segments corresponding to the spliced ​​data, and then obtain the correlation between the multiple target text data segments.

[0109] Specifically, the step of determining the relevance between the multiple segments of target text data based on the relevance further includes steps S360 to S380:

[0110] Step S360, based on any two segments of target text data corresponding to the correlation, respectively determining the first semantics and the second semantics corresponding to any two segments of target text data corresponding to the correlation;

[0111] It should be noted that the first semantics and the second semantics respectively indicate the topics corresponding to any two segments of target text data corresponding to the association.

[0112] Step S370, using a word vector embedding module to determine whether the first semantics is similar to the second semantics;

[0113] It should be noted that the preset large language model includes a word vector Embedding module, which can convert text data into feature vectors.

[0114] Specifically, the step of using the word vector embedding module to determine whether the first semantics is similar to the second semantics also includes steps S371 to S372:

[0115] Step S371, using a word vector embedding module in a preset large language model to vectorize the first semantics and the second semantics to obtain a first feature vector and a second feature vector;

[0116] Step S372: Calculate the distance between the first feature vector and the second feature vector, and determine whether the first semantics is similar to the second semantics based on the distance and a preset distance threshold.

[0117] It should be noted that, in this embodiment, the smaller the distance between the first feature vector and the second feature vector, the closer the first semantics is to the second semantics. When the distance is less than or equal to a preset distance threshold, it can be determined that the first semantics is similar to the second semantics.

[0118] Step S380: If they are similar, any two segments of target text data corresponding to the correlation are concatenated to obtain document data after inference calculation.

[0119] In this embodiment, based on the semantic feature vector, the association relationship between the two associated target text data segments is further verified to improve the reliability of splicing.

[0120] Specifically, the step of concatenating any two segments of target text data corresponding to the relevance to obtain the document data after inference calculation also includes steps S381 to S382:

[0121] Step S381, inputting any two target text data corresponding to the correlation into a preset prompt word module to obtain a first prompt word and a second prompt word respectively;

[0122] It should be noted that the prompt word refers to the keywords in the two target text data, and the themes of the two target text data are determined based on the keywords. Furthermore, based on the first prompt word and the second prompt word, the themes of the two target text data are determined, and whether the themes of the two target text data are different.

[0123] Step S382: Based on the first prompt word and the second prompt word, any two sections of target text data corresponding to the relevance are concatenated to obtain document data after inference calculation.

[0124] In this embodiment, there are both document data after inference calculation and the subject of each segment of target text data, which is convenient for users to browse documents.

[0125] The present application proposes an inference calculation method and related equipment. In the related technology, in the actual inference calculation process, due to the huge scale and huge computing cost of LLMs, the computational complexity of the inference calculation is very high. Therefore, when the inference calculation equipment uses a large language model to infer and calculate longer data, it takes a long time and the computing efficiency is very low. In the present application, first, the document data to be inferred and calculated is obtained, and the document data is segmented to obtain multiple segments of text data. Then, based on a preset large language model, the multiple segments of text data are inferred and calculated to obtain multiple segments of target text data, wherein the preset large language model is the current large language model after training. Finally, based on a preset paragraph relationship determination algorithm, the correlation between the multiple segments of target text data is determined. Based on the correlation, the multiple segments of target text data are spliced ​​to obtain the document data after inference calculation.

[0126] It is understandable that in the related art, the reasoning and computing device uses a large language model to directly reason and calculate the document data to be reasoned and calculated, and obtains the document data after the reasoning is completed, which takes a lot of time and is very inefficient. In the present application, the document data is divided into multiple segments of text data, and the reasoning and computing device uses a large language model to reason and calculate multiple segments of text data, which can shorten the time of reasoning and improve the computing efficiency. After the reasoning and computing is completed, based on the normalized compression distance between the target text data after the reasoning and computing, the correlation between the target text data after the reasoning and computing is determined, and the associated target text data is spliced ​​to obtain the document data after the reasoning and computing.

[0127] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 Before step S200, the inference calculation method further includes steps S210 to S260:

[0128] Step S210, determining the current large language model;

[0129] Step S220, obtaining sample data, wherein the inference calculation result of the sample data is a first inference result;

[0130] It should be noted that the sample data is used to train the current large language model.

[0131] Step S230, performing inference calculation on the sample data based on the current large language model to obtain a second inference result between the sample data;

[0132] It is understandable that the first inference result is the actual inference result, and the second inference result is the result obtained after the current large language model infers and calculates the sample data, which contains errors.

[0133] Step S240, obtaining a quantized difference of reasoning results based on the first reasoning result and the second reasoning result, and determining whether the quantized difference of reasoning results is greater than a preset quantized difference of reasoning results;

[0134] It is understandable that if the quantized difference of the inference result is greater than the preset quantized difference of the inference result, it indicates that the error of the inference calculation result of the current large language model is large and does not meet expectations, and thus the current large language model needs to be trained.

[0135] Step S250, if the quantization difference of the inference result is greater than the preset quantization difference of the inference result, adjusting the parameters of the current large language model and recording the number of parameter adjustments;

[0136] Step S260, based on the current large language model after adjusting the parameters, return to the step of performing inference calculation on the sample data based on the current large language model to obtain a second inference result between the sample data, until the quantized difference of the inference result is less than or equal to the preset quantized difference of the inference result, or the number of parameter adjustments is equal to the preset number of adjustments, and the current large language model corresponding to the minimum value of the quantized difference of the inference result is selected as the preset large language model.

[0137] In this embodiment, the large language model is trained based on sample data, which effectively reduces the inference calculation results obtained by the large language model performing inference calculations on text data and improves the reliability of the inference calculation results.

[0138] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the reasoning and calculation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0139] This application also provides an inference computing device, please refer to Figure 4 , the inference computing device comprises:

[0140] An acquisition module 10, which is used to acquire document data to be inferred and calculated, and segment the document data to obtain multiple segments of text data;

[0141] An inference calculation module 20, which is used to infer and calculate the multiple text data segments based on a preset large language model to obtain multiple target text data segments, wherein the preset large language model is a current large language model after training;

[0142] The determination module 30 is used to determine the correlation between the multiple target text data segments based on a preset paragraph relationship determination algorithm, and to concatenate the multiple target text data segments based on the correlation to obtain document data after inference calculation.

[0143] In one embodiment, the determining module includes:

[0144] A first splicing unit, configured to splice any two segments of target text data among the multiple segments of target text data to obtain multiple segments of spliced ​​data;

[0145] A first determining unit, used to respectively determine a first length, a second length and a third length corresponding to the spliced ​​data and two segments of target text data corresponding to the spliced ​​data;

[0146] A second determination unit is used to determine a normalized compression distance between two segments of target text data corresponding to the spliced ​​data based on a preset paragraph relationship determination algorithm, the first length, the second length, and the third length;

[0147] A first judging unit, configured to judge the size of the compression distance and a preset compression distance threshold;

[0148] The third determining unit determines that there is a correlation between the two segments of target text data corresponding to the spliced ​​data if the compression distance is less than a preset compression distance threshold.

[0149] In one embodiment, the determining module further includes:

[0150] A fourth determining unit, configured to respectively determine the first semantics and the second semantics corresponding to the arbitrary two segments of target text data corresponding to the correlation based on the arbitrary two segments of target text data corresponding to the correlation;

[0151] A fifth determining unit, configured to determine whether the first semantics is similar to the second semantics by using a word vector embedding module;

[0152] The second concatenation unit concatenates any two segments of target text data corresponding to the relevance if they are similar, to obtain document data after inference calculation.

[0153] In one embodiment, the determining module further includes:

[0154] A vectorization unit, used to vectorize the first semantics and the second semantics using a word vector Embedding module in a preset large language model to obtain a first feature vector and a second feature vector;

[0155] The calculation unit is used to calculate the distance between the first feature vector and the second feature vector, and determine whether the first semantics is similar to the second semantics based on the distance and a preset distance threshold.

[0156] In one embodiment, the determining module further includes:

[0157] An input unit, used for inputting any two segments of target text data corresponding to the correlation into a preset prompt word module to obtain a first prompt word and a second prompt word respectively;

[0158] The obtaining unit is used to concatenate any two sections of target text data corresponding to the relevance based on the first prompt word and the second prompt word to obtain document data after inference calculation.

[0159] In one embodiment, the inference calculation module includes:

[0160] A sixth determining unit, configured to determine a current large language model;

[0161] An acquisition unit, configured to acquire sample data, wherein the inference calculation result of the sample data is a first inference result;

[0162] An inference calculation unit, configured to perform inference calculation on the sample data based on the current large language model to obtain a second inference result between the sample data;

[0163] A second judgment unit is used to obtain a quantitative difference of the reasoning result based on the first reasoning result and the second reasoning result, and judge whether the quantitative difference of the reasoning result is greater than a preset quantitative difference of the reasoning result;

[0164] an adjustment unit, which adjusts the parameters of the current large language model if the quantization difference of the inference result is greater than a preset quantization difference of the inference result, and records the number of parameter adjustments;

[0165] A training unit is used to return to the step of performing inference calculation on the sample data based on the current large language model after adjusting the parameters to obtain a second inference result between the sample data, until the quantized difference of the inference result is less than or equal to the preset quantized difference of the inference result, or the number of parameter adjustments is equal to the preset number of adjustments, and select the current large language model corresponding to the minimum value of the quantized difference of the inference result as the preset large language model.

[0166] The inference calculation device provided by the present application adopts the inference calculation method in the above embodiment to solve the technical problem of inference calculation. Compared with the prior art, the beneficial effects of the inference calculation device provided by the present application are the same as the beneficial effects of the inference calculation method provided by the above embodiment, and the other technical features in the inference calculation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0167] The present application provides an inference computing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the inference computing method in the above-mentioned embodiment one.

[0168] Reference below Figure 5 , which shows a schematic diagram of the structure of an inference computing device suitable for implementing the embodiment of the present application. The inference computing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The inference computing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0169] like Figure 5As shown, the inference computing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the inference computing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the inference computing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an inference computing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0170] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0171] The inference computing device provided by the present application adopts the inference computing method in the above embodiment to solve the technical problems of inference computing. Compared with the prior art, the beneficial effects of the inference computing device provided by the present application are the same as the beneficial effects of the inference computing method provided by the above embodiment, and the other technical features in the inference computing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0172] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0174] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the inference calculation method in the above-mentioned embodiment.

[0175] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0176] The computer-readable storage medium may be included in the inference computing device; or may exist independently without being assembled into the inference computing device.

[0177] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the inference computing device, the inference computing device:

[0178] Obtaining document data to be inferred and calculated, and segmenting the document data to obtain multiple segments of text data;

[0179] Based on a preset large language model, reasoning and calculating the multiple sections of text data to obtain multiple sections of target text data, wherein the preset large language model is a current large language model after training;

[0180] Based on a preset paragraph relationship determination algorithm, the correlation between the multiple segments of target text data is determined, and based on the correlation, the multiple segments of target text data are spliced ​​to obtain document data after inference calculation.

[0181] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0184] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned inference calculation method, and can solve the technical problems of inference calculation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the inference calculation method provided in the above-mentioned embodiment, and will not be repeated here.

[0185] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned reasoning calculation method when executed by a processor.

[0186] The computer program product provided by this application can solve the technical problem of inference calculation. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the inference calculation method provided by the above embodiment, which will not be repeated here.

[0187] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method of inference calculation, characterized in that: The inference calculation method comprises: Obtaining document data to be inferred and calculated, and segmenting the document data to obtain multiple segments of text data; Based on a preset large language model, reasoning and calculating the multiple sections of text data to obtain multiple sections of target text data, wherein the preset large language model is a current large language model after training; Based on a preset paragraph relationship determination algorithm, the correlation between the multiple segments of target text data is determined, and based on the correlation, the multiple segments of target text data are spliced ​​to obtain document data after inference calculation.

2. The inference calculation method according to claim 1, characterized in that: The step of determining the relevance between the plurality of target text data segments based on a preset paragraph relationship determination algorithm further includes: Based on any two segments of the target text data among the multiple segments of the target text data, they are all spliced ​​to obtain multiple segments of spliced ​​data; Respectively determining a first length, a second length and a third length corresponding to the spliced ​​data and two segments of target text data corresponding to the spliced ​​data; Determine a normalized compression distance between two segments of target text data corresponding to the spliced ​​data based on a preset paragraph relationship determination algorithm, a first length, a second length, and a third length; Determine the size of the compression distance and a preset compression distance threshold; If the compression distance is less than a preset compression distance threshold, it is determined that there is a correlation between the two segments of target text data corresponding to the spliced ​​data.

3. The inference calculation method according to claim 1, characterized in that: The step of determining the relevance between the multiple segments of target text data based on the relevance further includes: Based on any two segments of target text data corresponding to the correlation, respectively determine the first semantics and the second semantics corresponding to any two segments of target text data corresponding to the correlation; Using a word vector embedding module to determine whether the first semantics is similar to the second semantics; If they are similar, any two segments of target text data corresponding to the correlation are concatenated to obtain document data after inference calculation.

4. The inference calculation method according to claim 3, characterized in that: The step of using the word vector embedding module to determine whether the first semantics is similar to the second semantics also includes: Use the word vector Embedding module in the preset large language model to vectorize the first semantics and the second semantics to obtain a first feature vector and a second feature vector; The distance between the first feature vector and the second feature vector is calculated, and based on the distance and a preset distance threshold, it is determined whether the first semantics is similar to the second semantics.

5. The inference calculation method according to claim 3, characterized in that: The step of concatenating any two segments of target text data corresponding to the relevance to obtain document data after inference calculation also includes: Input any two segments of target text data corresponding to the correlation into a preset prompt word module to obtain a first prompt word and a second prompt word respectively; Based on the first prompt word and the second prompt word, any two segments of target text data corresponding to the relevance are concatenated to obtain document data after inference calculation.

6. The inference calculation method according to claim 1, characterized in that: Before the step of inferring and calculating the multiple short data segments based on the preset large language model to obtain the multiple short data segments after inference and calculation, the method further includes: Determine the current large language model; Acquire sample data, wherein the inference calculation result of the sample data is a first inference result; Based on the current large language model, performing reasoning calculation on the sample data to obtain a second reasoning result between the sample data; Based on the first reasoning result and the second reasoning result, obtaining a reasoning result quantization difference value, and judging whether the reasoning result quantization difference value is greater than a preset reasoning result quantization difference value; If the quantized difference of the inference result is greater than the preset quantized difference of the inference result, adjusting the parameters of the current large language model and recording the number of parameter adjustments; Based on the current large language model after adjusting the parameters, return to the step of performing inference calculation on the sample data based on the current large language model to obtain a second inference result between the sample data, until the quantized difference of the inference result is less than or equal to the preset quantized difference of the inference result, or the number of parameter adjustments is equal to the preset number of adjustments, and select the current large language model corresponding to the minimum value of the quantized difference of the inference result as the preset large language model.

7. An inference computing device, characterized in that: The inference computing device comprises: An acquisition module, the acquisition module is used to acquire document data to be inferred and calculated, and segment the document data to obtain multiple segments of text data; An inference calculation module, the inference calculation module is used to infer and calculate the multiple text data segments based on a preset large language model to obtain multiple target text data segments, wherein the preset large language model is a current large language model after training; A determination module is used to determine the correlation between the multiple segments of target text data based on a preset paragraph relationship determination algorithm, and based on the correlation, splice the multiple segments of target text data to obtain document data after inference calculation.

8. An inference computing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the inference calculation method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the inference calculation method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the inference computing method according to any one of claims 1 to 6 are implemented.

Images