Model data processing method and device and electronic equipment

Abstract

The embodiment of the invention provides a model data processing method and device and electronic equipment, and relates to the technical field of computers. According to the embodiment of the invention, at least one piece of spliced data can be determined according to the length threshold and the statement length of each input statement, and then the processing core is controlled to process the effective data of the spliced data. Therefore, according to the embodiment of the invention, when the processing core performs data processing, the length of the effective data is as close as possible to the maximum length of the model processing statement, so that the processing core can process a plurality of statements at a time, and meanwhile, a filling part can be prevented from participating in text processing of the model. Therefore, by determining the spliced data and processing the effective data of the spliced data, the text processing efficiency of the model can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model data processing method, device and electronic device. Background Art

[0002] In the field of Natural Language Processing (NLP), it is often necessary to use models with structures such as transformers to perform semantic understanding or feature extraction on text. The maximum length of sentences that can be processed by the above models is often fixed, and the models need to process text according to the maximum length when processing text.

[0003] When processing text, the related technology will pad the input text to the maximum length and uniformly process the input text according to the maximum length. In this process, although the padding part will not affect the result of text processing, since the padding part still participates in the model's processing of the text, it will have a certain impact on the efficiency of the model, that is, it will lead to a low efficiency in the model's processing of the text. Summary of the invention

[0004] In view of this, embodiments of the present application provide a model data processing method, device and electronic device to improve the model's processing efficiency for text.

[0005] In a first aspect, a model data processing method is provided, the method comprising:

[0006] Get at least one input statement.

[0007] According to a preset length threshold and the sentence length of each of the input sentences, the at least one input sentence is processed to obtain at least one concatenated data, wherein the length of the concatenated data is less than or equal to the length threshold.

[0008] Control at least one processing core to receive the concatenated data, so that the processing core processes valid data of the concatenated data to determine a data processing result, wherein the valid data is used to characterize data corresponding to an input statement in the processing core.

[0009] In some embodiments, the processing of the at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one concatenated data includes:

[0010] The input sentences are sequentially concatenated to determine a first sentence.

[0011] The first sentence is split according to the length threshold to determine at least one concatenated data.

[0012] In some embodiments, the concatenated data includes complete data and / or split data, wherein the complete data includes all data corresponding to at least one input sentence, and the split data includes partial data corresponding to the input sentence.

[0013] In some embodiments, the processing core processes the valid data of the spliced ​​data, and determining the data processing result includes:

[0014] In response to the data length of the concatenated data in the processing core being smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0015] The valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0016] The matrix multiplication result of the effective data and the preset target weight matrix is ​​calculated to determine the data processing result.

[0017] In some embodiments, the processing core processes the valid data of the spliced ​​data, and determining the data processing result includes:

[0018] In response to the data length of the concatenated data in the processing core being smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0019] The valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0020] A self-attention operation is performed on the valid data to determine a data processing result.

[0021] In some embodiments, the spliced ​​data at least includes split data, and the valid data includes sub-data corresponding to the split data.

[0022] The performing self-attention operation processing on the valid data to determine the data processing result includes:

[0023] According to the sub-data and a preset self-attention weight matrix, a first intermediate result corresponding to the sub-data is determined.

[0024] A second intermediate result corresponding to the sub-data is determined based on the remaining data in the other processing cores, wherein the remaining data is used to characterize the valid data corresponding to the remaining part of the split data in the other processing cores.

[0025] A data processing result is determined based on a preset activation function, the first intermediate result, and the second intermediate result.

[0026] In some embodiments, the split data corresponding to the same input sentence are arranged consecutively in each processing core.

[0027] In some embodiments, the method further comprises:

[0028] For each of the processing cores, in response to determining a data processing result corresponding to each input statement in the complete data, a memory corresponding to the complete data is released.

[0029] In a second aspect, a model data processing device is provided, the device comprising:

[0030] The input sentence acquisition module is configured to execute and acquire at least one input sentence.

[0031] The splicing data determination module is configured to process the at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one splicing data, wherein the length of the splicing data is less than or equal to the length threshold.

[0032] The data processing module is configured to execute and control at least one processing core to receive the spliced ​​data so that the processing core processes the valid data of the spliced ​​data and determines the data processing result, wherein the valid data is used to characterize the data corresponding to the input statement in the processing core.

[0033] In some embodiments, the splicing data determination module is specifically configured to execute:

[0034] The input sentences are sequentially concatenated to determine a first sentence.

[0035] The first sentence is split according to the length threshold to determine at least one concatenated data.

[0036] In some embodiments, the concatenated data includes complete data and / or split data, wherein the complete data includes all data corresponding to at least one input sentence, and the split data includes partial data corresponding to the input sentence.

[0037] In some embodiments, the data processing module is specifically configured to perform:

[0038] In response to the data length of the concatenated data in the processing core being smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0039] The valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0040] The matrix multiplication result of the effective data and the preset target weight matrix is ​​calculated to determine the data processing result.

[0041] In some embodiments, the data processing module is specifically configured to perform:

[0042] In response to the data length of the concatenated data in the processing core being smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0043] The valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0044] A self-attention operation is performed on the valid data to determine a data processing result.

[0045] In some embodiments, the spliced ​​data at least includes split data, and the valid data includes sub-data corresponding to the split data.

[0046] The data processing module is specifically configured to execute:

[0047] According to the sub-data and a preset self-attention weight matrix, a first intermediate result corresponding to the sub-data is determined.

[0048] A second intermediate result corresponding to the sub-data is determined based on the remaining data in the other processing cores, wherein the remaining data is used to characterize the valid data corresponding to the remaining part of the split data in the other processing cores.

[0049] A data processing result is determined based on a preset activation function, the first intermediate result, and the second intermediate result.

[0050] In some embodiments, the split data corresponding to the same input sentence are arranged consecutively in each processing core.

[0051] In some embodiments, the apparatus further comprises:

[0052] The release module is configured to execute, for each of the processing cores, in response to determining a data processing result corresponding to each input statement in the complete data, to release the memory corresponding to the complete data.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0055] The embodiment of the present application can determine at least one splicing data according to the length threshold and the sentence length of each input sentence. Therefore, the embodiment of the present application can make the length of the valid data as close as possible to the maximum length of the model processing sentence when the processing core performs data processing, so that the processing core can process multiple sentences at a time, thereby improving the processing efficiency of the model. At the same time, the embodiment of the present application can also control the processing core to process the valid data of the splicing data, so the embodiment of the present application can also avoid the padding part from participating in the model's processing of the text, further improving the processing efficiency of the model. In other words, the embodiment of the present application can effectively improve the model's processing efficiency for the text by determining the splicing data and processing the valid data of the splicing data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other purposes, features and advantages of the embodiments of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0057] Figure 1 Schematic diagram of the process of model data processing for related technologies;

[0058] Figure 2 A schematic diagram of a flow chart of a model data processing method according to an embodiment of the present application;

[0059] Figure 3 A flowchart of a model data processing method according to an embodiment of the present application;

[0060] Figure 4 A flowchart of another model data processing method according to an embodiment of the present application;

[0061] Figure 5 A schematic diagram of splicing data according to an embodiment of the present application;

[0062] Figure 6 A flowchart of another model data processing method according to an embodiment of the present application;

[0063] Figure 7 A flowchart of another model data processing method according to an embodiment of the present application;

[0064] Figure 8 A flowchart of another model data processing method according to an embodiment of the present application;

[0065] Fig. 9 A flowchart of another model data processing method according to an embodiment of the present application;

[0066] Fig.10 A flowchart of another model data processing method according to an embodiment of the present application;

[0067] Fig.11 A flowchart of another model data processing method according to an embodiment of the present application;

[0068] Fig.12 A flowchart of another model data processing method according to an embodiment of the present application;

[0069] Fig.13 A schematic diagram of the structure of a model data processing device according to an embodiment of the present application;

[0070] Fig.14 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0071] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0072] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0073] Unless the context clearly requires otherwise, the words "include", "comprising" and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning "including but not limited to".

[0074] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0075] In the field of Natural Language Processing (NLP), it is often necessary to perform semantic understanding or feature extraction on text through models with structures such as transformers. For example, related technologies can process input text through the attention layer in the Bidirectional Encoder Representation from Transformers (BERT) model to determine the feature sequence corresponding to the input text.

[0076] Take the Attention Layer in BERT as an example. Figure 1 As shown in the figure, after receiving the input text, the Attention Layer can multiply the matrix corresponding to the input text with three different weight matrices to determine the Query matrix, Key matrix and Value matrix. Among them, the matrix corresponding to the input text can be represented as [Batch, seq_len, Hidden_size], Batch is used to represent the number of sentences processed by a single processing core at a time (the number of sentences is a value greater than or equal to 0, and the value can be less than 1), seq_len is used to represent the sentence length, and Hidden_size is used to represent the hidden size. The weight matrix can be represented as [Hidden_size, Hidden_size]. By performing matrix multiplication operations on the matrix corresponding to the input text and the weight matrices with different parameters, the Attention Layer can determine the Query matrix, Key matrix and Value matrix. The Query matrix, Key matrix and Value matrix can be represented as [Batch, seq_len, Hidden_size]. Furthermore, the Attention Layer can reorganize the Query matrix, Key matrix and Value matrix, and the reorganized Query matrix, Key matrix and Value matrix can be represented as [Batch, seq_len, head, Hidden_size / head].

[0077] Furthermore, the Attention Layer can perform a (0, 2, 1, 3) transposition on the Query matrix, and the transposed Query matrix can be represented as [Batch, head, seq_len, Hidden_size / head]. At the same time, the Attention Layer can perform a (0, 2, 3, 1) transposition on the Key matrix, and the transposed Key matrix can be represented as [Batch, head, Hidden_size / head, seq_len]. Furthermore, the Attention Layer can perform a matrix multiplication operation on the above two transposed results to determine the score matrix [Batch, head, seq_len, seq_len].

[0078] Furthermore, after determining the score matrix, the Attention Layer can perform a superposition operation on the score matrix and the Mask matrix, so that the valid part of the score matrix remains unchanged and the invalid part of the score matrix becomes a minimum value. Furthermore, the Attention Layer can perform a Softmax calculation on the result of the above superposition operation in the lowest dimension to determine the Softmax result matrix [Batch, head, seq_len, seq_len]. In the Softmax result matrix, the value corresponding to the above invalid part is 0.

[0079] Furthermore, the Attention Layer can perform a (0, 2, 1, 3) transposition on the Value matrix, and the transposed Value matrix can be represented as [Batch, head, seq_len, Hidden_size / head]. Among them, the transposed result of the Value matrix is ​​the right matrix, and the above-mentioned Softmax result matrix is ​​the left matrix. Then, the Attention Layer can perform a matrix multiplication operation on the transposed result of the Value matrix and the Softmax result matrix to determine the operation result [Batch, head, seq_len, Hidden_size / head]. Furthermore, the Attention Layer can perform a (0, 2, 1, 3) transposition and reorganization on the operation result to determine the Context matrix [Batch, seq_len, hidden_size].

[0080] Furthermore, the Attention Layer can determine the local summation matrix, the local output matrix, the intermediate feature matrix, and the feature matrix in sequence through the Context matrix and the input text, and finally determine the output features.

[0081] It should be noted that in the related art, the maximum length of the sentence that can be processed by the model (such as the model of the above transformer structure) is often fixed, and the model needs to process the text according to the maximum length when processing the text. Therefore, when processing text, the related art will fill the input text to the maximum length and uniformly process the input text according to the maximum length. Taking the above-mentioned Attention Layer as an example, in the process of Attention Layer determining the Context matrix, the input text will be filled first, and then the filled part will be masked by the Mask matrix in the subsequent process to avoid the impact of the filled part.

[0082] In this process, although the padding part will not affect the results of text processing, since the padding part will still participate in the model's processing of the text, the padding part will have a certain impact on the efficiency of the model, which will lead to lower efficiency in text processing by the model.

[0083] Therefore, how to effectively improve the model's efficiency in processing text is an urgent problem that needs to be solved.

[0084] In order to solve the above problems, the embodiment of the present application provides a model data processing method to improve the model's processing efficiency for text. The method can be applied to an electronic device, which can be a terminal or a server. The terminal can be a smart phone, a tablet computer, a vehicle control terminal or a personal computer (PC), etc. The server can be a single server, a server cluster configured in a distributed manner, or a cloud server.

[0085] Specifically, Figure 2 As shown, the electronic device 22 can receive an input sentence 21, and determine a data processing result 23 corresponding to the input sentence 21 based on the above-mentioned model data processing method. Specifically, the electronic device 22 can obtain at least one input sentence, and process at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one spliced ​​data. Furthermore, the electronic device 22 can control at least one processing core to receive the spliced ​​data, so that the processing core processes the valid data of the spliced ​​data and determines the data processing result. Among them, the length of the spliced ​​data is less than or equal to the length threshold, and the valid data is used to characterize the data corresponding to the input sentence in the processing core. The embodiment of the present application can set the length threshold to be equal to or less than the maximum length of a single processing sentence of the model.

[0086] In this process, since the embodiment of the present application can determine at least one splicing data according to the length threshold and the sentence length of each input sentence. Therefore, the embodiment of the present application can make the length of the valid data as close as possible to the maximum length of the model processing sentence when the processing core performs data processing, so that the processing core can process multiple sentences at a time, thereby improving the processing efficiency of the model. At the same time, since the embodiment of the present application can control the processing core to process the valid data of the splicing data, the embodiment of the present application can also avoid the padding part from participating in the model's processing of the text, further improving the processing efficiency of the model. In other words, the embodiment of the present application can effectively improve the model's processing efficiency for the text by determining the splicing data and processing the valid data of the splicing data.

[0087] The model data processing method provided in the embodiment of the present application will be described in detail below in conjunction with specific implementation methods. Figure 3 As shown, the specific steps are as follows:

[0088] In step S100, at least one input sentence is obtained.

[0089] The input sentence may be a sentence directly input by a user, a sentence acquired through speech recognition, or a sentence acquired through other means (such as a corpus).

[0090] In an optional implementation, after obtaining the input sentence, the embodiment of the present application can preprocess the input sentence, wherein the preprocessing can include paragraph division, word segmentation, word normalization, deletion of stop words, and deletion of invalid characters. Through preprocessing, the embodiment of the present application can make the input sentence easier for electronic devices to process, thereby improving the model's processing efficiency for text.

[0091] In step S200, at least one input sentence is processed according to a preset length threshold and the sentence length of each input sentence to obtain at least one concatenated data.

[0092] The length of the concatenated data is less than or equal to a length threshold, and the length threshold can be determined based on the maximum length of a single statement processed by the model. For example, the embodiment of the present application can set the length threshold to be equal to or less than the maximum length of a single statement processed by the model.

[0093] That is to say, since the model needs to process the text according to the maximum length of a single processing statement when processing text, the embodiment of the present application can use a length threshold to make the input data of the model (i.e., the spliced ​​data) meet the above-mentioned maximum length as much as possible, so that the model can process as much valid data as possible at one time, thereby improving the model's processing efficiency for text.

[0094] In an optional implementation, for a process of determining splicing data, such as Figure 4 As shown, the above step S200 may include the following steps:

[0095] In step S210, the input sentences are concatenated in sequence to determine the first sentence.

[0096] Among them, the embodiments of the present application can splice each input statement in the order in which each input statement is received, or can splice each input statement in a predetermined order (for example, the sequence numbers corresponding to each input statement), or can splice each input statement in other applicable ways.

[0097] In step S220, the first sentence is split according to the length threshold to determine at least one concatenated data.

[0098] The length threshold may be equal to the maximum length of a sentence processed by the model at a time, or may be less than the maximum length of a sentence processed by the model at a time. That is, the embodiment of the present application may combine and split each input sentence based on the length threshold, so that the input data (i.e., concatenated data) of the model meets the above maximum length as much as possible, so that the model can process as much valid data as possible at one time, thereby improving the model's processing efficiency for text.

[0099] In an optional implementation, the concatenated data may include complete data and / or split data, wherein the complete data includes all data corresponding to at least one input sentence, and the split data includes partial data corresponding to the input sentence.

[0100] That is to say, since the embodiment of the present application combines and splits each input sentence based on the length threshold, the spliced ​​data after the combination and splitting process may include only the complete data, only the split data, or both the complete data and the split data. Figure 5 As shown, Figure 5 It includes three schematic diagrams of spliced ​​data (ie, schematic diagrams corresponding to spliced ​​data 51, spliced ​​data 52 and spliced ​​data 53 respectively).

[0101] In one case, as shown in the concatenated data 51, the concatenated data 51 includes 3 complete data (sentence 1, sentence 2 and sentence 3) and a padding part, that is, the concatenated data 51 belongs to the case of only including complete data. Further, since the maximum length of a sentence processed by the model at a time and the sentence length of each sentence are known, the embodiment of the present application can determine the valid data in the concatenated data 51 by the maximum length and the sentence length of each sentence. Further, the embodiment of the present application can control the processing and check the valid data of the concatenated data 51 to determine the data processing result, avoid the padding part from participating in the model's processing of the text, and improve the processing efficiency of the model.

[0102] In another case, as shown in the concatenated data 52, the concatenated data 52 includes 3 complete data (sentences 4, 5, and 6) and 1 split data (the upper half of sentence 7), that is, the concatenated data 52 belongs to the case of including both complete data and split data. Further, the embodiment of the present application can determine the valid data in the concatenated data 52 by the maximum length and the sentence length of each sentence, and control the processing to check the valid data of the concatenated data 52 to determine the data processing result, avoid the padding part from participating in the model's processing of the text, and improve the processing efficiency of the model.

[0103] Among them, Figure 5 As shown, the upper part of statement 7 is only used to represent the part of statement 7 after being split, and does not limit the specific length of the split data. In other words, the specific length of the split data is not a fixed value, which is specifically determined by the above-mentioned length threshold.

[0104] In another case, as shown in the concatenated data 53, the concatenated data 53 includes 1 split data (the lower half of the sentence 7), that is, the concatenated data 53 belongs to the case of only including split data. Further, the embodiment of the present application can determine the valid data in the concatenated data 53 by the maximum length and the sentence length of each sentence, and control the processing to check the valid data of the concatenated data 53 to determine the data processing result, avoid the padding part from participating in the model's processing of the text, and improve the processing efficiency of the model.

[0105] Need to explain, Figure 5 The examples provided in the embodiments of the present application for three cases of splicing data do not limit the specific composition of the splicing data. For example, if the splicing of the above statements 1, 2, and 3 is exactly equal to the maximum length, then the embodiments of the present application do not need to fill the splicing data, that is, the splicing data 51 may not include the filling part. For another example, if the length of a statement is too long, the embodiments of the present application can split the statement into multiple parts greater than 2 according to the length threshold, that is, the split data of the embodiments of the present application is not limited to the upper and lower parts.

[0106] In an optional implementation, if the embodiment of the present application splits the same input statement, the split data corresponding to the same input statement can be arranged continuously in each processing core. In this way, the embodiment of the present application can control the processing core to process data by calling the overall calculation instruction once, reducing the number of instruction calls and improving the efficiency of data processing.

[0107] In an optional implementation, the embodiment of the present application may also release the memory corresponding to the complete data for each processing core in response to determining the data processing results corresponding to each input statement in the complete data.

[0108] That is to say, in the case of insufficient memory in the processing core, the embodiment of the present application can first control the processing core to determine the data processing result of the complete statement (i.e., complete data), and release the memory corresponding to the complete data, and then use the released memory to determine the data processing result corresponding to the split data. In this way, the embodiment of the present application can process the split data in time after processing the complete data, thereby improving the efficiency of data processing.

[0109] In step S300, at least one processing core is controlled to receive the spliced ​​data, so that the processing core processes valid data of the spliced ​​data and determines a data processing result.

[0110] The valid data is used to characterize the data corresponding to the input statement in the processing core. Specifically, since the maximum length of the statement processed by the model at a time and the length of each statement are known, the embodiment of the present application can determine the valid data in the spliced ​​data by the maximum length and the length of each statement. Furthermore, the embodiment of the present application can control the processing core to process the valid data of the spliced ​​data to determine the data processing result.

[0111] In this process, since the embodiment of the present application can determine at least one splicing data according to the length threshold and the sentence length of each input sentence. Therefore, the embodiment of the present application can make the length of the valid data as close as possible to the maximum length of the model processing sentence when the processing core performs data processing, so that the processing core can process multiple sentences at a time, thereby improving the processing efficiency of the model. At the same time, since the embodiment of the present application can control the processing core to process the valid data of the splicing data, the embodiment of the present application can also avoid the padding part from participating in the model's processing of the text, further improving the processing efficiency of the model. In other words, the embodiment of the present application can effectively improve the model's processing efficiency for the text by determining the splicing data and processing the valid data of the splicing data.

[0112] In an optional embodiment, if Figure 6 As shown, the above step S300 may include the following steps:

[0113] In step S310 , in response to the fact that the length of the concatenated data in the processing core is smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0114] by Figure 5 For example, since the total length of statements 1, 2, and 3 is less than the maximum length of a single statement processed by the model, the embodiment of the present application can fill the concatenated data 51 so that the concatenated data 51 is the same as the maximum length. Similarly, the lower half of statement 7 is also less than the maximum length of a single statement processed by the model, so the embodiment of the present application can fill the concatenated data 53 so that the concatenated data 53 is the same as the maximum length.

[0115] It should be noted that if the concatenated data is equal to the maximum length of a single processing statement of the model, the embodiment of the present application does not need to pad the concatenated data.

[0116] In step S320, valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0117] Among them, since the maximum length of a statement processed by the model at a single time and the sentence length of each statement are known, the embodiment of the present application can determine the valid data in the spliced ​​data through the maximum length and the sentence length of each statement.

[0118] In step S330, a matrix multiplication result of the effective data and a preset target weight matrix is ​​calculated to determine a data processing result.

[0119] That is to say, the embodiment of the present application can directly determine the data processing result through matrix multiplication operation, and the data processing result at this time is the result matrix corresponding to each input statement.

[0120] For example, Figure 7 As shown, after determining the spliced ​​data 71 and filling the spliced ​​data 71 (if the spliced ​​data 71 is equal to the maximum length, no filling is required), the embodiment of the present application can determine the valid data in the spliced ​​data 71. Further, the embodiment of the present application can perform matrix multiplication operations on the statement 711, the statement 712, and the statement 713 with the target weight matrix 72, and respectively determine the result matrix 731, the result matrix 732, and the result matrix 733, wherein the result matrix 731, the result matrix 732, and the result matrix 733 are the data processing results.

[0121] In this process, since the embodiment of the present application can determine at least one spliced ​​data according to the length threshold and the sentence length of each input sentence. Therefore, the embodiment of the present application can make the length of the valid data as close as possible to the maximum length of the model processing sentence when the processing core performs data processing, so that the processing core can process multiple sentences at a time, thereby improving the processing efficiency of the model. At the same time, since the embodiment of the present application can control the processing core to process the valid data of the spliced ​​data, the embodiment of the present application can also avoid the padding part from participating in the model's processing of the text, further improving the processing efficiency of the model.

[0122] In an optional embodiment, if Figure 8 As shown, the above step S300 may include the following steps:

[0123] In step S340 , in response to the fact that the length of the concatenated data in the processing core is smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0124] Among them, if the spliced ​​data is equal to the maximum length of a single processing statement of the model, then the embodiment of the present application does not need to fill the spliced ​​data.

[0125] In step S350, valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0126] Among them, since the maximum length of a statement processed by the model at a single time and the sentence length of each statement are known, the embodiment of the present application can determine the valid data in the spliced ​​data through the maximum length and the sentence length of each statement.

[0127] In step S360, self-attention operation is performed on the valid data to determine the data processing result.

[0128] That is to say, the embodiment of the present application can determine the data processing result based on the self-attention mechanism of the transformer model. Figure 1 Taking the Attention Layer shown as an example, the data processing result can be the Context matrix output by the Attention Layer.

[0129] For example, Fig. 9 As shown, after determining the valid data 91, the valid data 91 can be multiplied with three different self-attention weight matrices (self-attention weight 921, self-attention weight 922 and self-attention weight 923) respectively to determine the Query matrix 931, Key matrix 932 and Value matrix 933 corresponding to the valid data 91.

[0130] Furthermore, the embodiment of the present application can perform reorganization, transposition, matrix multiplication and other processing on the Query matrix 931 and the Key matrix 932 to determine the score matrix 94 of the Query matrix 931 and the Key matrix 932.

[0131] Furthermore, the embodiment of the present application may perform Softmax calculation on the score matrix 94, and perform matrix multiplication operation on the calculation result and the Value matrix 933 to determine the data processing result 95. The data processing result 95 may be a Context matrix.

[0132] contrast Figure 1 and Fig. 9 It can be seen that in the process of determining the Context matrix, the related technology will mask the filled part through the Mask matrix to avoid the impact of the filled part. However, since the filled part will still participate in the model's processing of the text, the filled part will have a certain impact on the efficiency of the model. The embodiment of the present application can determine the valid data of the spliced ​​data before performing the self-attention operation. Therefore, the embodiment of the present application does not need to mask the filled part through the Mask matrix (that is, the embodiment of the present application does not perform operations on the filled part), avoiding the filled part from participating in the model's processing of the text and improving the processing efficiency of the model.

[0133] In an optional implementation, the spliced ​​data at least includes the split data, and the valid data includes the sub-data corresponding to the split data. Figure 5 For example, in the case where the concatenated data only includes split data or includes both complete data and split data (i.e., concatenated data 53 or concatenated data 52), the valid data may include partial data of a certain sentence. For example, concatenated data 52 includes the upper half of sentence 7, and concatenated data 53 includes the lower half of sentence 7. The upper half of sentence 7 and the lower half of sentence 7 are the sub-data corresponding to the above split data.

[0134] Further, such as Fig.10 As shown, the above step S360 may include the following steps:

[0135] In step S361, a first intermediate result corresponding to the sub-data is determined based on the sub-data and a preset self-attention weight matrix.

[0136] Taking the Attention Layer in BERT as an example, the first intermediate result corresponding to the sub-data can be a score matrix of the sub-data determined by the Query matrix and the Key matrix.

[0137] For example, Fig.11As shown, the concatenated data received by processing core 1 (core1) includes 2 complete data (statement a and statement b) and 1 split data (statement c_up), where statement c_up is used to represent the upper half of statement c. The concatenated data received by processing core 2 (core2) includes 1 complete data (statement d) and 1 split data (statement c_down), where statement c_down is used to represent the lower half of statement c. In addition, the upper and lower parts of statement c are only used to represent the part of statement c after being split, and do not limit the specific length of the split data.

[0138] Furthermore, after receiving the concatenated data, if the length of the concatenated data is less than the maximum length of a single processing statement of the model, the concatenated data can be padded. Furthermore, the processing core 1 and the processing core 2 can determine the valid data of the concatenated data according to the sentence length of the input sentence. Among them, the sentence c_up and the sentence c_down are the sub-data corresponding to the split data.

[0139] Furthermore, processing core 1 and processing core 2 can determine the query matrix, key matrix and value matrix corresponding to the valid data according to the preset self-attention weight matrix. Fig.11 As shown, matrix 1101 is the Query matrix corresponding to the valid data in processing core 1 (including complete data and sub-data corresponding to split data), matrix 1102 is the Query matrix corresponding to the valid data in processing core 2, matrix 1103 is the Key matrix corresponding to the valid data in processing core 1, and matrix 1104 is the Key matrix corresponding to the valid data in processing core 2.

[0140] Further, processing core 1 and processing core 2 can each perform reorganization, transposition, and matrix multiplication on the Query matrix and the Key matrix to determine the score matrix corresponding to the complete data and the score matrix corresponding to the sub-data. Specifically, matrix 1105 includes the score matrix corresponding to statements a and b, and the score matrix corresponding to the upper half of statement c (i.e., statement c_uu). Matrix 1106 includes the score matrix corresponding to statement d, and the score matrix corresponding to the lower half of statement c (i.e., statement c_dd). Among them, uu is used to represent the matrix multiplication result of the Query matrix of the upper half of the statement and the Key matrix of the upper half of the statement, and dd is used to represent the matrix multiplication result of the Query matrix of the lower half of the statement and the Key matrix of the lower half of the statement.

[0141] In step S362, the second intermediate result corresponding to the sub-data is determined according to the remaining data in other processing cores.

[0142] The remaining data is used to represent valid data corresponding to the remaining part of the split data in other processing cores.

[0143] For example, Fig.12 As shown, after determining the first intermediate result, processing core 1 can obtain valid data corresponding to the lower half of statement c (for example, the Key matrix of statement c_down) from processing core 2. Furthermore, processing core 1 can determine the second intermediate result corresponding to the sub-data (i.e., statement c_ud) based on the Query matrix of statement c_up and the Key matrix of statement c_down. Among them, ud is used to represent the matrix multiplication result of the Query matrix of the upper half of the statement and the Key matrix of the lower half of the statement. Similarly, after determining the first intermediate result, processing core 2 can also obtain valid data corresponding to the upper half of statement c from processing core 1, thereby determining the second intermediate result corresponding to the sub-data (i.e., statement c_du). Among them, du is used to represent the matrix multiplication result of the Query matrix of the lower half of the statement and the Key matrix of the upper half of the statement.

[0144] In step S363, the data processing result is determined based on the preset activation function, the first intermediate result and the second intermediate result.

[0145] by Fig.12 For example, after determining the first intermediate result (i.e., statement c_uu) and the second intermediate result (i.e., statement c_ud), processing core 1 can perform Softmax calculation on the first intermediate result and the second intermediate result through the Softmax function (activation function), and perform matrix multiplication operation on the calculation result and the Value matrix of statement c_up to determine the data processing result. Similarly, after determining the first intermediate result (i.e., statement c_dd) and the second intermediate result (i.e., statement c_du), processing core 2 can perform Softmax calculation on the first intermediate result and the second intermediate result, and perform matrix multiplication operation on the calculation result and the Value matrix of statement c_down to determine the data processing result.

[0146] Through the embodiments of the present application, the split data in each processing core can be effectively processed. In other words, the embodiments of the present application can make the length of the effective data as close as possible to the maximum length of the model processing statement when the processing core processes the data, so that the processing core can process multiple statements at a time, thereby improving the processing efficiency of the model.

[0147] Based on the same technical concept, the embodiment of the present application also provides a model data processing device, such as Fig.13 As shown, the device includes: an input sentence acquisition module 1301, a splicing data determination module 1302 and a data processing module 1303.

[0148] The input sentence acquisition module 1301 is configured to execute and acquire at least one input sentence.

[0149] The splicing data determining module 1302 is configured to process the at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one splicing data, wherein the length of the splicing data is less than or equal to the length threshold.

[0150] The data processing module 1303 is configured to execute control of at least one processing core to receive the spliced ​​data so that the processing core processes the valid data of the spliced ​​data and determines the data processing result, wherein the valid data is used to represent the data corresponding to the input statement in the processing core.

[0151] In some embodiments, the splicing data determination module 1302 is specifically configured to execute:

[0152] The input sentences are sequentially concatenated to determine a first sentence.

[0153] The first sentence is split according to the length threshold to determine at least one concatenated data.

[0154] In some embodiments, the concatenated data includes complete data and / or split data, wherein the complete data includes all data corresponding to at least one input sentence, and the split data includes partial data corresponding to the input sentence.

[0155] In some embodiments, the data processing module 1303 is specifically configured to perform:

[0156] In response to the data length of the concatenated data in the processing core being smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0157] The valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0158] The matrix multiplication result of the effective data and the preset target weight matrix is ​​calculated to determine the data processing result.

[0159] In some embodiments, the data processing module 1303 is specifically configured to perform:

[0160] In response to the data length of the concatenated data in the processing core being smaller than the maximum length of a single processing statement of the model, the concatenated data received by the processing core is padded.

[0161] The valid data of the concatenated data is determined according to the sentence length of the input sentence.

[0162] A self-attention operation is performed on the valid data to determine a data processing result.

[0163] In some embodiments, the spliced ​​data at least includes split data, and the valid data includes sub-data corresponding to the split data.

[0164] The data processing module 1303 is specifically configured to execute:

[0165] According to the sub-data and a preset self-attention weight matrix, a first intermediate result corresponding to the sub-data is determined.

[0166] A second intermediate result corresponding to the sub-data is determined based on the remaining data in the other processing cores, wherein the remaining data is used to characterize the valid data corresponding to the remaining part of the split data in the other processing cores.

[0167] A data processing result is determined based on a preset activation function, the first intermediate result, and the second intermediate result.

[0168] In some embodiments, the split data corresponding to the same input sentence are arranged consecutively in each processing core.

[0169] In some embodiments, the apparatus further comprises:

[0170] The release module is configured to execute, for each of the processing cores, in response to determining a data processing result corresponding to each input statement in the complete data, to release the memory corresponding to the complete data.

[0171] The embodiment of the present application can determine at least one splicing data according to the length threshold and the sentence length of each input sentence. Therefore, the embodiment of the present application can make the length of the valid data as close as possible to the maximum length of the model processing sentence when the processing core performs data processing, so that the processing core can process multiple sentences at a time, thereby improving the processing efficiency of the model. At the same time, the embodiment of the present application can also control the processing core to process the valid data of the splicing data, so the embodiment of the present application can also avoid the padding part from participating in the model's processing of the text, further improving the processing efficiency of the model. In other words, the embodiment of the present application can effectively improve the model's processing efficiency for the text by determining the splicing data and processing the valid data of the splicing data.

[0172] Fig.14 Schematic diagram of an electronic device according to an embodiment of the present application. Fig.14 As shown, Fig.14The electronic device shown is a general address query device, which includes a general computer hardware structure, which includes at least a processor 1401 and a memory 1402. The processor 1401 and the memory 1402 are connected via a bus 1403. The memory 1402 is suitable for storing instructions or programs executable by the processor 1401. The processor 1401 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 1401 executes the instructions stored in the memory 1402, thereby executing the method flow of the embodiment of the present application as described above to realize the processing of data and the control of other devices. The bus 1403 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 1404 and the display device and the input / output (I / O) device 1405. The input / output (I / O) device 1405 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 1405 is connected to the system through an input / output (I / O) controller 1406.

[0173] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0175] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0176] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0177] Another embodiment of the present application relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0178] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0179] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A model data processing method, characterized in that: The method comprises: Get at least one input sentence; processing the at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one concatenated data, wherein the length of the concatenated data is less than or equal to the length threshold; Control at least one processing core to receive the spliced ​​data, so that the processing core processes valid data of the spliced ​​data and determines a data processing result, wherein the valid data is used to represent data corresponding to an input statement in the processing core.

2. The method according to claim 1, characterized in that The processing of the at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one spliced ​​data includes: sequentially concatenating the input sentences to determine a first sentence; The first sentence is split according to the length threshold to determine at least one concatenated data.

3. The method according to claim 1 or 2, characterized in that The spliced ​​data includes complete data and / or split data, wherein the complete data includes all data corresponding to at least one input sentence, and the split data includes partial data corresponding to the input sentence.

4. The method according to claim 3, characterized in that The processing core processes the valid data of the spliced ​​data to determine the data processing result, including: In response to the data length of the spliced ​​data in the processing core being less than the maximum length of a single processing statement of the model, padding the spliced ​​data received by the processing core; Determining valid data of the spliced ​​data according to the sentence length of the input sentence; Calculate the matrix multiplication result of the effective data and the preset target weight matrix to determine the data processing result.

5. The method according to claim 3 or 4, characterized in that The processing core processes the valid data of the spliced ​​data to determine the data processing result, including: In response to the data length of the spliced ​​data in the processing core being less than the maximum length of a single processing statement of the model, padding the spliced ​​data received by the processing core; Determining valid data of the spliced ​​data according to the sentence length of the input sentence; Perform self-attention operation processing on the valid data to determine the data processing result.

6. The method according to claim 5, characterized in that The spliced ​​data at least includes split data, and the valid data includes sub-data corresponding to the split data; The performing self-attention operation on the valid data to determine the data processing result includes: Determining a first intermediate result corresponding to the sub-data according to the sub-data and a preset self-attention weight matrix; Determine, based on remaining data in other processing cores, a second intermediate result corresponding to the sub-data, wherein the remaining data is used to represent valid data corresponding to a remaining portion of the split data in the other processing cores; A data processing result is determined based on a preset activation function, the first intermediate result, and the second intermediate result.

7. The method according to any one of claims 3 to 6, characterized in that The split data corresponding to the same input sentence are arranged continuously in each processing core.

8. The method according to any one of claims 4 to 7, characterized in that The method further comprises: For each of the processing cores, in response to determining a data processing result corresponding to each input statement in the complete data, the memory corresponding to the complete data is released.

9. A model data processing device, characterized in that: The device comprises: An input statement acquisition module is configured to execute and acquire at least one input statement; a spliced ​​data determining module configured to process the at least one input sentence according to a preset length threshold and the sentence length of each input sentence to obtain at least one spliced ​​data, wherein the length of the spliced ​​data is less than or equal to the length threshold; The data processing module is configured to execute control of at least one processing core to receive the spliced ​​data so that the processing core processes valid data of the spliced ​​data and determines a data processing result, wherein the valid data is used to represent data corresponding to an input statement in the processing core.

10. An electronic device comprising a memory and a processor, characterized in that: The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 8.

Images