Data processing method and device and related equipment

Abstract

The embodiment of the invention discloses a data processing method and device and related equipment, and the method comprises the steps: obtaining to-be-tested data, and carrying out the splitting processing of the to-be-tested data according to a preset splitting rule, and obtaining a plurality of pieces of sub-test data corresponding to the to-be-tested data; determining original index data corresponding to each piece of sub test data according to a preset mapping relationship between characters and index values; performing fusion processing on the original index data corresponding to each piece of sub-test data to obtain fused index data; and performing conversion processing on the fused index data to obtain a feature vector matrix, the feature vector matrix being used as an input of the to-be-optimized model to optimize the to-be-optimized model. In this way, the computing load of the computing device can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus and related equipment. Background Art

[0002] In a deep learning neural network model, all discrete features or non-numerical features of the model's input data need to be converted into unique embedding vectors before they can participate in the model's calculation. Embedding vectors are high-dimensional feature vectors used to represent objects. At present, deep learning neural network models can be used in application scenarios such as data recommendation, data search, and data prediction. Most of the embedding feature vectors in deep learning neural network models are calculated by grouping and calculating the embedding feature vectors corresponding to each discrete feature.

[0003] However, this grouping method of calculating the embedding feature vector corresponding to each discrete feature has a high computational load and low computational efficiency. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus and related equipment, which can reduce the computing load of a computing device.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0006] Acquire the data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested;

[0007] Determine the original index data corresponding to each sub-test data according to the preset mapping relationship between characters and index values;

[0008] Performing fusion processing on the original index data corresponding to each sub-test data to obtain fused index data;

[0009] The fused index data is converted to obtain a feature vector matrix, and the feature vector matrix is ​​used as an input of the model to be optimized to optimize the model to be optimized.

[0010] In a second aspect, an embodiment of the present application provides a text data processing method, comprising:

[0011] Obtaining text data to be processed, and obtaining a feature vector matrix corresponding to the text data to be processed;

[0012] Inputting the feature vector matrix corresponding to the text data to be processed into the optimization model to obtain text response data corresponding to the text data to be processed;

[0013] Outputting the generated text response data so as to display the text response data;

[0014] The optimization model is obtained after optimizing the model to be optimized, and the optimization of the model to be optimized is completed based on the characteristic vector matrix determined for the test data as the input of the model to be optimized.

[0015] In a third aspect, an embodiment of the present application provides a data processing device, including:

[0016] A splitting unit, used for acquiring the data to be tested, and splitting the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested;

[0017] A determination unit, used to determine the original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values;

[0018] A fusion unit, used for fusing the original index data corresponding to each sub-test data to obtain fused index data;

[0019] A processing unit is used to convert the fused index data to obtain a feature vector matrix, and the feature vector matrix is ​​used as an input of the model to be optimized to optimize the model to be optimized.

[0020] In a fourth aspect, an embodiment of the present application provides a text data processing device, including:

[0021] An acquisition unit, used to acquire the text data to be processed and acquire a feature vector matrix corresponding to the text data to be processed;

[0022] A processing unit, used for inputting the feature vector matrix corresponding to the text data to be processed into an optimization model to obtain text response data corresponding to the text data to be processed;

[0023] an output unit, used for outputting the generated text response data so as to display the text response data;

[0024] The optimization model is obtained after optimizing the model to be optimized, and the optimization of the model to be optimized is completed based on the characteristic vector matrix determined for the test data as the input of the model to be optimized.

[0025] In a fifth aspect, an embodiment of the present application provides a computing device, comprising: a processor and a memory, wherein the processor is used to execute the method described in the first aspect or the second aspect above.

[0026] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are executed, the method described in the first aspect or the second aspect above is implemented.

[0027] In a seventh aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0028] The embodiment of the present application can obtain the data to be tested, and split the data to be tested according to the preset splitting rules to obtain multiple sub-test data corresponding to the data to be tested; determine the original index data corresponding to each sub-test data according to the preset mapping relationship between the character and the index value; fuse the original index data corresponding to each sub-test data to obtain the fused index data; convert the fused index data to obtain the feature vector matrix, which is used as the input of the model to be optimized to optimize the model to be optimized. In this way, the computing load of the computing device can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application;

[0031] Figure 2 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0032] Figure 3 It is a schematic diagram of a process of model optimization provided in an embodiment of the present application;

[0033] Figure 4 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0034] Figure 5 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0035] Figure 6 It is a flowchart of a text data processing method provided in an embodiment of the present application;

[0036] Figure 7is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0037] Figure 8 is a structural schematic diagram of a text data processing device provided in an embodiment of the present application;

[0038] Fig. 9 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0041] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0042] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0043] In a specific implementation, the computing device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with a touch surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch surface (e.g., a touch screen display and / or a touch pad).

[0044] The embodiment of the present application provides a data processing method, device and related equipment, and the data processing method can be applied to application scenarios such as text recommendation, text search, video recommendation, and video search. The embodiment of the present application can obtain the data to be tested, and split the data to be tested according to the preset splitting rules to obtain multiple sub-test data corresponding to the data to be tested; determine the original index data corresponding to each sub-test data according to the preset mapping relationship between the character and the index value; fuse the original index data corresponding to each sub-test data to obtain the fused index data; convert the fused index data to obtain the eigenvector matrix, which is used as the input of the model to be optimized to optimize the model to be optimized. By fusing the original index data of each sub-test data of the test data, it is possible to avoid grouping the index data of each sub-test data and avoid frequent switching of different grouping calculations, which helps to reduce the computational load of the computing device in the process of calculating the eigenvector matrix and improve the computing efficiency. The fusion index data obtained by the fusion processing is further converted to obtain a feature vector matrix, which is used as an input of the model to be optimized to optimize the model to be optimized, which helps to improve the reliability, accuracy and efficiency of the optimized model obtained by optimizing the model to be optimized.

[0045] The data processing method provided in the embodiments of the present application can be applied to a data processing device, which can be set in a computing device. In some embodiments, the computing device can include but is not limited to smart phones, tablet computers, laptop computers, desktop computers, vehicle-mounted smart terminals, smart watches and other terminal devices; in other embodiments, the computing device can include but is not limited to independent servers, cluster servers or cloud servers, etc.

[0046] The data processing method provided in the embodiment of the present application is schematically described below with reference to the accompanying drawings.

[0047] Please attend Figure 1 , Figure 1 is a schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application, the application scenario includes a terminal device 11 ( Figure 1 The scenario shown takes the case where the terminal device 11 is a desktop computer as an example).

[0048] In practical applications, the terminal device 11 can obtain the text data to be processed, and obtain the feature vector matrix corresponding to the text data to be processed, and obtain the text response data corresponding to the text data to be processed by inputting the feature vector matrix corresponding to the text data to be processed into the optimization model; and output the generated text response data to facilitate display of the text response data and for the user to view the text response data. Among them, the optimization model is obtained after optimizing the model to be optimized, and the optimization of the model to be optimized is completed by using the feature vector matrix determined for the test data as the input of the model to be optimized.

[0049] In one embodiment, when the terminal device 11 obtains the feature vector matrix corresponding to the text data to be processed, it can split the text data to be processed according to a preset splitting rule to obtain multiple sub-text data corresponding to the text data to be processed; determine the original index data corresponding to each sub-text data according to a preset mapping relationship between characters and index values; fuse the original index data corresponding to each sub-text data to obtain fused index data corresponding to each sub-text data; convert the fused index data corresponding to each sub-text data to obtain the feature vector matrix corresponding to the text data to be processed.

[0050] Furthermore, when determining the characteristic vector matrix of the data to be tested, the terminal device 11 can obtain the data to be tested, and split the data to be tested according to the preset splitting rules to obtain multiple sub-test data corresponding to the data to be tested; determine the original index data corresponding to each sub-test data according to the preset mapping relationship between the character and the index value; fuse the original index data corresponding to each sub-test data to obtain the fused index data; convert the fused index data to obtain the characteristic vector matrix of the data to be tested. The characteristic vector matrix of the data to be tested is used as the input of the model to be optimized to optimize the model to be optimized.

[0051] It should be understood that Figure 1The application scenarios shown are only examples. In actual applications, in addition to the data processing method provided by the embodiment of the present application being independently executed by the terminal device 11, the data processing method provided by the embodiment of the present application can also be independently executed by the server; the data processing method provided by the embodiment of the present application can also be executed by the terminal device and the server in collaboration, for example, the terminal device receives a data processing instruction, the data processing instruction carries the text data to be processed, and the terminal device sends the text data to be processed to the server, so that the server converts the text data to be processed to obtain a feature vector matrix corresponding to the text data to be processed, and inputs the feature vector matrix into the optimization model to obtain text response data corresponding to the text data to be processed, and sends the generated text response data to the terminal device, so that the terminal device displays the text response data on the screen of the terminal device in the text data processing scenario. In some embodiments, the application scenario of the text processing method can be an application scenario such as text recommendation, text search, and text prediction. The application does not specifically limit the application scenario of the data processing method.

[0052] See also Figure 2 , Figure 2 is a flow chart of a data processing method provided in an embodiment of the present application, such as Figure 2 The data processing method in the embodiment shown can be applied to a data processing device, which is arranged in a computing device. The method may at least include the following steps.

[0053] S201: Acquire the data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested.

[0054] In the embodiment of the present application, the computing device can obtain the data to be tested, and split the data to be tested according to the preset splitting rules to obtain multiple sub-test data corresponding to the data to be tested. The data to be tested can include but is not limited to any one or more data such as text, image, video, etc.

[0055] In some embodiments, the preset splitting rule may include but is not limited to splitting the test data according to data type, wherein the data type may include but is not limited to gender, age, location, time, event, etc. For example, assuming that the test data is text data of "Ms. Zhang went on a business trip to Beijing on October 21, 2024", the computing device may split the test data into four sub-test data, namely, "Ms. Zhang", "October 21, 2024", "Beijing" and "business trip" according to the data type.

[0056] S202: Determine original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values.

[0057] In the embodiment of the present application, the computing device can determine the original index data corresponding to each sub-test data according to the mapping relationship between the preset characters and the index values. The characters may include but are not limited to words, letters, etc., and the index values ​​may be any one of 0 and 1. The original index data corresponding to each sub-test data is any one or more integers determined according to 0 and / or 1 in the index value, such as 0, 1, 2, ....n, where n is a positive integer.

[0058] S203: performing fusion processing on the original index data corresponding to each sub-test data to obtain fused index data.

[0059] In the embodiment of the present application, the computing device may perform fusion processing on the original index data corresponding to each sub-test data to obtain fused index data.

[0060] In one embodiment, when the computing device performs fusion processing on the original index data corresponding to each sub-test data to obtain fused index data, it can perform data conversion on the original index data corresponding to each sub-test data according to a first conversion rule to obtain index data corresponding to each original index data; and perform splicing processing on each index data according to a first splicing rule to obtain the fused index data.

[0061] In one embodiment, when the computing device performs data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain the index data corresponding to each original index data, it can select a target value from the original index data corresponding to each sub-test data, wherein the target data includes but is not limited to a maximum value or a minimum value; calculate the product of the kth original index data in each original index data and the target value and k, and determine the product of the kth original index data and the target value and k as the index data corresponding to the kth original index data, wherein k is a positive integer.

[0062] In some optional embodiments, when the computing device performs data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain the index data corresponding to each original index data, the maximum value can be selected from the original index data corresponding to each sub-test data; the product of the kth original index data and the maximum value and k in each original index data is calculated, and the product of the kth original index data and the maximum value and k is determined as the index data corresponding to the kth original index data, where k is a positive integer. Wherein, each original index data and each index data are matrices.

[0063] For example, assuming that there are original index data corresponding to k sub-test data, k is a positive integer, and the maximum value among the k original index data is 80, then the product 80A1 of the first original index data A1 and 80 and 1 can be determined as the index data corresponding to the first original index data, and the product A2*80*2 of the second original index data A2 and 80 and 2, that is, 160A2, can be determined as the index feature data corresponding to the second original index data. Similarly, the kth original index data A k The product of 80 and k is A k *80*k is the index data corresponding to the kth original index data.

[0064] In some optional embodiments, when the computing device performs data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain the index data corresponding to each original index data, it can also select the minimum value from the original index data corresponding to each sub-test data; calculate the product of the kth original index data and the minimum value and k in each original index data, and determine the product of the kth original index data and the minimum value and k as the index data corresponding to the kth original index data, where k is a positive integer. The specific embodiment example is similar to the above example and will not be repeated here.

[0065] In some optional embodiments, the computing device may also calculate the product of the kth original index data in each original index data and the preset weight value and k according to the preset weight value, and determine the product of the kth original index data and the preset weight value and k as the index data corresponding to the kth original index data, where k is a positive integer. The specific embodiment example is similar to the above example and will not be repeated here.

[0066] In other embodiments, the computing device may also use other first conversion rules to perform data conversion on the original index data corresponding to each sub-test data. The embodiment of the present application does not specifically limit the first conversion rule.

[0067] The embodiment of the present application can reduce the situation where different types of sub-test data correspond to the same original index data by converting the original index data corresponding to each sub-test data into corresponding index data, avoid data confusion, and help to better distinguish the index data corresponding to each sub-test data.

[0068] In one embodiment, when the computing device splices the various index data according to the first splicing rule to obtain the fused index data, it can obtain the position arrangement order of the various sub-test data in the data to be tested; according to the position arrangement order of the various sub-test data in the data to be tested, the various index data are spliced ​​in a splicing direction from left to right to obtain the fused index data.

[0069] In an example, assuming that the data to be tested is "Ms. Zhang is on a business trip to Beijing on October 21, 2024", the sub-test data corresponding to the data to be tested are "Ms. Zhang", "October 21, 2024", "Beijing" and "business trip", where the position arrangement order of each sub-test data in the data to be tested is "Ms. Zhang", "October 21, 2024", "Beijing", and "business trip". If the index data corresponding to "Ms. Zhang" is matrix [X], the index data corresponding to "October 21, 2024" is matrix [Y], the index data corresponding to "Beijing" is matrix [Z], and the index data corresponding to "business trip" is matrix [F], then the index data can be spliced ​​from left to right to obtain the fused index data as matrix [XYZF].

[0070] The embodiment of the present application obtains fused index data by splicing the various index data, which can avoid the subsequent calculation of each index data separately to determine the characteristic vector matrix, thereby reducing the amount of calculation, reducing the calculation complexity, and improving the calculation efficiency.

[0071] S204: converting the fused index data to obtain a feature vector matrix, which is used as an input of the model to be optimized to optimize the model to be optimized.

[0072] In an embodiment of the present application, the computing device may convert the fusion index data to obtain a feature vector matrix, which is used as an input of the model to be optimized to optimize the model to be optimized. The feature vector matrix is ​​an embedding feature vector matrix, and the model to be optimized may be a deep neural network (DNN) model.

[0073] In one embodiment, when the computing device converts the fused index data to obtain a feature vector matrix, it can determine the feature vectors corresponding to each index value in the fused index data based on a preset mapping relationship between the index value and the vector, and determine that each feature vector constitutes a first feature matrix corresponding to the fused index data; and convert the first feature matrix according to a second conversion rule to obtain a feature vector matrix.

[0074] The computing device can determine the feature vectors corresponding to each index value in the fused index data from the mapping relationship between the preset index value and the vector (ie, the embedding vector) through the gather function, and determine that each feature vector constitutes a first feature matrix corresponding to the fused index data.

[0075] It should be noted that the method usually used to extract the feature vector of the data to be tested is to split the data to be tested into multiple sub-test data, and use a gather function to extract the feature vector corresponding to each sub-test data, and then fuse the feature vectors extracted by each gather function to obtain the feature vector of the data to be tested. This method requires multiple gather functions to obtain the feature vector of each sub-test data, and fuse the feature vectors, which requires a large amount of data calculation. The embodiment of the present application utilizes the index data of each sub-test data, fuses the index data of each sub-test data, obtains a fused index data, and uses a gather function to extract the first feature matrix corresponding to the fused index data, so that the calculation of determining the first feature matrix occurs within an operator, avoiding the use of multiple gather functions to determine the feature vectors corresponding to each sub-test data, and calculating multiple feature vectors, reducing the amount of data calculation, making the storage of the first feature matrix more compact, improving the accuracy of the calculation, and helping to improve the calculation efficiency of the first feature matrix.

[0076] In one embodiment, when the computing device transforms the first characteristic matrix according to the second transformation rule to obtain the characteristic vector matrix, it can multiply the first characteristic matrix with the first characteristic matrix to obtain the second characteristic matrix; determine the splitting rule corresponding to the first splicing rule, and split the second characteristic matrix according to the splitting rule to obtain multiple third characteristic matrices; and splice each third characteristic matrix according to the second splicing rule to obtain the characteristic vector matrix.

[0077] In some embodiments, the first splicing rule is to splice each index data in a left-to-right splicing direction according to the position arrangement order of each sub-test data in the data to be tested. Then, the splitting rule corresponding to the first splicing rule can be determined as to split the first feature matrix corresponding to the fused index data in a left-to-right splitting direction according to the position arrangement order of each sub-test data in the data to be tested.

[0078] In an example, assuming that the position arrangement order of each sub-test data in the data to be tested is "Ms. Zhang", "October 21, 2024", "Beijing", and "Business trip", if the index data corresponding to "Ms. Zhang" is matrix [X], the index data corresponding to "October 21, 2024" is matrix [Y], the index data corresponding to "Beijing" is matrix [Z], and the index data corresponding to "Business trip" is matrix [F], the index data are spliced ​​from left to right to obtain the fused index data as matrix [XYZF]. If the root According to the mapping relationship between the preset index value and the vector, the first characteristic matrix corresponding to the fused index data [XYZF] is determined to be [X'Y'Z'F']. The computing device can determine that the second characteristic matrix is ​​[X'Y'Z'F']*[X'Y'Z'F'] which is equal to [X"Y"Z"F"]. According to the position arrangement order of each sub-test data in the data to be tested, the second characteristic matrix [X"Y"Z"F"] is split from left to right to obtain four third characteristic matrices, namely [X"], [Y"], [Z"], and [F"].

[0079] The embodiment of the present application reduces the calling overhead of the self-multiplication operator by multiplying the first characteristic matrix by the first characteristic matrix, increases the scale of the entire calculation characteristic vector matrix, and thus improves the CPU utilization efficiency.

[0080] In one embodiment, when the computing device splices each third characteristic matrix according to the second splicing rule to obtain the characteristic vector matrix, the position arrangement order of each third characteristic matrix in the second characteristic matrix can be obtained; and each third characteristic matrix is ​​spliced ​​in a top-down splicing direction according to the position arrangement order of each third characteristic matrix in the second characteristic matrix to obtain the characteristic vector matrix.

[0081] For example, assuming that four third characteristic matrices are determined to be [X”], [Y”], [Z”], and [F”] according to the position arrangement order of each third characteristic matrix in the second characteristic matrix, the computing device can splice each third characteristic matrix in the splicing direction from top to bottom according to the position arrangement order of each third characteristic matrix in the second characteristic matrix to obtain the characteristic vector matrix

[0082] The embodiment of the present application reduces the amount of calculation of the eigenvector matrix, improves the computational efficiency of calculating the eigenvector matrix, reduces the computational load of the computing device, and achieves optimization of the eigenvector matrix by splicing the third eigenmatrices.

[0083] In one embodiment, after the computing device determines the feature vector matrix corresponding to the data to be tested using the fusion index data corresponding to the data to be tested, the determined feature vector matrix corresponding to the data to be tested can be input into the model to be optimized to obtain the response data corresponding to the data to be tested. The computing device can directly obtain the feature vectors corresponding to each sub-test data corresponding to the data to be tested and input them into the model to be optimized to obtain the original response data corresponding to the data to be tested. In some embodiments, the response data and the original response data may include but are not limited to the probability of recommended data, the probability of search data, the probability of predicted data, etc.

[0084] The computing device may compare the response data corresponding to the data to be tested with the original response data. If the comparison result is that the response data corresponding to the data to be tested is better than the original response data, and the loss function value determined according to the comparison result is less than the function threshold, then the model optimization is determined to be successful, and the optimized model is determined to be obtained. If the comparison result is that the response data corresponding to the data to be tested is better than the original response data, but the loss function value determined according to the comparison result is greater than or equal to the function threshold, the model parameters of the model to be optimized may be adjusted according to the loss function value, and the eigenvector matrix corresponding to the data to be tested may be input into the model to be optimized after the model parameters are adjusted. If the new response data obtained is better than the original response data, and the loss function value determined according to the comparison result is less than the function threshold, then the model optimization is determined to be successful, and the optimized model is determined to be obtained.

[0085] When the computing device determines the loss function value of the model based on the comparison result, it can calculate the difference between the response data corresponding to the test data and the original response data, and determine the loss function value between the response data corresponding to the test data and the original response data based on the difference.

[0086] Specific can Figure 3 Take the optimization process of the optimization model to be optimized as an example to illustrate. Figure 3 is a schematic diagram of a model optimization process provided by an embodiment of the present application, such as Figure 3As shown, the test data 31 is input into the feature vector optimization module 32 and the original feature vector extraction module 33 respectively, and the feature vector optimization module 32 splits the test data 31 according to the preset splitting rules to obtain multiple sub-test data corresponding to the test data 31; the original index data corresponding to each sub-test data is determined according to the preset mapping relationship between the character and the index value; the original index data corresponding to each sub-test data is fused to obtain the fused index data; the fused index data is converted to obtain the feature vector matrix 321 corresponding to the test data 31, and the feature vector matrix 321 is input into the model to be optimized 34 to obtain the response data 35. On the other hand, the feature vector matrix 331 corresponding to each sub-test data corresponding to the test data 31 is obtained by the original feature vector extraction module 33, and the feature vector matrix corresponding to each sub-test data is input into the model to be optimized 34 to obtain the original response data 36 corresponding to the test data 31. Further, the optimization model 37 is determined according to the response data 35 and the original response data 36.

[0087] The embodiment of the present application optimizes the model to be optimized by inputting the spliced ​​eigenvector matrix into the model to be optimized, which can avoid the problem of large amount of calculation caused by inputting multiple third eigenmatrices into the model to be optimized for calculation, and helps to reduce the amount of calculation of the model and improve the data processing efficiency of the model.

[0088] In the embodiment of the present application, the computing device obtains the data to be tested, and splits the data to be tested according to the preset splitting rules to obtain multiple sub-test data corresponding to the data to be tested; determines the original index data corresponding to each sub-test data according to the preset mapping relationship between the character and the index value; fuses the original index data corresponding to each sub-test data to obtain the fused index data, which can avoid the subsequent calculation of each index data separately to determine the eigenvector matrix, and can reduce the amount of calculation, reduce the calculation complexity, and improve the calculation efficiency. By converting the fused index data to obtain the eigenvector matrix, the amount of calculation of the eigenvector matrix can be reduced, the calculation efficiency of the eigenvector matrix can be improved, the calculation load of the computing device can be reduced, and the optimization of the eigenvector matrix can be achieved.

[0089] See also Figure 4 , Figure 4 1 is a flow chart of another data processing method provided by an embodiment of the present application. The data processing method of the embodiment of the present application can be executed by a data processing device, wherein the data processing device is set in a computing device, wherein the specific explanation of the computing device is as described above. Specifically, the embodiment of the present application is mainly an explanation of the detailed process of data processing, and the method of the embodiment includes the following steps.

[0090] S401: Acquire the data to be tested, and split the data to be tested according to a preset splitting rule to obtain N sub-test data corresponding to the data to be tested, where N is a positive integer.

[0091] In an embodiment of the present application, the computing device can split the test data according to a preset splitting rule to obtain N sub-test data corresponding to the test data, wherein the N sub-test data can be sub-test data 1, sub-test data 2, ..., sub-test data N.

[0092] S402: determining original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values, and performing data conversion on the original index data corresponding to each sub-test data according to a first conversion rule to obtain index data corresponding to each original index data.

[0093] In the embodiment of the present application, the computing device may determine the original index data corresponding to sub-test data 1 according to the mapping relationship between preset characters and index values, and perform data conversion on the original index data corresponding to sub-test data 1 according to the first conversion rule to obtain index data 1 corresponding to the original index data corresponding to sub-test data 1. Similarly, the computing device may determine the original index data corresponding to sub-test data 2 according to the mapping relationship between preset characters and index values, and perform data conversion on the original index data corresponding to sub-test data 2 according to the first conversion rule to obtain index data 2 corresponding to the original index data corresponding to sub-test data 2. And the computing device may determine the original index data corresponding to sub-test data N according to the mapping relationship between preset characters and index values, and perform data conversion on the original index data corresponding to sub-test data N according to the first conversion rule to obtain index data N corresponding to the original index data corresponding to sub-test data N.

[0094] S403: According to the position arrangement order of each sub-test data in the data to be tested, each index data is spliced ​​in a splicing direction from left to right to obtain fused index data.

[0095] In an embodiment of the present application, the computing device can splice index data 1, index data 2, ..., index data N in the order of position arrangement of sub-test data 1, sub-test data 2, ..., sub-test data N in the data to be tested, and obtain fused index data in a splicing direction from left to right.

[0096] S404: Determine the feature vectors corresponding to the respective index values ​​in the fused index data according to the preset mapping relationship between the index values ​​and the vectors, and determine that the feature vectors constitute a first feature matrix corresponding to the fused index data.

[0097] S405: Perform product processing on the first characteristic matrix and the second characteristic matrix to obtain a second characteristic matrix.

[0098] S406: Determine a splitting rule corresponding to the first concatenation rule, and split the second characteristic matrix according to the splitting rule to obtain a plurality of third characteristic matrices.

[0099] S407: Obtain the position arrangement order of each third characteristic matrix in the second characteristic matrix; and splice each third characteristic matrix in a splicing direction from top to bottom according to the position arrangement order of each third characteristic matrix in the second characteristic matrix to obtain a characteristic vector matrix.

[0100] S408: Input the characteristic vector matrix corresponding to the data to be tested into the optimization model to obtain the response data corresponding to the data to be tested.

[0101] In one example, specifically Figure 5 Take this as an example to illustrate: Figure 5 is a flow chart of another data processing method provided in an embodiment of the present application, such as Figure 5 As shown, the input data Input 51 corresponds to Figure 4The data to be tested in S401, input data 1input1 521, input data 2input2 522, ..., input data NinputN 523 correspond to the N sub-test data obtained by splitting the data to be tested, i.e., input data Input 51, according to the preset splitting rule in S401. Index data 1Add1 531 corresponds to the index data corresponding to input data 1input1 521 determined in S402, index data 2Add2 532 corresponds to the index data corresponding to input data 2input2 522 determined in S402, and index data NAddN 533 corresponds to the index data corresponding to input data NinputN 523 determined in S402. Concatenation processing Concat concatenation direction dim=0 54 is used to indicate that each index data is concatenated according to the concatenation direction indicated by dim=0 to obtain fused index data. Feature screening operation Gather55 corresponds to determining the first feature matrix from the fused index data through feature screening operation Gather55 in S404. The self-multiplication processing Mul56 corresponds to the multiplication of the first characteristic matrix with the first characteristic matrix in S405 to obtain the second characteristic matrix. The splitting processing Split57 corresponds to the splitting processing of the second characteristic matrix according to the splitting rule in S406 to obtain multiple third characteristic matrices. The concatenation processing Concat concatenation direction dim=1 58 is used to indicate the concatenation processing of each third characteristic matrix according to the concatenation direction indicated by dim=1 to obtain the characteristic vector matrix. The deep learning neural network model DNN59 corresponds to the input of the characteristic vector matrix into the DNN model in S408 to obtain the output data Output510, and the output data Output510 corresponds to the response data in S408.

[0102] In an embodiment of the present application, a computing device obtains data to be tested, and splits the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested; determines original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values, and performs data conversion on the original index data corresponding to each sub-test data according to a first conversion rule to obtain index data corresponding to each original index data; and splices each index data from left to right in the order in which each sub-test data is arranged in the data to be tested to obtain fused index data, thereby avoiding subsequent calculations of each index data separately to determine a feature vector matrix, thereby reducing the amount of calculations, reducing the calculation complexity, and improving the calculation efficiency. By determining the eigenvectors corresponding to the respective index values ​​in the fused index data according to the mapping relationship between the preset index values ​​and the vectors, and determining the first eigenmatrix corresponding to the fused index data composed of the eigenvectors, the first eigenmatrix is ​​multiplied with the first eigenmatrix to obtain the second eigenmatrix, and the third eigenmatrix is ​​spliced ​​from top to bottom according to the position arrangement order of the third eigenmatrix in the second eigenmatrix to obtain the eigenvector matrix, the amount of calculation of the eigenvector matrix can be reduced, the calculation efficiency of the eigenvector matrix can be improved, the calculation load of the computing device can be reduced, and the optimization of the eigenvector matrix can be realized. By inputting the eigenvector matrix corresponding to the data to be tested into the optimization model, the response data corresponding to the data to be tested can be obtained, which helps to improve the accuracy of the response data obtained by the optimization model.

[0103] See also Figure 6 , Figure 6 1 is a flowchart of a text data processing method provided by an embodiment of the present application. The text data processing method of the embodiment of the present application can be executed by a text data processing device, wherein the text data processing device is set in a computing device, wherein the specific explanation of the computing device is as described above. Specifically, the method of the embodiment of the present application includes the following steps.

[0104] S601: Obtain text data to be processed, and obtain a feature vector matrix corresponding to the text data to be processed.

[0105] In the embodiment of the present application, the text data processing device can obtain the text data to be processed and obtain the feature vector matrix corresponding to the text data to be processed. The text data to be processed may include but is not limited to any one or more characters such as text, letters, numbers, punctuation marks, etc. The process of obtaining the feature vector matrix corresponding to the text data to be processed is the same as the process of obtaining the feature vector matrix of the data to be tested, which will not be repeated here.

[0106] S602: Inputting the feature vector matrix corresponding to the text data to be processed into the optimization model to obtain text response data corresponding to the text data to be processed.

[0107] In the embodiment of the present application, the text data processing device can input the feature vector matrix corresponding to the text data to be processed into the optimization model to obtain the text response data corresponding to the text data to be processed. The optimization model is obtained after optimizing the model to be optimized. The optimization of the model to be optimized is completed by using the feature vector matrix determined for the data to be tested as the input of the model to be optimized. The specific process of obtaining the optimization model based on the model to be optimized is as follows: Figure 3 The corresponding embodiments are described and will not be repeated here. In some embodiments, the model to be optimized may include but is not limited to a DNN model, and the text response data may be text recommendation data in a text recommendation scenario or text search data in a text search scenario. The application embodiments of the present application do not specifically limit the application scenarios of the text response data.

[0108] S603: Output the generated text response data to facilitate display of the text response data.

[0109] In an embodiment of the present application, the text data processing device can output the text response data obtained by the optimization model to the terminal device, so that the terminal device can display the text response data for the user to view, etc.

[0110] The embodiment of the present application obtains text data to be processed and obtains a feature vector matrix corresponding to the text data to be processed; inputs the feature vector matrix corresponding to the text data to be processed into an optimization model to obtain text response data corresponding to the text data to be processed; and outputs the generated text response data to facilitate display of the text response data, thereby reducing the computing load of the computing device and improving computing efficiency.

[0111] See also Figure 7 , Figure 7 701 is a structural diagram of a data processing device provided in an embodiment of the present application. Specifically, the data processing device is set in a computing device, and the data processing device includes: a splitting unit 701, a determining unit 702, a fusion unit 703 and a processing unit 704;

[0112] The splitting unit 701 is used to obtain the data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested;

[0113] A determination unit 702, configured to determine original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values;

[0114] A fusion unit 703 is used to perform fusion processing on the original index data corresponding to each sub-test data to obtain fused index data;

[0115] The processing unit 704 is used to perform conversion processing on the fused index data to obtain a feature vector matrix, and the feature vector matrix is ​​used as an input of the model to be optimized to optimize the model to be optimized.

[0116] Furthermore, the fusion unit 703 performs fusion processing on the original index data corresponding to each sub-test data to obtain fused index data, which is specifically used for:

[0117] Performing data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain index data corresponding to each original index data;

[0118] The various index data are spliced ​​according to a first splicing rule to obtain the fused index data.

[0119] Further, the fusion unit 703 performs data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain the index data corresponding to each original index feature data, specifically for:

[0120] Selecting a target value from the original index data corresponding to each sub-test data, wherein the target value includes a maximum value or a minimum value;

[0121] The product of the kth original index data among the original index data and the target value plus k is calculated, and the product of the kth original index data and the target value plus k is determined as the index data corresponding to the kth original index data, wherein k is a positive integer.

[0122] Further, the fusion unit 703 performs splicing processing on the respective index data according to the first splicing rule to obtain the fused index data, which is specifically used for:

[0123] Obtaining the position arrangement order of each sub-test data in the data to be tested;

[0124] According to the position arrangement order of each sub-test data in the data to be tested, each index data is spliced ​​in a splicing direction from left to right to obtain the fused index data.

[0125] Furthermore, when the processing unit 704 performs conversion processing on the fused index data to obtain a feature vector matrix, it is specifically used to:

[0126] According to a preset mapping relationship between index values ​​and vectors, a feature vector corresponding to each index value in the fused index data is determined, and each feature vector constitutes a first feature matrix corresponding to the fused index data;

[0127] The first characteristic matrix is ​​transformed according to a second transformation rule to obtain the characteristic vector matrix.

[0128] Furthermore, the processing unit 704 performs a transformation process on the first feature matrix according to a second transformation rule to obtain the feature vector matrix, which is specifically used for:

[0129] Multiplying the first characteristic matrix by the second characteristic matrix to obtain a second characteristic matrix;

[0130] Determine a splitting rule corresponding to the first splicing rule, and split the second characteristic matrix according to the splitting rule to obtain a plurality of third characteristic matrices;

[0131] Each third characteristic matrix is ​​concatenated according to the second concatenation rule to obtain the characteristic vector matrix.

[0132] Further, when the processing unit 704 performs splicing processing on each of the plurality of third characteristic matrices according to the second splicing rule to obtain the characteristic vector matrix, it is specifically used to: obtain the position arrangement order of each of the third characteristic matrices in the second characteristic matrix;

[0133] According to the position arrangement order of the third characteristic matrices in the second characteristic matrix, the third characteristic matrices are spliced ​​from top to bottom to obtain the characteristic vector matrix.

[0134] In an embodiment of the present application, a data processing device can obtain the data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested; determine the original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values; fuse the original index data corresponding to each sub-test data to obtain fused index data; convert the fused index data to obtain a feature vector matrix, which is used as an input of a model to be optimized to optimize the model to be optimized. In this way, the computational load of the computing device in the process of calculating the feature vector matrix corresponding to the data to be tested can be reduced, and the computational efficiency can be improved.

[0135] See also Figure 8 , Figure 8801 is a schematic diagram of the structure of a text data processing device provided in an embodiment of the present application. Specifically, the text data processing device is set in a computing device, and the text data processing device includes: an acquisition unit 801, a processing unit 802, and an output unit 803;

[0136] An acquisition unit 801 is used to acquire text data to be processed and acquire a feature vector matrix corresponding to the text data to be processed;

[0137] A processing unit 802 is used to input the feature vector matrix corresponding to the text data to be processed into an optimization model to obtain text response data corresponding to the text data to be processed;

[0138] An output unit 803, used for outputting the generated text response data, so as to display the text response data;

[0139] The optimization model is obtained after optimizing the model to be optimized, and the optimization of the model to be optimized is completed based on the characteristic vector matrix determined for the test data as the input of the model to be optimized.

[0140] In the embodiment of the present application, the text data processing device can obtain the text data to be processed and obtain the feature vector matrix corresponding to the text data to be processed; input the feature vector matrix corresponding to the text data to be processed into the optimization model to obtain the text response data corresponding to the text data to be processed; and output the generated text response data to facilitate display of the text response data. In this way, the computing load of the computing device can be reduced and the computing efficiency can be improved.

[0141] See also Fig. 9 , Fig. 9 1 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. As shown in the figure, the computing device in this embodiment may include: one or more processors 901, memory 902, bus 903 and communication interface 904, the processor 901, communication interface 904 and memory 902 are connected via bus 903; memory 902 is used to store programs; processor 901 is used to call the program stored in the above memory via bus 903 and perform the following steps:

[0142] Acquire the data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested;

[0143] Determine the original index data corresponding to each sub-test data according to the preset mapping relationship between characters and index values;

[0144] Performing fusion processing on the original index data corresponding to each sub-test data to obtain fused index data;

[0145] The fused index data is converted to obtain a feature vector matrix, and the feature vector matrix is ​​used as an input of the model to be optimized to optimize the model to be optimized.

[0146] Furthermore, the processor 901 performs fusion processing on the original index data corresponding to each sub-test data to obtain fused index data, which is specifically used to:

[0147] Performing data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain index data corresponding to each original index data;

[0148] The various index data are spliced ​​according to a first splicing rule to obtain the fused index data.

[0149] Further, the processor 901 performs data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain the index data corresponding to each original index feature data, specifically for:

[0150] Selecting a target value from the original index data corresponding to each sub-test data, wherein the target value includes a maximum value or a minimum value;

[0151] The product of the kth original index data among the original index data and the target value plus k is calculated, and the product of the kth original index data and the target value plus k is determined as the index data corresponding to the kth original index data, wherein k is a positive integer.

[0152] Further, the processor 901 performs splicing processing on the respective index data according to the first splicing rule to obtain the fused index data, specifically for:

[0153] Obtaining the position arrangement order of each sub-test data in the data to be tested;

[0154] According to the position arrangement order of each sub-test data in the data to be tested, each index data is spliced ​​in a splicing direction from left to right to obtain the fused index data.

[0155] Further, when the processor 901 performs conversion processing on the fusion index data to obtain a feature vector matrix, it is specifically used to:

[0156] According to a preset mapping relationship between index values ​​and vectors, a feature vector corresponding to each index value in the fused index data is determined, and each feature vector constitutes a first feature matrix corresponding to the fused index data;

[0157] The first characteristic matrix is ​​transformed according to a second transformation rule to obtain the characteristic vector matrix.

[0158] Further, the processor 901 performs conversion processing on the first feature matrix according to the second conversion rule to obtain the feature vector matrix, which is specifically used for:

[0159] Multiplying the first characteristic matrix by the second characteristic matrix to obtain a second characteristic matrix;

[0160] Determine a splitting rule corresponding to the first splicing rule, and split the second characteristic matrix according to the splitting rule to obtain a plurality of third characteristic matrices;

[0161] Each third characteristic matrix is ​​concatenated according to the second concatenation rule to obtain the characteristic vector matrix.

[0162] Further, when the processor 901 performs splicing processing on each of the plurality of third characteristic matrices according to the second splicing rule to obtain the characteristic vector matrix, it is specifically used to: obtain the position arrangement order of each of the third characteristic matrices in the second characteristic matrix;

[0163] According to the position arrangement order of the third characteristic matrices in the second characteristic matrix, the third characteristic matrices are spliced ​​from top to bottom to obtain the characteristic vector matrix.

[0164] In an embodiment of the present application, a computing device can obtain data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested; determine the original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values; fuse the original index data corresponding to each sub-test data to obtain fused index data; convert the fused index data to obtain a feature vector matrix, which is used as an input of a model to be optimized to optimize the model to be optimized. In this way, the computing load of the computing device in the process of calculating the feature vector matrix corresponding to the data to be tested can be reduced, and the computing efficiency can be improved.

[0165] It should be understood that in the embodiment of the present application, the processor 901 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0166] The memory 902 may include a read-only memory and a random access memory, and provide instructions and data to the processor 901. A portion of the memory 902 may also include a non-volatile random access memory. For example, the memory 902 may also store information about the device type.

[0167] In a specific implementation, the processor 901 described in the embodiment of the present application can execute the implementation method described in the data processing method or text data processing method provided in the embodiment of the present application, and can also execute the implementation method of the data processing device or text data processing device described in the embodiment of the present application, which will not be repeated here.

[0168] An embodiment of the present application also provides a computer-readable storage medium, in which program instructions are stored, and when the program instructions are executed, any of the data processing methods described above is implemented.

[0169] The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the mobile device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0170] The embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes the methods provided in the above various embodiments.

[0171] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed computing devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0174] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0175] The units in the terminal in the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0176] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0177] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0178] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0179] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: include: Acquire the data to be tested, and split the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested; Determine the original index data corresponding to each sub-test data according to the preset mapping relationship between characters and index values; Performing fusion processing on the original index data corresponding to each sub-test data to obtain fused index data; The fused index data is converted to obtain a feature vector matrix, and the feature vector matrix is ​​used as an input of the model to be optimized to optimize the model to be optimized.

2. The method according to claim 1, characterized in that The fusing the original index data corresponding to each sub-test data to obtain fused index data includes: Performing data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain index data corresponding to each original index data; The various index data are spliced ​​according to a first splicing rule to obtain the fused index data.

3. The method according to claim 2, characterized in that The step of performing data conversion on the original index data corresponding to each sub-test data according to the first conversion rule to obtain index data corresponding to each original index feature data includes: Selecting a target value from the original index data corresponding to each sub-test data, wherein the target data includes a maximum value or a minimum value; The product of the kth original index data among the original index data and the target value plus k is calculated, and the product of the kth original index data and the target value plus k is determined as the index data corresponding to the kth original index data, wherein k is a positive integer.

4. The method according to claim 3, characterized in that The step of performing splicing processing on the respective index data according to the first splicing rule to obtain the fused index data includes: Obtaining the position arrangement order of each sub-test data in the data to be tested; According to the position arrangement order of each sub-test data in the data to be tested, each index data is spliced ​​from left to right to obtain the fused index data.

5. The method according to claim 4, characterized in that The converting process of the fused index data to obtain a feature vector matrix includes: According to a preset mapping relationship between index values ​​and vectors, a feature vector corresponding to each index value in the fused index data is determined, and each feature vector constitutes a first feature matrix corresponding to the fused index data; The first characteristic matrix is ​​transformed according to a second transformation rule to obtain the characteristic vector matrix.

6. The method according to claim 5, characterized in that The converting the first characteristic matrix according to the second conversion rule to obtain the characteristic vector matrix includes: Multiplying the first characteristic matrix by the second characteristic matrix to obtain a second characteristic matrix; Determine a splitting rule corresponding to the first splicing rule, and split the second characteristic matrix according to the splitting rule to obtain a plurality of third characteristic matrices; Each third characteristic matrix is ​​concatenated according to the second concatenation rule to obtain the characteristic vector matrix.

7. The method according to claim 6, characterized in that The step of performing splicing processing on each of the plurality of third characteristic matrices according to the second splicing rule to obtain the characteristic vector matrix includes: Obtaining the position arrangement order of each of the third characteristic matrices in the second characteristic matrix; According to the position arrangement order of the third characteristic matrices in the second characteristic matrix, the third characteristic matrices are spliced ​​from top to bottom to obtain the characteristic vector matrix.

8. A text data processing method, characterized in that: include: Obtaining text data to be processed, and obtaining a feature vector matrix corresponding to the text data to be processed; Inputting the feature vector matrix corresponding to the text data to be processed into the optimization model to obtain text response data corresponding to the text data to be processed; Outputting the generated text response data so as to display the text response data; The optimization model is obtained after optimizing the model to be optimized, and the optimization of the model to be optimized is completed based on the characteristic vector matrix determined for the test data as the input of the model to be optimized.

9. A data processing device, characterized in that: include: A splitting unit, used for acquiring the data to be tested, and splitting the data to be tested according to a preset splitting rule to obtain a plurality of sub-test data corresponding to the data to be tested; A determination unit, used to determine the original index data corresponding to each sub-test data according to a preset mapping relationship between characters and index values; A fusion unit, used for fusing the original index data corresponding to each sub-test data to obtain fused index data; A processing unit is used to convert the fused index data to obtain a feature vector matrix, and the feature vector matrix is ​​used as an input of the model to be optimized to optimize the model to be optimized.

10. A text data processing device, characterized in that: include: An acquisition unit, used to acquire the text data to be processed and acquire a feature vector matrix corresponding to the text data to be processed; A processing unit, used for inputting the feature vector matrix corresponding to the text data to be processed into an optimization model to obtain text response data corresponding to the text data to be processed; an output unit, used for outputting the generated text response data so as to display the text response data; The optimization model is obtained after optimizing the model to be optimized, and the optimization of the model to be optimized is completed based on the characteristic vector matrix determined for the test data as the input of the model to be optimized.

11. A computing device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, and the processor is configured to call the computer program to execute the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, which, when executed, implement the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that The computer program product comprises program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images