Data processing method and device, electronic equipment, medium and product

Abstract

The present disclosure relates to the technical field of data processing, and discloses a data processing method, device, electronic equipment, medium and product. The present disclosure provides a data processing method, comprising: obtaining a first numerical feature sequence of original data; performing rotation position coding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotation position coding vector of the original data; determining a first model numerical feature sequence corresponding to the target dimension in a target data processing model; determining a target numerical feature vector based on a fusion result of the rotation position coding vector and the first model numerical feature sequence; and processing the target numerical feature vector through the target data processing model to obtain a target result. The ability to capture the relationship between numerical features can be enhanced, thereby helping to enhance the generalization ability of the target data processing model and ensuring the accuracy of model processing.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a data processing method and device, electronic equipment, medium and product. BACKGROUND

[0002] In the related art, when a target data processing model processes original data, the original data is processed based on numerical features corresponding to the original data. However, the numerical features are relatively independent of each other, and thus subsequent data processing using numerical feature data may affect the accuracy of model processing. SUMMARY

[0003] Therefore, the present disclosure provides a data processing method and device, electronic equipment, medium and product to solve the problem of low accuracy of model processing data.

[0004] In a first aspect, the present disclosure provides a data processing method, which comprises:

[0005] obtaining a first numerical feature sequence of original data;

[0006] performing rotation position coding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotation position coding vector of the original data;

[0007] determining a first model numerical feature sequence corresponding to the target dimension in a target data processing model;

[0008] determining a target numerical feature vector based on a fusion result of the rotation position coding vector and the first model numerical feature sequence;

[0009] processing the target numerical feature vector through the target data processing model to obtain a target result.

[0010] In a second aspect, the present disclosure provides a data processing device, which comprises:

[0011] an acquisition module configured to obtain a first numerical feature sequence of original data;

[0012] a first processing module configured to perform rotation position coding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotation position coding vector of the original data;

[0013] a determination module configured to determine a first model numerical feature sequence corresponding to the target dimension in a target data processing model;

[0014] a second processing module configured to determine a target numerical feature vector based on a fusion result of the rotation position coding vector and the first model numerical feature sequence;

[0015] The analysis module is used to analyze the target numerical feature vector through the target data processing model to obtain the target result.

[0016] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the data processing method described in the first aspect or any corresponding embodiment.

[0017] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the data processing method described in the first aspect or any corresponding embodiment thereof.

[0018] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the data processing method described in the first aspect or any corresponding embodiment thereof.

[0019] The data processing method provided in this embodiment can clarify the positional relationship between each first numerical feature by performing rotational position encoding on the first numerical feature sequence. Then, based on the fusion result of the first model numerical feature sequence corresponding to the target dimension in the target data processing model, the target numerical feature vector is determined and input into the target data processing model for processing. This enables the target data processing model to enhance its ability to capture the relationship between each numerical feature during the processing, thereby helping to enhance the generalization ability of the target data processing model and ensuring the accuracy of model processing. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the data processing method provided according to embodiments of this disclosure;

[0022] Figure 2 This is a flowchart illustrating another data processing method provided according to an embodiment of the present disclosure;

[0023] Figure 3 This is a flowchart illustrating yet another data processing method provided according to an embodiment of the present disclosure;

[0024] Figure 4is a flowchart of a self-attention processing method according to an embodiment of the present disclosure;

[0025] Figure 5 is a flowchart of another data processing method according to an embodiment of the present disclosure;

[0026] Figure 6 is a structural block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0027] Figure 7 is a hardware structure diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described below in connection with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, any other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0029] In the related art, when processing original data by a target data processing model, the original data corresponding numerical features are directly processed. However, since each numerical feature is relatively independent, the generalization ability of the target data processing model is relatively weak when processing, thereby affecting the accuracy of the model processing. The processing direction includes but is not limited to data mining, statistical analysis, machine learning, etc.

[0030] In view of this, the embodiments of the present disclosure provide a data processing method, which can enhance the ability of capturing the relationship between each numerical feature in the processing process of the target data processing model, thereby helping to enhance the generalization ability of the target data processing model and ensuring the accuracy of the model processing.

[0031] According to the embodiments of the present disclosure, a data processing method is provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] In the present embodiment, a data processing method is provided, which can be used in the above-mentioned electronic devices such as tablets, computers, etc. Figure 1 is a flowchart of a data processing method according to an embodiment of the present disclosure, as shown in Figure 1 The flowchart includes the following steps:

[0033] Step S101, a first numerical feature sequence of the original data is obtained.

[0034] The original data refers to data that needs to be processed and analyzed, such as text, image, audio, etc. The fields corresponding to the original data include but are not limited to: image processing, natural language processing, financial prediction, life information science, etc.

[0035] The first numerical feature sequence is obtained by feature extraction on the original data, which is used to describe the characteristics and attributes of the original data.

[0036] Step S102, based on the preset target dimension, the first numerical feature sequence is processed by rotating position coding to obtain a rotating position coding vector of the original data.

[0037] The target dimension refers to the dimension that needs to be processed by rotating position coding, such as time dimension, space dimension, etc. The specific dimension number of the target dimension can be determined according to actual needs.

[0038] In order to determine the position relationship between each first numerical feature in the first numerical feature sequence, the first numerical feature sequence is processed by rotating position coding based on the preset target dimension, so as to increase the representation ability and generalization ability of the features by rotating and position coding the first numerical feature sequence, and then achieve the purpose of establishing the relationship between each first numerical feature, thereby helping to enhance the ability of the target data processing model to understand and process the original data.

[0039] Step S103, determine the first model numerical feature sequence corresponding to the target dimension in the target data processing model.

[0040] The target data processing model is a model used to process the original data. The processing direction includes but is not limited to: classification, clustering, regression and dimension reduction, etc. For example: the target data processing model can be a neural network model, a regression analysis model, a learning model, etc.

[0041] In order to facilitate subsequent targeted processing of the original data by the target data processing model, the first model numerical feature sequence corresponding to the target dimension in the target data processing model is determined, so that the ability of the target data processing model can be better utilized in the subsequent process, and the original data can be processed more accurately and effectively.

[0042] Step S104, based on the fusion result of the rotating position coding vector and the first model numerical feature sequence, determine the target numerical feature vector.

[0043] By fusing the rotation position encoding vector and the first model numerical feature sequence, the representation ability and generalization ability of the features can be increased, and then processing and analyzing the obtained fusion result can enable the obtained target numerical feature vector to more comprehensively and comprehensively express the original data from multiple dimensions, so that the target data processing model can more comprehensively understand and analyze the original data when subsequent processing, and the performance and accuracy of the target data processing model can be effectively improved.

[0044] In some examples, the method of fusing the rotation position encoding vector and the first model numerical feature sequence can include but is not limited to vector splicing or vector addition. For example, vector splicing is to connect two vectors together to form a longer vector; vector addition is to add corresponding elements of two vectors to obtain a new vector.

[0045] In step S105, the target numerical feature vector is processed by the target data processing model to obtain a target result.

[0046] The target numerical feature vector is input into the target data processing model, and the target data processing model is used to process the obtained target numerical feature vector, and then a target result corresponding to the original data is obtained to provide support for subsequent decision-making and analysis.

[0047] For example, if the target data processing model is a classification model, the target data processing model classifies the target numerical feature vector, and then obtains a classification result corresponding to the original data. If the target data processing model is a clustering model, the target data processing model clusters the target numerical feature vector, and then obtains a clustering result corresponding to the original data. If the target data processing model is a regression model, the target data processing model regresses the target numerical feature vector, and then obtains a regression result corresponding to the original data.

[0048] The data processing method provided in this embodiment can determine the position relationship between each first numerical feature by performing rotation position encoding processing on the first numerical feature sequence, and then determine the target numerical feature vector based on the fusion result of the first model numerical feature sequence corresponding to the target dimension in the target data processing model, and input it into the target data processing model for processing, which can enhance the ability of the target data processing model to capture the relationship between each numerical feature during processing, thereby helping to enhance the generalization ability of the target data processing model and ensuring the accuracy of model processing.

[0049] In some optional implementation scenarios, taking an image recognition task as an example, the rotation position encoding vector can provide position information of the image, and the first model numerical feature sequence can contain other features of the image, such as color, texture, and the like, and then by fusing the two vectors, the target data processing model can consider the position and other features of the image at the same time in the process of feature processing, thereby effectively improving the accuracy of image recognition.

[0050] A data processing method is provided in the embodiment, which can be used in the electronic device as described above, such as a tablet computer, and the like, Figure 2 is a flowchart of the data processing method according to the embodiment of the present disclosure, as shown in Figure 2 , the flow includes the following steps:

[0051] Step S201, obtaining a first numerical feature sequence of original data. For details, please refer to step S101 of the embodiment shown in Figure 1 , which will not be repeated here.

[0052] Step S202, based on a preset target dimension, performing rotation position encoding processing on the first numerical feature sequence to obtain a rotation position encoding vector of the original data.

[0053] Specifically, the above step S202 includes:

[0054] Step S2021, determining a rotation angle of the first numerical feature under each target dimension to obtain a rotation angle vector corresponding to the first numerical feature.

[0055] In order to facilitate the determination of the rotation angle relationship between the first numerical features, the rotation angle of each first numerical feature under each target dimension is determined, and then the rotation angle relationship between the multiple first numerical features under different target dimensions can be determined, so that the relative position relationship between the numerical features can be described in more detail and accurately, thereby helping to increase the representation ability of the rotation position encoding.

[0056] For example, taking the first numerical feature sequence of the original data = as , W i represents the i-th first numerical feature in the original numerical feature sequence. The formula for calculating the rotation angle vector can be as follows:

[0057] wherein d is the target dimension, and then Θ d / 2-1 is obtained by repeating the elements at each position. d = {θ0, θ0, θ1, θ1, …, θ d / 2-1 , θ d / 2-1}.

[0058] In step S2022, a rotation position encoding vector of the original data is obtained based on the product between each first numerical feature and the corresponding rotation angle vector.

[0059] By multiplying each first numerical feature with the corresponding rotation angle vector, the rotation position encoding vector of the original data can be obtained to represent the rotation of the original data in each target dimension, thereby achieving the purpose of establishing the relationship between each first numerical feature in the original data, and thus helping to increase the representation ability and generalization ability of the original data.

[0060] In some optional embodiments, the step S2022 includes:

[0061] In step a1, an intermediate vector corresponding to the original data is obtained according to the product between each first numerical feature and the corresponding rotation angle vector.

[0062] In step a2, the intermediate vector is subjected to cosine processing to obtain a first rotation position encoding vector.

[0063] In step a3, the intermediate vector is subjected to sine processing to obtain a second rotation position encoding vector, and the rotation position encoding of the original data includes the first rotation position encoding vector and the second rotation position encoding vector.

[0064] Specifically, each first numerical feature is multiplied with the corresponding rotation angle vector to combine the first numerical feature with the corresponding rotation angle, and an intermediate vector is obtained. For example, the expression of the intermediate vector P e R M×d may be P i = X i × Θ d .

[0065] By subjecting the intermediate vector to cosine processing, the vector can be mapped to a cosine space, and a first rotation position encoding vector representing the cosine distribution of the corresponding intermediate vector can be obtained. For example, the calculation formula of the first rotation position encoding vector is P cos = cos(P).

[0066] By subjecting the intermediate vector to sine processing, the vector can be mapped to a sine space, and a second rotation position encoding vector representing the sine distribution of the corresponding intermediate vector can be obtained. For example, the calculation formula of the second rotation position encoding vector is P sin = sin(P).

[0067] By the first rotation position encoding vector and the second rotation position encoding vector, the position of the original data under different rotation angles can be comprehensively expressed, thereby better representing the features and relationships of the data subsequently.

[0068] Step S203, determine the first model numerical feature sequence corresponding to the target dimension in the target data processing model. For details, please refer to Figure 1 Step S103 of the embodiment shown, which will not be repeated here.

[0069] Step S204, determine the target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence. For details, please refer to Figure 1 Step S104 of the embodiment shown, which will not be repeated here.

[0070] Step S205, process the target numerical feature vector through the target data processing model to obtain the target result. For details, please refer to Figure 1 Step S105 of the embodiment shown, which will not be repeated here.

[0071] The data processing method provided in this embodiment further enhances the ability to capture the relationship between numerical features by determining the rotation angle of each first numerical feature under each target dimension and obtaining the rotation position encoding vector of the original data based on the product between each first numerical feature and the corresponding rotation angle vector. Then, based on the rotation position encoding vector obtained after the rotation position encoding processing, the target numerical feature vector is determined, which can provide more abundant information for the target data processing model, enhance the ability of the model to capture the relationship between numerical features, and thus help the target data processing model to better understand and process the original data, ensuring the accuracy of the model processing.

[0072] In this embodiment, a data processing method is provided, which can be used in the electronic device described above, such as a tablet, a computer, etc. Figure 3 is a flowchart of the data processing method according to the embodiments of the present disclosure, which includes the following steps: Figure 3

[0073] Step S301, obtain the first numerical feature sequence of the original data.

[0074] Step S302, perform rotation position encoding processing on the first numerical feature sequence based on the pre-set target dimension to obtain the rotation position encoding vector of the original data.

[0075] Step S303, determine the first model numerical feature sequence corresponding to the target dimension in the target data processing model.

[0076] Step S304, determine the target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence.

[0077] ​Specifically, the first model numerical feature sequence includes a first numerical feature sequence and a second numerical feature sequence. The first numerical feature sequence refers to a group of numerical feature sequences corresponding to the target dimension in the target data processing model. The second numerical feature sequence refers to another group of numerical feature sequences corresponding to the target dimension in the target data processing model. The first numerical feature sequence corresponds to a first numerical position, and the second numerical feature sequence corresponds to a second numerical position. The first numerical position and the second numerical position appear alternately in the first model numerical feature sequence. That is, in the first model numerical feature sequence, if the first numerical position is an even position and the second numerical position is an odd position, the first model numerical feature sequence has a first numerical feature at the first position, the next position of the first numerical feature is a second numerical feature, and the next position of the second numerical feature is a first numerical feature. If the second numerical position is an even position and the first numerical position is an odd position, the first model numerical feature sequence has a second numerical feature at the first position, the next position of the second numerical feature is a first numerical feature, and the next position of the first numerical feature is a second numerical feature.

[0078] In some optional examples, the step S304 includes:

[0079] In step S3041, the first numerical feature sequence is processed to obtain a third numerical feature sequence.

[0080] The reverse processing refers to taking the opposite of each numerical value in a numerical feature sequence, i.e., changing a positive number to a negative number and changing a negative number to a positive number.

[0081] Each numerical value in the first numerical feature sequence is traversed, and for each numerical value, the opposite is taken to obtain a corresponding reverse numerical value. All reverse numerical values are then combined to form a new sequence to obtain the third numerical feature sequence.

[0082] In step S3042, the third numerical feature sequence is replaced with the second numerical feature sequence in the position of the first model numerical feature sequence to obtain a second model numerical feature sequence.

[0083] The third numerical feature sequence is placed in the position originally occupied by the second numerical feature sequence in the first model numerical feature sequence, and the second numerical feature sequence is replaced with the third numerical feature sequence. The purpose of this is to further introduce diversity and changes by changing the arrangement order of the numerical feature sequence to improve the learning ability of the model.

[0084] For example, if the first model numerical feature sequence is E M = {q0, q1, q2, q3, …, q d-2 , q d-1} ∈ R d, the first digit is an odd digit, and the second digit is an even digit. The third digit value feature sequence is obtained by inverting the first digit value feature sequence, and replacing the obtained third digit value feature sequence at the position of the first model value feature sequence with the second digit value feature sequence, so that the second model value feature sequence is:

[0085]

[0086] In step S3043, the first model value feature sequence and the first model value feature sequence fused with the rotation position encoding vector are used to obtain a fusion result.

[0087] The first model value feature sequence is fused with the rotation position encoding vector to combine the features and position information of the original data.

[0088] In some optional embodiments, the rotation position encoding of the original data includes a first rotation position encoding vector and a second rotation position encoding vector, and the step S3043 includes:

[0089] In step b1, the first model value feature sequence is fused with the first rotation position encoding vector to obtain a first fusion sub-result.

[0090] In step b2, the second model value feature sequence is fused with the second rotation position encoding vector to obtain a second fusion sub-result.

[0091] In step b3, the sum of the first fusion sub-result and the second fusion sub-result is taken as the fusion result.

[0092] Specifically, the first model value feature sequence is fused with the first rotation position encoding vector. The fusion can be realized in various ways, such as element-level addition, multiplication or other custom fusion operations. Through fusion, the model value feature and the rotation position encoding information can be combined to obtain a first fusion sub-result containing position information.

[0093] Similarly, the second model value feature sequence is fused with the second rotation position encoding vector to obtain a second fusion sub-result. In this way, the second fusion sub-result also contains the model value feature and the rotation position encoding information.

[0094] The first fusion sub-result and the second fusion sub-result are added to obtain the final fusion result. Through the summation operation, the fusion information of the first model value feature sequence and the second model value feature sequence with the rotation position encoding vector can be considered comprehensively to obtain a comprehensive fusion result, which is helpful for subsequent processing and analysis.

[0095] In some optional implementation scenarios, the expression of the fusion result can be as follows:

[0096] Step S3044: Perform self-attention processing on the fusion result to obtain the target numerical feature vector.

[0097] Self-attention is a mechanism used to capture dependencies between elements in a sequence. By applying self-attention to the fusion result, the numerical features of the original data can be more comprehensively mined, thereby obtaining global information about the original data.

[0098] In some optional implementations, step S3044 above includes:

[0099] Step c1: Determine the query vector, key vector, and value vector corresponding to the fusion result;

[0100] Step c2: Determine the similarity between the query vector and the key vector, and determine the self-attention weight information corresponding to the fusion result based on the similarity.

[0101] Step c3: Based on the self-attention weight information and the value vector, the target numerical feature vector is obtained.

[0102] The query vector is used to retrieve information from the fusion result, the key vector represents key information in the fusion result, and the value vector represents value information in the fusion result. In one example, the query vector, key vector, and value vector corresponding to the fusion result can be obtained by performing linear transformations or other operations on the fusion result.

[0103] The similarity between the query vector and the key vector can be determined by calculating the dot product, cosine similarity, or other similarity metrics between the vectors. Based on the similarity, the self-attention weights corresponding to the fusion result can be determined. These self-attention weights represent the importance of each element in the fusion result; a higher weight indicates a more important element.

[0104] For example: Figure 4 As shown, taking the query vector corresponding to the fusion result as Q, the corresponding key vector as K, and the corresponding value vector as V as an example, the similarity between the query vector and the key vector is A = K. T ·Q, where K T Let Q be the transposed key vector and Q be the query vector. The self-attention weights are the result of normalizing this similarity. For example, the normalization function could be the softmax function. Self-attention weights

[0105] The self-attention weight information is multiplied with the value vector to obtain a target numerical feature that can contain information of important elements in the fusion result, thereby improving the performance and expression ability of the model in subsequent model processing. The expression of the target numerical feature vector A Y can be as follows:

[0106]

[0107] In some optional examples, the step c1 includes:

[0108] Step c11, obtaining a preset query parameter matrix, a key parameter matrix, and a value parameter matrix;

[0109] Step c12, obtaining a query vector according to the product of the query parameter matrix and the fusion result;

[0110] Step c13, obtaining a key vector according to the product of the key parameter matrix and the fusion result;

[0111] Step c14, obtaining a value vector according to the product of the value parameter matrix and the fusion result.

[0112] Specifically, the query parameter matrix, the key parameter matrix, and the value parameter matrix are learned in the model training process. The specific dimensions and numerical values of the query parameter matrix, the key parameter matrix, and the value parameter matrix depend on the actual model structure and requirements, which are not limited here.

[0113] The query parameter matrix corresponding to the fusion result is W Q q ∈R d×d The corresponding key parameter matrix is W K k ∈R d×d And the corresponding value parameter matrix is W V v ∈R d×d For example, the determination expressions of the query vector Q, the key vector K, and the value vector V are as follows:

[0114] Q=W q ·Y M ; K=W k ·Y M ; V=W v ·Y M .

[0115] Through the above steps, the query vector, the key vector, and the value vector can be obtained according to the preset parameter matrix and the fusion result, and then the self-attention weight calculation and the generation of the target numerical feature vector are performed using the above vectors, which helps to improve the performance and accuracy of the model.

[0116] Step S305, processing the target numerical feature vector by the target data processing model to obtain a target result.

[0117] The data processing method provided by the embodiment increases the diversity and variability of data by performing reverse processing and replacement operation on the first numerical feature sequence, and further fuses the rotation position encoding vector to introduce the position information of the original data, which can enhance the understanding and capturing ability of the model for the position relationship between features, and through the self-attention processing mechanism, important features can be adaptively focused and emphasized, so that the obtained target numerical feature vector is more expressive and adaptive, thereby effectively improving the performance and accuracy of the target data processing model.

[0118] As one or more specific application embodiments of the present disclosure, the process of data processing on the original data after obtaining the original data can be as shown in the following: Figure 5 The first numerical feature sequence of the original data is obtained based on a preset target dimension, rotation position encoding processing is performed on the first numerical feature sequence to obtain a rotation position encoding vector of the original data. The first model numerical feature sequence corresponding to the target dimension in the target data processing model is obtained, and the first model numerical feature sequence is fused with the rotation position encoding vector to obtain a fusion result. The fusion result is subjected to self-attention processing to obtain a target numerical feature vector, which is input to the target data processing model for processing, so that the distance information between numerical features can be better preserved and expressed, and the encoding effect of numerical feature vectorization is further improved. Not only can the capturing ability of the model for the relationship between numerical features be enhanced, but also the robustness and generalization ability are better, which is used for more general data analysis and machine learning tasks.

[0119] In the embodiment, a data processing apparatus is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0120] The embodiment provides a data processing apparatus, as shown in the following: Figure 6 The apparatus comprises:

[0121] The acquisition module 601 is configured to acquire a first numerical feature sequence of original data.

[0122] The first processing module 602 is configured to perform rotation position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotation position encoding vector of the original data.

[0123] The determination module 603 is configured to determine a first model numerical feature sequence corresponding to the target dimension in a target data processing model.

[0124] The second processing module 604 is configured to determine a target numerical feature vector based on a fusion result of the rotation position encoding vector and the first model numerical feature sequence.

[0125] The analysis module 605 is configured to analyze the target numerical feature vector by using a target data processing model to obtain a target result.

[0126] In some optional embodiments, the first processing module 602 includes:

[0127] The first determining unit is configured to determine a rotation angle of the first numerical feature in each target dimension to obtain a rotation angle vector corresponding to the first numerical feature.

[0128] The first processing unit is configured to obtain a rotation position encoding vector of the original data based on a product between each first numerical feature and the corresponding rotation angle vector.

[0129] In some optional embodiments, the first processing unit includes:

[0130] The first execution unit is configured to obtain an intermediate vector corresponding to the original data based on a product between each first numerical feature and the corresponding rotation angle vector.

[0131] The second execution unit is configured to perform cosine processing on the intermediate vector to obtain a first rotation position encoding vector.

[0132] The third execution unit is configured to perform sine processing on the intermediate vector to obtain a second rotation position encoding vector, and the rotation position encoding of the original data includes the first rotation position encoding vector and the second rotation position encoding vector.

[0133] In some optional embodiments, the first model numerical feature sequence includes a first digit numerical feature sequence and a second digit numerical feature sequence, and the second processing module 604 includes:

[0134] The second processing unit is configured to perform reverse processing on the first digit numerical feature sequence to obtain a third digit numerical feature sequence.

[0135] The third processing unit is configured to replace the third digit numerical feature sequence at a position of the first model numerical feature sequence with the second digit numerical feature sequence to obtain a second model numerical feature sequence.

[0136] The fusion unit is configured to fuse the first model numerical feature sequence and the first model numerical feature sequence with the rotation position encoding vector to obtain a fusion result.

[0137] The fourth processing unit is configured to perform self-attention processing on the fusion result to obtain a target numerical feature vector.

[0138] In some optional embodiments, the rotation position encoding of the original data includes a first rotation position encoding vector and a second rotation position encoding vector; the fusion unit includes:

[0139] a fourth execution unit configured to fuse the first model numerical feature sequence and the first rotation position encoding vector to obtain a first fusion sub-result;

[0140] a fifth execution unit configured to fuse the second model numerical feature sequence and the second rotation position encoding vector to obtain a second fusion sub-result;

[0141] a sixth execution unit configured to take a sum of the first fusion sub-result and the second fusion sub-result as the fusion result.

[0142] In some optional embodiments, the fourth processing unit includes:

[0143] a second determination unit configured to determine a query vector, a key vector and a value vector corresponding to the fusion result;

[0144] a third determination unit configured to determine a similarity between the query vector and the key vector, and determine self-attention weight information corresponding to the fusion result based on the similarity;

[0145] a seventh execution unit configured to obtain a target numerical feature vector based on the self-attention weight information and the value vector.

[0146] In some optional embodiments, the second determination unit includes:

[0147] an acquisition unit configured to acquire a preset query parameter matrix, a key parameter matrix and a value parameter matrix;

[0148] an eighth execution unit configured to obtain the query vector according to a product of the query parameter matrix and the fusion result;

[0149] a ninth execution unit configured to obtain the key vector according to a product of the key parameter matrix and the fusion result;

[0150] a tenth execution unit configured to obtain the value vector according to a product of the value parameter matrix and the fusion result.

[0151] Further function descriptions of the above-mentioned various modules and units are the same as those of the above-mentioned corresponding embodiments, which will not be described here again.

[0152] The data processing apparatus in the embodiment is presented in the form of functional units. The units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0153] The embodiment of the present disclosure also provides an electronic device having the above Figure 6 data processing apparatus.

[0154] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of an electronic device provided by an optional embodiment of the present disclosure, as Figure 7 shown, the electronic device includes one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common motherboard or in other manners as needed. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device, such as a display device coupled to the interface. In some optional embodiments, multiple processors and / or buses can be used with multiple memories and multiple memories, if necessary. Similarly, multiple electronic devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 The processor 10 in the embodiment is taken as an example.

[0155] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a general array logic, or any combination thereof.

[0156] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0157] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system and applications required for at least one function. The data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one of a magnetic disk storage device, a flash memory device, or other non-transitory solid state storage device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the electronic device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0158] The memory 20 can include a volatile memory such as a random access memory, and can further include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 20 can also include a combination of the above-mentioned types of memory.

[0159] The electronic device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected through a bus or other means, Figure 7 The connection through the bus is taken as an example.

[0160] The input device 30 can receive input digital or character information, and generate key signal input with respect to user settings and function controls of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a haptic feedback device (e.g., a vibration motor), etc. The display device includes, but is not limited to, a liquid crystal display, a light emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0161] The embodiments of the present disclosure further provide a computer readable storage medium, and the method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code, which, when accessed and executed by the computer, processor or hardware, implements the method shown in the above embodiments.

[0162] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, the method and / or technical solutions according to the present application can be called or provided. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file, etc. Correspondingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0163] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scene, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0164] For example, in response to receiving the user's active request, prompt information is sent to the user to explicitly prompt the user that the operation requested to be executed will need to obtain and use the user's personal information. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as electronic device, application program, server or storage medium that executes the operation of the technical solutions of the present disclosure according to the prompt information.

[0165] As an optional but non-limiting implementation, in response to receiving the active request of the user, the manner of sending the prompt information to the user may be, for example, a pop-up window manner, and the prompt information may be presented in the pop-up window in the form of text. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0166] It can be understood that the above notification and user authorization obtaining process is only illustrative and does not limit the implementation of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0167] Although the embodiments of the present disclosure are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present disclosure, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the first numerical feature sequence of the raw data. The raw data refers to the data that needs to be processed and analyzed, including any one of text, images, and audio. Based on a preset target dimension, the first numerical feature sequence is subjected to rotational position encoding processing to obtain the rotational position encoding vector of the original data. Determine the first model numerical feature sequence in the target data processing model that corresponds to the target dimension; Based on the fusion result of the rotation position encoding vector and the numerical feature sequence of the first model, the target numerical feature vector is determined; The target numerical feature vector is processed using the target data processing model to obtain the target result; The step of encoding the first numerical feature sequence based on a preset target dimension number to obtain the rotation position encoding vector of the original data includes: Determine the rotation angle of the first numerical feature in each target dimension to obtain the rotation angle vector corresponding to the first numerical feature; The rotation position encoding vector of the original data is obtained by multiplying each of the first numerical features with the corresponding rotation angle vector.

2. The method according to claim 1, characterized in that, The step of obtaining the rotation position encoding vector of the original data based on the product of each of the first numerical features and the corresponding rotation angle vector includes: Based on the product between each of the first numerical features and the corresponding rotation angle vector, an intermediate vector corresponding to the original data is obtained; The intermediate vector is cosine processed to obtain the first rotation position encoding vector; The intermediate vector is sinusoidally processed to obtain the second rotation position encoding vector. The rotation position encoding of the original data includes the first rotation position encoding vector and the second rotation position encoding vector.

3. The method according to claim 1, characterized in that, The first model numerical feature sequence includes a first digit numerical feature sequence and a second digit numerical feature sequence. The step of determining the target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence includes: The first digit numerical feature sequence is inverted to obtain the third digit numerical feature sequence. The third digit numerical feature sequence is replaced with the second digit numerical feature sequence at the position of the first model numerical feature sequence to obtain the second model numerical feature sequence; The fusion result is obtained by fusing the first model numerical feature sequence with the rotation position encoding vector; The fusion result is subjected to self-attention processing to obtain the target numerical feature vector.

4. The method according to claim 3, characterized in that, The rotation position encoding of the original data includes a first rotation position encoding vector and a second rotation position encoding vector; The fusion result obtained by fusing the first model numerical feature sequence and the first model numerical feature sequence with the rotation position encoding vector includes: The first model numerical feature sequence is fused with the first rotation position encoding vector to obtain the first fusion sub-result; The second model numerical feature sequence is fused with the second rotation position encoding vector to obtain the second fusion sub-result; The sum of the first fusion result and the second fusion result is taken as the fusion result.

5. The method according to claim 3, characterized in that, The step of performing self-attention processing on the fusion result to obtain the target numerical feature vector includes: Determine the query vector, key vector, and value vector corresponding to the fusion result; Determine the similarity between the query vector and the key vector, and determine the self-attention weight information corresponding to the fusion result based on the similarity; Based on the self-attention weight information and the value vector, the target numerical feature vector is obtained.

6. The method according to claim 5, characterized in that, The determination of the query vector, key vector, and value vector corresponding to the fusion result includes: Retrieve the preset query parameter matrix, key parameter matrix, and value parameter matrix; The query vector is obtained by multiplying the query parameter matrix with the fusion result. The key vector is obtained by multiplying the key parameter matrix with the fusion result; The value vector is obtained by multiplying the value parameter matrix with the fusion result.

7. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the first numerical feature sequence of the raw data. The raw data refers to the data that needs to be processed and analyzed, including any one of text, images, and audio. The first processing module is used to perform rotation position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain the rotation position encoding vector of the original data. The determination module is used to determine the first model numerical feature sequence in the target data processing model corresponding to the target dimension; The second processing module is used to determine the target numerical feature vector based on the fusion result of the rotation position encoding vector and the numerical feature sequence of the first model; The analysis module is used to analyze the target numerical feature vector through the target data processing model to obtain the target result; The first processing module includes: The first determining unit is used to determine the rotation angle of the first numerical feature in each target dimension, and to obtain the rotation angle vector corresponding to the first numerical feature. The first processing unit is used to obtain the rotation position encoding vector of the original data based on the product between each of the first numerical features and the corresponding rotation angle vector.

8. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the data processing method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the data processing method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the data processing method according to any one of claims 1 to 6.

Images