Data processing method and device, electronic equipment, medium and product
By rotating position encoding and fusion of the numerical feature sequences in the data processing model, the target numerical feature vector is generated, which solves the problem of low accuracy of the data processing model and improves the generalization ability and processing effect.
Patent Information
- Application Number
- CN202411161929.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-22
AI Technical Summary
When the existing data processing model processes original data, due to the independence of numerical features, the model processing accuracy is low and the generalization ability is weak, which affects the effectiveness of data mining, statistical analysis and machine learning.
By performing rotation position encoding processing on the numerical feature sequence of the original data, integrating the rotation position encoding vector and the model numerical feature sequence, generating the target numerical feature vector, and inputting it into the data processing model for processing, to enhance the relationship between the logarithmic features Capture ability.
It improves the generalization ability and accuracy of the data processing model, enhances the understanding and processing ability of original data, and improves the performance of data analysis and machine learning tasks.
Smart Images

Figure CN119004382A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to data processing methods, devices, electronic devices, media and products. Background Art
[0002] In the related art, when the target data processing model performs model processing on the original data, it processes the original data based on the numerical features corresponding to the original data. However, the numerical features are relatively independent of each other, and the subsequent use of the numerical feature data for data processing may affect the accuracy of the model processing. Summary of the invention
[0003] In view of this, the present disclosure provides a data processing method, device, electronic device, medium and product to solve the problem of low accuracy of model processing data.
[0004] In a first aspect, the present disclosure provides a data processing method, the method comprising:
[0005] Obtaining a first numerical feature sequence of original data;
[0006] Based on a preset target dimension, performing rotational position encoding processing on the first numerical feature sequence to obtain a rotational position encoding vector of the original data;
[0007] Determine a first model numerical feature sequence corresponding to the target dimension in the target data processing model;
[0008] Determine a target numerical feature vector based on a fusion result of the rotation position encoding vector and the numerical feature sequence of the first model;
[0009] The target numerical feature vector is processed through the target data processing model to obtain the target result.
[0010] In a second aspect, the present disclosure provides a data processing device, the device comprising:
[0011] An acquisition module, used for acquiring a first numerical feature sequence of original data;
[0012] A first processing module, configured to perform rotational position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotational position encoding vector of the original data;
[0013] A determination module, used to determine a first model numerical feature sequence corresponding to a target dimension in a target data processing model;
[0014] A second processing module is used to determine a target numerical feature vector based on a fusion result of the rotation position encoding vector and the first model numerical feature sequence;
[0015] The analysis module is used to analyze the target numerical feature vector through the target data processing model to obtain the target result.
[0016] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0017] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data processing method of the first aspect or any corresponding embodiment thereof.
[0018] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data processing method of the first aspect or any corresponding embodiment thereof.
[0019] The data processing method provided in this embodiment can clarify the positional relationship between each first numerical feature by performing rotation position encoding processing on the first numerical feature sequence, and then determine the target numerical feature vector based on the fusion result of the first model numerical feature sequence corresponding to the target dimension in the target data processing model, and input it into the target data processing model for processing. During the processing, the target data processing model can enhance its ability to capture the relationship between each numerical feature, thereby helping to enhance the generalization ability of the target data processing model and ensure the accuracy of model processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 is a flowchart of a data processing method provided according to an embodiment of the present disclosure;
[0022] Figure 2 is a flowchart of another data processing method provided according to an embodiment of the present disclosure;
[0023] Figure 3 is a flowchart of another data processing method provided according to an embodiment of the present disclosure;
[0024] Figure 4is a flowchart of a self-attention processing method provided according to an embodiment of the present disclosure;
[0025] Figure 5 is a flowchart of another data processing method provided according to an embodiment of the present disclosure;
[0026] Figure 6 is a structural block diagram of a data processing device provided according to an embodiment of the present disclosure;
[0027] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0029] In the related art, when the target data processing model processes the original data, the numerical features corresponding to the original data are directly used for processing. However, since the numerical features are relatively independent, the generalization ability of the target data processing model is relatively weak, thus affecting the accuracy of the model processing. The processing directions include but are not limited to: data mining, statistical analysis, machine learning, etc.
[0030] In view of this, the embodiments of the present disclosure provide a data processing method, which can enhance the ability of the target data processing model to capture the relationship between various numerical features during the processing process, thereby helping to enhance the generalization ability of the target data processing model and ensure the accuracy of model processing.
[0031] According to an embodiment of the present disclosure, a data processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0032] In this embodiment, a data processing method is provided, which can be used in the above-mentioned electronic devices, such as tablets, computers, etc. Figure 1 is a flow chart of a data processing method according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:
[0033] Step S101, obtaining a first numerical feature sequence of original data.
[0034] Raw data refers to data that needs to be processed and analyzed, such as text, images, audio, etc. The fields corresponding to raw data include but are not limited to: image processing, natural language processing, financial forecasting, life informatics, etc.
[0035] The first numerical feature sequence is obtained by extracting features from the original data and is used to describe the features and attributes of the original data.
[0036] Step S102: performing rotational position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotational position encoding vector of the original data.
[0037] The target dimension refers to the dimension that needs to be processed by rotational position encoding, such as the time dimension, the space dimension, etc. The specific number of dimensions of the target dimension can be determined according to actual needs.
[0038] In order to determine the positional relationship between each first numerical feature in the first numerical feature sequence, the first numerical feature sequence is rotated and position encoded based on the preset target dimension. By rotating and position encoding the first numerical feature sequence, the representation ability and generalization ability of the features are increased, thereby achieving the purpose of establishing the connection between each first numerical feature, thereby helping to enhance the target data processing model's ability to understand and process the original data.
[0039] Step S103, determining a first model numerical feature sequence corresponding to the target dimension in the target data processing model.
[0040] The target data processing model is a model used to process raw data. The processing directions include but are not limited to: classification, clustering, regression and dimensionality reduction, etc. For example: the target data processing model can be a neural network model, a regression analysis model, a learning model, etc.
[0041] In order to enable the target data processing model to perform targeted processing on the original data in the future, the first model numerical feature sequence corresponding to the target dimension in the target data processing model is determined, so that the capabilities of the target data processing model can be better utilized in the future to facilitate more accurate and effective processing of the original data.
[0042] Step S104, determining a target numerical feature vector based on a fusion result of the rotation position encoding vector and the first model numerical feature sequence.
[0043] By fusing the rotation position encoding vector with the first model numerical feature sequence, the representation and generalization capabilities of the features can be increased, and then the fusion results can be processed and analyzed, so that the target numerical feature vector can express the original data more comprehensively and comprehensively from a multi-dimensional perspective, so that in subsequent processing, the target data processing model can understand and analyze the original data more comprehensively, which can effectively improve the performance and accuracy of the target data processing model.
[0044] In some examples, the method of fusing the rotation position encoding vector with the first model numerical feature sequence may include but is not limited to: vector concatenation or vector addition. For example, vector concatenation is to connect two vectors together to form a longer vector; vector addition is to add corresponding elements of two vectors to obtain a new vector.
[0045] Step S105, processing the target numerical feature vector through the target data processing model to obtain the target result.
[0046] The target numerical feature vector is input into the target data processing model, and the target numerical feature vector is processed by the target data processing model to obtain the target result corresponding to the original data to provide support for subsequent decision-making and analysis.
[0047] For example, if the target data processing model is a classification model, the target data processing model performs classification processing on the target numerical feature vector, and then obtains a classification result corresponding to the original data. If the target data processing model is a clustering model, the target data processing model performs clustering processing on the target numerical feature vector, and then obtains a clustering result corresponding to the original data. If the target data processing model is a regression model, the target data processing model performs regression processing on the target numerical feature vector, and then obtains a regression result corresponding to the original data.
[0048] The data processing method provided in this embodiment can clarify the positional relationship between each first numerical feature by performing rotation position encoding processing on the first numerical feature sequence, and then determine the target numerical feature vector based on the fusion result of the first model numerical feature sequence corresponding to the target dimension in the target data processing model, and input it into the target data processing model for processing. During the processing, the target data processing model can enhance its ability to capture the relationship between each numerical feature, thereby helping to enhance the generalization ability of the target data processing model and ensure the accuracy of model processing.
[0049] In some optional implementation scenarios, taking image recognition tasks as an example, the rotation position encoding vector can provide the position information of the image, and the first model numerical feature sequence can include other features of the image, such as color, texture, etc., and then by fusing these two vectors, the target data processing model can simultaneously consider the position and other features of the image during the feature processing process, thereby effectively improving the accuracy of image recognition.
[0050] In this embodiment, a data processing method is provided, which can be used in the above-mentioned electronic devices, such as tablets, computers, etc. Figure 2 is a flow chart of a data processing method according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:
[0051] Step S201, obtain the first numerical feature sequence of the original data. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.
[0052] Step S202: performing rotational position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotational position encoding vector of the original data.
[0053] Specifically, the above step S202 includes:
[0054] Step S2021, determining the rotation angle of the first numerical feature in each target dimension, and obtaining the rotation angle vector corresponding to the first numerical feature.
[0055] In order to facilitate the determination of the rotation angle relationship between the first numerical features, the rotation angle of each first numerical feature in each target dimension is determined respectively, and then the rotation angle relationship between multiple first numerical features in different target dimensions can be clarified, so that the relative position relationship between the numerical features can be described in more detail and accurately, thereby helping to increase the representation capability of rotational position encoding.
[0056] For example: Taking the first numerical feature sequence of the original data as For example, W i represents the first numerical feature of the i-th in the original numerical feature sequence. The formula for calculating the rotation angle vector can be as follows:
[0057] in d is the target dimension, and then Θ d / 2-1 The elements at each position are repeated, and we get Θ d ={θ0,θ0,θ1,θ1,…,θ d / 2-1 ,θ d / 2-1}.
[0058] Step S2022: obtaining a rotation position encoding vector of the original data based on the product between each first numerical feature and the corresponding rotation angle vector.
[0059] By multiplying each first numerical feature with the corresponding rotation angle vector, the rotation position encoding vector of the original data can be obtained to represent the rotation of the original data in each target dimension, thereby achieving the purpose of establishing a connection between each first numerical feature in the original data, thereby helping to increase the representation and generalization capabilities of the original data.
[0060] In some optional implementations, the above step S2022 includes:
[0061] Step a1, obtaining an intermediate vector corresponding to the original data according to the product between each first numerical feature and the corresponding rotation angle vector;
[0062] Step a2, performing cosine processing on the intermediate vector to obtain a first rotation position encoding vector;
[0063] Step a3, performing sinusoidal processing on the intermediate vector to obtain a second rotational position encoding vector. The rotational position encoding of the original data includes the first rotational position encoding vector and the second rotational position encoding vector.
[0064] Specifically, each first numerical feature is multiplied by the corresponding rotation angle vector to combine the first numerical feature with the corresponding rotation angle to obtain an intermediate vector. For example: the intermediate vector P∈R M×d The expression of P can be: i =X i ×Θ d .
[0065] By performing cosine processing on the intermediate vector, the vector can be mapped into a cosine space, thereby obtaining a first rotation position encoding vector that can represent the cosine distribution of the corresponding intermediate vector. For example, the calculation formula of the first rotation position encoding vector is: cos =cos(P).
[0066] By performing sinusoidal processing on the intermediate vector, the vector can be mapped into a sinusoidal space, thereby obtaining a second rotation position encoding vector that can represent the sinusoidal distribution of the corresponding intermediate vector. For example, the calculation formula of the second rotation position encoding vector is: sin =sin(P).
[0067] The first rotation position encoding vector and the second rotation position encoding vector can fully express the position of the original data at different rotation angles, so that the characteristics and relationships of the data can be better represented later.
[0068] Step S203: Determine the first model numerical feature sequence corresponding to the target dimension in the target data processing model. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.
[0069] Step S204: Determine the target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.
[0070] Step S205: Process the target numerical feature vector through the target data processing model to obtain the target result. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.
[0071] The data processing method provided in this embodiment further enhances the ability to capture the relationship between each numerical feature by determining the rotation angle of the first numerical feature in each target dimension and obtaining the rotation position encoding vector of the original data based on the product of each first numerical feature and the corresponding rotation angle vector. Then, based on the rotation position encoding vector obtained after the rotation position encoding process, the target numerical feature vector is determined, which can provide richer information for the target data processing model and enhance the model's ability to capture the relationship between numerical features, thereby helping the target data processing model to better understand and process the original data and ensure the accuracy of the model processing.
[0072] In this embodiment, a data processing method is provided, which can be used in the above-mentioned electronic devices, such as tablets, computers, etc. Figure 3 is a flow chart of a data processing method according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:
[0073] Step S301, obtaining a first numerical feature sequence of original data.
[0074] Step S302: performing rotational position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotational position encoding vector of the original data.
[0075] Step S303, determining a first model numerical feature sequence corresponding to the target dimension in the target data processing model.
[0076] Step S304: determine a target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence.
[0077] Specifically, the first model numerical feature sequence includes a first digit numerical feature sequence and a second digit numerical feature sequence. The first digit numerical feature sequence refers to a set of numerical feature sequences corresponding to the target dimension in the target data processing model. The second digit numerical feature sequence refers to another set of numerical feature sequences corresponding to the target dimension in the target data processing model. The first digit corresponding to the first digit numerical feature sequence and the second digit corresponding to the second digit numerical feature sequence appear alternately in the first model numerical feature sequence. That is, in the first model numerical feature sequence, if the first digit is an even digit and the second digit is an odd digit, the first model numerical feature of the first model numerical feature sequence is the first digit numerical feature, the next digit of the first digit numerical feature is the second digit numerical feature, and the next digit of the second digit numerical feature is the first digit numerical feature. If the second digit is an even digit and the first digit is an odd digit, the first model numerical feature of the first model numerical feature sequence is the second digit numerical feature, the next digit of the second digit numerical feature is the first digit numerical feature, and the next digit of the first digit numerical feature is the second digit numerical feature.
[0078] In some optional examples, the above step S304 includes:
[0079] Step S3041, inverting the first digit numerical feature sequence to obtain a third digit numerical feature sequence.
[0080] Inversion processing refers to taking the opposite of each value in a numerical feature sequence, that is, changing positive numbers into negative numbers and negative numbers into positive numbers.
[0081] Each value in the first digital numerical feature sequence is traversed, and for each value, the opposite number is taken to obtain the corresponding inverted value, and then all the inverted values are combined into a new sequence to obtain the third digital numerical feature sequence.
[0082] Step S3042: Replace the third digit numerical feature sequence with the second digit numerical feature sequence at the position of the first model numerical feature sequence to obtain a second model numerical feature sequence.
[0083] The third digit numerical feature sequence is placed in the position of the original second digit numerical feature sequence in the first model numerical feature sequence, and the second digit numerical feature sequence is replaced by the third digit numerical feature sequence. The purpose of this is to further introduce diversity and changes by changing the arrangement order of the numerical feature sequence to improve the learning ability of the model.
[0084] For example: If the numerical feature sequence of the first model is E M ={q0,q1,q2,q3,…,q d-2 ,q d-1}∈R d, the first digit is an odd digit, the second digit is an even digit, and the first digit numerical feature sequence is inverted, and the obtained third digit numerical feature sequence is replaced with the second digit numerical feature sequence in the position of the first model numerical feature sequence, so that the second model numerical feature sequence is obtained as follows:
[0085]
[0086] Step S3043, obtaining a fusion result based on the first model numerical feature sequence and the fusion of the first model numerical feature sequence and the rotation position encoding vector.
[0087] The first model numerical feature sequence is fused with the rotation position encoding vector to combine the features and position information of the original data.
[0088] In some optional implementations, the rotational position encoding of the original data includes a first rotational position encoding vector and a second rotational position encoding vector, and the above step S3043 includes:
[0089] Step b1, fusing the first model numerical feature sequence with the first rotation position encoding vector to obtain a first fusion sub-result;
[0090] Step b2, fusing the second model numerical feature sequence with the second rotation position encoding vector to obtain a second fusion sub-result;
[0091] Step b3: taking the sum of the first fusion sub-result and the second fusion sub-result as the fusion result.
[0092] Specifically, the first model numerical feature sequence is fused with the first rotation position encoding vector. Fusion can be implemented in a variety of ways, such as element-level addition, multiplication or other custom fusion operations. Through fusion, the model numerical features can be combined with the rotation position encoding information to obtain a first fusion sub-result containing position information.
[0093] Similarly, the second model numerical feature sequence is fused with the second rotation position encoding vector to obtain a second fused sub-result. In this way, the second fused sub-result also contains information of the model numerical features and the rotation position encoding.
[0094] The first fusion sub-result and the second fusion sub-result are added to obtain the final fusion result. Through the summation operation, the fusion information of the first model numerical feature sequence and the second model numerical feature sequence and the rotation position encoding vector can be comprehensively considered to obtain a comprehensive fusion result, which is helpful for subsequent processing and analysis.
[0095] In some optional implementation scenarios, the expression of the fusion result can be as follows:
[0096] Step S3044, perform self-attention processing on the fusion result to obtain the target numerical feature vector.
[0097] Self-attention processing is a mechanism for capturing dependencies between elements in a sequence. By performing self-attention processing on the fusion result, the numerical features of the original data can be more comprehensively mined, thereby obtaining the global information of the original data.
[0098] In some optional implementations, the above step S3044 includes:
[0099] Step c1, determining the query vector, key vector and value vector corresponding to the fusion result;
[0100] Step c2, determining the similarity between the query vector and the key vector, and determining the self-attention weight information corresponding to the fusion result based on the similarity;
[0101] Step c3, based on the self-attention weight information and the value vector, obtain the target numerical feature vector.
[0102] The query vector is used to query the information in the fusion result, the key vector is used to represent the key information in the fusion result, and the value vector is used to represent the value information in the fusion result. In one example, the query vector, key vector, and value vector corresponding to the fusion result can be obtained by performing linear transformation or other operations on the fusion result.
[0103] The similarity between the query vector and the key vector can be determined by calculating the dot product, cosine similarity or other similarity metrics between the vectors. Based on the similarity, the self-attention weight information corresponding to the fusion result can be determined. Among them, the self-attention weight information represents the importance of each element in the fusion result. The higher the weight, the more important the element.
[0104] For example: Figure 4 As shown, the query vector corresponding to the fusion result is Q, the corresponding key vector is K, and the corresponding value vector is V. The similarity between the query vector and the key vector is A=K T Q, where K T is the transposed key vector, Q is the query vector. The self-attention weight information is the result of normalizing the similarity. For example, the normalization function can be a softmax function.
[0105] By weighted multiplication of the self-attention weight information and the value vector, the target numerical feature can contain the information of important elements in the fusion result, which can improve the performance and expression ability of the model in subsequent model processing. Among them, the expression for determining the target numerical feature vector AY can be as follows:
[0106]
[0107] In some optional examples, the above step c1 includes:
[0108] Step c11, obtaining a preset query parameter matrix, a key parameter matrix, and a value parameter matrix;
[0109] Step c12, obtaining a query vector according to the product of the query parameter matrix and the fusion result;
[0110] Step c13, obtaining a key vector according to the product of the key parameter matrix and the fusion result;
[0111] Step c14, obtaining a value vector according to the product of the value parameter matrix and the fusion result.
[0112] Specifically, the query parameter matrix, key parameter matrix, and value parameter matrix are learned during the model training process. The specific dimensions and values of the query parameter matrix, key parameter matrix, and value parameter matrix depend on the actual model structure and requirements, and are not limited here.
[0113] The query parameter matrix corresponding to the fusion result is W q ∈R d×d , the corresponding key parameter matrix is W k ∈R d×d And the corresponding value parameter matrix is W v ∈R d×d For example, the expressions for determining the query vector Q, the key vector K, and the value vector V are as follows:
[0114] Q=W q ·Y M ; K = W k ·Y M ; V = W v ·Y M .
[0115] Through the above steps, we can obtain the query vector, key vector and value vector according to the preset parameter matrix and fusion results, and then use the above vectors to calculate the self-attention weight and generate the target numerical feature vector, which helps to improve the performance and accuracy of the model.
[0116] Step S305, processing the target numerical feature vector through the target data processing model to obtain the target result.
[0117] The data processing method provided in this embodiment increases the diversity and variability of the data by inverting and replacing the first digit numerical feature sequence, and then fuses the rotated position encoding vector to introduce the position information of the original data, which can enhance the model's ability to understand and capture the positional relationship between features, and adaptively focus on and emphasize important features through the self-attention processing mechanism, which can make the obtained target numerical feature vector more expressive and adaptable, thereby effectively improving the performance and accuracy of the target data processing model.
[0118] As one or more specific application embodiments of the embodiments of the present disclosure, after the original data is obtained, the process of processing the original data can be as follows: Figure 5 As shown: the first numerical feature sequence of the original data is obtained based on the preset target dimension, and the first numerical feature sequence is subjected to rotational position encoding processing to obtain the rotational position encoding vector of the original data. The first model numerical feature sequence corresponding to the target dimension in the target data processing model is obtained, and the first model numerical feature sequence is fused with the rotational position encoding vector to obtain a fusion result. The fusion result is subjected to self-attention processing to obtain a target numerical feature vector, which is input into the target data processing model for processing, so as to better retain and express the distance information between numerical features, and further improve the encoding effect of numerical feature vectorization. It can not only enhance the model's ability to capture the relationship between numerical features, but also has greater robustness and generalization ability, and is used for more extensive data analysis and machine learning tasks.
[0119] In the present embodiment, a data processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0120] This embodiment provides a data processing device, such as Figure 6 As shown, including:
[0121] An acquisition module 601 is used to acquire a first numerical feature sequence of original data;
[0122] The first processing module 602 is used to perform rotation position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotation position encoding vector of the original data;
[0123] A determination module 603 is used to determine a first model numerical feature sequence corresponding to a target dimension in a target data processing model;
[0124] The second processing module 604 is used to determine the target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence;
[0125] The analysis module 605 is used to analyze the target numerical feature vector through the target data processing model to obtain the target result.
[0126] In some optional implementations, the first processing module 602 includes:
[0127] A first determining unit, used to determine a rotation angle of the first numerical feature in each target dimension, and obtain a rotation angle vector corresponding to the first numerical feature;
[0128] The first processing unit is used to obtain the rotation position encoding vector of the original data based on the product between each first numerical feature and the corresponding rotation angle vector.
[0129] In some optional implementations, the first processing unit includes:
[0130] A first execution unit, configured to obtain an intermediate vector corresponding to the original data according to the product between each first numerical feature and the corresponding rotation angle vector;
[0131] A second execution unit is used to perform cosine processing on the intermediate vector to obtain a first rotation position encoding vector;
[0132] The third execution unit is used to perform sinusoidal processing on the intermediate vector to obtain a second rotational position encoding vector, and the rotational position encoding of the original data includes the first rotational position encoding vector and the second rotational position encoding vector.
[0133] In some optional implementations, the first model numerical feature sequence includes a first digit numerical feature sequence and a second digit numerical feature sequence, and the second processing module 604 includes:
[0134] A second processing unit is used to perform inversion processing on the first digit numerical feature sequence to obtain a third digit numerical feature sequence;
[0135] A third processing unit is used to replace the third digit numerical feature sequence with the second digit numerical feature sequence at the position of the first model numerical feature sequence to obtain a second model numerical feature sequence;
[0136] A fusion unit, used for fusing the first model numerical feature sequence and the first model numerical feature sequence with the rotation position encoding vector to obtain a fusion result;
[0137] The fourth processing unit is used to perform self-attention processing on the fusion result to obtain a target numerical feature vector.
[0138] In some optional implementations, the rotational position encoding of the original data includes a first rotational position encoding vector and a second rotational position encoding vector; and the fusion unit includes:
[0139] A fourth execution unit, used for fusing the first model numerical feature sequence with the first rotation position encoding vector to obtain a first fusion sub-result;
[0140] A fifth execution unit, configured to fuse the second model numerical feature sequence with the second rotation position encoding vector to obtain a second fusion sub-result;
[0141] The sixth execution unit is used to take the sum of the first fusion sub-result and the second fusion sub-result as the fusion result.
[0142] In some optional implementations, the fourth processing unit includes:
[0143] A second determining unit, configured to determine a query vector, a key vector, and a value vector corresponding to the fusion result;
[0144] A third determining unit is used to determine the similarity between the query vector and the key vector, and determine the self-attention weight information corresponding to the fusion result based on the similarity;
[0145] The seventh execution unit is used to obtain a target numerical feature vector based on the self-attention weight information and the value vector.
[0146] In some optional implementations, the second determining unit includes:
[0147] An acquisition unit, used to acquire a preset query parameter matrix, a key parameter matrix, and a value parameter matrix;
[0148] an eighth execution unit, configured to obtain a query vector according to the product of the query parameter matrix and the fusion result;
[0149] A ninth execution unit, configured to obtain a key vector according to the product of the key parameter matrix and the fusion result;
[0150] The tenth execution unit is used to obtain a value vector according to the product of the value parameter matrix and the fusion result.
[0151] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0152] The data processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0153] The present disclosure also provides an electronic device having the above Figure 6 The data processing device shown.
[0154] See also Figure 7 , Figure 7 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present disclosure, such as Figure 7 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.
[0155] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0156] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0157] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0158] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0159] The electronic device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.
[0160] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.
[0161] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0162] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.
[0163] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0164] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0165] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0166] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0167] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A data processing method, characterized in that: The method comprises: Obtaining a first numerical feature sequence of original data; Based on a preset target dimension, performing rotational position encoding processing on the first numerical feature sequence to obtain a rotational position encoding vector of the original data; Determining a first model numerical feature sequence corresponding to the target dimension in the target data processing model; Determine a target numerical feature vector based on a fusion result of the rotation position encoding vector and the first model numerical feature sequence; The target numerical feature vector is processed by the target data processing model to obtain a target result.
2. The method according to claim 1, characterized in that The encoding process of the first numerical feature sequence based on the preset target dimension number to obtain the rotation position encoding vector of the original data includes: Determine a rotation angle of the first numerical feature in each target dimension, and obtain a rotation angle vector corresponding to the first numerical feature; Based on the product between each of the first numerical features and the corresponding rotation angle vector, a rotation position encoding vector of the original data is obtained.
3. The method according to claim 2, characterized in that The step of obtaining the rotation position encoding vector of the original data based on the product between each of the first numerical features and the corresponding rotation angle vector comprises: Obtaining an intermediate vector corresponding to the original data according to the product between each of the first numerical features and the corresponding rotation angle vector; Performing cosine processing on the intermediate vector to obtain a first rotation position encoding vector; The intermediate vector is subjected to sinusoidal processing to obtain a second rotational position encoding vector, and the rotational position encoding of the original data includes the first rotational position encoding vector and the second rotational position encoding vector.
4. The method according to claim 1, characterized in that: The first model numerical feature sequence includes a first digit numerical feature sequence and a second digit numerical feature sequence, and the determining of the target numerical feature vector based on the fusion result of the rotation position encoding vector and the first model numerical feature sequence includes: Performing inversion processing on the first digit numerical characteristic sequence to obtain a third digit numerical characteristic sequence; Replace the position of the third digital numerical feature sequence in the first model numerical feature sequence with the second digital numerical feature sequence to obtain a second model numerical feature sequence; Obtaining a fusion result based on the first model numerical feature sequence and the fusion of the first model numerical feature sequence and the rotation position encoding vector; Perform self-attention processing on the fusion result to obtain the target numerical feature vector.
5. The method according to claim 4, characterized in that The rotational position encoding of the original data includes a first rotational position encoding vector and a second rotational position encoding vector; The step of obtaining a fusion result based on the first model numerical feature sequence and fusing the first model numerical feature sequence with the rotation position encoding vector includes: Fusing the first model numerical feature sequence with the first rotation position encoding vector to obtain a first fused sub-result; Fusing the second model numerical feature sequence with the second rotation position encoding vector to obtain a second fused sub-result; The sum of the first fusion sub-result and the second fusion sub-result is taken as the fusion result.
6. The method according to claim 4, characterized in that The performing self-attention processing on the fusion result to obtain the target numerical feature vector includes: Determining a query vector, a key vector, and a value vector corresponding to the fusion result; Determine the similarity between the query vector and the key vector, and determine the self-attention weight information corresponding to the fusion result based on the similarity; Based on the self-attention weight information and the value vector, the target numerical feature vector is obtained.
7. The method according to claim 6, characterized in that The determining of the query vector, the key vector, and the value vector corresponding to the fusion result includes: Get the preset query parameter matrix, key parameter matrix, and value parameter matrix; Obtaining the query vector according to the product of the query parameter matrix and the fusion result; Obtaining the key vector according to the product of the key parameter matrix and the fusion result; The value vector is obtained according to the product of the value parameter matrix and the fusion result.
8. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first numerical feature sequence of original data; A first processing module, configured to perform rotational position encoding processing on the first numerical feature sequence based on a preset target dimension to obtain a rotational position encoding vector of the original data; A determination module, used to determine a first model numerical feature sequence corresponding to the target dimension in the target data processing model; A second processing module, used for determining a target numerical feature vector based on a fusion result of the rotation position encoding vector and the first model numerical feature sequence; The analysis module is used to analyze the target numerical feature vector through the target data processing model to obtain the target result.
9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data processing method according to any one of claims 1 to 7.
11. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the data processing method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Improved BERT training model and named entity identification method and system
CN112560484A
Data processing method and device, electronic equipment and storage medium
CN115115913A
Classification method and device for multi-modal data
CN116049518A
Address information updating method and device, electronic equipment and storage medium
CN116595023A
Multi-modal business process prediction method and system fusing flow chart and process log
CN118134232A
