Data internal feature vector generation method, medium and system

Abstract

The application discloses a data internal feature vector generation method, medium and system, wherein the method comprises the following steps: obtaining to-be-processed data, and preprocessing the data to obtain a standard input vector; disassembling the standard input vector to obtain a subvector set, independently encoding each subvector to obtain a corresponding embedded vector set, and generating an embedded matrix according to the embedded vector set; performing linear transformation on the embedded matrix to obtain a linear transformation result, learning to obtain corresponding attention weights, and generating a fusion vector by weighted fusion according to the attention weights; constructing a semantic graph, calculating a structure preservation loss term according to the semantic graph; calculating a main task loss term, and jointly optimizing the structure preservation loss term and the main task loss term to generate a structure enhanced vector, thereby generating a final internal feature vector; the internal feature of the data can be effectively extracted, and the discriminability and generalization of the finally generated feature vector are improved.

Description

Technical Field

[0001] The present application relates to the technical field of data internal feature extraction, and in particular to a method, medium and system for generating data internal feature vectors. Background Art

[0002] At present, in intelligent systems, especially in the fields of psychological assessment, recommendation systems, personalized service platforms, etc., internal feature vector generation has become an indispensable basic link.

[0003] In related technologies, when generating internal feature vectors, most simply semantically encode the data being processed, then simply concatenate the semantic encoding results to obtain the internal feature representation. While this approach can meet basic semantic expression requirements in most applications and provide support for subsequent matching, recommendation, and evaluation, as application scenarios become more complex, this simple concatenation approach leads to a large amount of information redundancy and a significant waste of computing resources. Furthermore, this approach prevents the full capture of hidden fine-grained relationships between fields, reducing the discriminative power and generalizability of the final vector. Summary of the Invention

[0004] The present invention aims to solve, at least to some extent, one of the technical problems in the related art. To this end, one object of the present invention is to propose a method for generating internal feature vectors of data that can effectively extract the internal features of the data and improve the discriminative power and generalization of the resulting feature vectors.

[0005] In a first aspect, an embodiment of the present invention proposes a method for generating an internal feature vector of data, comprising the following steps: obtaining data to be processed, and preprocessing the data to be processed to obtain a standard input vector; disassembling the standard input vector to obtain a corresponding sub-vector set, and independently encoding each sub-vector in the sub-vector set to obtain a corresponding embedding vector set, and generating a corresponding embedding matrix based on the embedding vector set; performing a linear transformation on the embedding matrix to obtain a corresponding linear transformation result, and learning based on the linear transformation result to obtain a corresponding attention weight, and performing weighted fusion based on the attention weight to generate a fusion vector; constructing a semantic graph based on the linear transformation result, and calculating a structure preservation loss term based on the semantic graph; calculating a corresponding main task loss term based on the fusion vector, and performing joint optimization based on the structure preservation loss term and the main task loss term to generate a structure enhancement vector, and generating a final internal feature vector based on the structure enhancement vector.

[0006] According to an embodiment of the present invention, a method for generating an internal feature vector of data is as follows: first, to-be-processed data is obtained and pre-processed to obtain a standard input vector; then, the standard input vector is disassembled to obtain a corresponding set of sub-vectors, and each sub-vector in the set of sub-vectors is independently encoded to obtain a corresponding set of embedding vectors, and a corresponding embedding matrix is ​​generated based on the set of embedding vectors; then, a linear transformation is performed on the embedding matrix to obtain a corresponding linear transformation result, and learning is performed based on the linear transformation result to obtain a corresponding attention weight, and weighted fusion is performed based on the attention weight to generate a fusion vector; then, a semantic graph is constructed based on the linear transformation result, and a structure-preserving loss term is calculated based on the semantic graph; then, a corresponding main task loss term is calculated based on the fusion vector, and a structure-preserving loss term and the main task loss term are jointly optimized to generate a structure-enhanced vector, and a final internal feature vector is generated based on the structure-enhanced vector; thereby effectively extracting the internal features of the data and improving the discriminability and generalization of the finally generated feature vector.

[0007] In some embodiments, preprocessing the data to be processed includes: constructing an original input tensor set based on the data to be processed; performing numerical formatting validity check and range boundary truncation on each numerical field in the original input tensor set, and performing spelling normalization and semantic label mapping on text fields to obtain a standardized tensor set; normalizing the numerical fields in the standardized tensor set; and splicing all processed fields to obtain a standard input vector.

[0008] In some embodiments, preprocessing the data to be processed also includes: for each field in the original input tensor set, calculating the missing rate corresponding to the field; judging whether the missing rate is greater than a first preset missing rate threshold; if the missing rate is greater than the first preset missing rate threshold, determining that the field corresponding to the missing rate is an unqualified field; if the missing rate is less than or equal to the first preset missing rate threshold, judging whether the missing rate is greater than a second preset missing rate threshold and the field is an important field; if the missing rate is greater than the second preset missing rate threshold and the field is an important field, determining the field as an unqualified field.

[0009] In some embodiments, weighted fusion is performed according to the following formula:

[0010] ;

[0011] in, represents the attention weight, Indicates the embedding matrix The linear transformation result of the fields, represents the query vector;

[0012] ;

[0013] in, Represents the fusion vector.

[0014] In some embodiments, after generating the fusion vector, the method further includes: determining whether an abnormality occurs in the weighted fusion process; if so, adjusting the gating coefficient according to the abnormality to control the information weight of the weighted fusion result based on the gating coefficient.

[0015] In some embodiments, calculating the corresponding main task loss term based on the fusion vector includes:

[0016] Performing discriminant enhancement on the fusion vector based on a discriminant enhancement module to obtain a discriminant enhancement vector, wherein the discriminant enhancement module includes two layers of fully connected networks and one layer of residual connection layer;

[0017] The discriminant enhancement vector is expressed by the following formula:

[0018] ;

[0019] in, represents the discriminant enhancement module, represents the discriminant enhancement vector, represents a feedforward network composed of nonlinear activations, Represents normalization operation;

[0020] ;

[0021] in, represents the main task loss term, represents the number of samples, represents the true label, Represents the model prediction results.

[0022] In some embodiments, in the semantic graph, the node set represents the embedded representation of the field, the edge set represents the semantic adjacency relationship between the fields, and the edge weight represents the semantic similarity between the fields;

[0023] The semantic similarity is calculated using the following formula:

[0024] ;

[0025] in, Representation field With fields The semantic similarity between Representation field The linear transformation result of Represents a vector The transpose of .

[0026] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium on which a data internal feature vector generation program is stored. When the data internal feature vector generation program is executed by a processor, the data internal feature vector generation method described above is implemented.

[0027] In a third aspect, an embodiment of the present invention proposes a data internal feature vector generation system, comprising: a preprocessing module, the preprocessing module being used to obtain data to be processed and preprocess the data to be processed to obtain a standard input vector; an encoding module, the encoding module being used to decompose the standard input vector to obtain a corresponding sub-vector set, and independently encode each sub-vector in the sub-vector set to obtain a corresponding embedding vector set, and generate a corresponding embedding matrix based on the embedding vector set; a linear transformation module, the linear transformation module being used to perform a linear transformation on the embedding matrix to obtain a corresponding linear transformation result; a fusion module, the fusion module being used to learn according to the linear transformation result to obtain a corresponding attention weight, and perform weighted fusion according to the attention weight to generate a fusion vector; a construction module, the construction module being used to construct a semantic graph based on the linear transformation result, and calculate a structure preservation loss term based on the semantic graph; a calculation module, the calculation module being used to calculate a corresponding main task loss term based on the fusion vector, and jointly optimize the structure preservation loss term and the main task loss term to generate a structure enhancement vector, and generate a final internal feature vector based on the structure enhancement vector.

[0028] In some embodiments, preprocessing the data to be processed includes: constructing an original input tensor set based on the data to be processed; performing numerical formatting validity check and range boundary truncation on each numerical field in the original input tensor set, and performing spelling normalization and semantic label mapping on text fields to obtain a standardized tensor set; normalizing the numerical fields in the standardized tensor set; and splicing all processed fields to obtain a standard input vector.

[0029] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 1 is a flow chart of a method for generating an internal feature vector of data according to an embodiment of the present invention;

[0031] Figure 24 is a block diagram of a system for generating internal feature vectors of data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0033] The following describes a method for generating a data internal feature vector according to an embodiment of the present invention with reference to the accompanying drawings.

[0034] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for generating an internal feature vector of data according to an embodiment of the present invention. Figure 1 As shown, the method for generating the internal feature vector of the data includes the following steps:

[0035] S101 , obtaining data to be processed and preprocessing the data to be processed to obtain a standard input vector.

[0036] That is to say, after obtaining the data to be processed, the data to be processed is subjected to data structuring, field format standardization, numerical verification and normalization processing to obtain the corresponding standard input vector.

[0037] In some embodiments, preprocessing the data to be processed includes: constructing an original input tensor set based on the data to be processed; performing numerical formatting validity check and range boundary truncation on each numerical field in the original input tensor set, and performing spelling normalization and semantic label mapping on text fields to obtain a standardized tensor set; normalizing the numerical fields in the standardized tensor set; and splicing all processed fields to obtain a standard input vector.

[0038] As an example, first, in the user interface, an information input template is preset to prompt the user's information input behavior; specifically, the user can be prompted to enter the following fields in sequence:

[0039] Basic personal attributes (name, age, gender); educational background (education background, graduation school, major); job-seeking intention (target position, industry, expected job content).

[0040] Next, construct the original input tensor based on the received information:

[0041] ;

[0042] in, represents the original input tensor set, Indicates the The original value of the input field.

[0043] Then, for numeric fields (such as age and length of service), the numerical format validity check and range boundary truncation are performed; for text fields (such as professional name and job title), the standard vocabulary is introduced to perform spelling normalization and semantic label mapping to obtain a standardized tensor set:

[0044] ;

[0045] in, represents a collection of normalized tensors, Indicates the The formatting results corresponding to the input fields.

[0046] Next, all numerical fields in the normalized tensor set are linearly normalized. The process is defined as:

[0047] ;

[0048] Normalization ensures that scale differences will not cause feature imbalance in the subsequent unified encoding space. At the same time, all text fields are uniformly converted into system-internal standard semantic labels for subsequent embedding operations.

[0049] Then, a standard input vector with a clear structure is constructed based on all the standardized fields, which is recorded as:

[0050] ;

[0051] in, Represents a vector concatenation operation, and the length of the final generated standard input vector is , which will serve as the input of the subsequent semantic encoding module. This vector is the result of information structure standardization and does not yet have semantic representation capabilities. It is still in the data preparation stage.

[0052] It should be noted that in order to ensure the integrity of the system process, the standard input vector can be transmitted to the semantic encoding module through the system's predefined cache interface. Specifically, first, the standard input vector is stored in the database table user_input_tensor; then, the source information such as the source, normalization type, and encoded index position of each field is marked in the table structure to facilitate subsequent tracking and debugging; then, the system sets the timestamp field to ensure that the time-related status of each input remains consistent.

[0053] In some embodiments, preprocessing the data to be processed also includes: for each field in the original input tensor set, calculating the missing rate corresponding to the field; judging whether the missing rate is greater than a first preset missing rate threshold; if the missing rate is greater than the first preset missing rate threshold, determining that the field corresponding to the missing rate is an unqualified field; if the missing rate is less than or equal to the first preset missing rate threshold, judging whether the missing rate is greater than a second preset missing rate threshold and the field is an important field; if the missing rate is greater than the second preset missing rate threshold and the field is an important field, determining the field as an unqualified field.

[0054] That is to say, first, a first preset missing rate threshold is set to make a first judgment on the missing rate of the field; if the missing rate of the field is greater than the first preset missing rate threshold; it means that the missing rate of the field is too large, and the field is directly defined as an unqualified field; then, if the missing rate of the field is less than or equal to the first preset missing rate threshold, it is further judged whether the missing rate of the field is greater than the second preset missing rate threshold, and the field is an important field (for example, name, position, etc.); if so, the field is determined to be an unqualified field, so that relevant personnel can handle the unqualified field (for example, manual confirmation, filling, etc.); in this way, the missing of important field information can be avoided.

[0055] In some embodiments, whether a field is important can be determined by its importance score; specifically, a field importance scoring mechanism based on task relevance can be introduced. The importance score of each field can be obtained by evaluating the information gain, correlation coefficient, or contribution to baseline model training. The importance score can be calculated using the following formula:

[0056] ;

[0057] in, represents label entropy, represents the conditional entropy, Representation field Importance rating.

[0058] S102: Decompose the standard input vector to obtain a corresponding set of sub-vectors, independently encode each sub-vector in the sub-vector set to obtain a corresponding set of embedded vectors, and generate a corresponding embedding matrix based on the embedded vector set.

[0059] As an example, first, the underlying embedding function is called, which is implemented by the embedding layer of the pre-trained large language model, and can encode text or numerical data into corresponding semantic vectors; then, for the standard input vector , decompose it into a collection of field-level sub-vectors:

[0060] ;

[0061] The system feeds the concatenated vector into the embedding function field by field for independent encoding, thereby obtaining a set of embedded vectors:

[0062] in, ;

[0063] in, Represents the underlying embedding function, which is implemented by the embedding layer of the pre-trained large language model and can encode text or numerical data into corresponding semantic vectors. Represents the standardized input fields of the function; Indicates the embedding vectors, Represents a set of embedding vectors.

[0064] Each embedding vector in the embedding vector set is a dense vector with a context-sensitive structure and its length is , with stable representation capabilities in the model's pre-trained semantic space. This embedding operation preserves the details of the field's original semantics while compressing it into a single dimensional space, ensuring mathematical uniformity and scale alignment for subsequent processing.

[0065] Then, because each vector in the set maintains a differentiable structure in the embedding space, it can be directly used to calculate distance, similarity, or fusion operations. To further advance the linear process, the system reassembles the set into an embedding matrix in sequence:

[0066] ;

[0067] The matrix is ​​a OK The matrix structure enables the system to uniformly dispatch this semantic representation in subsequent modules for operations such as multi-layer fusion, graph structure modeling, and discriminant enhancement.

[0068] It should be noted that at this stage, the system will complete the following three guarantee operations: First, the embedding cache registration mechanism: all embedding vectors will be retained in the memory until the final internal feature vector is generated, so as to achieve cross-module sharing and reverse tracking; then, the embedding domain verification is completed: the system verifies whether each embedding vector conforms to the predefined semantic category range to prevent the embedding vector from leaving the model training domain and causing semantic drift; then, the semantic space alignment mechanism: if the embedding model called by the system supports multiple sub-domains (for example, position semantic space, industry semantic space), the corresponding sub-model is selected according to the field type to improve the local semantic resolution.

[0069] In this way, this step can complete the transformation from the preliminary semantic vector domain of the structured field item, and construct a coding matrix (i.e., embedding matrix) that has semantic information but has not yet been integrated; the semantic representation is still dimensionally separated, and its information structure does not form a coordinated expression.

[0070] S103: Perform a linear transformation on the embedding matrix to obtain a corresponding linear transformation result, perform learning based on the linear transformation result to obtain a corresponding attention weight, and perform weighted fusion based on the attention weight to generate a fusion vector.

[0071] In other words, the embedding matrix is ​​further fused into a unified, more discriminative internal feature representation vector (i.e., fusion vector). This process not only requires dimensionality reduction and compression, but also requires the introduction of a nonlinear mapping mechanism to make the interaction between information more expressive.

[0072] In some embodiments, weighted fusion is performed according to the following formula:

[0073] ;

[0074] in, represents the attention weight, Indicates the embedding matrix The linear transformation result of the fields, Representation query vector (this parameter is automatically learned by the ship during the model training phase. It is the global attention query vector in the attention mechanism and is a learnable model parameter used to measure the importance of each field representation in the final fusion vector);

[0075] ;

[0076] in, Represents the fusion vector.

[0077] As an example, first, the embedding matrix is ​​transformed using the standard linear transformation mechanism, and the transformation matrix is ​​set and bias , perform the following linear fusion operation:

[0078] ;

[0079] in, Represents an element-by-element activation function, which is used to increase nonlinear expression capabilities while suppressing the propagation of redundant features. Linear transformation results Represents the intermediate representation after a linear and nonlinear fusion.

[0080] Next, in order to improve the efficiency of information interaction between features, an adaptive multi-channel attention mechanism is introduced to capture the saliency and complementarity of different fields. Indicates the The system will learn a set of attention weights to complete the global weighted fusion based on the linear transformation results of the fields:

[0081] ;

[0082] in, represents the attention weight, Indicates the embedding matrix The linear transformation result of the fields, represents the query vector;

[0083] ;

[0084] in, Represents the fusion vector.

[0085] In some embodiments, after generating the fusion vector, the method further includes: determining whether an abnormality occurs in the weighted fusion process; if so, adjusting the gating coefficient according to the abnormality to control the information weight of the weighted fusion result based on the gating coefficient.

[0086] As an example, in multi-channel attention or other nonlinear fusion processes, if the fusion operation fails (such as gradient explosion, training non-convergence) or generates noise, it may lead to performance degradation of downstream tasks. At this time, the system should automatically enable the backoff strategy.

[0087] Specifically, we introduce residual connections in the fusion layer, bypassing the original input vector and passing it directly to subsequent layers. When anomalies occur in the fusion branch, the network automatically redirects more information flow to this path using learned gating weights, ensuring that the model's worst-case performance is at least as good as that achieved using only the original features.

[0088] Residual connection formula description:

[0089] ;

[0090] in:

[0091] is the original feature vector before fusion,

[0092] To fuse sub-networks (such as attention or nonlinear mapping),

[0093] is a learnable gating coefficient that controls the information weight of the fusion branch and the bypass branch.

[0094] During the training process, if the fusion branch effect is not good, the model will automatically reduce The value of , will rely more on the direct bypass of the information Otherwise, it will increase , to fully exploit the expressive power brought by fusion.

[0095] S104: construct a semantic graph based on the linear transformation result, and calculate the structure preservation loss term based on the semantic graph.

[0096] In some embodiments, in a semantic graph, a set of nodes represents an embedded representation of a field, a set of edges represents a semantic adjacency relationship between fields, and the weight of the edge represents a semantic similarity between fields;

[0097] The semantic similarity is calculated using the following formula:

[0098] ;

[0099] in, Representation field With fields The semantic similarity between Representation field The linear transformation result of Represents a vector The transpose of .

[0100] As an example, it can be understood that after the embedded information is fused, the global integration characteristics of the averaging or attention weighting mechanism may cause the microstructural differences between different fields to be overly compressed, thereby weakening the personalized semantic expression and the semantic tension between dimensions. Therefore, after feature enhancement, a structure-preserving regularization mechanism is introduced to force the system to preserve the local semantic graph structure between input fields while learning unified features, making the representation more structure-aware and discriminative.

[0101] Specifically, first, based on the linear transformation results , build a semantic relationship graph between fields , where: node set Represents the embedded representation of each field, edge set It represents the semantic adjacency relationship between fields, and the edge weight represents the semantic similarity between fields. The semantic similarity is calculated by the following formula:

[0102] ;

[0103] in, Representation field With fields The semantic similarity between Representation field The linear transformation result of Represents a vector Preferably, the adjacency graph is not a static prior structure, but is adaptively generated based on the user's current input information to ensure that individual differences are fully expressed at the structural level.

[0104] In order to preserve the graph structure information during training, a structure preservation loss term is introduced , its goal is to make the semantic structure relationship as undamaged as possible when the final internal vector is generated. Assuming that the final output of the system is the global fusion vector , the system will construct an auxiliary mapping , project the global vector back to the local field vector space and calculate the deviation between its graph structure and the original graph:

[0105] ;

[0106] in, Represents the first This structure-preserving loss term forces the system to maintain the relative semantic structure between fields, so that the final fusion result not only has global expressiveness but also has structural sensitivity to field relationships, preventing semantic ambiguity caused by over-fusion.

[0107] S105: Calculate the corresponding main task loss term based on the fusion vector, and perform joint optimization based on the structure preservation loss term and the main task loss term to generate a structure enhancement vector, and generate a final internal feature vector based on the structure enhancement vector.

[0108] In some embodiments, calculating the corresponding main task loss term based on the fusion vector includes:

[0109] The fusion vector is discriminatively enhanced based on the discriminative enhancement module to obtain a discriminative enhancement vector, wherein the discriminative enhancement module includes two layers of fully connected networks and one layer of residual connection layer;

[0110] The discriminant enhancement vector is expressed by the following formula:

[0111] ;

[0112] in, represents the discriminant enhancement module, represents the discriminant enhancement vector, represents a feedforward network composed of nonlinear activations, Represents normalization operation;

[0113] ;

[0114] in, represents the main task loss term, represents the number of samples, represents the true label, Represents the model prediction results.

[0115] As an example, first, we perform discriminative enhancement on the fusion vector through the discriminative enhancement module to obtain a discriminative enhancement vector, where the discriminative enhancement module includes two layers of fully connected networks and one layer of residual connection layer;

[0116] The discriminant enhancement vector is expressed by the following formula:

[0117] ;

[0118] in, represents the discriminant enhancement module, represents the discriminant enhancement vector, represents a feedforward network composed of nonlinear activations, Represents normalization operation;

[0119] ;

[0120] in, represents the main task loss term, represents the number of samples, represents the true label, Represents the model prediction results.

[0121] It should be noted that this main task loss term is more applicable in the scenario of matching the user's internal feature vector with job features or labels.

[0122] In scenarios where the system needs to give a continuous score and has real scoring or review data for supervision, the main task loss term can be defined as:

[0123] ;

[0124] in, represents the model prediction score, Represents the true score given in manual / historical data.

[0125] Furthermore, the system jointly trains the structure preservation loss term with the main task loss term to form the following overall goal:

[0126] ;

[0127] in, Represents a hyperparameter used to balance the weight between the main task objective and the structure-preserving regularization term. Its value can be set based on experimental statistical results and task sensitivity; it is usually between [0.01, 0.1].

[0128] It should be noted that after the structure-preserving mechanism is trained, the system will use the fusion module to output the final internal feature vector, which will be used as the unique user-personalized semantic representation in all subsequent modules (for example, question matching, dynamic adjustment, and score calculation) and has the following characteristics: 1. Complete global expression, covering all fields in user input; 2. Field structure relationships are retained, suitable for semantically sensitive downstream matching tasks; 3. Fusion behavior is explainable, and the system can provide a field impact analysis view.

[0129] In some embodiments, after the final internal feature vector is embedded, extracted, semantically fused, and optimized for structure preservation, the data is stored for subsequent processes to complete the closed loop from the original data to be processed to the unified semantic vector expression, ensuring the consistency of the data at the semantic level, the stability of the structural level, and the continuity of the system's internal processes. First, the final internal feature vector has the following characteristics: 1. Dimensional uniformity. Regardless of the content of the field filled in by the user, the dimension of the final internal feature vector is fixed, ensuring the consistency of the system input interface; 2. High semantic concentration. Through multiple rounds of deep fusion and structure preservation, the final internal feature vector condenses the contextual associations and internal semantic distribution of each field, which is suitable for subsequent high-semantic tasks; 2. Structural perception: After introducing the graph structure regularization term, the final internal feature vector maintains the micro-semantic differences between each field, which facilitates more accurate distinction of different types of user features. Then, in order to ensure a one-to-one mapping relationship between the final internal feature vector and the user identity, the system binds the final internal feature vector with the user identification information to construct the following tuple:

[0130] ;

[0131] in, Indicates the system's unique identification number. represents the final internal eigenvector, Indicates the timestamp, used to record the evaluation time. Indicates auxiliary information such as original fields, fusion weights, model version numbers, etc., which are used for subsequent debugging and backtracking.

[0132] The final internal feature vector is then written to the corresponding feature table in the system database, automatically triggering the activation logic of subsequent processing modules. Once written, the system calls the vector registration service and publishes the event type and event content (for example, user ID, vector address, version number, etc.). The system uses an event bus mechanism to broadcast this final internal feature vector to all required submodules. Each module retrieves the final internal feature vector for the corresponding user from the database or cache as the basic input for its subsequent semantic tasks. This creates a complete linear closed loop from raw user information input to semantic vector archiving, ensuring that the transmission of user features within the system is structured, standardized, and computable.

[0133] In summary, according to the method for generating internal feature vectors of data in an embodiment of the present invention, first, the data to be processed is obtained and preprocessed to obtain a standard input vector; then, the standard input vector is disassembled to obtain a corresponding sub-vector set, and each sub-vector in the sub-vector set is independently encoded to obtain a corresponding embedding vector set, and a corresponding embedding matrix is ​​generated based on the embedding vector set; then, the embedding matrix is ​​linearly transformed to obtain a corresponding linear transformation result, and learning is performed based on the linear transformation result to obtain a corresponding attention weight, and weighted fusion is performed based on the attention weight to generate a fusion vector; then, a semantic graph is constructed based on the linear transformation result, and a structure preservation loss term is calculated based on the semantic graph; then, the corresponding main task loss term is calculated based on the fusion vector, and the structure preservation loss term and the main task loss term are jointly optimized to generate a structure enhancement vector, and a final internal feature vector is generated based on the structure enhancement vector; thereby effectively extracting the internal features of the data and improving the discriminability and generalization of the finally generated feature vector.

[0134] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium on which a data internal feature vector generation program is stored. When the data internal feature vector generation program is executed by a processor, the data internal feature vector generation method described above is implemented.

[0135] In a third aspect, an embodiment of the present invention proposes a data internal feature vector generation system, such as Figure 2 As shown, the data internal feature vector generation system includes: a preprocessing module 10, an encoding module 20, a linear transformation module 30, a fusion module 40, a construction module 50 and a calculation module 60.

[0136] The preprocessing module 10 is used to obtain the data to be processed and preprocess the data to be processed to obtain a standard input vector;

[0137] The encoding module 20 is used to disassemble the standard input vector to obtain a corresponding sub-vector set, independently encode each sub-vector in the sub-vector set to obtain a corresponding embedding vector set, and generate a corresponding embedding matrix based on the embedding vector set;

[0138] The linear transformation module 30 is used to perform a linear transformation on the embedded matrix to obtain a corresponding linear transformation result;

[0139] The fusion module 40 is used to learn based on the linear transformation results to obtain the corresponding attention weights, and perform weighted fusion based on the attention weights to generate a fusion vector;

[0140] The construction module 50 is used to construct a semantic graph based on the linear transformation result and calculate the structure preservation loss term according to the semantic graph;

[0141] The calculation module 60 is used to calculate the corresponding main task loss term based on the fusion vector, and perform joint optimization based on the structure preservation loss term and the main task loss term to generate a structure enhancement vector, and generate a final internal feature vector based on the structure enhancement vector.

[0142] In some embodiments, preprocessing the data to be processed includes: constructing an original input tensor set based on the data to be processed; performing numerical formatting validity check and range boundary truncation on each numerical field in the original input tensor set, and performing spelling normalization and semantic label mapping on text fields to obtain a standardized tensor set; normalizing the numerical fields in the standardized tensor set; and splicing all processed fields to obtain a standard input vector.

[0143] It should be noted that the above description of the method for generating the internal feature vector of data is also applicable to the system for generating the internal feature vector of data, and will not be elaborated here.

[0144] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0145] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0146] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0147] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0148] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0149] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0150] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0151] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for generating internal feature vectors of data, characterized in that: The following steps are involved: Acquiring data to be processed and preprocessing the data to be processed to obtain a standard input vector; Decomposing the standard input vector to obtain a corresponding set of sub-vectors, independently encoding each sub-vector in the set of sub-vectors to obtain a corresponding set of embedded vectors, and generating a corresponding embedding matrix according to the set of embedded vectors; Performing a linear transformation on the embedding matrix to obtain a corresponding linear transformation result, performing learning based on the linear transformation result to obtain a corresponding attention weight, and performing weighted fusion based on the attention weight to generate a fusion vector; Constructing a semantic graph based on the linear transformation result, and calculating a structure preservation loss term according to the semantic graph; Calculating a corresponding main task loss term based on the fusion vector, and performing joint optimization based on the structure preservation loss term and the main task loss term to generate a structure enhancement vector, and generating a final internal feature vector based on the structure enhancement vector; Preprocessing of the data to be processed includes: Constructing an original input tensor set according to the data to be processed; Performing a value formatting validity check and range boundary truncation on each numeric field in the original input tensor set, and performing spelling normalization and semantic label mapping on text fields to obtain a standardized tensor set; Normalizing the numerical fields in the standardized tensor set; Concatenate all processed fields to obtain the standard input vector; Calculating the corresponding main task loss term based on the fusion vector includes: Performing discriminant enhancement on the fusion vector based on a discriminant enhancement module to obtain a discriminant enhancement vector, wherein the discriminant enhancement module includes two layers of fully connected networks and one layer of residual connection layer; The discriminant enhancement vector is expressed by the following formula: ； in, represents the fusion vector, represents the discriminant enhancement module, represents the discriminant enhancement vector, represents a feedforward network composed of nonlinear activations, Represents normalization operation; ； in, represents the main task loss term, represents the number of samples, represents the true label, Represents the model prediction results.

2. The method for generating internal feature vectors of data according to claim 1, wherein: Preprocessing the data to be processed further includes: For each field in the original input tensor set, calculate the missing rate corresponding to the field; Determining whether the missing rate is greater than a first preset missing rate threshold; If the missing rate is greater than a first preset missing rate threshold, determining that the field corresponding to the missing rate is an unqualified field; If the missing rate is less than or equal to the first preset missing rate threshold, determining whether the missing rate is greater than a second preset missing rate threshold and the field is an important field; If the missing rate is greater than a second preset missing rate threshold and the field is an important field, the field is determined to be an unqualified field.

3. The method for generating internal feature vectors of data according to claim 1, wherein: Weighted fusion is performed according to the following formula: ； in, represents the attention weight, Indicates the embedding matrix The linear transformation result of the fields, represents the query vector, Indicates the total number of fields, Representation field The linear transformation result of ; ； in, Represents the fusion vector.

4. The method for generating internal feature vectors of data according to claim 1, wherein: After generating the fusion vector, it also includes: Determine whether there is any abnormality in the weighted fusion process; If yes, the gating coefficient is adjusted according to the abnormal situation to control the information weight of the weighted fusion result based on the gating coefficient.

5. The method for generating internal feature vectors of data according to claim 1, wherein: In the semantic graph, the node set represents the embedded representation of the field, the edge set represents the semantic adjacency relationship between the fields, and the edge weight represents the semantic similarity between the fields; The semantic similarity is calculated using the following formula: ； in, Representation field With fields The semantic similarity between Representation field The linear transformation result of Represents a vector The transpose of .

6. A computer-readable storage medium, characterized in that A data internal feature vector generation program is stored thereon, and when the data internal feature vector generation program is executed by a processor, the data internal feature vector generation method according to any one of claims 1 to 5 is implemented.

7. A data internal feature vector generation system, characterized in that: include: A preprocessing module, the preprocessing module is used to obtain the data to be processed and preprocess the data to be processed to obtain a standard input vector; an encoding module configured to decompose the standard input vector to obtain a corresponding set of sub-vectors, independently encode each sub-vector in the set of sub-vectors to obtain a corresponding set of embedded vectors, and generate a corresponding embedding matrix based on the set of embedded vectors; A linear transformation module, configured to perform a linear transformation on the embedding matrix to obtain a corresponding linear transformation result; A fusion module is configured to learn based on the linear transformation result to obtain corresponding attention weights, and perform weighted fusion based on the attention weights to generate a fusion vector; A construction module, wherein the construction module is used to construct a semantic graph based on the linear transformation result and calculate a structure preservation loss term according to the semantic graph; a calculation module, the calculation module being configured to calculate a corresponding main task loss term based on the fusion vector, and to perform joint optimization based on the structure-preserving loss term and the main task loss term to generate a structure enhancement vector, and to generate a final internal feature vector based on the structure enhancement vector; Preprocessing of the data to be processed includes: Constructing an original input tensor set according to the data to be processed; Performing a value formatting validity check and range boundary truncation on each numeric field in the original input tensor set, and performing spelling normalization and semantic label mapping on text fields to obtain a standardized tensor set; Normalizing the numerical fields in the standardized tensor set; Concatenate all processed fields to obtain the standard input vector; Calculating the corresponding main task loss term based on the fusion vector includes: Performing discriminant enhancement on the fusion vector based on a discriminant enhancement module to obtain a discriminant enhancement vector, wherein the discriminant enhancement module includes two layers of fully connected networks and one layer of residual connection layer; The discriminant enhancement vector is expressed by the following formula: ； in, represents the fusion vector, represents the discriminant enhancement module, represents the discriminant enhancement vector, represents a feedforward network composed of nonlinear activations, Represents normalization operation; ； in, represents the main task loss term, represents the number of samples, represents the true label, Represents the model prediction results.

Images