Data completion method and device, equipment, storage medium and product

By injecting training data adjustment matrix and large model feature vector adjustment in specific fields into the data completion model, combined with preset knowledge base and regular expressions, the problem of inaccurate data dictionary completion in the existing technology is solved, and more efficient and accurate data completion is achieved.

CN120492824APending Publication Date: 2025-08-15CHINA MOBILE INFORMATION TECHNOLOGY CO LTD +1
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202510668362.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-22
Publication Date
2025-08-15

AI Technical Summary

Technical Problem

In the prior art, machine learning models have poor processing capabilities for highly nonlinear data relationships and insufficient generalization capabilities, resulting in inaccurate data dictionary completion results, and high cost of manual data dictionary completion, machine translation cannot understand the contextual meaning of data dictionary, and existing methods consume resources and have low accuracy.

Method used

By injecting the adjustment matrix determined by specific domain training data into the data completion model, the large model uses the feature vector adjustment of the complete data, user problems and similar data, and extracting missing data in combination with the preset knowledge base and regular expressions, the model's adaptability in specific domains and the accuracy of the completion results.

Benefits of technology

The accuracy of missing data is improved, the accuracy of data completion results is enhanced, problems in the prior art are avoided, and the adaptability and completion efficiency of the model in specific fields is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120492824A_ABST
    Figure CN120492824A_ABST
Patent Text Reader

Abstract

The invention discloses a data completion method and device, equipment, a storage medium and a product. The method comprises the steps that data, user questions and similar data with the first similarity with the data larger than a preset value are obtained, and the user questions comprise missing data items of the data; inputting the data, the user problem and the similar data into a data completion model, adjusting first feature vectors corresponding to the data, the user problem and the similar data by using an adjustment matrix in the data completion model to obtain a second feature vector, and determining missing data according to the second feature vector; combining the data and the missing data to obtain a first completion result; wherein the adjustment matrix is obtained by training based on training data of a preset field. The adaptive capacity of the model in the specific field is improved through the adjustment matrix determined by the training data in the specific field, the missing data is determined on the basis of the missing data items in the user question, the accuracy of the missing data is improved, and the problem that the accuracy of a completion result is low is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data completion method, device, equipment, storage medium and product. Background Art

[0002] Data is a crucial factor in production, and a high-quality data dictionary is a crucial information infrastructure for data inventory and efficient development. Currently, with numerous business systems and a heavy workload for data import, data dictionaries often contain incomplete and inaccurate data.

[0003] Currently, data completion methods for data dictionaries primarily rely on machine learning algorithms. For example, a metadata file to be completed is obtained; the metadata file is then fed into a pre-trained metadata completion model, which then completes the metadata file. The metadata completion model comprises a machine learning model trained on several pre-processed metadata samples. However, machine learning models are overly dependent on the quantity and quality of training data, making them difficult to handle highly nonlinear data relationships and exhibiting poor generalization capabilities. If the model's training data is insufficient or contains errors, the completion results will be inaccurate. Summary of the Invention

[0004] The embodiments of the present application provide a data completion method, apparatus, device, storage medium, and product, which can improve the accuracy of the completion results.

[0005] In a first aspect, an embodiment of the present application provides a data completion method, comprising:

[0006] Acquire data and a user question, as well as similar data having a first similarity with the data greater than a preset value, wherein the user question includes a missing data item of the data;

[0007] Inputting the data, user questions, and similar data into a data completion model, adjusting first eigenvectors corresponding to the data, user questions, and similar data using an adjustment matrix in the data completion model to obtain a second eigenvector, and determining missing data based on the second eigenvector, where the missing data includes missing data items;

[0008] Combine the data and missing data to obtain the first completion result;

[0009] The adjustment matrix is obtained by training based on training data in a preset field.

[0010] In one possible embodiment, before inputting the data, user questions, and similar data into the data completion model, the method further includes:

[0011] Obtain initial data in a preset domain from a preset knowledge base;

[0012] Generate a data sample and a user question sample in a preset format according to the initial data, and obtain a similar data sample whose second similarity with the data sample is greater than a preset value;

[0013] Input the data sample, user question sample, and similar data sample into the initial data completion model, use the initial adjustment matrix in the initial data completion model to adjust the third eigenvectors corresponding to the data sample, user question sample, and similar data sample to obtain a fourth eigenvector, and determine the predicted data based on the fourth eigenvector;

[0014] Combine the data sample and the predicted data to obtain the second completion result;

[0015] Determine the loss value of the initial data completion model based on the initial data and the second completion result;

[0016] When the loss value does not meet the training stop condition, the initial adjustment matrix is adjusted, and the third eigenvectors corresponding to the data samples, user question samples and similar data samples are adjusted using the adjusted initial adjustment matrix. The predicted data and loss value are updated until the loss value meets the training stop condition, and the data completion model is obtained.

[0017] In one possible embodiment, the preset field includes at least one sub-field, and the initial data includes initial sub-data of at least one sub-field;

[0018] Based on the initial data and the second completion result, the loss value of the initial data completion model is determined, including:

[0019] Obtaining the loss weight of at least one sub-field;

[0020] Determine the sub-loss value of the initial data completion model based on the initial sub-data and the second completion result of each sub-field;

[0021] Determine the loss value of the initial data completion model based on the loss weight and sub-loss value.

[0022] In one possible embodiment, adjusting the initial adjustment matrix includes:

[0023] The initial adjustment matrix is adjusted using the maximum acquisition function.

[0024] In one possible embodiment, the initial adjustment matrix and the adjustment matrix are within the search space; before adjusting the initial adjustment matrix, the method further includes:

[0025] Get hardware performance metrics;

[0026] Generate a search space that meets preset conditions, where the preset conditions include hardware performance indicators.

[0027] In one possible embodiment, obtaining similar data having a first similarity with the data greater than a preset value includes:

[0028] Obtain initial data in a preset domain from a preset knowledge base;

[0029] A first similarity between the data and the initial data is calculated, and similar data having a first similarity greater than a preset value is selected from the initial data.

[0030] In one possible embodiment, after combining the data and the missing data to obtain a first completion result, the method further includes:

[0031] When no missing data is detected in the first completion result, regular expressions are used to extract keywords from the missing data;

[0032] Combine keywords and data to get the third completion result.

[0033] In a second aspect, an embodiment of the present application provides a data completion device, including:

[0034] an acquisition module, configured to acquire data and user questions, as well as similar data having a first similarity with the data greater than a preset value, wherein the user questions include missing data items of the data;

[0035] a determination module, configured to input the data, user questions, and similar data into a data completion model, adjust first eigenvectors corresponding to the data, user questions, and similar data using an adjustment matrix in the data completion model to obtain a second eigenvector, and determine missing data based on the second eigenvector, where the missing data includes missing data items;

[0036] A combination module, used to combine data and missing data to obtain a first completion result;

[0037] The adjustment matrix is obtained by training based on training data in a preset field.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising:

[0039] A processor and a memory storing computer program instructions; a method for completing data of any one of the above items when the processor executes the computer program instructions.

[0040] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method for completing data of any one of the above items is implemented.

[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute any of the above-mentioned data completion methods.

[0042] The data completion method, device, equipment, storage medium and product of the embodiment of the present application obtain data and user questions, as well as similar data with a first similarity greater than a preset value to the data, wherein the user questions include missing data items of the data; the data, user questions and similar data are input into the data completion model, and the adjustment matrix in the data completion model is used to adjust the first eigenvectors corresponding to the data, user questions and similar data to obtain a second eigenvector, and the missing data is determined based on the second eigenvector, wherein the missing data includes missing data items; the data and the missing data are combined to obtain a first completion result; wherein the adjustment matrix is trained based on training data of a preset field. First, the adjustment matrix determined by training data of a specific field is injected into the model to improve the adaptability of the model in a specific field. Secondly, based on the missing data items in the user questions, the similar data of the data is used as the input of the model, and the features corresponding to the data to be completed, the user questions and similar data are adjusted to determine the missing data. The accuracy of the missing data is improved, thereby improving the accuracy of the completion result. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 This is a schematic diagram of the structure of a data completion system provided by an embodiment of the present application;

[0045] Figure 2 is a flowchart of a data completion method provided in another embodiment of the present application;

[0046] Figure 3 This is a flowchart of a data completion method provided in another embodiment of the present application;

[0047] Figure 4 1 is a flow chart of a data completion method provided in yet another embodiment of the present application;

[0048] Figure 5 is a structural diagram of a data completion device provided in yet another embodiment of the present application;

[0049] Figure 6 This is a structural diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0050] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0051] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0052] It should be noted that the acquisition, storage, use and processing of data in the embodiments of this application are in compliance with the relevant provisions of national laws and regulations.

[0053] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0054] As mentioned in the background, machine learning models are overly dependent on the quantity and quality of training data. They struggle to handle highly nonlinear data relationships and have poor generalization capabilities. If the model's training data is insufficient or contains errors, the completion results will be inaccurate.

[0055] In addition to machine learning models, existing technologies for data dictionary completion using manual methods are costly, and the completeness and accuracy of the information cannot be guaranteed. Data dictionary completion based on a preset vocabulary often has a limited range of vocabulary and terminology, and cannot effectively support field names that fall outside the vocabulary. Data dictionary completion based on machine translation cannot accurately understand the contextual meaning of field names and descriptions in the data dictionary, performs poorly when handling complex contexts and domain-specific terminology, and produces inaccurate translation results. Furthermore, data dictionaries typically involve domain-specific knowledge and terminology, and using machine translation for data dictionary completion often requires customized development and repeated training, which consumes a lot of resources.

[0056] In order to solve the problems of the prior art, the embodiments of the present application provide a data completion method, apparatus, device, storage medium and product. The data completion method provided in the embodiments of the present application can be applied to a data completion system. Figure 1 As shown, the data completion system 100 includes a professional knowledge base module 110, a vectorization module 120, a retrieval and positioning module 130, an input module 140, a large model module 150 and an output module 160. The professional knowledge base module 110 includes a preset knowledge base, and the data in the preset knowledge base is used to train the model. The preset knowledge base is constructed using data related to the data dictionary completion task. Among them, the data related to the data dictionary completion task is the processed data. The vectorization module 120 is used to vectorize the data to be completed obtained by the input module 140 and forward it to the retrieval and positioning module 130. The retrieval and positioning module 130 is used to obtain similar data from the preset knowledge base whose first similarity of the vector corresponding to the data to be completed is greater than a preset value. The large model module 150 is used to determine the missing data based on the data to be completed, user questions and similar data, and the output module 160 outputs the completion result.

[0057] In some embodiments, the input module 140 is used to receive and cache the data to be completed and the user questions.

[0058] In the embodiments provided in this application, first, an adjustment matrix determined by training data from a specific field is injected into the model to improve the model's adaptability in a specific field. Secondly, based on the missing data items in the user's question, similar data of the data is used as the input of the model. After adjusting the features corresponding to the data to be completed, the user's question, and the similar data, the missing data is determined. This improves the accuracy of the missing data, thereby improving the accuracy of the completion results. At the same time, the problems existing in the prior art will not occur.

[0059] The following first introduces the data completion method provided in the embodiment of the present application.

[0060] Figure 2FIG. 1 shows a flow chart of a data completion method provided by an embodiment of the present application. Figure 2 As shown, the data completion method provided in the embodiment of the present application includes the following steps S210 to S230.

[0061] S210: Acquire data, user questions, and similar data having a first similarity with the data greater than a preset value, wherein the user questions include missing data items of the data.

[0062] The data is the data to be completed. The similar data is the initial data in the preset knowledge base. The initial data in the preset knowledge base is set in advance. The preset value is set in advance.

[0063] In some embodiments, Figure 1 The input module 140 is used to receive and cache the data to be completed and the user questions. Figure 1 The retrieval and positioning module 130 is used to obtain similar data whose first similarity with the vector corresponding to the data to be completed is greater than a preset value.

[0064] In some embodiments, the data to be completed may be data input by a user, including the English name of a table, the Chinese name of a table, and the English name of a field. The user question may be input by a user.

[0065] As an example, a user's question is "Please complete the Chinese name and meaning of the field." The Chinese name and meaning of the field are missing data items. For example, as shown in Table 1, the missing data items to be completed are the Chinese name and meaning of the field.

[0066] Table 1

[0067]

[0068]

[0069] The AAA may be a Business Activity Monitoring (BAM) system.

[0070] In some embodiments, the data may be complete. When the data is complete, the user may not need to enter a user question. The user question is determined by comparing the data. The data comparison process includes:

[0071] The data is compared with initial data in a preset knowledge base. If the initial data in the preset knowledge base does not contain target data whose sub-data similarity with the target data item in the data exceeds a similarity threshold, the target data item is determined to be a missing data item. User questions are generated that include the missing data item.

[0072] In some embodiments, the professional knowledge base module includes a preset knowledge base, and the similar data is initial data in the preset knowledge base.

[0073] S220. Input the data, user questions, and similar data into the data completion model, use the adjustment matrix in the data completion model to adjust the first eigenvectors corresponding to the data, user questions, and similar data to obtain a second eigenvector, and determine the missing data based on the second eigenvector, where the missing data includes missing data items.

[0074] Among them, the data completion model is a large model trained in advance. The data completion model uses an adjustment matrix to fine-tune parameters, that is, a low-rank adaptation (LoRA) adaptation layer (also called a LoRA module) is injected into the large model.

[0075] In some embodiments, Figure 1 The large model module 150 includes a data completion model, which is used to determine missing data based on the data to be completed, user questions and similar data.

[0076] In some embodiments, the data completion model can be a self-developed large model or an open source large model. The open source large model is, for example, the Tongyi Qianwen large model.

[0077] In some embodiments, by using a fine-tuned large model, missing data such as data elements, data structures, data tables, fields, etc. in the data dictionary can be supplemented, and a clear and accurate definition can be provided for each data element, including data type, value range, business meaning, etc.

[0078] In some embodiments, using Figure 1 The vectorization module 120 in the

[15] vectorizes the user question, the data to be completed, and similar data. Vectorization is the process of converting text into numerical vectors. After text preprocessing (removing noise, spelling errors, word segmentation, and stop words), word embedding is used to convert it into a vector form in a high-dimensional space.

[0079] The large model first understands the user's question, analyzing its semantics and context to determine the data the user wishes to complete. Specifically, the large model performs preprocessing operations such as word segmentation and part-of-speech tagging on the user's question, converting the text into a machine-readable format. By calculating the semantic vectors of the words, analyzing the syntactic structure, and considering contextual relationships, the model accurately grasps the user's true intent. The question understanding phase outputs a structured question representation, providing strong support for subsequent information retrieval and answer generation.

[0080] It is understandable that by capturing the contextual information of the data and understanding the contextual content, large models can more accurately infer the implicit meaning in the data. With the help of reasoning capabilities, in addition to simple vocabulary matching and grammatical analysis, they can also infer the deeper meaning and intention behind the text based on clues and logical relationships in the data.

[0081] In some embodiments, the computational process after inputting data, user questions, and similar data into the big model includes:

[0082] 1) Feature Extraction: The large model first extracts features from the input data (including the data to be completed, user questions, and similar data), using methods such as word embedding and sentence embedding. This converts the text data into a representation in a high-dimensional vector space to capture the semantic and grammatical features of the text.

[0083] 2) Model Reasoning: Based on feature extraction, the fine-tuned large model performs reasoning, analyzing the context, grammatical structure, and semantic meaning of the input data to infer the data corresponding to missing or erroneous data items. The erroneous data items are target data items, which are data items in the preset knowledge base that do not have corresponding sub-data with a similarity greater than a similarity threshold.

[0084] 3) Answer generation: Based on the inference results, the large model outputs missing data. This missing data may include missing field Chinese names, field business meanings, etc.

[0085] As an example, the output of missing data is as follows:

[0086] Field Chinese name: Cost center code, cost center name; Field meaning: The company's cost center code, used for cost management and control.

[0087] In some embodiments, during feature extraction, the input data is first organized into a format that can be processed by the large model in preparation for entering the model inference phase. A string in a format that can be processed by the large model is, for example, "System: AAA, Table English Name: T_DIM_D_MGT_PB_GUEST_OPEN_ORGAN, Table Chinese Name: Enterprise Organization Structure Information Table, Field English Name: CostCentrCode, Field Chinese Name: , Field Meaning: ". Then, the string is segmented using the Tokenization algorithm to generate a token sequence. For example, [101, 1392, 4396, 8911]. The token sequence is then positionally encoded, and the positional encoding is used to capture the position information of the token in the input sequence. The positional encoding vector is such as [0.1, 0.2.....0.05]. Each token is added with the corresponding positional encoding through the embedding layer to obtain an embedding vector. For example, for the input "user_id", the corresponding embedding vector is: [0.2, -0.1, 0.4, ...]. Among them, the embedding vector is used as the first feature vector.

[0088] In some embodiments, during model inference, the embedding vector is first fine-tuned through the LoRA module, that is, a low-rank matrix is introduced through the LoRA module for efficient feature learning. During the LoRA module fine-tuning process, the embedding vector and the low-rank matrix are combined to produce adjusted features, namely the second feature vector. Next, using the self-attention mechanism, the model calculates the relationship between the input tokens to capture contextual information. Each token is determined to be important by calculating similarity (such as dot product) with other tokens. The contextual information passes through the feedforward network and an activation function such as ReLU is applied. This process expands and maps the features to a higher-dimensional space. After the self-attention layer and the feedforward network, a normalization layer (LayerNormalization) is used to improve training stability and accelerate convergence. Finally, output tokens are generated for missing data items. The output tokens are passed through the Softmax layer to generate a probability distribution, and the output tokens are such as [5120, 6789, 2345, ...]. When the answer is generated, the output token is converted into readable text to obtain the final generated result "Field Chinese name: Cost center code, field meaning: The company's cost center code, used for cost management and control."

[0089] In some embodiments, parameters of the data completion model are adjusted based on an adaptive learning mechanism.

[0090] S230: Combine the data and the missing data to obtain a first completion result.

[0091] The adjustment matrix is obtained by training based on training data in a preset field.

[0092] In some embodiments, Figure 1 The output module 160 in outputs the first completion result.

[0093] This embodiment of the application first improves the model's adaptability in a specific domain by injecting an adjustment matrix determined using domain-specific training data into the model. Secondly, based on the missing data items in the user's questions, similar data is used as the model input. After adjusting the features corresponding to the data to be completed, the user's questions, and the similar data, the missing data is determined. This improves the accuracy of the missing data, thereby improving the accuracy of the completion results.

[0094] Based on this, in some embodiments, such as Figure 3 As shown, before the above S220, the method may further include S310 to S360.

[0095] S310: Obtain initial data of a preset domain from a preset knowledge base.

[0096] in, Figure 1 The professional knowledge base module 110 in the system includes a preset knowledge base. From the preset knowledge base, initial data of the preset field is obtained, and the initial data is used to train the initial data completion model. The training process of the initial data completion model is as follows: Figure 1 The large model module 150 is shown as being implemented therein.

[0097] In some embodiments, the preset knowledge base includes BOMS domain data, and the construction process of the preset knowledge base is as follows:

[0098] (1) Data collection: Use SQL query, file parsing, and other methods to collect the original data of the data dictionary from the operator's BOMS domain, including field names, data types, etc. The BOMS domain includes the business domain (Business Support System), operation domain (Operation Support System), and management domain (Management Support System), where S represents the support system corresponding to each domain;

[0099] (2) Data cleaning: Since the original data may have problems such as inconsistent format, missing values, and outliers, regular expressions, data filling, and data smoothing are used to clean the data to ensure consistency in data format, handle missing values (null values) and outliers (data that do not meet format requirements), and delete problematic original data.

[0100] (3) Preset Knowledge Base Construction: Based on the operator's business rules and data standards, the original data of the data dictionary is sorted and a series of processes such as data cleaning, classification, labeling, and indexing are carried out to form a preset knowledge base. An example is shown in Table 2.

[0101] Table 2

[0102] Field English name UserID Field Chinese name User ID Field meaning A number that uniquely identifies a user.

[0103] It should be noted that the initial data is the data in the preset knowledge base that has been constructed, not the collected original data.

[0104] S320: Generate a data sample and a user question sample in a preset format according to the initial data, and obtain a similar data sample whose second similarity with the data sample is greater than a preset value.

[0105] Here, the preset format is set in advance.

[0106] In some embodiments, a portion of the initial data is selected, and the target sub-data in the portion of the data is deleted to obtain a data sample to be completed, and a user question sample including the data items in the target sub-data is generated.

[0107] S330. Input the data sample, user question sample and similar data sample into the initial data completion model, use the initial adjustment matrix in the initial data completion model to adjust the third eigenvectors corresponding to the data sample, user question sample and similar data sample to obtain the fourth eigenvector, and determine the predicted data based on the fourth eigenvector.

[0108] In some embodiments, an application programming interface (API) provided by a library such as Hugging Face is used to load a pre-trained initial data completion model. The initial data completion model can be the Tongyi Qianwen model.

[0109] Fine-tuning is a key step in the initial data completion model training. It aims to further train the pre-trained large model with a domain-specific dataset to improve the model's accuracy and efficiency when handling specific tasks (such as data dictionary completion). The core idea of fine-tuning with the LoRA module is to inject the LoRA module into specific layers of the pre-trained large model (such as fully connected layers and self-attention layers) (for example, defining low-rank matrices A and B). While keeping most of the parameters of the pre-trained large model unchanged, the model is fine-tuned by training a small number of low-rank matrices. The adjustment matrix is a low-rank matrix.

[0110] S340: Combine the data sample and the predicted data to obtain a second completion result.

[0111] S350: Determine the loss value of the initial data completion model based on the initial data and the second completion result.

[0112] In some embodiments, the LoRA module uses a fixed module architecture, meaning that the rank (r) of the low-rank matrix, the number of rows in the matrix, and the number of columns in the matrix remain unchanged. However, a fixed module architecture is difficult to adapt to changes in operator financial data, especially the emergence of new data tables and fields. Therefore, the LoRA module architecture uses Loss Function-Driven Model Architecture Search (LF-MAS) technology to achieve dynamic adjustment of its architecture.

[0113] S360. When the loss value does not meet the training stop condition, adjust the initial adjustment matrix, use the adjusted initial adjustment matrix to adjust the third eigenvectors corresponding to the data samples, user question samples and similar data samples, update the predicted data and loss value until the loss value meets the training stop condition, and obtain the data completion model.

[0114] In some embodiments, the specific training process of the data completion model is as follows:

[0115] 1) Loading data samples for a predefined domain: The predefined knowledge base contains initial data such as terms, concepts, entities, and their relationships within the predefined domain. This serves as an important source of knowledge for completing the data dictionary. You can retrieve the required data by calling the API through methods such as HTTP requests.

[0116] 2) Construct a training set: Based on the initial data, generate data samples and user question samples in a preset format, and obtain similar data samples whose second similarity with the data samples is greater than a preset value. Organize them into a training set format suitable for fine-tuning the LoRA module, and fine-tune the pre-trained large model on this basis.

[0117] 3) Inject the LoRA module into the pre-trained large model: Inject the LoRA module into specific layers (such as fully connected layers and self-attention layers) of the pre-trained large model.

[0118] 4) LoRa Module Parameter Update: During training, the original parameters of the pre-trained large model remain unchanged. The LoRa module parameters are updated. The gradients of the LoRa module parameters (i.e., matrices A and B) are calculated using the backpropagation algorithm. An optimization algorithm (such as the Adaptive Moment Estimation (Adam) algorithm) is applied to update the LoRa module parameters to minimize the loss. By adjusting the low-rank matrix, the original model parameters are fine-tuned.

[0119] The loss function of the initial data completion model can be a binary cross-entropy loss function (BCE). The binary cross-entropy loss function is shown in formula (1):

[0120] BCE=-(ylog(p)+(1-y)log(1-p)) (1)

[0121] Where y is the true label, which can be 0 or 1. P is the probability that the predicted label is 1.

[0122] It can be understood that the model building and application process includes: problem definition and data collection, data preprocessing, feature selection, feature engineering, model selection and construction, model training and evaluation, and prediction and result analysis. Specifically, problem definition and data collection: clearly define the prediction goal; collect relevant data: based on the prediction goal, collect relevant historical data. Data preprocessing: data cleaning, remove duplicate data, handle missing values, correct outliers, and ensure data quality. Feature selection: select features from historical data that are highly relevant to the prediction goal. Feature engineering: transform and scale the selected features to improve the model's predictive performance. Model selection and construction: select an appropriate prediction model based on the nature of the problem and the characteristics of the data. Model construction: use the selected model framework and the preprocessed data to build a prediction model. Model training and evaluation: train the model using the training dataset, adjust the model parameters through iterative optimization algorithms to ensure that the model better fits the training data, and evaluate the trained model using the validation or test dataset to verify the model's predictive and generalization capabilities. Prediction and result analysis: Use the trained model to predict new or future data and obtain the prediction results; result analysis, analyze the prediction results and evaluate their rationality and accuracy.

[0123] In some embodiments, an independent test set is used to evaluate the fine-tuned data completion model to examine its accuracy and efficiency in the data dictionary completion task. Evaluation metrics may include, but are not limited to, precision, recall, and F1 score. For example, using an independent test set, the Chinese names and meanings of the fields are extracted for comparison. The fine-tuned data completion model is used to generate the Chinese names and meanings of the fields. The results generated by the data completion model are then compared with the original Chinese names and meanings of the fields, and the precision, recall, and F1 scores are calculated.

[0124] The embodiment of the present application trains the data completion model through training data in a preset field, thereby improving the accuracy and efficiency of the model in processing specific tasks.

[0125] Based on this, in some embodiments, the preset field includes at least one sub-field, and the initial data includes initial sub-data of at least one sub-field;

[0126] The above S350 may specifically include:

[0127] Obtaining the loss weight of at least one sub-field;

[0128] Determine the sub-loss value of the initial data completion model based on the initial sub-data and the second completion result of each sub-field;

[0129] Determine the loss value of the initial data completion model based on the loss weight and sub-loss value.

[0130] The loss weights are set in advance and can be the same or different for each sub-field.

[0131] In some embodiments, through multi-task learning, in a multi-task learning scenario, designing a loss value that integrates multiple tasks can improve the performance of the model on each task.

[0132] Specifically, data samples generated from the initial sub-data of different sub-domains, user question samples, and corresponding similar data samples are input into the initial data completion model to determine their sub-loss values. The loss value of the initial data completion model is determined based on the loss weights and sub-loss values corresponding to the initial sub-data of different sub-domains.

[0133] In some embodiments, the loss value of the initial data completion model is determined based on the loss weight and the sub-loss values, including: based on the loss weight, calculating the weighted average of the sub-loss values to obtain the loss value of the initial data completion model.

[0134] In some embodiments, the loss value of the initial data completion model is determined based on the loss weight and the sub-loss values, including: based on the loss weight, calculating the weighted sum of the sub-loss values to obtain the loss value of the initial data completion model.

[0135] The embodiment of the present application uses a loss function weighting method to weight the losses of different categories according to the imbalance of data sets in different fields, so as to improve the model's recognition ability for minority categories.

[0136] Based on this, in some embodiments, in the above S360, adjusting the initial adjustment matrix may specifically include:

[0137] The initial adjustment matrix is adjusted using the maximum acquisition function.

[0138] In some embodiments, a loss function-driven model architecture search technique (LF-MAS) is used to dynamically adjust the LoRA module architecture. The specific process of dynamic adjustment of the LoRA module architecture is as follows:

[0139] Obtain the search space for LoRA module parameters (i.e., the adjustment matrix) and randomly select an initial LoRA module parameter as the starting point for the search. Use a Bayesian optimization algorithm to explore the search space for LoRA module parameters. Adopt a surrogate model (such as a Gaussian process) and construct a prediction distribution based on the existing LoRA module parameters and corresponding loss values in the search space. Based on the prediction distribution, the loss value under different LoRA module parameters can be estimated.

[0140] By maximizing the acquisition function, the next LoRA module parameter to be evaluated is selected. A Bayesian optimization algorithm is used to search within the search space, with the goal of maximizing the acquisition function, to find the next LoRA module parameter that is most likely to bring performance improvement for evaluation.

[0141] The evaluation results are recorded and the acquisition function and initial adjustment matrix are updated. The optimization is iterated until the predetermined number of evaluations is reached or the loss value satisfies the training stop condition. This new data point (the point in the search space that represents the adjustment matrix) is then added to the proxy model.

[0142] Among them, the search space refers to all possible parameters of the LoRA module, including the rank number (r) of the low-rank matrix, the number of rows of the matrix, the number of columns of the matrix, the matrix initialization method (random initialization, zero initialization, pre-training initialization, etc.), whether to apply regularization to the matrix, the learning rate setting, etc.

[0143] The choice of acquisition function depends on the evaluation objective and the nature of the optimization problem. The expected improvement (EI) acquisition function is used, which considers not only the probability of performance improvement but also the magnitude of performance improvement.

[0144] In some embodiments, among all explored LoRA module parameters, the LoRA module parameters with the smallest loss value on the validation set are selected as the final selected LoRA module architecture.

[0145] The embodiment of the present application maximizes the acquisition function and adjusts the initial adjustment matrix, taking into account not only the probability of performance improvement but also the magnitude of performance improvement, thereby improving the training efficiency of the model.

[0146] Based on this, in some embodiments, the initial adjustment matrix and the adjustment matrix are within the search space; before the above S360, the method may further include:

[0147] Get hardware performance metrics;

[0148] Generate a search space that meets preset conditions, where the preset conditions include hardware performance indicators.

[0149] Here, hardware performance indicators are related to the hardware system.

[0150] In some embodiments, a search space for LoRA module parameters is defined, and the parameter value range is set according to actual hardware and application requirements.

[0151] Specifically, within the range allowed by the hardware performance indicators, the maximum number of data points in the search space is ensured as much as possible, that is, when the search space range meets the hardware performance indicators, the distribution range of data points in the search space is as large as possible and the step size is as small as possible.

[0152] The embodiment of the present application determines the search space of the adjustment matrix based on the hardware performance indicators, associates the hardware performance indicators with the training process of the model, avoids the situation where the training process cannot be completed, and at the same time maximizes the use of resources and improves resource utilization.

[0153] Based on this, in some embodiments, in the above S210, obtaining similar data having a first similarity with the data greater than a preset value may specifically include:

[0154] Obtain initial data in a preset domain from a preset knowledge base;

[0155] A first similarity between the data and the initial data is calculated, and similar data having a first similarity greater than a preset value is selected from the initial data.

[0156] In some embodiments, similar data can be determined through search and positioning. Search and positioning is used to filter out the data that is relevant to the current task from a large amount of data, reducing the time and cost of manual screening, excluding irrelevant data, and only inputting relevant data into the model for processing, thereby improving completion efficiency.

[0157] Retrieval and positioning include vector retrieval and indexing. Specific computational steps include query understanding and preprocessing, building indexes and inverted indexes, retrieval algorithm design and implementation, sorting and result optimization, and result display and interaction.

[0158] Among them, vector retrieval includes: feature extraction, vector encoding, index construction and similarity calculation. Specifically, feature extraction: use deep learning models to extract features from text, images and other data, and convert them into high-dimensional vector representations. Vector encoding: encode the extracted features into vectors of fixed length for efficient calculation and storage. Index construction: use efficient index structures (such as Facebook AI Similarity Search (FAISS), Approximate Nearest Neighbors Oh Yeah, Annoy), etc.) to index vectors to speed up the retrieval process. Similarity calculation: during online queries, the query vector is compared with the vectors in the index library to find the document or data object corresponding to the most similar vector.

[0159] Indexes include: B+Tree indexes, Hash indexes, or full-text indexes. B+Tree is a balanced multi-way search tree that supports fast location, sequential access, and range queries. Hash indexes use a hash function to map key values to a location in the table, enabling fast equality queries. Full-text indexes are indexes for text fields. Based on an inverted index structure, they support complex queries and efficient text searches.

[0160] This embodiment of the application uses the missing data items in the user's question as the basis, uses the determined similar data as the input of the model, and adjusts the corresponding features of the data to be completed, the user's question, and the similar data to determine the missing data. This improves the accuracy of the missing data and, in turn, the accuracy of the completion results.

[0161] Based on this, in some embodiments, such as Figure 4 As shown, after the above S230, the method may further include:

[0162] S240: When no missing data is detected in the first completion result, extract keywords from the missing data using a regular expression;

[0163] S250: Combine the keyword and the data to obtain a third completion result.

[0164] In some embodiments, if no missing data is detected in the first completion result, regular expressions are used to extract keywords from the missing data. For example, "field Chinese name" and "field meaning" are extracted. The extracted data is combined with the data to be completed, and the results are shown in Table 3.

[0165] Table 3

[0166]

[0167]

[0168] In some embodiments, Figure 1 The output module 160 in outputs the third completion result.

[0169] In an embodiment of the present application, when the large model cannot automatically complete the missing data, keywords can be extracted through regular expressions to complete the missing data, thereby improving the accuracy of the completion results.

[0170] In the embodiment of the present application, first, an existing data dictionary is collected from the BOMS domain. Data cleaning and deduplication are performed on it to form initial data. Next, a pre-trained large model (such as Tongyi Qianwen) is selected, and the model architecture search technology is introduced to dynamically adjust the architecture of the LoRA module. The pre-trained large model is fine-tuned using the LoRA module after the architecture adjustment. Finally, the data dictionary is completed based on the fine-tuned large model.

[0171] The embodiment of the present application uses a large model to complete the data dictionary to solve the problems of high cost, high resource consumption and poor generalization ability encountered in the current data dictionary completion process. Fine-tuning is a key step, which aims to further train the pre-trained large model through data sets in specific fields to improve the accuracy and efficiency of the model when processing specific tasks (such as data dictionary information completion). The dynamic adjustment of the LoRA module architecture is achieved through model architecture search technology, and the Bayesian optimization algorithm is used to optimize the loss value to find the model architecture that best suits the given task and improve the adaptability of the model to different data sets.

[0172] In the embodiment provided in the present application, the large model module selects the Tongyi Qianwen large model as the basic model, adopts the LoRA module, and introduces a preset knowledge base for fine-tuning to adapt to the data characteristics of the preset field, so as to enhance the adaptability of the large model in a specific field. The loss function-driven model architecture search technology is used to realize the dynamic adjustment of the LoRA module architecture. A large model application for data dictionary completion is created, the average time consumption for single-table dictionary information collation is shortened, the dictionary information completeness rate of the data directory is improved, the dictionary information accuracy rate is improved, and the labor cost of data dictionary collation is saved. Through data collection and cleaning of the original data of the data dictionary, a preset knowledge base is constructed after ensuring its consistency. The basic large model is fine-tuned by the LoRA module and the dynamic structure optimization of the model architecture search technology driven by the loss function is introduced to improve the model accuracy. Finally, the fine-tuned large model is used to realize the effective completion of the data dictionary, and the Chinese name and meaning of the missing field are generated by the table field name entered by the user, thereby improving the data processing capability of the operator. Specifically, the model architecture search technology is used to achieve dynamic adjustment of the LoRA module architecture. It can automatically adjust the model architecture according to the characteristics of different data sets, making it more suitable for specific application scenarios and more adaptable to changes in the operator's financial data brought about by the growth of the operator's new business, data governance and optimization of internal management.

[0173] Based on the data completion method provided in the above embodiment, the present application also provides a specific implementation of a data completion device. Please refer to the following embodiment.

[0174] See also Figure 5 The data completion device 400 provided in the embodiment of the present application includes:

[0175] An acquisition module 410 is configured to acquire data, a user question, and similar data having a first similarity with the data greater than a preset value, wherein the user question includes a missing data item of the data;

[0176] a determination module 420 for inputting the data, user questions, and similar data into a data completion model, adjusting first eigenvectors corresponding to the data, user questions, and similar data using an adjustment matrix in the data completion model to obtain second eigenvectors, and determining missing data based on the second eigenvectors, where the missing data includes missing data items;

[0177] A combining module 430 is used to combine the data and the missing data to obtain a first completion result;

[0178] The adjustment matrix is obtained by training based on training data in a preset field.

[0179] Based on this, in some embodiments, the apparatus 400 may further include:

[0180] The acquisition module 410 is further configured to acquire initial data in a preset domain from a preset knowledge base before inputting the data, user questions, and similar data into the data completion model;

[0181] A generating module, configured to generate a data sample and a user question sample in a preset format according to the initial data, and to obtain a similar data sample having a second similarity with the data sample greater than a preset value;

[0182] The determination module 420 is further configured to input the data sample, the user question sample, and the similar data sample into the initial data completion model, adjust the third eigenvectors corresponding to the data sample, the user question sample, and the similar data sample using the initial adjustment matrix in the initial data completion model to obtain a fourth eigenvector, and determine the predicted data based on the fourth eigenvector;

[0183] The combining module 430 is further configured to combine the data sample and the predicted data to obtain a second completion result;

[0184] The determination module 420 is further configured to determine a loss value of the initial data completion model based on the initial data and the second completion result;

[0185] Determination module 420 is also used to adjust the initial adjustment matrix when the loss value does not meet the training stop condition, use the adjusted initial adjustment matrix to adjust the third eigenvectors corresponding to the data samples, user question samples and similar data samples, update the predicted data and loss value until the loss value meets the training stop condition, and obtain the data completion model.

[0186] Based on this, in some embodiments, the preset field includes at least one sub-field, and the initial data includes initial sub-data of at least one sub-field;

[0187] The determination module 420 may be specifically configured to:

[0188] Obtaining the loss weight of at least one sub-field;

[0189] Determine the sub-loss value of the initial data completion model based on the initial sub-data and the second completion result of each sub-field;

[0190] Determine the loss value of the initial data completion model based on the loss weight and sub-loss value.

[0191] Based on this, in some embodiments, the determination module 420 may be specifically configured to:

[0192] The initial adjustment matrix is adjusted using the maximum acquisition function.

[0193] Based on this, in some embodiments, the initial adjustment matrix and the adjustment matrix are within the search space; the apparatus 400 may further include:

[0194] The acquisition module 410 is further configured to acquire hardware performance indicators before adjusting the initial adjustment matrix;

[0195] The generation module is used to generate a search space that meets preset conditions, where the preset conditions include hardware performance indicators.

[0196] Based on this, in some embodiments, the acquisition module 410 may be specifically used to:

[0197] Obtain initial data in a preset domain from a preset knowledge base;

[0198] A first similarity between the data and the initial data is calculated, and similar data having a first similarity greater than a preset value is selected from the initial data.

[0199] Based on this, in some embodiments, the apparatus 400 may further include:

[0200] an extraction module for, after combining the data and the missing data to obtain a first completion result, extracting keywords from the missing data using a regular expression if no missing data is detected in the first completion result;

[0201] The combining module 430 is further configured to combine keywords and data to obtain a third completion result.

[0202] The various modules of the data completion device provided in the embodiment of the present application can implement the functions of the various steps of the data completion method provided above and achieve the corresponding technical effects. For the sake of brevity, they will not be repeated here.

[0203] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0204] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0205] The electronic device may include a processor 501 and a memory 502 storing computer program instructions.

[0206] Specifically, the processor 501 may include a central processing unit (CPU) or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0207] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 502 may include removable or non-removable (or fixed) media. Where appropriate, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0208] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0209] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any one of the data completion methods in the above embodiments.

[0210] In one example, the electronic device may further include a communication interface 503 and a bus 510. Figure 6 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0211] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0212] The bus 510 includes hardware, software, or both, and couples the components of the electronic device to each other. For example, and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Linear Predictive Coding (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (Peripheral Component Interconnect-X, PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VESA Local Bus, VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 510 may include one or more buses. Although the embodiments of the present application describe and illustrate a specific bus, the present application contemplates any suitable bus or interconnection. The electronic device can execute the data completion method of the embodiments of the present invention, thereby implementing the above-mentioned data completion method.

[0213] In addition, in conjunction with the data completion method in the above embodiments, embodiments of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the data completion methods in the above embodiments is implemented.

[0214] The present application also provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes each process of implementing any one of the above-mentioned data completion method embodiments.

[0215] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0216] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link via a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable read-only memories (EROMs), floppy disks, compact disc read-only memories (CD-ROMs), optical discs, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet and intranets.

[0217] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0218] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0219] The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application.

Claims

1. A data completion method, characterized in that: include: Acquire data and a user question, and similar data having a first similarity with the data greater than a preset value, wherein the user question includes a missing data item of the data; inputting the data, the user question, and the similar data into a data completion model, adjusting first eigenvectors corresponding to the data, the user question, and the similar data using an adjustment matrix in the data completion model to obtain a second eigenvector, and determining missing data based on the second eigenvector, where the missing data includes the missing data item; Combining the data and the missing data to obtain a first completion result; The adjustment matrix is obtained by training based on training data in a preset field.

2. The data completion method according to claim 1, characterized in that: Before inputting the data, the user question, and the similar data into a data completion model, the method further includes: Acquiring initial data of the preset domain from a preset knowledge base; Generate a data sample and a user question sample in a preset format according to the initial data, and obtain a similar data sample having a second similarity with the data sample greater than the preset value; Inputting the data sample, the user question sample, and the similar data sample into an initial data completion model, adjusting the third eigenvectors corresponding to the data sample, the user question sample, and the similar data sample using an initial adjustment matrix in the initial data completion model to obtain a fourth eigenvector, and determining predicted data based on the fourth eigenvector; Combining the data sample and the predicted data to obtain a second completion result; Determining a loss value of the initial data completion model according to the initial data and the second completion result; When the loss value does not meet the training stop condition, the initial adjustment matrix is adjusted, and the third eigenvectors corresponding to the data sample, the user question sample and the similar data sample are adjusted using the adjusted initial adjustment matrix, and the predicted data and the loss value are updated until the loss value meets the training stop condition, thereby obtaining the data completion model.

3. The data completion method according to claim 2, characterized in that: The preset field includes at least one sub-field, and the initial data includes initial sub-data of the at least one sub-field; The determining, based on the initial data and the second completion result, a loss value of the initial data completion model includes: obtaining a loss weight of the at least one sub-field; Determining a sub-loss value of the initial data completion model according to the initial sub-data of each sub-field and the second completion result; Determine the loss value of the initial data completion model based on the loss weight and the sub-loss value.

4. The data completion method according to claim 2, characterized in that: The adjusting the initial adjustment matrix includes: The initial adjustment matrix is adjusted by maximizing the acquisition function.

5. The data completion method according to claim 2, characterized in that: The initial adjustment matrix and the adjustment matrix are within a search space; and before adjusting the initial adjustment matrix, the method further includes: Get hardware performance metrics; The search space that meets a preset condition is generated, where the preset condition includes the hardware performance indicator.

6. The data completion method according to claim 1, characterized in that: Acquiring similar data having a first similarity with the data greater than a preset value, comprising: Acquiring initial data of the preset domain from a preset knowledge base; A first similarity between the data and the initial data is calculated, and similar data having the first similarity greater than a preset value is selected from the initial data.

7. The data completion method according to claim 1, characterized in that: After combining the data and the missing data to obtain a first completion result, the method further includes: When the missing data cannot be detected in the first completion result, extracting keywords from the missing data using a regular expression; The keyword and the data are combined to obtain a third completion result.

8. A data completion device, characterized in that: include: an acquisition module, configured to acquire data and a user question, and similar data having a first similarity with the data greater than a preset value, wherein the user question includes a missing data item of the data; a determination module, configured to input the data, the user question, and the similar data into a data completion model, adjust first eigenvectors corresponding to the data, the user question, and the similar data using an adjustment matrix in the data completion model to obtain a second eigenvector, and determine missing data based on the second eigenvector, the missing data including the missing data item; a combining module, configured to combine the data and the missing data to obtain a first completion result; The adjustment matrix is obtained by training based on training data in a preset field.

9. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data completion method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data completion method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the data completion method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Optical storage system control method based on AI data processing, program product and equipment

    CN121012086A

  • Map data completion method and device, equipment, storage medium and program product

    CN121387879A

  • Map data completion methods, devices, equipment, storage media, and software products

    CN121387879B