A multi-dimensional data matching training processing method based on multi-type database files

Abstract

The application discloses a kind of multi-dimensional data matching training processing methods based on multiple types database files, obtain the text data of new upload update in database file interior and carry out cleaning, the format of update text data is standardized and unified, update text data data is preprocessed, according to the similarity of different update text data segmented optimization deduplication, eliminate invalid data;Secondly, the matching weight coefficient of different categories in the existing multiple type database is analyzed by the update text data and the sample data under different categories, the data matching level framework is constructed, the update text data and its corresponding sample data category are matched and processed, at the same time, the keyword extraction is carried out according to the semantic information of index for the update text data with low matching weight coefficient, and it is added as a new category inside the database file, so as to realize the matching addition of data by effectively identifying the category variable, ensure the flexibility of data upload addition in database.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular to a multi-dimensional data matching training processing method based on multi-type database files. BACKGROUND

[0002] Multi-type data classification is the first step of data asset management. Whether it is cataloging, standardizing data assets, or data rights management, or providing data asset services, effective data classification is the primary task. Data classification is more from the perspective of business or data management, including industry dimension, business field dimension, data source dimension, sharing dimension, data opening dimension, etc. At the same time, according to these dimensions, data with the same attributes or characteristics is classified according to certain principles and methods.

[0003] Data category matching refers to ensuring that categories in different data sets are consistent in definition, label, coding or structure during data processing, integration or analysis, so as to enable effective comparison, integration or analysis. It is an important part of data preprocessing, aiming to eliminate data conflicts, analysis errors or result deviations caused by inconsistent categories. Among them, identifying category variables is a key step in data analysis and preprocessing, which refers to finding out those variables representing classification or grouping from the data set. The characteristics of category variables are discrete categories or labels with a limited number of values, rather than continuous numerical values. The purpose of identifying category variables is to better understand the data structure and lay the foundation for subsequent data cleaning, conversion and analysis.

[0004] Category variables refer to variables used to represent data classification or grouping, whose values are usually discrete, limited categories or labels. Currently, multi-type databases can only manually select categories to add data, and cannot accurately and automatically identify complex category variables in multi-type databases for data matching. The present application aims to automatically identify categories of newly updated text data in multi-type databases, dynamically match multiple categories of new text data, and analyze the matching weight coefficients of different added updated text and different categories in the multi-type database by updating the text data and sample data under different categories, thereby realizing effective identification of category variables for data matching and ensuring the flexibility of data uploading and adding in the database. SUMMARY

[0005] The present application aims to provide a multi-dimensional data matching training processing method based on multi-type database files to solve the problems in the prior art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0007] A multi-dimensional data matching training processing method based on a multi-type database file:

[0008] S1: Obtain newly uploaded updated text data inside the database file for cleaning, standardize the format of the updated text data, simultaneously perform multi-field similarity analysis on different updated text data, normalize data with a similarity higher than a threshold, segmentally optimize and remove duplicate data according to the similarity of different updated text data, and remove invalid data;

[0009] S2: Extract sample data inside an existing category in the multi-type database, extract and mark the data features of the optimized updated text data and the sample data, match and analyze the sample data content corresponding to the keyword field information in the optimized updated text data, and analyze the matching weight coefficient with the existing category in the multi-type database;

[0010] S3: Obtain the matching weight coefficient of the updated text data and all sample data, construct a data matching level framework including a complete matching level, a similar matching level, and a fuzzy matching level, obtain the updated text data content corresponding to the complete matching level, and match the updated text data with the corresponding sample data category;

[0011] S4: Obtain the updated text data content corresponding to the similar matching level and the fuzzy matching level, calculate the accurate matching ratio of different updated text data and corresponding sample data, when the accurate matching ratio is higher than a set threshold, perform full-range association matching training on the complete data content inside the updated text data and the corresponding sample data category, upload the matching training data to the data matching level framework, and perform secondary matching operation;

[0012] S5: When the accurate matching ratio is lower than the set threshold, index the semantic information of the uploaded updated data, extract keywords according to the indexed semantic information, and mark as a completely new category inside the database file.

[0013] Further setting: In S1, obtain newly uploaded updated text data inside the database file for cleaning, standardize the format of the updated text data, simultaneously perform multi-field similarity analysis on different updated text data, normalize data with a similarity higher than a threshold, segmentally optimize and remove duplicate data according to the similarity of different updated text data, and remove invalid data, including the following steps:

[0014] S11: Obtain updated text data to be uploaded inside the multi-type database, standardize the updated text data according to the standardized format, including unifying the text format, separator, removing white spaces, removing common meaningless words, and removing invalid text data;

[0015] S12: divide the updated text data in a unified format into different fields, and perform similarity comparison on different fields of the updated text data, and set the number of fields divided by the current two strings of updated text data respectively 、 , wherein the number of similar fields of the two strings of updated text data is k, and the set similarity threshold is , when , it is determined that the similarity of the current two strings of updated text data is high, the data fields of the two strings of updated text data which are not determined to be similar are recognized as synonyms, and similarity analysis is performed again after recognizing the synonyms, when the similarity threshold of the two strings of updated text data fields is equal to 100%, it is determined that the current two strings of updated text data are extremely similar, and one of the updated text data is removed for deduplication, when the similarity threshold of the two strings of updated text data fields is greater than or equal to the set threshold, the similar fields of the two strings of updated text data are normalized and combined, and the repeated fields are removed.

[0016] Further setting: in S2, sample data inside the existing category in the multi-type database is extracted, and the data features of the optimized updated text data and the sample data are extracted and labeled respectively, the keyword field information in the optimized updated text data is matched and analyzed with the sample data content, the matching weight coefficient with the existing category in the multi-type database is analyzed, and the following steps are further included:

[0017] S21: the optimized and deduplicated updated text data and the sample data inside the existing category in the multi-type database are obtained respectively, the internal syntax and semantics of the updated text data and the sample data are captured, the high-frequency words inside the updated text data are collected and labeled, the high-frequency words inside the updated text data are matched with the text semantics, and the data field with matching semantic information and high-frequency words is marked as a keyword field;

[0018] S22: the keyword field marked inside the updated text data is matched with different sample data in terms of semantic coincidence degree, the sample data with high coincidence degree is marked after preliminary screening, and the associated words of the keyword field are analyzed according to the sample data content of the preliminary screening, the associated words analyzed by matching each keyword field in the sample data are obtained, the keyword field with associated words is replaced with the original data text in the keyword field, the keyword field with replaced associated words is marked as a preset keyword field, the keyword field and the preset keyword field are matched with the preliminary screening sample data respectively, and the matching weight coefficient of different keyword fields and different sample data categories is analyzed.

[0019] Further setting: in S22, the following steps are included:

[0020] S22-1: The keyword field marked in the update text data is analyzed for coincidence with different sample data. When the keyword field marked in the update text data is completely coincident with the text data in the sample data, the matching weight coefficient of the keyword field is defined as 1. ;

[0021] S22-2: The preset keyword field analyzed according to the text content of the sample data is obtained, and the preset keyword field is analyzed for coincidence. When the preset keyword field replaced according to the keyword field in the update text data is completely coincident with the text data in the sample data, the matching weight coefficient of the keyword field is defined as 0.8 ;

[0022] S22-3: When the keyword field marked in the update text data is coincident with the text data in the sample data, the matching weight coefficient of the keyword field is defined as 0.6 ;

[0023] S22-4: When the replaced preset keyword field is coincident with the text data in the sample data, the matching weight coefficient of the keyword field is defined as 0.5 ;

[0024] 22-5: When the keyword field marked in the update text data is coincident with the preset keyword field and different sample data, the matching weight coefficient of the keyword field is defined as 0;

[0025] S22-6: The matching weight coefficients of each update text data and sample data are analyzed according to the number of keyword fields marked in the different update text data. When the matching weight coefficients of all keyword fields in a certain update text data and sample data are , it is checked whether the update text data and the sample data are repeated, and the update text data and the sample data are deleted when they are repeated.

[0026] Further setting: In S3, the matching weight coefficients of the update text data and all sample data are obtained, a data matching level framework is constructed, including a complete matching level, a similar matching level, and a fuzzy matching level, the content of the update text data corresponding to the complete matching level is obtained, and the update text data and the sample data corresponding thereto are matched. The method further comprises the following steps:

[0027] S31: The matching weight coefficients of different update text data are analyzed according to the keyword fields marked in the update text data and the matching weight coefficients of different sample data. According to different matching weight coefficients, a data matching level framework is constructed, and the matching weight coefficient of the update text data and the sample data is 0.8 ~ defined as the complete matching level, the matching weight coefficient of the updated text data and the sample data is 0.5 0.8 defined as the similar matching level, the matching weight coefficient of the updated text data and the sample data is 0-0.5 defined as the fuzzy matching level

[0028] S32: Obtain the updated sample data of the complete matching level and the sample data matched therewith, and according to the category of the matched sample data, match mark the category of the updated sample data of the complete matching level.

[0029] Further setting: in S4, obtain the updated text data content corresponding to the similar matching level and the fuzzy matching level, calculate the accurate matching ratio of different updated text data and the corresponding sample data, when the accurate matching ratio is higher than the set threshold value, perform full-range association matching training on the complete data content of the updated text data and the corresponding sample data category, upload the matching training data to the data matching level framework, and perform secondary matching operation, including the following steps:

[0030] Obtain the updated text data of the matching weight level of the similar matching level and the fuzzy matching level, set the number of keyword fields in a certain updated text data as M, and the matching weight coefficient of each keyword field and the preset keyword field and a certain matched sample data is respectively , , , , , Calculate the accurate matching ratio of the updated text data, set the accurate ratio threshold value of the current different updated text data and the sample data matching as 0.6 Set the accurate matching ratio of the current updated text data According to the formula:

[0031]

[0032] When the matching weight level of the updated text data matching the sample data is close to the matching level and the fuzzy matching level, and the accurate ratio of the updated text data to the matched sample data is greater than the set threshold, the complete data in the category corresponding to the current sample data is opened, the keyword field coincidence analysis of the complete text data in the current sample data category is performed on the updated text data with the current accurate ratio greater than the set threshold, the associated word analysis is performed again according to the content of the complete text data, the preset keyword field is constructed, the keyword field in the updated text data and the preset keyword field are matched with the complete text data in the full range, the matching weight coefficient of different keyword fields and the complete text data in the full range is analyzed, and the matching weight coefficient of the analyzed updated text data is sent to the data matching level framework for re-determination.

[0033] When the matching result is the complete matching level, the category of the updated text data is marked according to the determination result, and when the matching result is the close matching level, the accurate ratio of the updated text data to the matched sample data and the matching weight coefficient of the complete text data in the category of the corresponding sample data are sent to the remote PC end for manual discrimination.

[0034] Further setting: in S5, when the accurate matching ratio is lower than the set threshold, the semantic information of the uploaded updated data is indexed, the keyword is extracted according to the indexed semantic information, and the new category in the database file is marked, including the following steps:

[0035] S51: the updated text data with the accurate matching ratio lower than the set threshold is obtained, the keyword field in the updated text data is extracted, the updated text data is marked as a new category data in the multi-type database, and the keyword field with the matching weight coefficient of 0 in the sample data to be matched is extracted.

[0036] S52: the extracted keyword field is indexed according to the semantic information of the updated text data, the keyword field with prominent semantic information is marked, and is defined as the new category index of the updated text data.

[0037] Compared with the prior art, the beneficial effects of the present application are: aiming at automatic category recognition of the newly updated text data in the multi-type database, dynamic multi-category matching of the newly updated text data, preprocessing of the updated text data, segmented optimization and deduplication according to the similarity of different updated text data, and elimination of invalid data.

[0038] Secondly, by updating the text data and the sample data in different categories, the matching weight coefficients of the different added update texts and the existing different categories in the multi-type database are analyzed, a data matching level framework is constructed, the update text data and the corresponding sample data categories are matched, and the keyword extraction is performed on the update text data with low matching weight coefficients according to the semantic information of the index, and the new categories are added to the database file, so that the matching and addition of data are realized by effectively identifying the category variables, and the flexibility of data uploading and adding in the database is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments and in combination with the drawings.

[0040] Figure 1 A total step schematic diagram of the multi-dimensional data matching training processing method based on the multi-type database file of the present application;

[0041] Figure 2 A specific step schematic diagram of S1 in the multi-dimensional data matching training processing method based on the multi-type database file of the present application;

[0042] Figure 3 A specific step schematic diagram of S2 in the multi-dimensional data matching training processing method based on the multi-type database file of the present application;

[0043] Figure 4 A specific step schematic diagram of S22 in the multi-dimensional data matching training processing method based on the multi-type database file of the present application;

[0044] Figure 5 A specific step schematic diagram of S3 in the multi-dimensional data matching training processing method based on the multi-type database file of the present application;

[0045] Figure 6 A specific step schematic diagram of S5 in the multi-dimensional data matching training processing method based on the multi-type database file of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0047] Please refer to Figures 1-6 In the embodiments of the present application, a multi-dimensional data matching training processing method based on a multi-type database file:

[0048] S1: Obtain the newly uploaded update text data inside the database file for cleaning, standardize the update text data format, analyze the similarity of different update text data in multiple fields, normalize the data with a similarity higher than the threshold, segment the optimization and deduplicate according to the similarity of different update text data, and eliminate invalid data.

[0049] As shown in Figure 2 , step S1 needs further explanation.

[0050] S11: Obtain the update text data to be uploaded inside the multi-type database, and standardize the update text data according to the standardized format, including unifying the text format, separator, removing white space characters, and removing common meaningless words, and cleaning invalid text data.

[0051] S12: Divide the update text data after unifying the format into different fields, compare the similarity of different update text data fields, and set the number of fields divided by the current two strings of update text data respectively 、 , where the number of similar fields of the two strings of update text data is k, and the set similarity threshold is , when , it is determined that the similarity of the current two strings of update text data is high, the data fields of the two strings of update text data that are not determined to be similar are recognized as synonyms, and the similarity analysis is performed again after recognizing the synonyms, when the similarity threshold of the two strings of update text data fields is equal to 100%, it is determined that the current two strings of update text data are extremely similar, and one of the update text data is eliminated and deduplicated, when the similarity threshold of the two strings of update text data fields is greater than or equal to the set threshold, the similar fields of the two strings of update text data are normalized and combined, and the repeated fields are eliminated.

[0052] S2: Extract sample data inside the existing categories in the multi-type database, extract and label the data features of the optimized update text data and sample data, match and analyze the keyword field information in the optimized update text data with the sample data content, and analyze the matching weight coefficient with the existing categories in the multi-type database.

[0053] As shown in Figure 3 , step S2 needs further explanation.

[0054] S21: Obtain the updated text data after optimization and the sample data in the existing categories in the multi-type database respectively, capture the syntax and semantics in the updated text data and the sample data, label the high-frequency words in the updated text data, correspond the high-frequency words in the updated text data with the text semantics, and mark the data field with matching semantics and high-frequency words as a keyword field;

[0055] S22: Perform semantic matching of the keyword field marked in the updated text data with different sample data, mark the sample data with high coincidence degree, and perform associated word analysis on the keyword field according to the sample data content, obtain the associated words analyzed by each keyword field in the sample data, replace the original data text in the keyword field with the associated words, mark the keyword field with the replaced associated words as a preset keyword field, and perform coincidence matching of the keyword field and the preset keyword field with the sample data, and analyze the matching weight coefficient of different keyword fields and different sample data categories.

[0056] Among them, such as Figure 4 It also needs to be specifically pointed out that in step S22, the following steps are included:

[0057] S22-1: Perform coincidence analysis of the keyword field marked in the updated text data with different sample data, when the keyword field marked in the updated text data is completely coincident with the text data in the sample data, the matching weight coefficient of the keyword field is defined as 1 ;

[0058] S22-2: Obtain the preset keyword field analyzed according to the text content of the sample data, and perform coincidence analysis on the preset keyword field, when the preset keyword field replaced by the keyword field in the updated text data is completely coincident with the text data in the sample data, the matching weight coefficient of the keyword field is defined as 0.8 ;

[0059] S22-3: When the keyword field marked in the updated text data is semantically similar to the text data in the sample data, the matching weight coefficient of the keyword field is defined as 0.6 ;

[0060] S22-4: When the replaced preset keyword field is semantically similar to the text data in the sample data, the matching weight coefficient of the keyword field is defined as 0.5 ;

[0061] 22-5: When the keyword field of the internal label of the updated text data is not fuzzy coincided with the preset keyword field of the different sample data, the matching weight coefficient of the keyword field is defined as 0;

[0062] S22-6: According to the number of keyword fields of the internal label of the different updated text data, the matching weight coefficient of each updated text data and sample data is analyzed, wherein when the matching weight coefficient of all keyword fields in the internal label of the updated text data and the sample data is , the repeated data of the updated text data and the sample data is screened, and the repeated data is deleted.

[0063] S3: Obtain the matching weight coefficient of the updated text data and all sample data, construct a data matching level framework including a complete matching level, a similar matching level and a fuzzy matching level, obtain the content of the updated text data corresponding to the complete matching level, and match the updated text data and the corresponding sample data category;

[0064] As shown in Figure 5 , step S3 needs to be further explained;

[0065] S31: According to the matching weight coefficient of the keyword field of the internal label of the updated text data and the different sample data, the matching weight coefficient of the different updated text data is analyzed, and the data matching level framework is constructed according to the different matching weight coefficients. The matching weight coefficient of the updated text data and the sample data is 0.8 ~ , defined as a complete matching level, the matching weight coefficient of the updated text data and the sample data is 0.5 ~0.8 , defined as a similar matching level, the matching weight coefficient of the updated text data and the sample data is 0~0.5 , defined as a fuzzy matching level;

[0066] S32: Obtain the updated sample data of the complete matching level and the sample data matched therewith, and match the category of the updated sample data of the complete matching level according to the category of the matched sample data.

[0067] S4: Obtain the content of the updated text data corresponding to the similar matching level and the fuzzy matching level, calculate the accurate matching ratio of different updated text data and corresponding sample data, when the accurate matching ratio is higher than a set threshold, perform full-range associated matching training on the complete data content of the category of the updated text data and the corresponding sample data, upload the matching training data to the data matching level framework, and perform secondary matching operation;

[0068] Step S4 needs to be further explained;

[0069] obtaining the update text data with the matching weight level of the similar matching level and the fuzzy matching level, setting the number of keyword fields in the update text data as M, and the matching weight coefficient of each keyword field and the preset keyword field with the matched sample data is 、 、 、 、 、 calculating the accurate matching ratio of the update text data, setting the accurate matching ratio threshold of the current different update text data and the sample data as 0.6 , setting the accurate matching ratio of the current update text data According to the formula:

[0070]

[0071] When the accurate ratio of the update text data with the matching weight level of the similar matching level and the fuzzy matching level and the matched sample data is greater than the set threshold, the complete data in the category corresponding to the current sample data is opened, the update text data with the current accurate ratio greater than the set threshold and the full range complete text data under the current sample data category are analyzed for keyword field coincidence, the associated word analysis is performed again according to the content of the complete text data, the preset keyword field is constructed, and the keyword field in the update text data and the preset keyword field are matched with the full range complete text data, the matching weight coefficient of different keyword fields and the full range complete text data is analyzed, and the analyzed matching weight coefficient of the update text data is sent to the data matching level framework for re-determination.

[0072] When the matching result is the complete matching level, the category of the update text data is marked according to the determination result, and when the matching result is the similar matching level, the accurate ratio of the update text data and the matched sample data and the matching weight coefficient of the full range complete text data under the category of the corresponding sample data are sent to the remote PC end for manual discrimination.

[0073] S5: When the accurate matching ratio is lower than the set threshold, the uploaded data semantic information is indexed, the keyword is extracted according to the indexed semantic information, and is marked as a new category in the database file.

[0074] As shown in Figure 6 , step S5 needs to be further explained.

[0075] S51: obtaining the update text data matched with the sample data and having a precision ratio lower than a set threshold, extracting a keyword field in the update text data, marking the update text data as new category data in the multi-type database, and extracting the keyword field with a matching weight coefficient of 0 in the sample data to be matched;

[0076] S52: indexing the screened keyword field according to semantic information of the update text data, marking the keyword field with highlighted semantic information, and defining the keyword field as a new category index of the update text data.

[0077] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments but can be implemented in other embodiments without departing from the scope of the application. The embodiments are to be considered in all respects as illustrative and not restrictive, and the scope of the application is defined by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No reference signs in the claims should be considered as limiting the scope of the claims.

Claims

1. A multi-dimensional data matching training processing method based on a multi-type database file, characterized by: S1: obtaining newly uploaded updated text data inside the database file for cleaning, standardizing and unifying the format of the updated text data, and performing multi-field similarity analysis on different updated text data, normalizing data with a similarity higher than a threshold, and segmenting and optimizing the de-duplication according to the similarity of different updated text data, and eliminating invalid data; S2: extracting sample data inside the existing categories in the multi-type database, extracting and labeling the data features of the optimized updated text data and sample data, respectively, matching and analyzing the sample data content corresponding to the keyword field information in the optimized updated text data, and analyzing the matching weight coefficient with the existing categories in the multi-type database; S3: obtaining the matching weight coefficient of the updated text data and all sample data, constructing a data matching level framework, including a complete matching level, a similar matching level, and a fuzzy matching level, obtaining the updated text data content corresponding to the complete matching level, and matching the updated text data with its corresponding sample data category; S4: obtaining the updated text data content corresponding to the similar matching level and the fuzzy matching level, calculating the accurate matching ratio of different updated text data and corresponding sample data, when the accurate matching ratio is higher than the set threshold, performing full-range association matching training on the complete data content inside the updated text data and its corresponding sample data category, uploading the matching training data to the data matching level framework, and performing secondary matching operation; S5: when the accurate matching ratio is lower than the set threshold, indexing the semantic information of the uploaded updated data, extracting keywords according to the indexed semantic information, and marking as a completely new category inside the database file. 2.The multi-dimensional data matching training processing method based on multi-type database files according to claim 1, characterized in that In S1, the newly uploaded updated text data inside the database file is cleaned, the format of the updated text data is standardized and unified, and multi-field similarity analysis is performed on different updated text data, the data with a similarity higher than a threshold is normalized, and the invalid data is eliminated by segmenting and optimizing the de-duplication according to the similarity of different updated text data, including the following steps: S11: obtaining the updated text data to be uploaded inside the multi-type database, normalizing the updated text data according to the standardized format, including unifying the text format, separator, removing white space, and removing common meaningless words, and cleaning invalid text data; S12: The updated text data in a unified format is divided into different fields, and similarity comparison is performed on different fields of the updated text data. The number of fields divided for the current two strings of updated text data is respectively , , wherein the number of similarity fields of the two strings of updated text data is k, and the set similarity threshold is , when , it is determined that the similarity of the current two strings of updated text data is high. Synonyms are recognized in the data fields of the two strings of updated text data which are not determined to be similar. After the synonyms are recognized, similarity analysis is performed again. When the similarity threshold of the fields of the two strings of updated text data is equal to 100%, it is determined that the current two strings of updated text data are extremely similar. One of the updated text data is removed for deduplication. When the similarity threshold of the fields of the two strings of updated text data is greater than or equal to the set threshold, the similar fields of the two strings of updated text data are normalized and combined, and the repeated fields are removed. 3.The multi-dimensional data matching training processing method based on multi-type database files according to claim 1, characterized in that In S2, sample data inside the existing categories in the multi-type database is extracted, the data features of the optimized updated text data and sample data are extracted and labeled, respectively, the sample data content corresponding to the keyword field information in the optimized updated text data is matched and analyzed, and the matching weight coefficient with the existing categories in the multi-type database is analyzed, further comprising the following steps: S21: Obtain the updated text data after optimization and the sample data in the existing category in the multi-type database respectively, capture the syntax and semantics in the updated text data and the sample data, label the high-frequency words in the updated text data, correspond the high-frequency words in the updated text data with the text semantics, and mark the data field with matching semantics and high-frequency words as a keyword field; S22: Perform semantic matching of the keyword field marked in the updated text data with different sample data, mark the sample data with high coincidence degree, and perform associated word analysis on the keyword field according to the sample data content, obtain the associated words analyzed by each keyword field in the sample data, mark the keyword field with associated words, replace the original data text in the keyword field with associated words in advance, mark the keyword field with replaced associated words as a preset keyword field, and perform coincidence matching of the keyword field and the preset keyword field with the sample data, and analyze the matching weight coefficient of different keyword fields and different sample data categories.

4. The multi-dimensional data matching training processing method based on multi-type database files according to claim 3, characterized in that In S22, the following steps are included: S22-1: The keyword field of the internal label of the update text data is analyzed for coincidence with different sample data, and when the keyword field of the internal label of the update text data is completely coincident with the text data inside the sample data, the matching weight coefficient of the keyword field is defined as ; S22-2: Obtain the preset keyword field according to the text content analysis of the sample data, and perform coincidence analysis on the preset keyword field. When the preset keyword field replaced according to the keyword field inside the updated text data is completely coincident with the text data inside the sample data, the matching weight coefficient of the keyword field is defined as 0.8 ; S22-3: When the keyword field of the updated text data internal label is similar to the text data semantics inside the sample data, the matching weight coefficient of the keyword field is defined as 0.6 ; S22-4: When the replaced preset keyword field is similar to the text data semantics inside the sample data, the matching weight coefficient of the keyword field is defined as 0.5 ; 22-5: When the keyword field marked in the updated text data and the preset keyword field are fuzzy coincident with different sample data, the matching weight coefficient of the keyword field is defined as 0. S22-6: According to the number of keyword fields marked in the different update text data, the matching weight coefficient of each update text data and the sample data is analyzed, wherein when all the keyword fields in the update text data are matched with the sample data , the update text data and the sample data are screened for duplication, and the duplicate data is deleted.

5. The multi-dimensional data matching training processing method based on multi-type database files according to claim 1, characterized in that In S3, the matching weight coefficient of the updated text data and all sample data is obtained, a data matching level framework is constructed, including a complete matching level, a similar matching level, and a fuzzy matching level, the updated text data content corresponding to the complete matching level is obtained, the updated text data is matched with the corresponding sample data category, and the following steps are further included: S31: According to the analysis of the matching weight coefficients of different update text data according to the matching of the keyword field of the update text data internal label and different sample data, the data matching level framework is constructed according to the different matching weight coefficients, and the matching weight coefficient of the update text data and the sample data is 0.8 , defined as complete matching level, the matching weight coefficient of the update text data and the sample data is 0.5 ~0.8 , defined as similar matching level, the matching weight coefficient of the update text data and the sample data is 0~0.5 , defined as fuzzy matching level;​ S32: Obtain the updated sample data of the complete matching level and the sample data matched therewith, and match and mark the updated sample data category of the complete matching level according to the category of the matched sample data.

6. The multi-dimensional data matching training processing method based on multi-type database files according to claim 1, characterized in that In S4, the updated text data content corresponding to the similar matching level and the fuzzy matching level is obtained, the accurate matching ratio of different updated text data and corresponding sample data is calculated, when the accurate matching ratio is higher than a set threshold, the updated text data is associated with the complete data content in the corresponding sample data category for full-range matching training, the matching training data is uploaded to the data matching level framework, and secondary matching operation is performed, including the following steps: The update text data with the matching weight level of the similar matching level and the fuzzy matching level is acquired, the number of keyword fields in the update text data is set as M, and the matching weight coefficients of each keyword field and the preset keyword field and the matching sample data are respectively , , , , , , the accurate matching ratio of the update text data is calculated, the accurate matching ratio threshold of the current different update text data and the sample data is set as 0.6 , the accurate matching ratio of the current update text data is set as , according to the formula: When the matching weight level of the updated text data matching the sample data is close to the matching level and the fuzzy matching level, and the accurate ratio of the updated text data to the matched sample data is greater than the set threshold, the complete data in the category corresponding to the current sample data is opened, the keyword field coincidence analysis is performed on the updated text data with the current sample data category under the full range complete text data, the associated word analysis is performed again according to the content of the complete text data, the preset keyword field is constructed, the keyword field in the updated text data and the preset keyword field are matched with the full range complete text data, the matching weight coefficient of different keyword fields and the full range complete text data is analyzed, and the matching weight coefficient of the analyzed updated text data is sent to the data matching level framework for re-determination. When the matching result is the complete matching level, the category of the updated text data is marked according to the determination result, and when the matching result is the close matching level, the accurate ratio of the updated text data to the matched sample data and the matching weight coefficient of the full range complete text data under the corresponding sample data category are sent to the remote PC end for manual discrimination.

7. The method of claim 1, wherein the method further comprises: In S5, when the accurate matching ratio is lower than the set threshold, the uploaded updated data semantic information is indexed, the keyword is extracted according to the indexed semantic information, and the new category in the database file is marked, including the following steps: ​ S51: The updated text data with the accurate matching ratio lower than the set threshold is obtained, the keyword field in the updated text data is extracted, the updated text data is marked as a new category data in the multi-type database, and the keyword field with the matching weight coefficient of 0 in the sample data to be matched is extracted; S52: The screened keyword field is indexed according to the semantic information of the updated text data, the keyword field with the semantic information is marked, and the new category index of the updated text data is defined.

Images