Mapping template generation method, device and equipment

Abstract

The invention discloses a mapping template generation method, device and equipment, and the method comprises the steps: obtaining a first data set corresponding to a source message, and a second data set corresponding to a target message; based on a classification result obtained by classifying the first data set and the second data set, a mapping information set is constructed, and mapping fragments in the mapping information set are used for mapping the first data of the same type into second data; determining at least one matching mapping fragment matched with the field of the source message in the mapping information set; and generating a mapping template for mapping the source message based on the at least one matched mapping fragment, wherein the mapping template is used for mapping the source message into the target message.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data exchange, and relate to but are not limited to a mapping template generation method, device and equipment. Background Art

[0002] The existing technical solution is. As enterprise application systems become increasingly complex, the workload of B2B system integration is increasing. Especially for companies with a large number of overseas customers, the workload of system integration development is large and cumbersome during data exchange due to the different industries of customers and the inconsistent electronic data interchange (EDI) standards used. The traditional manual development of integration mapping templates is inefficient and has uneven quality. How to improve the generation of mapping templates has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In view of this, embodiments of the present application provide a method, apparatus and device for generating a mapping template.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for generating a mapping template, comprising:

[0006] Acquire a first data set corresponding to the source message and a second data set corresponding to the target message;

[0007] Based on the classification results obtained by classifying the first data set and the second data set, a mapping information set is constructed, wherein the mapping fragments in the mapping information set are used to map the first data of the same category to the second data;

[0008] Determining in the mapping information set at least one matching mapping fragment that matches a field of the source message;

[0009] A mapping template for mapping the source message is generated based on at least one matching mapping fragment, and the mapping template is used to map the source message to a target message.

[0010] In a second aspect, an embodiment of the present application provides a mapping template generating device, including:

[0011] An acquisition module, used to acquire a first data set corresponding to a source message and a second data set corresponding to a target message;

[0012] A classification module, configured to construct a mapping information set based on a classification result obtained by classifying the first data set and the second data set, wherein a mapping segment in the mapping information set is used to map the first data of the same category to the second data;

[0013] A matching module, used to determine at least one matching mapping fragment matching a field of a source message in a mapping information set;

[0014] A generating module is used to generate a mapping template for mapping a source message based on at least one matching mapping fragment, wherein the mapping template is used to map the source message to a target message.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the above method is implemented when the processor executes the program.

[0016] In a fourth aspect, an embodiment of the present application provides a storage medium storing executable instructions for implementing the above method when executed by a processor.

[0017] In a fifth aspect, a computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps in the above method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of an implementation flow of a mapping template generation method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of an implementation process of constructing a mapping information set provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of an implementation flow of a mapping template generation method provided in an embodiment of the present application;

[0021] Figure 4A A schematic diagram of the implementation flow of the mapping template generation method provided in the embodiment of the present application;

[0022] Figure 4B A flowchart of a method for generating a mapping template is provided for an embodiment of the present application;

[0023] Figure 4C A schematic diagram of a flow chart of deduplication of an existing mapping template provided in an embodiment of the present application;

[0024] Figure 4D A schematic diagram of a sample data structure provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of a mapping template generating device provided in an embodiment of the present application;

[0026] Figure 6 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the embodiments of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0028] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0029] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0031] The present application embodiment provides a method for generating a mapping template, such as Figure 1 As shown, the method includes:

[0032] Step S110: Acquire a first data set corresponding to the source message and a second data set corresponding to the target message;

[0033] Here, B2B system integration refers to connecting the business processes, data and information between different enterprises or systems through standardized interfaces and protocols to achieve information sharing and process automation. The source message is the message provided by the source system, and the target message is the message provided by the target system.

[0034] During the implementation process, the data between the source system and the target system are connected through interfaces and protocols, and the source message is converted into the target message to achieve information sharing.

[0035] For example, the source message includes the following content:

[0036] “<OrderRequestHeader orderID="PO24032800723"type="new"> ”

[0037] Used to describe the order number of the source system. The source tag is "OrderRequestHeader".

[0038] The target message includes the following contents:

[0039] “<PO_NUMBER_SOLD> PO24032800723< / PO_NUMBER_SOLD> ”

[0040] Used to describe the order number of the target system. The target tag is "PO_NUMBER_SOLD".

[0041] During the implementation process, a mapping relationship between the source tag "OrderRequestHeader" and the target tag "PO_NUMBER_SOLD" can be established to achieve conversion between the source message and the target message.

[0042] The first data set corresponding to the source message is the data set used by the source system; the second data set corresponding to the target message is the data set used by the target system. Since the source system and the target system have different definitions of field usage, the first data set is different from the second data set.

[0043] In some embodiments, a data set may include label data and / or attribute data.

[0044] Step S120: construct a mapping information set based on the classification results obtained by classifying the first data set and the second data set, wherein the mapping fragments in the mapping information set are used to map the first data of the same category to the second data;

[0045] During implementation, the classification model may be used to classify the first data in the first data set and the second data in the second data set, and the data used to describe the same meaning may be classified into the same category to obtain a classification result.

[0046] For example, fields such as "phone", "telephone", "mobile phone", "cell phone", and "phone number" used to describe data in telephone number fields can be classified into the same category.

[0047] The mapping information set may be composed of multiple mapping fragments that can map the same type of first data into second data.

[0048] Step S130: Determine in the mapping information set at least one matching mapping fragment that matches the field of the source message;

[0049] In the implementation process, the field in the source message can be used to find the matching mapping fragment matching the field in the mapping information set. Since the source message may include multiple different fields, the matching mapping fragments corresponding to the multiple different fields can be obtained.

[0050] Step S140: Generate a mapping template for mapping the source message based on the at least one matching mapping fragment, wherein the mapping template is used to map the source message to the target message.

[0051] Here, the mapping template is a template used to map a source message and a target message. For example, it can be a template used to translate a source system order into a target system order.

[0052] In the embodiment of the present application, firstly, a first data set corresponding to the source message and a second data set corresponding to the target message are obtained; then, a mapping information set is constructed based on the classification results obtained by classifying the first data set and the second data set; at least one matching mapping fragment matching the field of the source message is determined in the mapping information set; finally, a mapping template for mapping the source message is generated based on at least one matching mapping fragment, and the mapping template is used to map the source message to the target message. In this way, the mapping template for converting the source message to the target message can be generated by using the classification method, effectively improving the efficiency of integrated mapping development, and the method for generating the mapping template can be efficiently reused, and the quality of the generated mapping template is controllable.

[0053] In some embodiments, in the above step S120, "building a mapping information set based on the classification results obtained by classifying the first data set and the second data set", such as Figure 2 As shown, this can be achieved by following the steps below:

[0054] Step S210: split the first data set and the second data set based on field types to obtain a first field library corresponding to the first data set and a second field library corresponding to the second data set;

[0055] In some embodiments, the collected raw data in the first data set and the second data set may be cleaned and normalized first, and the processed data may be stored in a structured data set to provide high-quality input for subsequent discrete models.

[0056] During implementation, the data may be split based on different field types, such as tags, attributes, keywords, etc., to obtain a first field library and a second field library.

[0057] Step S220: classify the first field library and the second field library using a classification model to obtain the classification result;

[0058] Here, the classification model is used to classify the samples in the data set into different categories, including Naive Bayes classifier, decision tree, support vector machine, neural network and XGBoost model. Among them, XGBoost model, full name eXtremeGradient Boosting, is an ensemble learning algorithm based on Gradient Boosting Decision Tree (GBDT). It supports various types of tasks, including classification, regression and sorting.

[0059] During the implementation process, the classification model can be trained using the historical first field library, the historical second field library, and existing mapping templates (standard and custom), and then the trained classification model can be used to classify the updated first field library and second field library.

[0060] Step S230: construct the mapping information set based on the classification result.

[0061] Here, a mapping segment can be established based on each category in the classification result. The mapping segments corresponding to all categories in the classification result are mapping information sets.

[0062] In the embodiment of the present application, the first data set and the second data set are firstly split based on the field type to obtain the first field library corresponding to the first data set and the second field library corresponding to the second data set; then the first field library and the second field library are classified using the classification model to obtain the classification result; finally, a mapping information set is constructed based on the classification result. In this way, the first data set and the second data set can be split using different field types to obtain the first field library and the second field library, and the first field library and the second field library can be classified to obtain the classification result of the fields representing the same meaning, and a mapping information set for mapping the source message and the target message can be established based on the classification result.

[0063] In some embodiments, the above step S220 "classifying the first field library and the second field library using the classification model to obtain the classification result" can be implemented by the following steps:

[0064] Step 221: Encode the fields in the first field library and the second field library to obtain the encoded first field library and the second field library;

[0065] Here, when dealing with a dataset containing categorical data (such as text, category labels, etc.), the fields can be encoded. The purpose of encoding is to convert these categorical data into numerical form so that machine learning algorithms can process them.

[0066] During implementation, one-hot encoding can be used to binary encode discrete features (fields) and input into the XGBoost model for training.

[0067] Step 222: Classify the fields in the first field library and the second field library that have completed the encoding using a classification model to obtain the classification result.

[0068] Here, the encoded fields are input into the classification model, which enables the classification model to perform the classification task more efficiently.

[0069] In the embodiment of the present application, the fields in the first field library and the second field library are encoded to obtain the encoded first field library and the second field library. In this way, the fields in the encoded first field library and the second field library can be classified using the classification model to obtain the classification result.

[0070] In some embodiments, the field type includes at least one of the following: a tag field, an attribute field, and a keyword field; the above step S210 "splitting the first data set and the second data set based on the field type to obtain a first field library corresponding to the first data set and a second field library corresponding to the second data set" can be implemented by the following steps:

[0071] Step 211: split the first data set into at least one of the following fields based on the field type: a first tag field, a first attribute field, and a first keyword field; split the second data set into at least one of the following fields: a second tag field, a second attribute field, and a second keyword field;

[0072] Here, tags (or elements) are used to define the structure and content of data. Each tag corresponds to a specific part or component defined in the Commerce Extensible Markup Language (cXML) specification. The cXML specification is an extensible markup language (XML) XML used for e-commerce transactions. These tags are organized according to the cXML specification to describe the specific content of business transactions. Attributes provide additional information about the tag and can appear at the beginning of the tag in the form of "attribute name = attribute value" and are contained in the starting angle brackets of the tag. Keywords can be attribute values ​​or data within a specific tag.

[0073] During implementation, the data in the first data set and the second data set can be split based on the file formats of the source system and the target system. For example, if the source system and the target system are cXML, the first data set can be split into at least one of the following fields: a first tag field, a first attribute field, and a first keyword field; and the second data set can be split into at least one of the following fields: a second tag field, a second attribute field, and a second keyword field.

[0074] Step 212: Clear the first keyword field and the second keyword field;

[0075] Here, since the keyword can be an attribute value or data in a specific tag, that is, it is irrelevant to the mapping relationship, the first keyword field and the second keyword field can be cleared.

[0076] In some embodiments, interference items in the first data set and the second data set can also be removed, such as filtering, converting abbreviations, and removing stop words (-, _, ", ', etc.).

[0077] Step 213: obtain the first field library based on the first label field and / or the first attribute field corresponding to the first data set, and obtain the second field library based on the first label field and / or the first attribute field corresponding to the second data set.

[0078] In the embodiment of the present application, firstly, the first data set is split into at least one of the following fields based on the field type: the first label field, the first attribute field and the first keyword field, and the second data set is split into at least one of the following fields: the second label field, the second attribute field and the second keyword field; then the first keyword field and the second keyword field are cleared; finally, the first field library is obtained based on the first label field and / or the first attribute field corresponding to the first data set, and the second field library is obtained based on the first label field and / or the first attribute field corresponding to the second data set. In this way, the data set can be differentiated and interference items can be removed to construct the first field library and the second field library including valid fields for classification.

[0079] In some embodiments, in the above step S120, "building a mapping information set based on the classification results obtained by classifying the first data set and the second data set" further includes the following steps:

[0080] Step S240: in the process of constructing the mapping information set based on the classification result, determining the mapping times of each mapping fragment;

[0081] In the implementation process, a mapping segment is used to map data in a first data set to data in a second data set. For example, the first mapping segment can map "phone" in the first data set to "phonenumber", and the mapping times of the first mapping segment can be 20 times, and the second mapping segment can map "phone" in the first data set to "telephone", and the mapping times of the second mapping segment can be 2 times.

[0082] Step S250: Determine the weight of the corresponding mapping segment based on the mapping times.

[0083] Here, the greater the number of mappings, the greater the weight of the mapping segment can be determined.

[0084] During implementation, a conversion relationship between the mapping times and the weight can be established to determine the weight of the corresponding mapping segment based on the mapping times. For example, the weight of the mapping segment with a mapping time of 20 can be determined to be 20.

[0085] In the embodiment of the present application, the mapping times of each mapping fragment are determined in the process of constructing the mapping information set based on the classification results, and then the weight of the corresponding mapping fragment is determined based on the mapping times. In this way, the corresponding weight can be marked for each mapping fragment, so as to further select the mapping fragment using the weight.

[0086] In some embodiments, the above step S130 "determining at least one matching mapping fragment matching the field of the source message in the mapping information set" can be implemented by the following steps:

[0087] Step 131: Determine, in the mapping information set, a plurality of matching mapping fragments that match the first field in the source message;

[0088] Here, since the mapping relationship can be a one-to-many or many-to-many relationship, more than one mapping fragment matching the first field in the source message can be determined. For example, when the first field is "phone", the first mapping fragment that can match "phone" can map "phone" in the first data set to "phone number", and the mapping times of the first mapping fragment can be 20 times, and the second mapping fragment that matches "phone" can map "phone" in the first data set to "telephone", and the mapping times of the second mapping fragment can be 2 times.

[0089] Step 132: Based on the weights of the multiple matching mapping segments, determine the matching mapping segment with the highest weight as the matching mapping segment of the first field.

[0090] In the implementation process, the matching mapping information field with the highest weight can be the matching mapping information field of the first field. For example, the mapping times of the first mapping segment can be 20 times, and the mapping times of the second mapping segment can be 2 times. The first mapping segment can be used to map the first field. For example, if the first field is "phone", the first mapping segment can be used to map it to "phone number".

[0091] In the embodiment of the present application, firstly, multiple matching mapping fragments matching the first field in the source message are determined in the mapping information set; then, based on the weights of the multiple matching mapping fragments, the matching mapping information field with the highest weight is determined as the matching mapping information field of the first field. In this way, the weight can be used to select the most appropriate mapping fragment as the matching mapping fragment of the first field.

[0092] In some embodiments, the above step S140 "generating a mapping template for mapping the source message based on the at least one matching mapping fragment" is as follows: Figure 3 As shown, this can be achieved by following the steps below:

[0093] Step S310: Determine that at least one target first field of the source message does not match any mapping fragment in the mapping information set;

[0094] Here, the source message may contain a newly added field, and the newly added target first field cannot determine a matching mapping fragment in the mapping information set.

[0095] Step S320: Determine, in the mapping information set, a similarity mapping segment that matches the target first field using a similarity algorithm;

[0096] In the implementation process, a similarity algorithm (combined with the word vector model Word2Vec, EDI corpus, cosine similarity, etc.) can be used to recommend a close mapping relationship, i.e., a similarity mapping segment, for the newly added target first field. For example, when the newly added target first field is "TEL", the field "telephone" similar to "TEL" can be determined by the similarity algorithm, and the mapping segment matching "telephone" can be used as the similarity mapping segment of "TEL".

[0097] Step S330: Generate a mapping template for mapping the source message based on the at least one matching mapping segment and the similarity mapping segment.

[0098] In some embodiments, a target mapping segment corresponding to the target first field can be generated based on the similarity mapping segment. For example, "telephone" in the mapping segment matched by "telephone" can be modified to "TEL" as the target mapping segment of "TEL". A mapping template for mapping a source message with a newly added field can be generated based on the mapping segment and at least one matching mapping segment.

[0099] In the embodiment of the present application, it is first determined that at least one target first field of the source message does not match any mapping fragment in the mapping information set; then a similarity mapping fragment matching the target first field is determined in the mapping information set using a similarity algorithm; finally, a mapping template for mapping the source message is generated based on the at least one matching mapping fragment and the similarity mapping fragment. In this way, in the case of a newly added field, a similarity algorithm can be used to match the similarity mapping fragment of the newly added field, effectively improving the matching efficiency without affecting the matching accuracy.

[0100] In some embodiments, the mapping template generation method further includes the following steps:

[0101] Step S340: Generate a target mapping segment matching the target first field based on the similarity mapping segment and the target first field;

[0102] In some embodiments, a target mapping segment corresponding to the target first field may be generated based on the similarity mapping segment. For example, "telephone" in the mapping segment matching "telephone" may be modified to "TEL" as the target mapping segment of "TEL".

[0103] Step S350: Update the target mapping fragment to the mapping information set.

[0104] During implementation, the target mapping fragment may be added to the mapping information set to adapt to the newly added target field and expand the mapping information set.

[0105] In the embodiment of the present application, the similarity mapping fragment is updated to the mapping information set. In this way, the similarity reflow model can be used for automatic learning to improve the efficiency of integrated mapping development.

[0106] Figure 4A A flowchart of a mapping template generation method provided in an embodiment of the present application is shown as follows: Figure 4A As shown, the template can be generated by following the steps below:

[0107] Step S410: data collection and processing;

[0108] Data collection in the data preparation phase includes: historical raw data, historical target data, existing templates (standard and custom). Collect and archive structured data from multiple enterprise systems;

[0109] Data processing includes: cleaning and normalizing the collected raw data; storing the processed data in a structured data set to provide high-quality input for subsequent modeling.

[0110] Step S420: model training;

[0111] During implementation, the model can be trained using historical raw data, historical target data, and existing templates (standard and custom).

[0112] Step S430: model prediction;

[0113] Here, model prediction is the core step of template generation, which can be based on XGBoost classification prediction, use one-hot binary encoding of discrete features, train model data, analyze and create a mapping model between source data and target system data.

[0114] In some embodiments, the model can be used to calculate and, for data that has not been successfully matched, a similarity algorithm (combined with the word vector model Word2Vec, EDI corpus, cosine similarity, etc.) can be used to recommend similar results and fill in the model.

[0115] During the implementation process, the XGBoost model can be optimized in the following ways: enhance the core EDI fields that are most critical to the feature data, such as label case conversion and random letter insertion; adjust the model training method, such as adjusting the depth of the tree; continuously train the model and improve the quality of the generated template through similarity algorithm recommendations and the return of adjusted results.

[0116] In some embodiments, the following similarity algorithm optimizations can be performed: based on industry issues and standards, historical data and EDI standards can be used as corpora; the pre-trained word vector model GoogleNews-vectors-negative300.bin provided by Google can be used as a supplement to increase the recall rate. For data that does not match the results, a three-party model is used to match synonyms, and the results are limited according to rules to improve accuracy.

[0117] Step S440: template generation;

[0118] After creating a mapping model between source data and target system data, the mapping model can be used to obtain a matching result set of source data and target system data. According to at least one mapping code snippet in the matching result set, a pre-application template is inferred and generated through relationship mapping.

[0119] Step S450: Continue optimization.

[0120] In the embodiment of the present application, the similarity reflux model is used for automatic learning; templates are automatically generated to improve the efficiency of integrated mapping development; mapping knowledge is extracted and accumulated for efficient reuse; the quality of generated templates is controllable and does not rely on personal experience. By adopting machine learning, a more accurate effect is achieved through automatic learning backfilling models. It has obvious efficiency and advantages in automated mapping in the integration field.

[0121] Figure 4B A flowchart of a method for generating a mapping template is provided for an embodiment of the present application, such as Figure 4B As shown, the template can be generated by following the steps below:

[0122] Step S420: model training;

[0123] The corpus preparation phase before model training includes:

[0124] Corpus definition: Use historical data and EDI standard data to split tags, attributes, keywords, etc. as corpus;

[0125] Remove noise items: filter and convert abbreviations, and remove stop words (-, _, ", ', etc.).

[0126] The historical data, EDI standard data and existing templates (standard and customized) that have been removed from the items are classified based on the target system to obtain the type of input standards.

[0127] Figure 4C A schematic diagram of a flow chart of deduplication of an existing mapping template provided in an embodiment of the present application, such as Figure 4C As shown in the figure, you can use the following process to deduplicate the existing mapping template:

[0128] Step S1, parsing input file (data);

[0129] Here, the existing mapping template can be obtained by parsing the input file.

[0130] Step S2, extracting tags and attributes;

[0131] Step S3, removing duplicate data and adding levels;

[0132] Use fixed-length compression to compress the data for storage and point to the template for an existing mapping template.

[0133] Here, the fixed-length compression may be MD5 encryption. MD5 encryption is convenient and fast. By utilizing its fixed length, better comparison may be performed to remove duplicate existing mapping templates.

[0134] The compression of the existing mapping templates can be gizp compression (L277 compression + Huffman coding). Due to some interference factors (such as the lack of a certain attribute, etc.), the comparison of compressed values ​​is used at this time. If the probability of the same result between the compressed existing mapping templates is greater than the same threshold (such as: 90%), they are considered to be the same.

[0135] During the implementation process, if there are multiple mapping templates for the same original file and the data of the same target system, they are very likely to be the same and can be compared using the same compression method.

[0136] Constructing the relevant data model includes: splitting the above structured data into labels, attributes and values, removing interference items such as content changes, and constructing one-to-many and many-to-many relationships for the data record levels of the tree structure.

[0137] Creating a mapping relationship includes: creating a mapping relationship according to labels and attributes, pointing to a code snippet of a mapping template, and setting a weight value.

[0138] Training the XGBoost model includes: based on the processed data, using one-hot binary encoding of discrete features, inputting into the XGBoost model for training, and creating a mapping model between the source data and the target system data.

[0139] Figure 4D A schematic diagram of a sample data structure provided in an embodiment of the present application, such as Figure 4D As shown, the schematic diagram includes:

[0140] The training data structure is divided into the following features based on the following: name (label, attribute), hierarchical path, whether it is an attribute, original document type, target name (label, attribute), target hierarchical path, whether the target is an attribute, target document type, last edited time and weight value, and the mapping relationship corresponding to each different name path. Among them, the name is the data name in the source system, and the target name is the data name in the target system.

[0141] like Figure 4D As shown, three groups of different training data structures and their corresponding mapping relationships are shown.

[0142] Step S430: model prediction;

[0143] Data preparation: The system accesses new data and loads the mapping model (file) obtained during the model training phase;

[0144] Construct features: Format the input new data and construct features based on the formatted new data;

[0145] Model service: Through the core algorithm, the mapping relationship required for new data is found in the mapping model;

[0146] Processing of unmatched results: For data that has no mapping relationship in the mapping model, a similarity algorithm (combined with the word vector model Word2Vec, EDI corpus, cosine similarity, etc.) is used to recommend a close mapping relationship.

[0147] Step S440: template generation;

[0148] Generate template: According to at least one mapping code snippet in the matching result set, generate a pre-application template by inferring through relationship mapping;

[0149] Apply template: Apply pre-generated templates. Initially, due to errors, adjustments need to be made and the results are fed back into the model; as the model sample data is continuously enriched and improved, the template will be ready for use.

[0150] Step S450: Continue optimization.

[0151] Enhance the core EDI fields that are most critical to feature data, such as changing the case of labels and inserting random letters; adjust the model training method, such as adjusting the depth and number of trees; recommend and return the adjusted results through similarity algorithms; make full use of the company's existing data and templates for multi-faceted verification; continuously train the model, constantly improve the model algorithm, and improve the quality of generated templates.

[0152] In the embodiment of the present application, the mapping relationship between source data and target data is automatically learned, and a template is automatically generated. The efficiency of integrated mapping development is improved; mapping knowledge is extracted and accumulated, and reused efficiently; the quality of generated templates is controllable and does not rely on personal experience. It has obvious efficiency and advantages in automated mapping in the integration field.

[0153] Based on the foregoing embodiments, the embodiments of the present application provide a mapping template generation device, which includes the modules included, each module includes each sub-module, each sub-module includes a unit, which can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0154] Figure 5 A schematic diagram of the structure of the mapping template generating device provided in the embodiment of the present application is shown in FIG. Figure 5As shown, the device 500 includes:

[0155] An acquisition module 510 acquires a first data set corresponding to a source message and a second data set corresponding to a target message;

[0156] A classification module 520, configured to construct a mapping information set based on a classification result obtained by classifying the first data set and the second data set, wherein a mapping fragment in the mapping information set is used to map first data of the same category to second data;

[0157] A matching module 530, configured to determine, in the mapping information set, at least one matching mapping fragment that matches a field of the source message;

[0158] The generating module 540 is configured to generate a mapping template for mapping the source message based on the at least one matching mapping fragment, wherein the mapping template is used to map the source message to the target message.

[0159] In some embodiments, the classification module 520 includes a splitting submodule, a classification submodule and a construction submodule, wherein the splitting submodule is used to split the first data set and the second data set based on field type to obtain a first field library corresponding to the first data set and a second field library corresponding to the second data set; the classification submodule is used to classify the first field library and the second field library using a classification model to obtain the classification result; the construction submodule is used to construct the mapping information set based on the classification result.

[0160] In some embodiments, the classification submodule includes an encoding unit and a classification unit, wherein the encoding unit is used to encode the fields in the first field library and the second field library to obtain the encoded first field library and the second field library; the classification unit is used to classify the fields in the encoded first field library and the second field library using a classification model to obtain the classification result.

[0161] In some embodiments, the field type includes at least one of the following: a tag field, an attribute field and a keyword field; the splitting submodule includes a splitting unit, a clearing unit and a obtaining unit, wherein the splitting unit is used to split the first data set into at least one of the following fields based on the field type: a first tag field, a first attribute field and a first keyword field, and split the second data set into at least one of the following fields: a second tag field, a second attribute field and a second keyword field; the clearing unit is used to clear the first keyword field and the second keyword field; the obtaining unit is used to obtain the first field library based on the first tag field and / or the first attribute field corresponding to the first data set, and to obtain the second field library based on the first tag field and / or the first attribute field corresponding to the second data set.

[0162] In some embodiments, the classification module 520 also includes a first determination submodule and a second determination submodule, wherein the first determination submodule is used to determine the mapping number of each mapping fragment in the process of constructing the mapping information set based on the classification result; and the second determination submodule is used to determine the weight of the corresponding mapping fragment based on the mapping number.

[0163] In some embodiments, the matching module 530 includes a third determination submodule and a fourth determination submodule, wherein the third determination submodule is used to determine, in the mapping information set, multiple matching mapping fragments that match the first field in the source message; and the fourth determination submodule is used to determine, based on the weights of the multiple matching mapping fragments, that the matching mapping fragment with the highest weight is the matching mapping fragment of the first field.

[0164] In some embodiments, the generation module 540 includes a fifth determination submodule, a sixth determination submodule and a first generation submodule, wherein the fifth determination submodule is used to determine that at least one first field of the source message does not match any mapping fragment in the mapping information set; the sixth determination submodule is used to determine a similarity mapping fragment that matches the first field in the mapping information set using a similarity algorithm; and the first generation submodule is used to generate a mapping template for mapping the source message based on the at least one matching mapping fragment and the similarity mapping fragment.

[0165] In some embodiments, the generation module 540 also includes a second generation submodule and an update submodule, wherein the second generation submodule is used to generate a target mapping fragment that matches the target first field based on the similarity mapping fragment and the target first field; and the update submodule is used to update the target mapping fragment to the mapping information set.

[0166] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0167] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0168] Correspondingly, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the mapping template generation method provided in the above embodiment are implemented.

[0169] Correspondingly, an embodiment of the present application provides an electronic device, Figure 6 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the hardware entity of the device 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores a computer program that can be run on the processor 602, and the processor 602 implements the steps in the mapping template generation method provided in the above embodiment when executing the program.

[0170] The memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or processed by the processor 602 and various modules in the electronic device 600 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0171] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0172] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0173] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0174] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0175] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0176] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0177] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0178] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0179] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0180] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0181] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0182] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating a mapping template, the method comprising: Acquire a first data set corresponding to the source message and a second data set corresponding to the target message; Based on the classification results obtained by classifying the first data set and the second data set, a mapping information set is constructed, wherein the mapping fragments in the mapping information set are used to map the first data of the same category to the second data; Determining at least one matching mapping fragment that matches a field of the source message in the mapping information set; A mapping template for mapping the source message is generated based on the at least one matching mapping fragment, and the mapping template is used to map the source message to the target message.

2. The method according to claim 1, wherein the step of constructing a mapping information set based on the classification results obtained by classifying the first data set and the second data set comprises: Splitting the first data set and the second data set based on field types to obtain a first field library corresponding to the first data set and a second field library corresponding to the second data set; Using a classification model to classify the first field library and the second field library to obtain the classification result; The mapping information set is constructed based on the classification result.

3. The method according to claim 2, wherein the step of using a classification model to classify the first field library and the second field library to obtain the classification result comprises: Encoding the fields in the first field library and the second field library to obtain the encoded first field library and the second field library; The fields in the first field library and the second field library that have completed the encoding are classified using a classification model to obtain the classification result.

4. The method of claim 2, wherein the field type comprises at least one of the following: a tag field, an attribute field, and a keyword field; The step of splitting the first data set and the second data set based on field types to obtain a first field library corresponding to the first data set and a second field library corresponding to the second data set includes: Splitting the first data set into at least one of the following fields based on the field type: a first tag field, a first attribute field, and a first keyword field, and splitting the second data set into at least one of the following fields: a second tag field, a second attribute field, and a second keyword field; clearing the first keyword field and the second keyword field; The first field library is obtained based on the first label field and / or the first attribute field corresponding to the first data set, and the second field library is obtained based on the first label field and / or the first attribute field corresponding to the second data set.

5. The method of claim 2, further comprising: In the process of constructing the mapping information set based on the classification result, determining the mapping number of each mapping fragment; The weight of the corresponding mapping segment is determined based on the mapping number.

6. The method according to claim 5, wherein determining in the mapping information set at least one matching mapping fragment that matches a field of the source message comprises: Determining, in the mapping information set, a plurality of matching mapping fragments that match the first field in the source message; Based on the weights of the multiple matching mapping segments, a matching mapping segment with the highest weight is determined as the matching mapping segment of the first field.

7. The method according to claim 1, wherein generating a mapping template for mapping the source message based on the at least one matching mapping fragment comprises: Determining that at least one target first field of the source message does not match any mapping fragment in the mapping information set; Determine, in the mapping information set, a similarity mapping segment that matches the target first field using a similarity algorithm; A mapping template for mapping the source message is generated based on the at least one matching mapping segment and the similarity mapping segment.

8. The method of claim 7, further comprising: generating a target mapping segment matching the target first field based on the similarity mapping segment and the target first field; The target mapping fragment is updated to the mapping information set.

9. A mapping template generating device, the device comprising: An acquisition module, used to acquire a first data set corresponding to a source message and a second data set corresponding to a target message; A construction module, configured to construct a mapping information set based on a classification result obtained by classifying the first data set and the second data set, wherein a mapping fragment in the mapping information set is used to map first data of the same category to second data; A matching module, configured to determine at least one matching mapping fragment in the mapping information set that matches a field of the source message; A generating module is used to generate a mapping template for mapping the source message based on the at least one matching mapping fragment, wherein the mapping template is used to map the source message to the target message.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps in the method according to any one of claims 1 to 8 are implemented.

Images