Data conversion method, device and readable storage medium

Abstract

The present invention provides a data conversion method, device and readable storage medium. The method includes: in response to a data conversion request, obtaining multiple source entities, multiple target entities, entity types of multiple source entities and first data of multiple source entities contained in the data conversion request; determining a target data conversion model from multiple preset data conversion models based on the entity type; determining a mapping relationship between multiple source entities and multiple target entities based on the target data conversion model; determining a conversion order for performing data conversion on the first data of multiple source entities based on the target data conversion model; performing data conversion on the first data based on the target data conversion model, the conversion order and the mapping relationship, generating second data, and writing the second data into multiple target entities. The technical solution provided by the present invention does not rely on specific business systems and middleware products, and achieves good versatility.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data conversion method, a data conversion device and a readable storage medium. Background Art

[0002] In related technologies, source system files implement data replication and synchronization based on database tables through database middleware, which has a heavy reliance on the consistency of data structures and the middleware deployment architecture. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0004] To this end, a first aspect of the present invention provides a data conversion method.

[0005] The second aspect of the present invention further provides a data conversion device.

[0006] The third aspect of the present invention further provides a data conversion device.

[0007] A fourth aspect of the present invention further provides a readable storage medium.

[0008] In view of this, the first aspect of the present invention proposes a data conversion method, including: in response to a data conversion request, obtaining multiple source entities, multiple target entities, entity types of multiple source entities and first data of multiple source entities contained in the data conversion request; determining a target data conversion model from multiple preset data conversion models according to the entity type; determining a mapping relationship between multiple source entities and multiple target entities according to the target data conversion model; determining a conversion order for performing data conversion on the first data of multiple source entities according to the target data conversion model; performing data conversion on the first data according to the target data conversion model, the conversion order and the mapping relationship, generating second data, and writing the second data into multiple target entities.

[0009] In this technical solution, after receiving a data conversion request, multiple source entities contained in the data conversion request, multiple target entities corresponding to the multiple source entities, entity types of the multiple source entities, and first data stored in the multiple source entities are obtained, wherein the data conversion request indicates that the first data in the multiple source entities is converted and written into the corresponding multiple target entities.

[0010] Furthermore, based on the entity types of multiple source entities, a target data conversion model is determined among multiple preset data conversion models, wherein the preset data conversion model is a standard data conversion model pre-constructed based on different business scenarios, and the target data conversion model is a data conversion model corresponding to the source entity and the target entity.

[0011] Furthermore, the preset data conversion model includes mapping relationships between various business entities that are pre-configured according to business requirements. According to the determined target conversion model, the mapping relationship between the source entity and the target entity can be determined.

[0012] Furthermore, the data conversion of the first data is performed in a pipelined manner. The target data conversion model includes a pre-configured priority of the first data. The priority is a priority of the pipeline in the entire data conversion process, which is used to determine the conversion order of different first data. The conversion order of the first data is determined based on the conversion logic of the target data conversion model.

[0013] Furthermore, according to the target data conversion model, the conversion order of the first data, and the mapping relationship between the source entity and the target entity, the first data is converted into the second data, and then the second data is synchronized into the target entity.

[0014] The first data may include data attribute information, such as attribute name, attribute type, attribute classification, and measurement unit.

[0015] Through the above-mentioned method, a flexible configuration data conversion logic is provided based on the data conversion model, and the first data is converted according to the data conversion logic to realize the flexible conversion of business data and the rapid transmission of complex data relationships in the business system, which effectively improves the efficiency of data conversion. At the same time, through different business entities, the corresponding target data conversion model is determined to realize the call of cross-class data conversion models between different business scenarios, meet the data conversion processing under different business scenarios, and realize data conversion between heterogeneous business systems or different business entities. Compared with the prior art that relies on database middleware to realize data replication and synchronization based on database tables, and has a heavy dependence on the middleware deployment construction of data structures, this application selects a data conversion model for the business entity, and realizes the data conversion function in the business application based on the data conversion logic configured by different data conversion models. It does not rely on specific business systems and middleware products, achieves better versatility, and solves the problems of single system functions and tight coupling in the prior art.

[0016] The data conversion method provided by the present invention may also have the following additional technical features:

[0017] In the above technical solution, further, according to the target data conversion model, the step of determining the conversion order of the first data of multiple source entities specifically includes: determining the priority of the first data according to the target data conversion model; sorting the first data according to the priority to obtain the conversion order.

[0018] In this technical solution, the first data is related data with dependencies. Typically, the dependent first data in the related data is assigned a high priority and converted first, while the dependent first data is assigned a low priority and converted after the dependent data is generated. According to the conversion logic configured in the target data conversion model, the priority of the first data in the entire conversion process is determined, and the execution order of the first data is sorted according to the priority, thereby determining the conversion order of the first data. In addition, the various conversion steps do not necessarily have to be executed sequentially; they can also be executed simultaneously. Pipelines with the same priority can be executed in parallel during data conversion.

[0019] Through the above method, based on the conversion logic provided by the target conversion model, the priority of the first data in the entire data conversion process is determined to realize the configuration of parallel execution or serial execution between multiple pipelines. While improving the efficiency of data conversion execution, the conversion order of related and dependent data is guaranteed, the integrity of data conversion is ensured, confusion and errors are avoided, and the accuracy of data conversion is improved.

[0020] In any of the above technical solutions, further, the step of performing data conversion on the first data to generate the second data according to the target data conversion model, conversion order and mapping relationship specifically includes: determining the conversion method of the first data according to the mapping relationship; inputting the first data into the target data conversion model, and converting the first data into the second data according to the conversion method and conversion order.

[0021] In this technical solution, the data conversion method is automatically synchronized based on the mapping relationship between multiple source entities and multiple target entities. Specifically, the mapping relationship between source entities and target entities can be a one-to-one relationship, that is, one source entity corresponds to one target entity; it can also be a many-to-one relationship, that is, multiple source entities correspond to one target entity; or it can be a part of a source entity corresponds to one target entity. The first data is input into the target data conversion model, and based on the mapping relationship between different entities, the comparison relationship between the first data and the second data is automatically synchronized, and the first data is determined to be combined, split or mapped to generate the corresponding second data, and the generated second data is written to the target entity.

[0022] Through the above method, the conversion logic is defined based on the target data conversion model, which can quickly and conveniently realize the definition, configuration and application of data relationships. For complex data, there is no need to edit plug-ins, only simple configuration is required, which improves the breadth of data conversion, effectively covers data conversion of more business types, and quickly realizes the relationship transfer of complex data in heterogeneous systems, effectively improving the efficiency of data conversion.

[0023] In any of the above technical solutions, further, the conversion method includes at least one of the following: data combination, data splitting and data mapping.

[0024] In this technical solution, based on the mapping relationship between multiple source entities and multiple target entities, the data comparison relationship between multiple source entities and multiple target entities is determined, that is, a first data stored in the source entity is converted into a second data of the target entity. According to the comparison relationship between the first data and the second data, the first data is merged, split and mapped to generate the corresponding second data.

[0025] Through the above method, according to the conversion logic defined by the target data conversion model, data conversion can be automatically performed in actual application scenarios such as heterogeneous system integration, version upgrades of business systems, and the flow of business data within the system, thereby achieving the rapid transmission of complex data relationships in the business system and effectively improving the efficiency of data conversion.

[0026] In any of the above technical solutions, further, the step of writing the second data information into multiple target entities specifically includes: obtaining the data format of the second data; determining the standard data format of the target entity based on the target data conversion model; comparing the data format with the standard data format; writing the second data into the target entity based on the data format being the same as the standard data format; deleting the second data based on the data format being different from the standard data format.

[0027] In this technical solution, a preset data conversion model includes a standard data format preconfigured according to the format requirements of different business entities. The data format of the converted second data is obtained, and the standard data format corresponding to the target entity is determined based on the target data conversion model. The data format of the second data is compared with the standard data format of the target entity. If the data format of the second data and the standard data format are the same, it indicates that the data conversion between the multiple source entities and the multiple target entities is successful, and the second data is written to the target entity. If the data format of the second data and the standard data format are different, it indicates that the data conversion between the multiple source entities and the multiple target entities failed, and the second data is deleted.

[0028] In the above manner, when writing the second data into the target entity, the generated second data is verified based on the verification rules provided by the target data conversion model to ensure the correctness of the output second data and effectively reduce the error rate of the data in the business entity.

[0029] In any of the above technical solutions, further, in response to a data conversion request, before obtaining multiple source entities, multiple target entities, entity types of multiple source entities and first data of multiple source entities contained in the data conversion request, it also includes: obtaining converted historical data; determining the combination logic between multiple source entities and multiple target entities and the verification rules of the historical data based on the historical data; and creating a preset data conversion model based on the combination logic and the verification rules.

[0030] In this technical solution, for business data in different business scenarios, the historical data of different business entities, the combination logic between business entities, and business validation rules are imported into the model configuration interface to establish a visual data conversion model. Specifically, request nodes and conversion nodes are created in the model configuration interface, and the created request nodes and conversion nodes are connected. The node attributes of the request nodes and conversion nodes are configured accordingly based on the historical data and data format requirements of different business entities. Then, based on the configured nodes, combination logic, and validation rules, the data conversion model is established.

[0031] Through the above method, different data conversion models are abstractly defined based on the combination and data dependency relationships between different business entities, so that the created data conversion model can meet the data conversion requirements in different business scenarios and can automatically perform data conversion, avoiding the tedious process of manually rewriting the plug-in logic code. There is no need to pay attention to the source of the data, which to a certain extent avoids the repetitive development of various docking applications, saves system R&D resources, and reduces data conversion implementation time and cost.

[0032] In any of the above technical solutions, further, the verification rules include at least one of the following: data length verification, default data verification and whether the data is a null value verification.

[0033] In this technical solution, the target data conversion model includes validation rules corresponding to the configured business entity, including length validation, default value validation, and null check. Specifically, after the first data conversion generates second data, the second data is validated according to the data validation rules corresponding to the target entity contained in the target data conversion model.

[0034] In the above manner, the generated second data is verified based on the verification rules provided by the target data conversion model to ensure the validity of the output second data, thereby effectively improving the correctness of the data conversion output.

[0035] According to a second aspect of the present invention, a data conversion device is proposed, including: a data conversion model determination unit, used to determine a target data conversion model from a plurality of preset data conversion models according to the entity types of a plurality of source entities; the data conversion model determination unit is also used to determine a mapping relationship between a plurality of source entities and a plurality of target entities according to the target data conversion model; a data conversion configuration unit, used to determine a conversion order for performing data conversion on first data of a plurality of source entities according to the target data conversion model; an execution unit, used to perform data conversion on the first data according to the target data conversion model, the conversion order and the mapping relationship, generate second data, and write the second data information into a plurality of target entities.

[0036] In this technical solution, the data conversion device includes a data conversion model determination unit, a data conversion unit, a data conversion configuration unit, and an execution unit. After receiving a data conversion request, the data conversion device obtains multiple source entities contained in the data conversion request, target entities corresponding to the multiple source entities, entity types of the multiple source entities, and first data stored in the multiple source entities, wherein the data conversion request instructs to convert the first data in the multiple source entities and write them into the corresponding target entities. The data conversion model determination unit is used to determine a target data conversion model from a plurality of preset data conversion models based on the entity types of the multiple source entities, wherein the preset data conversion model is a standard data conversion model pre-constructed based on different business scenarios, and the target data conversion model is a data conversion model corresponding to the source entity and the target entity.

[0037] Furthermore, the data conversion model determination unit is also used to define the mapping relationships between multiple source entities and multiple target entities of the data to be converted. Specifically, in a business system, business data is composed of multiple different business entities, and there are combinations and dependencies between different business entities. These combinations and relationships are usually reflected in the definition of the database table. The preset data conversion model contains the mapping relationships between various business entities that are pre-configured according to business needs. Based on the determined target conversion model, the mapping relationships between multiple source entities and multiple target entities can be determined.

[0038] Furthermore, the data conversion configuration unit is used to specify the steps and conversion logic of data conversion, mainly including data conversion configuration and task configuration. Among them, the data conversion configuration defines the data conversion rules from the first data to the second data, and the data conversion rules include data mapping rules, data merging rules and data splitting rules. The task configuration defines the conversion order of the data conversion. Specifically, the data conversion of the first data is performed in the form of a pipeline, and the target data conversion model contains the pre-configured priority of the first data, wherein the priority is the priority of the pipeline in the entire data conversion, which is used to determine the conversion order of different first data. The conversion order of the first data is determined according to the conversion logic of the target data conversion model.

[0039] Furthermore, the execution unit is used to convert the first data into second data according to the target data conversion model, the conversion order of the first data, and the mapping relationship between the source entity and the target entity, and then synchronize the second data into the target entity.

[0040] Through the above-mentioned method, a flexible configuration data conversion logic is provided based on the data conversion model, and the first data is converted according to the data conversion logic to realize the flexible conversion of business data and the rapid transmission of complex data relationships in the business system, which effectively improves the efficiency of data conversion. At the same time, through different business entities, the corresponding target data conversion model is determined to realize the call of cross-class data conversion models between different business scenarios, meet the data conversion processing under different business scenarios, and realize data conversion between heterogeneous business systems or different business entities. Compared with the prior art that relies on database middleware to realize data replication and synchronization based on database tables, and has a heavy dependence on the middleware deployment construction of data structures, this application selects a data conversion model for the business entity, and realizes the data conversion function in the business application based on the data conversion logic configured by different data conversion models. It does not rely on specific business systems and middleware products, achieves better versatility, and solves the problems of single system functions and tight coupling in the prior art.

[0041] In any of the above technical solutions, further, the data conversion device also includes: a parsing unit, which is used to parse the data conversion request in response to the data conversion request, and obtain multiple source entities, multiple target entities, entity types of multiple source entities and first data of multiple source entities included in the data conversion request; an input data reading unit, which is used to filter the first data of the input multiple source entities; a data processing collection unit, which is used to merge, split, map and convert the first data according to the conversion order and conversion method of the first data; a data consistency processing unit, which is used to update the target data module according to the first data and the second data; and an output data generation unit, which is used to summarize the converted second data for subsequent output.

[0042] In this technical solution, the data conversion device also includes a parsing unit, an input data reading unit, a data processing aggregation unit, a data consistency processing unit and an output data generating unit.

[0043] Specifically, after receiving a data conversion request, the data conversion device uses a parsing unit to parse the data conversion request, obtain multiple source entities contained in the data conversion request, target entities corresponding to the multiple source entities, entity types of the multiple source entities, and the first data stored in the multiple source entities, and create a corresponding conversion job during the data conversion execution according to the configuration of the data conversion configuration unit, wherein the job includes the first data pipeline, the processing steps in any pipeline, and the instantiation of the extension plug-in.

[0044] Furthermore, the input data reading unit is used to read the input first data and perform processing such as filtering and paging on the first data according to the configuration parameters in the target data conversion model.

[0045] Furthermore, the data processing collection unit is used to implement the default data processing process and encapsulate the commonly used data conversion logic, wherein the default data processing process mainly includes: data merging process, data splitting process, context variable parsing process, data mapping process, primary key conversion process and type conversion process.

[0046] Furthermore, the data consistency processing unit is used to update the target entity and the target data module according to the user-defined idempotent verification rules or repeated update strategy. Specifically, the data conversion request includes the user-defined verification rules, and the user-defined verification rules are imported into the target data conversion model to verify the target entity. Among them, the verification rules can be idempotent verification rules or repeated update strategies. When the second data is written to the target entity, the target entity is traversed based on the second data, and based on the provided idempotent verification rules, it is determined whether a consistent data result is obtained when executing the conversion. In addition, the repeated update strategy can also be used to specify whether to update the target data conversion model after the first data conversion when repeatedly executing the data conversion.

[0047] Furthermore, the output data generation unit is used to implement data output control. Specifically, if the amount of first data is large, the first data is segmented and converted according to the amount of the first data, and then the second data is generated. The generated second data is then aggregated or fully generated for subsequent data output operations. Among them, the user can customize the immediate receipt of the converted second data during the data conversion process or receive the aggregated second data after the data conversion is completed according to needs.

[0048] Through the above method, the conversion logic and core processing steps in the data conversion process are provided, focusing on solving the conversion logic and data processing in the data conversion process, without relying on specific business systems and middleware products, and improving the versatility of the data conversion device.

[0049] In any of the above technical solutions, the data conversion device further includes: a rule expansion unit, which is used to respond to the expansion request, obtain the expansion rule for converting the first data contained in the expansion request, and update the target data conversion model according to the expansion rule.

[0050] In this technical solution, the rule extension unit is used to receive an extension request sent by a user and obtain the extension rules contained in the extension request, wherein the extension rules are an extension mechanism for controlling the data conversion flow program and logic provided by the user. The target data conversion model is updated according to the extension rules so that the general conversion logic in the target data conversion model is combined with the conversion mechanism required by the user.

[0051] Through the above method, plug-in expansion is achieved, which enables users to intervene in and control each link of data conversion, making the data conversion logic more in line with user needs, improving the reusability and flexibility of the data conversion logic to meet more business scenarios.

[0052] According to a third aspect of the present invention, a data conversion device is provided, comprising: a memory storing a program or instructions; and a processor connected to the memory, the processor being configured to implement the data conversion method of the first aspect when executing the program or instructions. Therefore, the data conversion device possesses all the beneficial effects of the data conversion method of the first aspect, and to avoid repetition, further details are omitted.

[0053] According to a fourth aspect of the present invention, a readable storage medium is provided, on which a program or instruction is stored. When executed by a processor, the program or instruction performs the data conversion method provided in the first aspect. Therefore, the readable storage medium has all the beneficial effects of the data conversion method provided in the first aspect, and to avoid repetition, further details are omitted.

[0054] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0056] Figure 1 FIG1 shows a flow chart of a data conversion method according to an embodiment of the present invention;

[0057] Figure 2 FIG2 shows a second flow chart of the data conversion method according to an embodiment of the present invention;

[0058] Figure 3 FIG3 shows a flow chart of a data conversion method according to an embodiment of the present invention;

[0059] Figure 4 FIG4 shows a fourth flow chart of the data conversion method according to an embodiment of the present invention;

[0060] Figure 5 FIG5 shows a fifth flow chart of a data conversion method according to an embodiment of the present invention;

[0061] Figure 6 A schematic diagram showing a flow chart of a data conversion method according to a specific embodiment of the present invention is shown;

[0062] Figure 7 A schematic block diagram of a data conversion device according to an embodiment of the present invention is shown;

[0063] Figure 8 A schematic diagram showing a data mapping relationship between a source entity and a target entity according to a specific embodiment of the present invention;

[0064] Figure 9 A schematic diagram showing a data mapping relationship between multiple source entities and multiple target entities according to a specific embodiment of the present invention;

[0065] Figure 10 A logic diagram of data conversion of first data according to a specific embodiment of the present invention is shown.

[0066] in, Figure 7 The corresponding relationship between the reference numerals and component names is as follows:

[0067] 700 data conversion device, 702 parsing unit, 704 input data reading unit, 706 data conversion model determination unit, 708 data conversion configuration unit, 710 data processing collection unit, 712 execution unit, 714 data consistency processing unit, 716 output data generation unit, 718 rule expansion unit. DETAILED DESCRIPTION

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0069] It should be noted that the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0070] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0071] Refer to the following Figures 1 to 10 The present invention describes a data conversion method, a data conversion device, and a readable storage medium according to some embodiments of the present invention.

[0072] Example 1:

[0073] like Figure 1 According to one embodiment of the present invention, a data conversion method is proposed, which includes:

[0074] Step 102: In response to the data conversion request, obtain multiple source entities, multiple target entities, entity types of the multiple source entities, and first data of the multiple source entities included in the data conversion request;

[0075] Step 104: Determine a target data conversion model from a plurality of preset data conversion models according to the entity type;

[0076] Step 106: Determine mapping relationships between multiple source entities and multiple target entities according to the target data conversion model;

[0077] Step 108: determining a conversion order for performing data conversion on the first data of the plurality of source entities according to the target data conversion model;

[0078] Step 110, performing data conversion on the first data according to the target data conversion model, conversion sequence and mapping relationship to generate second data;

[0079] Step 112: Write the second data into multiple target entities.

[0080] In this embodiment, after receiving a data conversion request, multiple source entities contained in the data conversion request, multiple target entities corresponding to the multiple source entities, entity types of the multiple source entities, and first data stored in the multiple source entities are obtained, wherein the data conversion request indicates that the first data in the multiple source entities is converted and written into the corresponding multiple target entities.

[0081] Furthermore, based on the entity types of multiple source entities, a target data conversion model is determined among multiple preset data conversion models, wherein the preset data conversion model is a standard data conversion model pre-constructed based on different business scenarios, and the target data conversion model is a data conversion model corresponding to the source entity and the target entity.

[0082] Furthermore, the preset data conversion model includes mapping relationships between various business entities that are pre-configured according to business requirements. According to the determined target conversion model, the mapping relationship between the source entity and the target entity can be determined.

[0083] Furthermore, the data conversion of the first data is performed in a pipelined manner. The target data conversion model includes a pre-configured priority of the first data. The priority is a priority of the pipeline in the entire data conversion process, which is used to determine the conversion order of different first data. The conversion order of the first data is determined based on the conversion logic of the target data conversion model.

[0084] Furthermore, according to the target data conversion model, the conversion order of the first data, and the mapping relationship between the source entity and the target entity, the first data is converted into the second data, and then the second data is synchronized into the target entity.

[0085] The first data may include data attribute information, such as attribute name, attribute type, attribute classification, and measurement unit.

[0086] Through the above-mentioned method, a flexible configuration data conversion logic is provided based on the data conversion model, and the first data is converted according to the data conversion logic to realize the flexible conversion of business data and the rapid transmission of complex data relationships in the business system, which effectively improves the efficiency of data conversion. At the same time, through different business entities, the corresponding target data conversion model is determined to realize the call of cross-class data conversion models between different business scenarios, meet the data conversion processing under different business scenarios, and realize data conversion between heterogeneous business systems or different business entities. Compared with the prior art that relies on database middleware to realize data replication and synchronization based on database tables, and has a heavy dependence on the middleware deployment construction of data structures, this application selects a data conversion model for the business entity, and realizes the data conversion function in the business application based on the data conversion logic configured by different data conversion models. It does not rely on specific business systems and middleware products, achieves better versatility, and solves the problems of single system functions and tight coupling in the prior art.

[0087] Example 2:

[0088] like Figure 2 According to one embodiment of the present invention, a data conversion method is proposed, which includes:

[0089] Step 202: In response to the data conversion request, obtain multiple source entities, multiple target entities, entity types of the multiple source entities, and first data of the multiple source entities included in the data conversion request;

[0090] Step 204: Determine a target data conversion model from a plurality of preset data conversion models according to the entity type;

[0091] Step 206: Determine mapping relationships between multiple source entities and multiple target entities according to the target data conversion model;

[0092] Step 208: Determine the priority of the first data according to the target data conversion model;

[0093] Step 210: sort the first data according to the priority to obtain a conversion order;

[0094] Step 212: performing data conversion on the first data according to the target data conversion model, conversion sequence, and mapping relationship to generate second data;

[0095] Step 214: write the second data into multiple target entities.

[0096] In this embodiment, the first data is associated data with dependencies. Generally, the dependent first data in the associated data is set to a high priority and converted first, and the first data that generates the dependency is set to a low priority, and the conversion is performed after the dependent data is generated. According to the conversion logic configured in the target data conversion model, the priority of the first data in the entire conversion process is determined, and the execution order of the first data is sorted according to the priority, thereby determining the conversion order of the first data. In addition, the various conversion steps do not necessarily have to be executed sequentially, but can also be executed simultaneously. Pipelines with the same priority can be executed in parallel during data conversion.

[0097] Through the above method, based on the conversion logic provided by the target conversion model, the priority of the first data in the entire data conversion process is determined to realize the configuration of parallel execution or serial execution between multiple pipelines. While improving the efficiency of data conversion execution, the conversion order of related and dependent data is guaranteed, the integrity of data conversion is ensured, confusion and errors are avoided, and the accuracy of data conversion is improved.

[0098] In a specific embodiment, Figure 10 As shown, it is an operation flow for data conversion of the first data, including data conversion processes and pipeline configuration, wherein a process is the processing process of a piece of data in a certain link, and the process includes: sequence, that is, the execution sequence of the specified process in the same pipeline; processing program, that is, the data conversion program of the current process. The pipeline is the complete conversion process of a piece of data, and the pipeline includes: priority, that is, the priority of the pipeline in the entire synchronous operation, which is used to determine the conversion order of different data; process set, that is, the processes that need to be executed for the data conversion of the specified pipeline. Specifically, the first data set is input, and the pipeline execution order is determined according to the priority of the first data. According to the execution sequence of the predetermined process in any pipeline, the first data is converted, the generated second data is summarized, and the second data set is output for subsequent operations.

[0099] Example 3:

[0100] like Figure 3 According to one embodiment of the present invention, a data conversion method is proposed, which includes:

[0101] Step 302: In response to the data conversion request, obtain multiple source entities, multiple target entities, entity types of the multiple source entities, and first data of the multiple source entities included in the data conversion request;

[0102] Step 304: Determine a target data conversion model from a plurality of preset data conversion models according to the entity type;

[0103] Step 306: Determine mapping relationships between multiple source entities and multiple target entities according to the target data conversion model;

[0104] Step 308: Determine the order of performing data conversion on the first data of the plurality of source entities according to the target data conversion model;

[0105] Step 310: determining a conversion method for the first data according to the mapping relationship;

[0106] Step 312: input the first data into the target data conversion model, and convert the first data into second data according to the conversion method;

[0107] Step 314: write the second data into multiple target entities.

[0108] In this embodiment, the data conversion method is automatically synchronized based on the mapping relationship between multiple source entities and multiple target entities. Specifically, the mapping relationship between source entities and target entities can be a one-to-one relationship, that is, one source entity corresponds to one target entity; it can also be a many-to-one relationship, that is, multiple source entities correspond to one target entity; or it can be a part of a source entity corresponds to one target entity. The first data is input into the target data conversion model, and based on the mapping relationship between different entities, the comparison relationship between the first data and the second data is automatically synchronized, and it is determined to combine, split or map the first data, generate corresponding second data, and write the generated second data into the target entity.

[0109] Through the above method, the conversion logic is defined based on the target data conversion model, which can quickly and conveniently realize the definition, configuration and application of data relationships. For complex data, there is no need to edit plug-ins, only simple configuration is required, which improves the breadth of data conversion, effectively covers data conversion of more business types, and quickly realizes the relationship transfer of complex data in heterogeneous systems, effectively improving the efficiency of data conversion.

[0110] In a specific embodiment, the data mapping rules configured in the target data conversion model are generated by combining preset data mapping rules with user-extended custom data mapping rules. Specifically, the data mapping rules include: automatic mapping, which is a simple mapping based on the same attributes. For example, when both the source entity and the target entity contain the "unit of measurement" attribute, a mapping is automatically established. Declarative mapping, which is a mapping relationship established by the user specifying the source attribute name and the target attribute name, such as the mapping of "material code" to "product code". Constant mapping, which is a constant assignment used when specifying the target attribute. Variable mapping, which is a variable value based on the context environment when specifying the target attribute during conversion, such as the current time, salesperson, etc. Expression mapping, which is an expression value calculated by specifying the target attribute during conversion, such as numerical aggregation, amount calculation, etc. In the expression configuration, context variables, source entity data, and target entity data can all be used as expression parameters. Extended mapping is used to implement user-defined mapping logic.

[0111] Furthermore, the conversion method includes at least one of the following: data combination, data splitting, and data mapping. Based on the mapping relationship between the multiple source entities and the multiple target entities, a corresponding relationship between the data of the multiple source entities and the multiple target entities is determined, that is, a first data stored in the source entity is converted into a second data of the target entity, and based on the corresponding relationship between the first data and the second data, the first data is combined, split, and mapped to generate the corresponding second data.

[0112] Through the above method, according to the conversion logic defined by the target data conversion model, data conversion can be automatically performed in actual application scenarios such as heterogeneous system integration, version upgrades of business systems, and the flow of business data within the system, thereby achieving the rapid transmission of complex data relationships in the business system and effectively improving the efficiency of data conversion.

[0113] Example 4:

[0114] like Figure 4 According to one embodiment of the present invention, a data conversion method is proposed, which includes:

[0115] Step 402: In response to the data conversion request, obtain multiple source entities, multiple target entities, entity types of the multiple source entities, and first data of the multiple source entities included in the data conversion request;

[0116] Step 404: Determine a target data conversion model from a plurality of preset data conversion models according to the entity type;

[0117] Step 406: Determine mapping relationships between multiple source entities and multiple target entities according to the target data conversion model;

[0118] Step 408: Determine the order of performing data conversion on the first data of the plurality of source entities according to the target data conversion model;

[0119] Step 410: Perform data conversion on the first data according to the target data conversion model, conversion sequence, and mapping relationship to generate second data;

[0120] Step 412, obtaining the data format of the second data;

[0121] Step 414, determining the standard data format of the target entity according to the target data conversion model;

[0122] Step 416, determine whether the data format is the same as the standard data format; if so, proceed to step 418, if not, proceed to step 420;

[0123] Step 418, writing the second data into the target entity;

[0124] Step 420: Delete the second data.

[0125] In this embodiment, the preset data conversion model includes a standard data format preconfigured according to the format requirements of different business entities. The data format of the converted second data is obtained, and the standard data format corresponding to the target entity is determined based on the target data conversion model. The data format of the second data is compared with the standard data format of the target entity. If the data format of the second data is the same as the standard data format, it indicates that the data conversion between the multiple source entities and the multiple target entities is successful, and the second data is written to the target entity. If the data format of the second data is different from the standard data format, it indicates that the data conversion between the multiple source entities and the multiple target entities has failed, and the second data is deleted.

[0126] In the above manner, when writing the second data into the target entity, the generated second data is verified based on the verification rules provided by the target data conversion model to ensure the correctness of the output second data and effectively reduce the error rate of the data in the business entity.

[0127] Example 5:

[0128] like Figure 5 According to one embodiment of the present invention, a data conversion method is proposed, which includes:

[0129] Step 502, obtaining converted historical data;

[0130] Step 504: Determine the combination logic between the multiple source entities and the multiple target entities and the verification rules of the historical data based on the historical data;

[0131] Step 506: Create a preset data conversion model based on the combination logic and verification rules;

[0132] Step 508 , in response to the data conversion request, obtaining a plurality of source entities, a plurality of target entities, entity types of the plurality of source entities, and first data of the plurality of source entities included in the data conversion request;

[0133] Step 510: Determine a target data conversion model from a plurality of preset data conversion models according to the entity type;

[0134] Step 512: Determine mapping relationships between multiple source entities and multiple target entities according to the target data conversion model;

[0135] Step 514: determining a conversion order for performing data conversion on the first data of the plurality of source entities according to the target data conversion model;

[0136] Step 516: Perform data conversion on the first data according to the target data conversion model, conversion sequence, and mapping relationship to generate second data;

[0137] Step 518: Write the second data into multiple target entities.

[0138] In this embodiment, for business data in different business scenarios, historical data from different business entities, the combination logic between business entities, and business validation rules are imported into the model configuration interface to establish a visual data conversion model. Specifically, request nodes and conversion nodes are created in the model configuration interface, and the created request nodes and conversion nodes are connected. The node attributes of the request nodes and conversion nodes are configured accordingly based on the historical data and data format requirements of different business entities. Then, based on the configured nodes, combination logic, and validation rules, the data conversion model is established.

[0139] Through the above method, different data conversion models are abstractly defined based on the combination and data dependency relationships between different business entities, so that the created data conversion model can meet the data conversion requirements in different business scenarios and can automatically perform data conversion, avoiding the tedious process of manually rewriting the plug-in logic code. There is no need to pay attention to the source of the data, which to a certain extent avoids the repetitive development of various docking applications, saves system R&D resources, and reduces data conversion implementation time and cost.

[0140] Furthermore, the validation rules include at least one of the following: data length validation, default data validation, and data null value validation. The target data conversion model includes validation rules corresponding to the configured business entity, including length validation, default value validation, and null value validation. Specifically, after the first data conversion generates second data, the second data is validated according to the data validation rules corresponding to the target entity included in the target data conversion model.

[0141] In the above manner, the generated second data is verified based on the verification rules provided by the target data conversion model to ensure the validity of the output second data, thereby effectively improving the correctness of the data conversion output.

[0142] Furthermore, the data conversion request may also include user-defined validation rules, which are imported into the target data conversion model to update the target entity and the target data conversion model. The validation rules may be idempotent validation rules or repeated update strategies. When the second data is written to the target entity, the target entity is traversed based on the second data, and based on the provided idempotent validation rules, it is determined whether a consistent data result is obtained when the conversion is performed. This avoids duplicate and redundant data in the target entity and reduces the storage space occupied by the target entity. In addition, the repeated update strategy can be used to specify whether to update the target data conversion model after the first data conversion when the data conversion is repeatedly performed, thereby improving the accuracy and consistency of the target data conversion model.

[0143] Example 6:

[0144] like Figure 6 As shown, according to a specific embodiment of the present invention, a data conversion method is proposed, which includes:

[0145] Step 602, read the input first data;

[0146] Step 604, creating a first data pipeline;

[0147] Step 606, performing job slicing on the first data;

[0148] Step 608 , determining whether there is an unexecuted first data job segment, if so, proceeding to step 610 , if not, proceeding to step 616 ;

[0149] Step 610, executing a first data pipeline process;

[0150] Step 612, performing a first data consistency check;

[0151] Step 614, generate second data and proceed to step 608;

[0152] Step 616: Generate second data to be output for aggregation.

[0153] In this embodiment, the number of jobs and job shards executed in parallel can be dynamically adjusted during the data conversion process based on resource allocation. The number of jobs refers to the number of first data points for which data conversion is executed in parallel, while the number of job shards refers to the number of processes executed in parallel within any job. Specifically, data conversion is performed on a job-by-job basis. To address the conversion and synchronization requirements for large amounts of data, the execution unit is designed to execute multiple jobs in parallel. Furthermore, multiple job shards within a job are supported in parallel, fully utilizing server resources and improving conversion efficiency.

[0154] In business systems, especially enterprise applications, business data is often structured by data conversion models, with different business entities containing different structure definitions. In real-world application scenarios, business data often requires structural conversion. For example, when upgrading business system versions or replacing old systems with new ones, existing business data needs to be migrated and upgraded. Initial data from project implementation phases needs to be synchronized or imported from heterogeneous systems or user data collected into the business system for the project implementation. Business data also needs to be converted from one structure to another as it circulates within the system, including expansion or conversion processes such as from materials to products, from potential customers to official customers, and from customers to franchisees. In response to the above scenario requirements, data processing methods or systems of related technologies can solve some data synchronization or conversion problems, but they generally have limitations and shortcomings, which are specifically manifested in the close coupling with related business system middleware. For example, traditional DTS (Data Transmission Service) relies on database middleware to implement data replication and synchronization based on database tables, and has a heavy reliance on the consistency of data structure and the middleware deployment architecture; most data conversion and synchronization systems run as independent service deployments, such as more commonly used open source systems, most of which require independent deployment, the configuration process is relatively cumbersome, and integration with business systems is difficult to achieve; data processing logic is not universal, for example, it only supports data processing based on database tables, or is only used for data processing in a certain business field or scenario, and has poor scalability.

[0155] Based on the above scenarios, this application provides a data conversion method and device based on a data conversion model to implement data conversion functions in business applications, and through configurable and scalable design, support application scenarios such as data conversion, synchronization, and migration of business systems. By defining the data conversion model and configuring the conversion logic based on the model, and based on the processing program provided by the built-in data mapping unit, flexible conversion of business data can be achieved. In addition, the conversion process and data processing logic can be customized through the configuration mechanism and extension mechanism. Common data combination, splitting, mapping, and other processing can be performed on the data according to the data conversion model and mapping relationship to meet the data conversion processing in different scenarios.

[0156] Specifically, the data conversion device includes a data conversion model determination unit for defining the entity structure and relationship of the input and output of the data to be converted. Usually in a business system, business data is composed of multiple different business entities, and there are combinations and dependencies between different business entities. These structures and relationships are usually reflected in the design and definition of database tables. In this application, the data conversion model is used as the basis for data conversion, and the configuration, extension and conversion execution of data conversion are all based on the data conversion model. The data conversion model is based on the abstract definition of business entity structure and relationship, and has nothing to do with the specific data storage medium. Among them, the complete data conversion model includes: business entity data, including entity attribute name and type, and business verification rules, such as length, default value, whether to allow null, uniqueness, etc.; mapping relationship between business entities, such as reference, inclusion and other combination logic between entity types.

[0157] Furthermore, the data conversion device also includes a data conversion configuration unit for specifying the steps and logic of data conversion, mainly including data mapping configuration and task configuration. The data mapping configuration defines the mapping rules from the first data of the source entity to the second data of the target entity, including data mapping rules, data merging rules, and data splitting rules. The task configuration defines the execution logic of the input, conversion, and output of the data conversion task, including specifying the first data acquisition device of the source entity, the execution order and dependency relationship of multiple first data when performing conversion, the data processing steps, and the data output device.

[0158] Among them, Figure 8 The following figure shows the data conversion mapping relationship from material to commodity. In the data mapping configuration, the data mapping rules define the attribute mapping relationship between different business entities, that is, an attribute of the source entity is converted to an attribute of the target entity.

[0159] Furthermore, the present application supports data splitting and merging between multiple sources and multiple targets. Specifically, the data conversion configuration unit supports merging or splitting of input and output data, such as splitting a first data of a source entity to generate multiple second data corresponding to target entities, or generating them into multiple different target entities. It also supports merging multiple source entities to generate a single target entity. Figure 9 As shown, the data merging and splitting from materials to commodities are as follows: the commodity data in the first commodity comes from the material data of two source entities, material and material atlas; the material data of the source entity generates the commodity data of two target entities in the mapping configuration.

[0160] Furthermore, the data conversion device also includes a rule expansion unit, which implements an extensible mechanism and enables user-defined control over the data input, conversion, and output processes. This expansion can cover a wider range of application scenarios, such as real-time data conversion in business systems involving data flow, heterogeneous data conversion during system migration, and real-time synchronization of heterogeneous databases using database archive logs.

[0161] Furthermore, the data conversion device also includes a core processing unit, which provides a flexible configuration and expansion mechanism, common operations and core logic, as well as flexible deployment and integration methods, solving the problems of complex system deployment and configuration in the past. At the same time, it realizes the conversion logic and core processing procedures in the data conversion process to support data conversion processing based on user configuration and extension definition, and ensure the correctness and consistency of the data conversion results.

[0162] Furthermore, the data conversion device also includes an execution unit for providing processing services during the data conversion operation.

[0163] Furthermore, the present application also provides two different deployment methods for the corresponding device, including: an integrable SDK (Software Development Kit) form to output an integrable development framework component that can be integrated into existing services, and configure the generated conversion jobs through interface calls and other methods, and automatically or manually perform the conversion. An independent toolkit form provides a toolkit in the form of outputting an executable file. Users can modify the configuration after deployment in any environment and execute it when needed. Through two different deployment methods, it can adapt to the data conversion needs in different business scenarios.

[0164] Example 7:

[0165] like Figure 7As shown, an embodiment of the second aspect of the present invention proposes a data conversion device 700, including: a data conversion model determination unit 706, used to determine a target data conversion model from multiple preset data conversion models according to the entity types of multiple source entities; the data conversion model determination unit 706 is also used to determine the mapping relationship between multiple source entities and multiple target entities according to the target data conversion model; a data conversion configuration unit 708 is used to determine the conversion order of performing data conversion on the first data of multiple source entities according to the target data conversion model; an execution unit 712 is used to perform data conversion on the first data according to the target data conversion model, the conversion order and the mapping relationship, generate second data, and write the second data information into multiple target entities.

[0166] In this embodiment, the data conversion device 700 includes a data conversion model determination unit 706, a data conversion unit, a data conversion configuration unit 708, and an execution unit 712. After receiving a data conversion request, the data conversion device 700 obtains multiple source entities contained in the data conversion request, target entities corresponding to the multiple source entities, entity types of the multiple source entities, and first data stored in the multiple source entities, wherein the data conversion request indicates that the first data in the multiple source entities should be converted and written into the corresponding target entities. The data conversion model determination unit 706 is used to determine a target data conversion model from a plurality of preset data conversion models based on the entity types of the multiple source entities, wherein the preset data conversion model is a standard data conversion model pre-constructed based on different business scenarios, and the target data conversion model is a data conversion model corresponding to the source entity and the target entity.

[0167] Furthermore, the data conversion model determination unit 706 is further configured to define mapping relationships between multiple source entities and multiple target entities of the data to be converted. Specifically, in a business system, business data is composed of multiple different business entities, and there are combinations and dependencies between different business entities. These combinations and relationships are usually reflected in the definition of database tables. The preset data conversion model contains mapping relationships between various business entities that are pre-configured according to business requirements. Based on the determined target conversion model, the mapping relationships between multiple source entities and multiple target entities can be determined.

[0168] Furthermore, the data conversion configuration unit 708 is used to specify the steps and conversion logic of data conversion, mainly including data conversion configuration and task configuration. Among them, the data conversion configuration defines the data conversion rules from the first data to the second data, and the data conversion rules include data mapping rules, data merging rules and data splitting rules. The task configuration defines the conversion order of the data conversion. Specifically, the data conversion of the first data is performed in the form of a pipeline, and the target data conversion model contains the pre-configured priority of the first data, wherein the priority is the priority of the pipeline in the entire data conversion, which is used to determine the conversion order of different first data. The conversion order of the first data is determined according to the conversion logic of the target data conversion model.

[0169] Furthermore, the execution unit 712 is configured to convert the first data into second data according to the target data conversion model, the conversion order of the first data, and the mapping relationship between the source entity and the target entity, and then synchronize the second data into the target entity.

[0170] Through the above-mentioned method, a flexible configuration data conversion logic is provided based on the data conversion model, and the first data is converted according to the data conversion logic to realize the flexible conversion of business data and the rapid transmission of complex data relationships in the business system, which effectively improves the efficiency of data conversion. At the same time, through different business entities, the corresponding target data conversion model is determined to realize the call of cross-class data conversion models between different business scenarios, meet the data conversion processing under different business scenarios, and realize data conversion between heterogeneous business systems or different business entities. Compared with the prior art that relies on database middleware to realize data replication and synchronization based on database tables, and has a heavy dependence on the middleware deployment construction of data structures, this application selects a data conversion model for the business entity, and realizes the data conversion function in the business application based on the data conversion logic configured by different data conversion models. It does not rely on specific business systems and middleware products, achieves better versatility, and solves the problems of single system functions and tight coupling in the prior art.

[0171] Example 8:

[0172] like Figure 7As shown, an embodiment of the present invention, based on the above embodiment, further: the data conversion device 700 also includes: a parsing unit 702, the parsing unit 702 is used to parse the data conversion request in response to the data conversion request, and obtain multiple source entities, multiple target entities, multiple source entity types and multiple source entity first data included in the data conversion request; an input data reading unit 704 is used to filter the input first data of multiple source entities; a data processing collection unit 710 is used to merge, split, map and convert the first data according to the conversion order and conversion method of the first data; a data consistency processing unit 714 is used to update the target data module according to the first data and the second data; an output data generation unit 716 is used to summarize the converted second data for subsequent output.

[0173] In this embodiment, the data conversion device 700 further includes a parsing unit 702 , an input data reading unit 704 , a data processing aggregation unit 710 , a data consistency processing unit 714 and an output data generating unit 716 .

[0174] Specifically, after the data conversion device 700 receives a data conversion request, the parsing unit 702 is used to parse the data conversion request, obtain multiple source entities contained in the data conversion request, target entities corresponding to the multiple source entities, entity types of the multiple source entities, and the first data stored in the multiple source entities, and create a corresponding conversion job during the data conversion execution according to the configuration of the data conversion configuration unit 708, wherein the job includes the first data pipeline, the processing steps in any pipeline, and the instantiation of the extension plug-in.

[0175] Furthermore, the input data reading unit 704 is used to read the input first data and perform processing such as filtering and paging on the first data according to the configuration parameters defined by the target data conversion model.

[0176] Furthermore, the data processing collection unit 710 is used to implement the default data processing process and encapsulate the commonly used data conversion logic, wherein the default data processing process mainly includes: data merging process, data splitting process, context variable parsing process, data mapping process, primary key conversion process and type conversion process.

[0177] Furthermore, the data consistency processing unit 714 is used to update the target entity and the target data module according to the user-defined idempotent verification rules or repeated update strategy. Specifically, the data conversion request includes the user-defined verification rules, and the user-defined verification rules are imported into the target data conversion model to verify the target entity. Among them, the verification rules can be idempotent verification rules or repeated update strategies. When the second data is written to the target entity, the target entity is traversed based on the second data, and based on the provided idempotent verification rules, it is determined whether a consistent data result is obtained when executing the conversion. In addition, the repeated update strategy can also be used to specify whether to update the target data conversion model after the first data conversion when repeatedly executing the data conversion.

[0178] Furthermore, the output data generation unit 716 is used to implement data output control. Specifically, if the amount of first data is large, the first data is segmented and converted according to the amount of first data, and then the second data is generated. The generated second data is then aggregated or fully generated for subsequent data output operations. Among them, the user can customize the reception of the converted second data immediately during the data conversion process or receive the aggregated second data after the data conversion is completed.

[0179] Through the above method, the conversion logic and core processing steps in the data conversion process are provided, focusing on solving the conversion logic and data processing in the data conversion process, without relying on specific business systems and middleware products, and improving the versatility of the data conversion device 700.

[0180] Example 9:

[0181] like Figure 7 As shown, an embodiment of the present invention, based on the above embodiment, further: the data conversion device 700 also includes: a rule extension unit 718, which is used to respond to the extension request, obtain the extension rule for data conversion of the first data contained in the extension request, and update the target data conversion model according to the extension rule.

[0182] In this embodiment, the rule extension unit 718 is used to receive an extension request sent by a user and obtain the extension rules contained in the extension request, wherein the extension rules are an extension mechanism for controlling the data conversion flow program and logic provided by the user. The target data conversion model is updated according to the extension rules so that the general conversion logic in the target data conversion model is combined with the conversion mechanism required by the user.

[0183] Through the above method, plug-in expansion is achieved, which enables users to intervene in and control each link of data conversion, making the data conversion logic more in line with user needs, improving the reusability and flexibility of the data conversion logic to meet more business scenarios.

[0184] Specifically, the extension mechanisms of the rule extension unit 718 include: custom data input, which allows users to customize the source of conversion data based on the data input interface provided by the target data conversion model, for example, reading data from a database instance, or obtaining input data from a file system or external service. Based on the interface context provided by the target data conversion model and context parameters, users can customize operations such as data paging and filtering. Custom data mapping, which allows users to extend the processing logic of data mapping. For example, when processing data mapping, users can customize complex functions or service calls to control and process the mapping process and generate the desired result values. Custom processing steps, which allow users to customize data processing steps and configure them to the conversion pipeline, processing the current data set within the steps. Custom data validation, which allows users to customize data validation rules. Custom data output, which allows users to customize data output processing based on the data output interface provided by the target data conversion model, such as persisting data to a database, saving to a file system, or synchronizing data to other business systems via an external service interface.

[0185] Example 10:

[0186] An embodiment of the third aspect of the present invention proposes a data conversion device, including a memory storing a program or instruction; a processor connected to the memory, and the processor is configured to implement the data conversion method proposed in the first aspect when executing the program or instruction.

[0187] In this embodiment, after receiving a data conversion request, multiple source entities contained in the data conversion request, multiple target entities corresponding to the multiple source entities, entity types of the multiple source entities, and first data stored in the multiple source entities are obtained, wherein the data conversion request indicates that the first data in the multiple source entities is converted and written into the corresponding multiple target entities.

[0188] Furthermore, based on the entity types of multiple source entities, a target data conversion model is determined among multiple preset data conversion models, wherein the preset data conversion model is a standard data conversion model pre-constructed based on different business scenarios, and the target data conversion model is a data conversion model corresponding to the source entity and the target entity.

[0189] Furthermore, the preset data conversion model includes mapping relationships between various business entities that are pre-configured according to business requirements. According to the determined target conversion model, the mapping relationship between the source entity and the target entity can be determined.

[0190] Furthermore, the data conversion of the first data is performed in a pipelined manner. The target data conversion model includes a pre-configured priority of the first data. The priority is a priority of the pipeline in the entire data conversion process, which is used to determine the conversion order of different first data. The conversion order of the first data is determined based on the conversion logic of the target data conversion model.

[0191] Furthermore, according to the target data conversion model, the conversion order of the first data, and the mapping relationship between the source entity and the target entity, the first data is converted into the second data, and then the second data is synchronized into the target entity.

[0192] The first data may include data attribute information, such as attribute name, attribute type, attribute classification, and measurement unit.

[0193] Through the above-mentioned method, a flexible configuration data conversion logic is provided based on the data conversion model, and the first data is converted according to the data conversion logic to realize the flexible conversion of business data and the rapid transmission of complex data relationships in the business system, which effectively improves the efficiency of data conversion. At the same time, through different business entities, the corresponding target data conversion model is determined to realize the call of cross-class data conversion models between different business scenarios, meet the data conversion processing under different business scenarios, and realize data conversion between heterogeneous business systems or different business entities. Compared with the prior art that relies on database middleware to realize data replication and synchronization based on database tables, and has a heavy dependence on the middleware deployment construction of data structures, this application selects a data conversion model for the business entity, and realizes the data conversion function in the business application based on the data conversion logic configured by different data conversion models. It does not rely on specific business systems and middleware products, achieves better versatility, and solves the problems of single system functions and tight coupling in the prior art.

[0194] Example 11:

[0195] An embodiment of the fourth aspect of the present invention provides a readable storage medium having a program or instruction stored thereon, which, when executed by a processor, implements the steps of the data conversion method as described in any of the above technical solutions.

[0196] The readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0197] The readable storage medium provided by the present invention implements the steps of the data conversion method of any of the above-mentioned technical solutions when the program or instruction is executed by the processor. Therefore, the readable storage medium includes all the beneficial effects of the data conversion method of any of the above-mentioned technical solutions, which will not be repeated here. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data conversion method, characterized in that: include: In response to a data conversion request, acquiring a plurality of source entities, a plurality of target entities, entity types of the plurality of source entities, and first data of the plurality of source entities included in the data conversion request; Determining a target data conversion model from a plurality of preset data conversion models according to the entity type; Determining mapping relationships between a plurality of source entities and a plurality of target entities according to the target data conversion model; determining, according to the target data conversion model, a conversion order for performing data conversion on the first data of the plurality of source entities; Performing data conversion on the first data according to the target data conversion model, the conversion order, and the mapping relationship to generate second data, and writing the second data into the multiple target entities; The step of writing the second data into the multiple target entities specifically includes: Acquire a data format of the second data; Determining a standard data format of the target entity according to the target data conversion model; comparing the data format with the standard data format; Based on the data format being the same as the standard data format, writing the second data into the target entity; Based on the fact that the data format is different from the standard data format, the second data is deleted.

2. The data conversion method according to claim 1, wherein: The step of determining, according to the target data conversion model, a conversion order for performing data conversion on the first data of the multiple source entities specifically includes: determining the priority of the first data according to the target data conversion model; The first data is sorted according to the priority to obtain the conversion order.

3. The data conversion method according to claim 1, wherein: The step of performing data conversion on the first data to generate the second data according to the target data conversion model, the conversion sequence and the mapping relationship specifically includes: Determining a conversion method for the first data according to the mapping relationship; The first data is input into the target data conversion model, and the first data is converted into the second data according to the conversion method and the conversion order.

4. The data conversion method according to claim 3, wherein: The conversion method includes at least one of the following: data combination, data splitting and data mapping.

5. The data conversion method according to any one of claims 1 to 4, characterized in that: Before obtaining the plurality of source entities, the plurality of target entities, the entity types of the plurality of source entities, and the first data of the plurality of source entities contained in the data conversion request in response to the data conversion request, the method further includes: Get converted historical data; Determining, based on the historical data, combination logic between the plurality of source entities and the plurality of target entities and verification rules for the historical data; The preset data conversion model is created according to the combination logic and the verification rules.

6. The data conversion method according to claim 5, characterized in that: The verification rules include at least one of the following: data length verification, default data verification, and data null value verification.

7. A data conversion device, characterized in that: include: a data conversion model determining unit, configured to determine a target data conversion model from a plurality of preset data conversion models according to entity types of a plurality of source entities; The data conversion model determining unit is further configured to determine a mapping relationship between a plurality of source entities and a plurality of target entities according to the target data conversion model; a data conversion configuration unit, configured to determine a conversion order for performing data conversion on the first data of the plurality of source entities according to the target data conversion model; an execution unit, configured to perform data conversion on the first data according to the target data conversion model, the conversion order, and the mapping relationship, generate second data, and write the second data into the plurality of target entities; Also includes: a parsing unit, configured to parse the data conversion request in response to the data conversion request, and obtain the multiple source entities, the multiple target entities, the entity types of the multiple source entities, and the first data of the multiple source entities included in the data conversion request; an input data reading unit, configured to filter the first data of the plurality of source entities input; a data processing aggregation unit, configured to perform data merging, data splitting, data mapping, and data conversion on the first data according to the conversion order and conversion method of the first data; a data consistency processing unit, configured to update the target data conversion model according to the first data and the second data; The output data generating unit is used to summarize the converted second data for subsequent output.

8. The data conversion device according to claim 7, characterized in that: Also includes: The rule extension unit is configured to, in response to an extension request, obtain an extension rule for performing data conversion on the first data contained in the extension request, and update the target data conversion model according to the extension rule.

9. A data conversion device, characterized in that: include: a memory storing programs or instructions; The processor is connected to the memory, and when the processor executes the program or instruction, the steps of the data conversion method according to any one of claims 1 to 6 are implemented.

10. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the data conversion method according to any one of claims 1 to 6 are implemented.

Images