A data synchronization engine and data query synchronization method

Abstract

The application relates to a method for constructing a data synchronization engine and data query synchronization, a method for constructing a data synchronization engine, which comprises the following steps: step one: configuring three annotations, the annotations comprising: @EnableIaaSDataEngine, @DataInterceptor and @DataBoxHandler; step two: configuring an intercepting processing class interface, which comprises: defining an intercepting processing class interface DataBox, defining processing logic after successful interception, and defining AbstractDataBox; step three: defining an interceptor, parameter management and registering the interceptor; and a method for data query synchronization, which comprises: using the data synchronization engine to query synchronization data. The application solves the problems of data inconsistency and disordered and inharmonious front-end interface display logic caused by switching different underlying IaaS resources under the same management interface.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a method for constructing a data synchronization engine and a data query synchronization method. Background Technology

[0002] In cloud computing scenarios, we often encounter situations where a single PaaS management console interfaces with and adapts to multiple IaaS resources. In these cases, the architecture of the cloud resource management and control platform is typically the same. Therefore, our multi-cloud adaptation management console faces a scenario where one resource pool interfaces with IaaS1, while another resource pool interfaces with IaaS2. If the data retrieved from the two IaaS pools on the same console is inconsistent, it will lead to inconsistencies in the content displayed on the front-end interface. Furthermore, the types and definitions of the data fields returned by different IaaS layers also differ. When we directly query the underlying IaaS layer data based on the API and display it on the front-end page, different resource pools will have different interfaces. Summary of the Invention

[0003] Based on the above problems, this application provides a method for constructing a data synchronization engine and data query synchronization.

[0004] A method for building a data synchronization engine and data query synchronization, wherein building the data synchronization engine includes the following three steps:

[0005] Step 1: Configure the data engine with three annotations: @EnableIaaSDataEngine, @DataInterceptor, and @DataBoxHandler.

[0006] Step 2: Configure the interception processing class interface in the data engine. Configuring the interception processing class interface includes: defining the interception processing class interface DataBox, defining the processing logic after successful interception, and defining AbstractDataBox.

[0007] Step 3: Define interceptors, manage parameters, and register interceptors;

[0008] The parameters include: the input parameters and return parameter types of the interception method;

[0009] Methods for data query and synchronization include: using a data synchronization engine to query and synchronize data.

[0010] Furthermore, in the above method for building a data synchronization engine and data query synchronization, the @EnableIaaSDataEngine annotation is placed on the project's configuration class or startup class to indicate that the data engine function is enabled; @EnableIaaSDataEngine includes the values ​​of the interceptors attribute, the handlers attribute, and the iaasType attribute.

[0011] The `interceptors` attribute value annotation needs to define the path to the Mybatis interception method and its interception mechanism.

[0012] The `handlers` attribute value specifies the directory of the classes to be intercepted and processed.

[0013] The IaaSType attribute value specifies the type of IaaS layer resource to be intercepted.

[0014] Furthermore, in the above method for building a data synchronization engine and data query synchronization, the @DataInterceptor annotation is placed on the DAO method of Mybatis and its interception mechanism that needs to be intercepted. The @DataInterceptor includes the bean attribute.

[0015] The `bean` attribute specifies the interceptor class corresponding to the method to be intercepted, as well as the name of the interceptor class.

[0016] Furthermore, in the aforementioned method for constructing a data synchronization engine and data query synchronization, the @DataBoxHandler annotation on the class corresponds to the bean attribute in the @DataInterceptor annotation, and is the interception processing class corresponding to the method to be intercepted.

[0017] Furthermore, in the above method for constructing a data synchronization engine and data query synchronization, the defined processing logic after successful interception includes: the interception processing class implements the execute method to customize the processing logic after successful interception.

[0018] Specifically, in this embodiment of the application, the interception processing class needs to implement the execute method to customize the processing logic after successful interception, such as validating or replacing the queried data.

[0019] Furthermore, in the aforementioned method for constructing a data synchronization engine and data query synchronization, AbstractDataBox is defined to intercept the types of IaaS layer resources that need to be intercepted, as specified in the IaaSType attribute value annotation of @EnableIaaSDataEngine.

[0020] Specifically, in this embodiment, the IsFilter method determines whether an application will be intercepted based on the iaasType attribute value configured in the @EnableIaaSDataEngine annotation. Generally, only the implementation of the processing method needs to be considered, while the interception judgment method can be expressed in a generic way: only those conforming to the defined IaasType are allowed to be intercepted, and the rest are not. Based on this, this application defines an abstract class AbstractDataBox to implement this method. When implementing the interception processing logic, this abstract class can be inherited.

[0021] Furthermore, in the aforementioned method for constructing a data synchronization engine and data query synchronization, the data synchronization engine is implemented based on Java annotations and the database persistence layer framework Mybatis and its interception mechanism.

[0022] The proposed method for building a data synchronization engine and data query synchronization is based on Java annotations. It only requires configuring the JDK environment and does not require any other additional dependencies. It features low coupling with other platforms, cross-platform operation, and portability.

[0023] Furthermore, in the above method for constructing a data synchronization engine and data query synchronization, registering an interceptor includes: based on the interception directory configured by the @EnableIaaSDataEngine annotation and the type of IaaS resource to be intercepted, obtaining the dao method annotated with @DataInterceptor and the interception processing class specified in the bean property in the project, wherein the interception processing class is annotated with @DataBoxHandler, obtaining and saving the input and return parameter types of the interception method, registering the interception processing class and saving a reference to the interception processing class.

[0024] Specifically, in this embodiment, based on the interception directory configured by the @EnableIaaSDataEngine annotation and the type of IaaS resource to be intercepted (determined by the IaasType attribute value), the project obtains the dao method annotated with @DataInterceptor and the interception processing class specified in the bean attribute (annotated with @DataBoxHandler), obtains and saves the input and return parameter types of the interception method, and registers and saves the reference to the interception processing class.

[0025] Furthermore, in the above-mentioned method for constructing a data synchronization engine and data query synchronization, the registration of interceptors relies on the Java language's class loader mechanism to load interception processing classes, the Spring framework's bean management mechanism to load bean properties, read environment variable parameters of bean properties, and read the classes and method implementations annotated with annotations.

[0026] Specifically, in this embodiment, the interceptor registration mechanism relies on the Java language's class loader mechanism to load the interceptor processing class, the Spring framework's bean management mechanism to load bean properties and read environment variable parameters of bean properties, and to read the classes and method implementations annotated with annotations.

[0027] Furthermore, in the above-mentioned method for constructing a data synchronization engine and synchronizing data queries, using the data synchronization engine to query synchronized data includes:

[0028] When a request is made to the corresponding intercepted database method, the defined interceptor, the registered interceptor, the input parameters and return parameter types of the interception method are used to call the execute method of the DataBox handler interface to perform interception processing, implement custom logic to obtain the processing result, and return the processing result according to the return parameter type.

[0029] Specifically, in this embodiment of the application, when a request is made to the corresponding intercepted database method, the data synchronization engine will execute the interception and complete the processing according to the custom processing logic, such as verifying and replacing the returned data after calling different IaaS resources.

[0030] The advantages of this application's embodiments are as follows: 1) Compared to existing multi-cloud adaptation management and control platforms, this application, while ensuring the integrity of the original system platform's functions, solves the problems of data inconsistency and chaotic front-end interface display logic caused by switching between different underlying IaaS resources under the same management interface. Furthermore, the method for building the data synchronization engine is implemented in Java, requiring only JDK environment configuration without other additional dependencies. It features low coupling with other platforms, cross-platform compatibility, and portability. The method for building the data synchronization engine uses annotation configuration and interception mechanisms. When introduced into a project, it can be freely enabled or disabled based on annotations, resulting in minimal intrusion into the original project and even no impact on the original logic. Moreover, deploying such a data engine requires very few resources, and its operation does not consume many resources. As a data engine system, it has the advantages of low intrusion and low resource consumption. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram illustrating the construction of a data synchronization engine, as provided in an embodiment of this application. Detailed Implementation

[0033] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0035] Figure 1 This is a schematic diagram illustrating the construction of a data synchronization engine, as provided in an embodiment of this application.

[0036] A method for building a data synchronization engine and data query synchronization, comprising building a data synchronization engine, and combining... Figure 1 It includes the following three steps, S101 to S103:

[0037] S101: The data engine is configured with three annotations: @EnableIaaSDataEngine, @DataInterceptor, and @DataBoxHandler.

[0038] Specifically, in this embodiment of the application, the @EnableIaaSDataEngine annotation is placed on the configuration class or startup class of the project to indicate that the data engine function is enabled; @EnableIaaSDataEngine includes: interceptors attribute value, handlers attribute value, and iaasType attribute value;

[0039] The `interceptors` attribute value annotation needs to define the path to the Mybatis interception method and its interception mechanism.

[0040] The `handlers` attribute value specifies the directory of the classes to be intercepted and processed.

[0041] The IaaSType attribute value specifies the type of IaaS layer resource to be intercepted.

[0042] The following is the code for the @EnableIaaSDataEngine annotation:

[0043] @Target(ElementType.TYPE)

[0044] @Retention(RetentionPolicy.RUNTIME)

[0045] @Import(IaaSDataEngineRegistrar.class)

[0046] public@interface EnableIaaSDataEngine{

[0047] / ** Directory of interception and handling classes...* /

[0048] String[]handlers()default{};

[0049] / ** The directory for the interception point, the same directory as the MyBatis DAO and mapping interfaces. * /

[0050] String[]interceptors()default{};

[0051] / ** The type of IaaS to connect to...* /

[0052] String[]iaasType()default{"cdesk"}

[0053] }

[0054] The @DataInterceptor annotation is placed on the DAO methods of Mybatis and its interception mechanism that need to be intercepted. @DataInterceptor includes: bean attribute;

[0055] The `bean` attribute specifies the interceptor class corresponding to the method to be intercepted, as well as the name of the interceptor class.

[0056] The following is the code for the @DataInterceptor annotation:

[0057] @Target(ElementType.METHOD)

[0058] @Retention(RetentionPolicy.RUNTIME)

[0059] public@interface DataInterceptor{

[0060] String bean()default"commonDataHandler";

[0061] String method()default"execute";

[0062] }

[0063] The @DataBoxHandler annotation on a class corresponds to the bean attribute in the @DataInterceptor annotation, and is the interception processing class for the method to be intercepted.

[0064] The following is the code for the @DataBoxHandler annotation:

[0065] @Target(ElementType.TYPE)

[0066] @Retention(RetentionPolicy.RUNTIME)

[0067] public@interface DataBoxHandler{

[0068] String value()default"";

[0069] }

[0070] S102: Configure the interception processing class interface of the data engine. The configuration of the interception processing class interface includes: defining the interception processing class interface DataBox, defining the processing logic after successful interception, and defining AbstractDataBox.

[0071] Specifically, in this embodiment, the interception processing class needs to implement the `execute` method to customize the processing logic after successful interception, such as validating or replacing the queried data. The `IsFilter` method determines whether an interception will occur, depending on the `iaasType` attribute value configured in the `@EnableIaaSDataEngine` annotation. Generally, only the implementation of the processing method needs to be considered, while the interception judgment method can be expressed in a general way: only those conforming to the defined `IaasType` type are allowed to be intercepted, and the rest are not intercepted. Based on this, this application defines an abstract class `AbstractDataBox` to implement this method. When implementing the interception processing logic, this abstract class can be inherited.

[0072] The following is the code for implementing the DataBox interface of the interception handling class:

[0073] public interface DataBox

[0074] <e>List <e>execute(HashMap args,E resultType);

[0075] <e>List <e>execute(HashMap args, Class resultClass) throws ClassNotFoundException, InstantiationE

[0076] boolean isFilter(HashMap args);

[0077] The following is the code for defining AbstractDataBox:

[0078] public abstract class AbstractDataBox implements DataBox{

[0079] @Override

[0080] public <e>List <e>execute(HashMapargsClass resultClaas)throwsClassNotFoundException,Insta

[0081] Object o=resultClaas.newInstance();

[0082] List ret=new ArrayList<>();

[0083] ret.add(o);return ret;

[0084] }

[0085] @Override

[0086] public boolean isFilter(HashMap args){

[0087] if(args.get("resPoolCode")==null){

[0088] return false;

[0089] }

[0090] BaseConfig baseConfig=SdkApplicationContextUtil.getBean(beanld:

[0091] "dataEngineBaseConfig")String resPoolCode=(String)args.get("resPoolCode");

[0092] try{

[0093] String iaasType=baseConfig.checkIaaSTypeByCode(resPoolCode):

[0094] return DataMappingUtil.getIaasType().contains(iaasType);

[0095] }catch(Exception e){

[0096] }

[0097] return false;

[0098] }

[0099] S103: Define interceptors, manage parameters, and register interceptors.

[0100] Specifically, in this embodiment of the application, the interceptor is defined through code, and the managed parameters include the input parameters and return parameter types of the interception method. The method for registering the interceptor is described below.

[0101] Methods for data query and synchronization include: using a data synchronization engine to query and synchronize data.

[0102] Furthermore, in the above method for constructing a data synchronization engine and data query synchronization, the defined processing logic after successful interception includes: the interception processing class implements the execute method to customize the processing logic after successful interception.

[0103] Specifically, in this embodiment of the application, the interception processing class needs to implement the execute method to customize the processing logic after successful interception, such as validating or replacing the queried data.

[0104] Furthermore, in the aforementioned method for constructing a data synchronization engine and data query synchronization, AbstractDataBox is defined to intercept the types of IaaS layer resources that need to be intercepted, as specified in the IaaSType attribute value annotation of @EnableIaaSDataEngine.

[0105] Specifically, in this embodiment, the IsFilter method determines whether an application will be intercepted based on the iaasType attribute value configured in the @EnableIaaSDataEngine annotation. Generally, only the implementation of the processing method needs to be considered, while the interception judgment method can be expressed in a generic way: only those conforming to the defined IaasType are allowed to be intercepted, and the rest are not. Based on this, this application defines an abstract class AbstractDataBox to implement this method. When implementing the interception processing logic, this abstract class can be inherited.

[0106] Furthermore, in the aforementioned method for constructing a data synchronization engine and data query synchronization, the data synchronization engine is implemented based on Java annotations and the database persistence layer framework Mybatis and its interception mechanism.

[0107] The proposed method for building a data synchronization engine and data query synchronization is based on Java annotations. It only requires configuring the JDK environment and does not require any other additional dependencies. It features low coupling with other platforms, cross-platform operation, and portability.

[0108] Furthermore, in the above method for constructing a data synchronization engine and data query synchronization, registering an interceptor includes: based on the interception directory configured by the @EnableIaaSDataEngine annotation and the type of IaaS resource to be intercepted, obtaining the dao method annotated with @DataInterceptor and the interception processing class specified in the bean property in the project, wherein the interception processing class is annotated with @DataBoxHandler, obtaining and saving the input and return parameter types of the interception method, registering the interception processing class and saving a reference to the interception processing class.

[0109] Specifically, in this embodiment, based on the interception directory configured by the @EnableIaaSDataEngine annotation and the type of IaaS resource to be intercepted (determined by the IaasType attribute value), the project obtains the dao method annotated with @DataInterceptor and the interception processing class specified in the bean attribute (annotated with @DataBoxHandler), obtains and saves the input and return parameter types of the interception method, and registers and saves the reference to the interception processing class.

[0110] The following is the code for interceptor registration:

[0111] PublicclassIaaSDataEngineRegistrar implenentsImportBeanDefinitionRegistrar,ResourceLoaderAware,BeanClassLoaderAware,Environmet

[0112] private static final Logger logger=LoggerFactory.getLogger(IaaSDataEngineRegistrar.class);

[0113] private ResourceLoader resourceLoader;

[0114] private classLoader classLoader;

[0115] private environment environment;

[0116] @Override

[0117] public void setBeanClassLoader(ClassLoader classLoader)(this.classLoader=classLoader;}

[0118] @Override

[0119] public void setEnviromment(Environment environment){this.environment=environment;}

[0120] @Override

[0121] public void registerBeanDefinitions(AnnotationMetadata metadata,BeanDefinitionRegistry registry){

[0122] AnnotationAttributesannoAttrs=AnnotationAttributes.fromap(metadata.getA notationAttributes(EnableIaaSDataEngine.class.ge

[0123] registerBoxHandler(metadata,registry.annoAttrs);

[0124] cacheDataInterceptor(annoAttrs);

[0125] Furthermore, in the above-mentioned method for constructing a data synchronization engine and data query synchronization, the registration of interceptors relies on the Java language's class loader mechanism to load interception processing classes, the Spring framework's bean management mechanism to load bean properties, read environment variable parameters of bean properties, and read the classes and method implementations annotated with annotations.

[0126] Specifically, in this embodiment, the interceptor registration mechanism relies on the Java language's class loader mechanism to load the interceptor processing class, the Spring framework's bean management mechanism to load bean properties and read environment variable parameters of bean properties, and to read the classes and method implementations annotated with annotations.

[0127] Furthermore, in the above-mentioned method for constructing a data synchronization engine and synchronizing data queries, using the data synchronization engine to query synchronized data includes:

[0128] When a request is made to the corresponding intercepted database method, the defined interceptor, the registered interceptor, the input parameters and return parameter types of the interception method are used to call the execute method of the DataBox handler interface to perform interception processing, implement custom logic to obtain the processing result, and return the processing result according to the return parameter type.

[0129] Specifically, in this embodiment of the application, when a request is made to the corresponding intercepted database method, the data synchronization engine will execute the interception and complete the processing according to the custom processing logic, such as verifying and replacing the returned data after calling different IaaS resources.

[0130] The following is the code for the interceptor to implement the interception logic:

[0131] @Intercepts({@Signature(type=Executor.class,method="query".args={MappedStatement.class,Object.class,

[0132] RowBounds.class, ResultHandler.class, CacheKey.class, BoundSq1.class})})

[0133] @s1f4j

[0134] @Component

[0135] public class IaaSDataInterceptor implements Interceptor{

[0136] private Properties properties;

[0137] public Object intercept(Invocation invocation)throws Throwable{

[0138] String sqlid = "";

[0139] Object returnValue = null;

[0140] log.info("intercept-----{}",invocation):

[0141] try{

[0142] Mappedstatement mappedStatement=(MappedStatement)invocation.getArgs()[0]

[0143] sqlId=mappedstatement.getid();

[0144] if(mappedStatement.getResultMaps(().isEmpty()){

[0145] log.info("----no result type--_________");

[0146] return invocation.proceed();

[0147] }

[0148] Class aClass=mappedStatement.getResultMaps().get(0).getType();

[0149] log.info("----type---_--{}----_",aClass);

[0150] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments.

[0151] Those skilled in the art will understand that the descriptions of the various embodiments have different focuses, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.< / e> < / e> < / e> < / e> < / e> < / e>

Claims

1. A method of building a data synchronization engine and data query synchronization, characterized by, The construction of the data synchronization engine includes the following three steps: Step 1: Configure the data engine with three annotations: @EnableIaaSDataEngine, @DataInterceptor, and @DataBoxHandler. Step 2: Configure the interception processing class interface in the data engine. The configuration of the interception processing class interface includes: defining the interception processing class interface DataBox, defining the processing logic after successful interception, and defining AbstractDataBox. Step 3: Define interceptors, manage parameters, and register interceptors; The parameters include: the input parameters and return parameter types of the interception method; The data query and synchronization method includes: using the data synchronization engine to query synchronized data; The @EnableIaaSDataEngine annotation is placed on the project's configuration or startup class to enable the data engine functionality; The @DataInterceptor annotation is placed on the DAO method of the Mybatis instance and its interception mechanism that needs to be intercepted; The @DataBoxHandler annotation on the class corresponds to the bean attribute in the @DataInterceptor annotation, and is the interception processing class corresponding to the method to be intercepted; The definition of AbstractDataBox is used to intercept the type of IaaS layer resources that need to be intercepted, as specified in the IaaSType attribute value annotation of @EnableIaaSDataEngine.

2. The method for constructing a data synchronization engine and data query synchronization according to claim 1, characterized in that, The @EnableIaaSDataEngine includes: interceptors attribute value, handlers attribute value, and IaaSType attribute value; The `interceptors` attribute annotation needs to define the path to the Mybatis object being intercepted and the interception mechanism method. The `handlers` attribute value annotation requires the directory of the interception and processing class; The IaaSType attribute value specifies the type of IaaS layer resource that needs to be intercepted.

3. The method for constructing a data synchronization engine and data query synchronization according to claim 1, characterized in that, The @DataInterceptor includes: bean attribute; The bean attribute annotation specifies the interception class corresponding to the method to be intercepted, as well as the name of the interception class corresponding to the method to be intercepted.

4. The method of claim 1, wherein the data synchronization engine and data query synchronization are constructed. The defined implementation of the processing logic after successful interception includes: the interception processing class implements the execute method to customize the processing logic after successful interception.

5. The method of building a data synchronization engine and data query synchronization of claim 1, wherein, The data synchronization engine is implemented based on Java annotations and the Mybatis database persistence layer framework and its interception mechanism.

6. The method of building a data synchronization engine and data query synchronization of claim 1, wherein, The registration interceptor includes: based on the interception directory configured by the @EnableIaaSDataEngine annotation and the type of IaaS resource to be intercepted, obtaining the dao method annotated with @DataInterceptor and the interception processing class specified in the bean property in the project, wherein the interception processing class is annotated with @DataBoxHandler, obtaining and saving the input and return parameter types of the interception method, registering the interception processing class and saving a reference to the interception processing class.

7. The method of building a data synchronization engine and data query synchronization of claim 1, wherein, The registered interceptor relies on Java's class loader mechanism to load the interception processing class, Spring's bean management mechanism to load bean properties, read environment variable parameters of bean properties, and read the classes and method implementations annotated with annotations.

8. The method of building a data synchronization engine and data query synchronization of claim 1, wherein, The process of querying synchronized data using the data synchronization engine includes: When a request is made to the corresponding intercepted database method, the defined interceptor, the registered interceptor, the input parameters and return parameter types of the interception method are used to call the execute method of the DataBox handler interface to perform interception processing, implement custom logic to obtain the processing result, and return the processing result according to the return parameter type.

Images