Method, device and equipment for verifying heterogeneous SQL (Structured Query Language) under micro-service architecture and storage medium

Through customized scripts and java agent mechanisms, automated verification and error notification of heterogeneous SQL under the microservice architecture is realized, resource waste and manual error problems are solved, and testing efficiency and accuracy are improved.

CN119065962AInactive Publication Date: 2024-12-03JIUXINGXING (WUHAN) INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411009353.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Under the microservice architecture, when verifying the correctness of heterogeneous SQL, there is waste of resources and high probability of manual errors, third-party tools are not processed in time and parameters are complex, resulting in inefficient testing.

Method used

Start SpringApplication of the master-slave application through custom scripts, intercept SQL requests and send them to a bounded queue of memory, use the java agent mechanism to transmit SQL requests, realize automated testing of heterogeneous database environments, and notify error messages through enterprise WeChat or email.

Benefits of technology

It saves startup, deployment and human resources costs, realizes simultaneous testing of multiple SQL heterogeneous applications, promptly handles error information, and reduces test labor costs and error handling time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065962A_ABST
    Figure CN119065962A_ABST
Patent Text Reader

Abstract

The invention discloses a heterogeneous SQL verification method, device and equipment under a micro-service architecture and a storage medium, and relates to the technical field of micro-services, and the method specifically comprises the steps that other heterogeneous configurations including a main application are started through a customized script; the SpringApplication of the master application and the SpringApplication of the slave application are started at the same time, the master application and the slave application are switched into a java agent, server.port ports of the slave application are different, and the slave application only receives an SQL request; the method comprises the following steps of: intercepting a query method and an update method in an org.apache. Ibasis.executor.BaseExecutor interface of a main mybatis, and sending an SQL (Structured Query Language) request object to a memory bounded queue; sending an SQL request object in the main application queue to other slave nodes, recording information of the slave nodes in a starting script, and transmitting the information in the starting script through an agent mechanism; receiving an SQL request object from the application, and after the SQL request object is transmitted into the spring context object, executing the SQL; after a new file is generated, the new file is sent to an enterprise WeChat group or a mailbox, and the content comprises a server IP and an error file path; the host monitoring service is used as an optional service; the httpserver service of the host serves as an optional service and supports checking or downloading of error files.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of microservice technology, and specifically to a method, device, equipment and storage medium for heterogeneous SQL verification under a microservice architecture. Background Art

[0002] In order to enhance the competitiveness of business products, various database versions need to be developed for the same business function. The resulting problem is that applications of various database versions need to be deployed to support multiple tests of the same function, which requires investment in human resources, time resources, and server resources.

[0003] Currently, in order to verify the correctness of heterogeneous SQL, there is a serious waste of resources and the possibility of human error. In addition, it relies on third-party tools, which leads to untimely processing and notification, and there are situations where parameter objects are complex and cannot be processed. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method, apparatus, device and storage medium for heterogeneous SQL verification under a microservice architecture, which solves the problems mentioned in the above background technology.

[0005] To achieve the above objectives, the present invention first provides a method for heterogeneous SQL verification under a microservice architecture, which specifically includes the following steps: starting other heterogeneous configurations including a main application through a customized script; starting the SpringApplication of the master and slave applications at the same time, the master and slave applications are both cut into the java agent, the server.port ports of the slave application are different, and the slave application only receives SQL requests; intercepting the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and sending the SQL request object to the bounded queue in the memory; sending the SQL request object in the main application queue to other slave nodes, the information of the slave nodes is recorded in the startup script, and passed in through the agent mechanism; receiving the SQL request object from the application, and after passing in the spring context object, executing the SQL; after generating a new file, sending it to the enterprise WeChat group or mailbox, the content includes the server IP and the error file path; the host monitoring service is an optional service; the host httpserver service is an optional service, which supports viewing or downloading error files.

[0006] With this setting, multiple SQL heterogeneous applications can be started with one click through the script, which can save startup, deployment, manpower and resource costs; only one test is required to execute simultaneously in multiple SQL heterogeneous applications, saving testing manpower costs; and as long as the main application is executed, the slave application can handle it in time; as long as the slave application SQL execution error occurs, a message will be immediately received in the enterprise WeChat group, which will inform the specific server IP address and the path of the execution error; you can also view and download files through the browser.

[0007] Furthermore, different directories are specified through spring.config.location to start the corresponding application instance, and the database configuration is modified so that each slave application instance has a different database environment.

[0008] Furthermore, when the spring application is loaded, the spring context object is recorded in the agent-related reference and the netty service is started.

[0009] Furthermore, the netty network encoding and decoding protocol uses the java serialization mechanism.

[0010] Furthermore, after executing httpserver_notice.sh, if a new error file is generated, the server IP and file path where the error file is located will be reported.

[0011] The present invention also provides a device for heterogeneous SQL verification under a microservice architecture, which is characterized in that it includes: a startup module, which starts other heterogeneous configurations including a main application through a customized script, and starts the SpringApplication of the main and slave applications at the same time. The main and slave applications are both cut into the java agent. The server.port ports of the slave application are different, and the slave application only receives SQL requests; a sending request module, which intercepts the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and sends the SQL request object to the bounded queue in the memory; the SQL request object in the main application queue is sent to other slave nodes, and the information of the slave nodes is recorded in the startup script and passed in through the agent mechanism; a receiving module, which receives the SQL request object from the application, and after passing in the spring context object, executes the SQL; a sending file module, after generating a new file, sends it to an enterprise WeChat group or mailbox, and the content includes the server IP and the error file path; the host monitoring service is an optional service, and the host httpserver service is an optional service, which supports viewing or downloading error files.

[0012] Furthermore, different directories are specified through spring.config.location to start the corresponding application instance, and the database configuration is modified so that each slave application instance has a different database environment.

[0013] Furthermore, when the spring application is loaded, the spring context object is recorded in the agent-related references, and the netty service is started; the netty network encoding and decoding protocol uses the java serialization mechanism; after executing httpserver_notice.sh, if a new error file is generated, the server IP and file path where the error file is located will be reported.

[0014] The present invention also provides an electronic device, comprising: a processor, and a memory connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method for heterogeneous SQL verification under the microservice architecture as described in the above technical solution.

[0015] The present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed, they are used to implement the method for heterogeneous SQL verification under the microservice architecture as described in the above technical solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 The present invention provides a technical solution: a method, device, equipment and storage medium for heterogeneous SQL verification under a microservice architecture, specifically including the following embodiments:

[0019] Embodiment 1

[0020] S1. Start other heterogeneous configurations including the main application through customized scripts;

[0021] S2. Start the SpringApplication of the master and slave applications at the same time. Both the master and slave applications are cut into the Java agent. The server.port of the slave application is different and is not registered with the service registration center to ensure that the slave application does not receive web requests but only SQL requests.

[0022] S3. Intercept the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and send the SQL request object to the memory bounded queue. The SQL request object includes the following parts:

[0023] / **

[0024] *

[0025] * /

[0026] private static final long serialVersionUID=1L;

[0027] public Object parameter0object;

[0028] public String ds;

[0029] public String sql;

[0030] public String statementId;

[0031] public int limit=Integer.MAX_VALUE;

[0032] public int offset = θ;

[0033] Where paramObject is the SQL parameter object, which is usually an entity;

[0034] ds is the current data source, which is adapted to the multi-data source mechanism of mybatis plus;

[0035] sql is the database SQL of the current main application version, statementId is the mybatis query ID, and offset and limit are used for paging, as shown below:

[0036] S4. The main application thread sends the SQL request objects in the queue to other slave nodes through Netty. The information of the slave nodes is recorded in the startup script and passed in through the agent mechanism.

[0037] S5, after receiving the SQL request object from the application and passing it into the spring context object mentioned in S2, execute the SQL;

[0038] S6. If a new file is generated, it will be sent to the enterprise WeChat group or email, including the server IP and error file path;

[0039] S7, host monitoring service as an optional service;

[0040] S8. The host httpserver service is an optional service. After executing httpserver_notice.sh, you can open a browser at the IP:10000 port to view or download the error file.

[0041] Embodiment 2

[0042] S1. Start other heterogeneous configurations including the main application through customized scripts, start different application instances by specifying different directories through spring.config.location, and ensure that each slave application instance has a different database environment by modifying the database configuration;

[0043] S2. Start the SpringApplication of the master and slave applications at the same time. Both the master and slave applications are cut into the Java agent. The server.port of the slave application is different and cannot be registered with the service registration center to ensure that the slave application does not receive web requests but only SQL requests. When the spring application is loaded, the spring context object will be recorded in the agent-related reference, and the netty service will be started. The netty service port number is 30000+server.port;

[0044] S3, intercepting the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and sending the SQL request object to the bounded memory queue. The SQL request object includes the same specific parts as shown in Example 1;

[0045] S4. The main application thread sends the SQL request object in the queue to other slave nodes through Netty. The information of the slave nodes is recorded in the startup script and passed in through the agent mechanism. The Netty network encoding and decoding protocol indirectly uses the Java serialization mechanism to ensure lossless transmission of parameters, otherwise it may be distorted.

[0046] S5. After receiving the SQL request object from the application and passing it into the spring context object mentioned in S2, the SQL is executed. There are several key points:

[0047] (1) Set the slave execution flag to cooperate with the master-slave judgment during S3 interception to ensure that the slave SQL is not executed in a loop;

[0048] (2) Set the data source in com.baomidou.dynamic.datasource.toolkit.DynamicDataSourceContextHolder as parameter ds to ensure that the sessionFactory and data source obtained later are correct;

[0049] (3) Get the transaction through sessionFactory-》Configuration-》Environment-》TransactionFactory->Transaction, and get the execution object through sessionFactory-》Configuration-》Environment-》Executor. After the execution is completed, clear the parameters set in steps 1 and 2;

[0050] (4) If an error occurs during execution, a new text file is generated to record the context of the error, including the original SQL, the SQL from the application database, parameters, and exceptions, for easy download and review in the future; the log file name is time error, as shown below:

[0051] After an SQL execution error occurs, a corresponding file will be generated in the server host directory. The directory structure of the file is usually / application / slave node configuration directory / mapperId. In this way, you can clearly understand which application's slave node executed the SQL and caused the error.

[0052] S6. If a new file is generated, it will be sent to the enterprise WeChat group or email, including the server IP and error file path;

[0053] S7. Host monitoring service is an optional service. After executing httpserver_notice.sh, as long as a new error file is generated, the relevant personnel will be notified via corporate WeChat or email, informing them of the server IP and file path where the error file is located.

[0054] S8. The host httpserver service is an optional service. After executing httpserver_notice.sh, you can open a browser at the IP:10000 port to view or download the error file.

[0055] Embodiment 3

[0056] This embodiment provides a device for heterogeneous SQL verification under a microservice architecture, including:

[0057] Start the module, start other heterogeneous configurations including the main application through customized scripts, specify different directories through spring.config.location to start the corresponding application instance, modify the database configuration so that each slave application instance has a different database environment, start the SpringApplication of the master and slave applications at the same time, both the master and slave applications cut into the Java agent, the server.port ports of the slave application are different, and the slave application only receives SQL requests; when the spring application is loaded, the spring context object is recorded in the agent-related reference, and the netty service is started;

[0058] The request sending module intercepts the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and sends the SQL request object to the bounded memory queue; the SQL request object in the main application queue is sent to other slave nodes through the netty method, and the information of the slave nodes is recorded in the startup script and passed in through the agent mechanism; the netty network encoding and decoding protocol uses the java serialization mechanism;

[0059] The receiving module receives the SQL request object from the application, passes it into the spring context object, and executes the SQL;

[0060] The file sending module generates a new file and sends it to the enterprise WeChat group or email address, including the server IP and error file path.

[0061] The host monitoring service is an optional service. After being executed using httpserver_notice.sh, if a new error file is generated, the server IP and file path where the error file is located will be reported. The host httpserver service is an optional service that supports viewing or downloading error files.

[0062] Embodiment 4

[0063] This embodiment provides an electronic device, including: a processor, and a memory connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement a method for heterogeneous SQL verification under a microservice architecture as in the above embodiment.

[0064] Embodiment 5

[0065] This embodiment provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed, they are used to implement the method for heterogeneous SQL verification under the microservice architecture of the above embodiment.

[0066] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0067] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0068] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for heterogeneous SQL verification under a microservice architecture, characterized in that: The specific steps include: Start other heterogeneous configurations including the main application through customized scripts; Start the SpringApplication of the master and slave applications at the same time. Both the master and slave applications are cut into the Java agent. The server.port ports of the slave application are different. The slave application only receives SQL requests. Intercept the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and send the SQL request object to the memory bounded queue; Send the SQL request objects in the main application queue to other slave nodes. The information of the slave nodes is recorded in the startup script and passed in through the agent mechanism. After receiving the SQL request object from the application and passing it into the spring context object, execute the SQL; After a new file is generated, it will be sent to the enterprise WeChat group or email, including the server IP and error file path; Host monitoring service is an optional service; The host httpserver service is an optional service that supports viewing or downloading error files.

2. According to a method for heterogeneous SQL verification under a microservice architecture according to claim 1, it is characterized in that: Use spring.config.location to specify different directories to start the corresponding application instance, and modify the database configuration so that each slave application instance has a different database environment.

3. According to a method for heterogeneous SQL verification under a microservice architecture according to claim 1, it is characterized in that: When the spring application is loaded, the spring context object is recorded in the agent-related reference and the netty service is started.

4. According to a method for heterogeneous SQL verification under a microservice architecture according to claim 1, it is characterized in that: The netty network encoding and decoding protocol uses the Java serialization mechanism.

5. According to a method for heterogeneous SQL verification under a microservice architecture according to claim 1, it is characterized in that: After executing httpserver_notice.sh, if a new error file is generated, the server IP and file path where the error file is located will be reported.

6. A device for heterogeneous SQL verification under a microservice architecture, characterized in that: include: Start the module, start other heterogeneous configurations including the master application through customized scripts, and start the SpringApplication of the master and slave applications at the same time. Both the master and slave applications cut into the Java agent. The server.port ports of the slave application are different, and the slave application only receives SQL requests. The request sending module intercepts the query and update methods in the org.apache.ibatis.executor.BaseExecutor interface of the main mybatis, and sends the SQL request object to the bounded memory queue; the SQL request object in the main application queue is sent to other slave nodes, and the information of the slave nodes is recorded in the startup script and passed in through the agent mechanism; The receiving module receives the SQL request object from the application, passes it into the spring context object, and executes the SQL; The file sending module generates a new file and sends it to the enterprise WeChat group or email address, including the server IP and error file path. The host monitoring service is an optional service, and the host httpserver service is an optional service that supports viewing or downloading error files.

7. The device for heterogeneous SQL verification under a microservice architecture according to claim 6, characterized in that: Use spring.config.location to specify different directories to start the corresponding application instance, and modify the database configuration so that each slave application instance has a different database environment.

8. The device for heterogeneous SQL verification under a microservice architecture according to claim 6, characterized in that: When the spring application is loaded, the spring context object is recorded in the agent-related references, and the netty service is started; the netty network encoding and decoding protocol uses the java serialization mechanism; after executing httpserver_notice.sh, if a new error file is generated, the server IP and file path where the error file is located will be reported.

9. An electronic device, characterized in that: include: A processor, and a memory connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method for heterogeneous SQL verification under the microservice architecture according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the method for heterogeneous SQL verification under the microservice architecture according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Data query method and device in distributed environment and storage medium

    CN109656968A

  • Mass data quick query method based on polymorphic composition

    CN110489445A

  • Multi-source heterogeneous data governance method and device

    CN112579625A

  • Data acquisition method and device

    CN113742159A

  • Micro-service link calling method, system, equipment and medium

    CN114237770A

Cited By

  • HTTP (Hyper Text Transport Protocol) request performance diagnosis method and system based on dynamic Agent mechanism and storage medium

    CN120658647A

  • HTTP request performance diagnosis method and system based on dynamic agent mechanism, and storage medium

    CN120658647B