Asynchronous data update system and method based on WebSocket

Abstract

The present invention relates to an asynchronous data update system and method based on WebSocket, belonging to the field of computer application technology, wherein the system is applied to a WebSocket server, and the system comprises: a first data processing module, a second data processing module and an asynchronous update data module: the first data processing module is used to receive an SQL statement sent by a client, and convert the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data, and send the first data to the second data processing module; the second data processing module is used to perform syntax analysis on the first data to obtain second data in the form of a key-value pair, and send the second data to the asynchronous update data module; the asynchronous update data module is used to asynchronously update and save the second data to a local database. The present invention converts the SQL statement into the first data of the key-value pair, parses the first data to obtain the second data of the key-value pair, reduces the transmission of parameters, and improves the execution efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer applications, and in particular to a WebSocket-based asynchronous data updating system and method. Background Art

[0002] In the prior art, when executing an SQL statement to change a database object, it is necessary to asynchronously update the corresponding object in the local database storage. The specific steps are as follows: (1) First, the SQL statement must be parsed to obtain the changed database object; (2) Then, the latest database object information is obtained through a JDBC connection; (3) The latest database object information is synchronized to the local database storage; (4) Finally, the corresponding database object information in the local cache is synchronously updated.

[0003] In the above process, many parameters need to be passed, such as SQL statements, database connections, user information, object information, cache objects, etc., and other SERVICE interfaces and methods need to be referenced to parse SQL interfaces, database object update interfaces, cache update methods, etc. If asynchronous threads are used to complete all operations, the threads will be bloated and complex, resulting in low execution efficiency. Summary of the invention

[0004] In view of this, it is necessary to provide a WebSocket-based asynchronous data update system and method to solve the problem of low execution efficiency caused by too many parameters being passed when SQL statements asynchronously update local databases in the prior art.

[0005] In order to solve the above problems, in a first aspect, the present invention provides an asynchronous data update system based on WebSocket, which is applied to a WebSocket server and includes: a first data processing module, a second data processing module and an asynchronous data update module: A first data processing module is used to receive an SQL statement sent by a client, convert the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data, and send the first data to a second data processing module; A second data processing module is used to perform syntax analysis on the first data to obtain second data in the form of key-value pairs, and send the second data to the asynchronous update data module; The asynchronous data update module is used to asynchronously update and save the second data to the local database.

[0006] In a possible implementation, the first data processing module is further configured to asynchronously execute the SQL statement based on a thread and update the statement in a remote database.

[0007] In a possible implementation, the format of the key-value pair is JSON format.

[0008] In a possible implementation, the data structure of the first data includes: a string of SQL data type and SQL statement, and the data structure of the second data includes: an update type and parsed data, and the parsed data includes: an operation type, a database name, a database type, and a table field.

[0009] In a second aspect, the present invention further provides a WebSocket-based asynchronous data update method, wherein the WebSocket-based asynchronous data update method is based on the WebSocket-based asynchronous data update system as described in any one of the above, comprising: Receive an SQL statement sent by a client, and convert the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data; Parsing the first data to obtain second data in the form of key-value pairs; Asynchronously update and save the second data to the local database.

[0010] In a possible implementation, the performing syntax parsing on the first data to obtain the second data in the form of a key-value pair includes: The character string of the SQL statement in the first data is parsed according to the SQL data type in the first data based on the thread class in the programming language to obtain the second data.

[0011] In a possible implementation, the step of asynchronously updating and saving the second data to a local database includes: Based on the thread class in the programming language, the corresponding business method is called according to the update type in the second data to asynchronously update the database object and save it in the local database.

[0012] In a possible implementation, the thread class in the programming language is the Thread class in Java.

[0013] In a third aspect, the present invention further provides a server, comprising the WebSocket-based asynchronous data update system as described in any one of the above.

[0014] The beneficial effects of the present invention are as follows: the present invention provides an asynchronous data update system based on WebSocket, which is applied to a WebSocket server. The system includes: a first data processing module, a second data processing module and an asynchronous update data module. The first data processing module receives an SQL statement sent by a client, and converts the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data. The first data is in the form of a key-value pair, which reduces the transmission of data objects and jdbc connection objects compared to the prior art, and sends the first data to the second data processing module. The second data processing module performs syntax analysis on the first data to obtain second data in the form of a key-value pair, and sends the second data to the asynchronous update data module. The second data is in the form of a key-value pair, so that the database content to be updated can be directly updated to a local database according to the key and value in the second data. The asynchronous update data module asynchronously updates and saves the second data to the local database. The present invention converts the SQL statement into the first data of the key-value pair, parses the first data to obtain the second data of the key-value pair, reduces the transmission of parameters, and improves the execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the structure of an embodiment of an asynchronous data update system based on WebSocket provided by the present invention; Figure 2 A method flow chart of an embodiment of an asynchronous data update method based on WebSocket provided by the present invention. DETAILED DESCRIPTION

[0016] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0017] Before presenting the embodiments, the following terms are explained.

[0018] WebSocket is a full-duplex communication protocol over a single TCP connection. It allows real-time two-way communication between the client and the server. WebSocket uses a long connection to maintain a persistent connection between the client and the server, so that data can be sent and received in real time. In WebSocket, the client and the server can send messages to each other. The client can use the WebSocket API in JavaScript to send messages to the server and receive messages from the server.

[0019] The present invention provides a WebSocket-based asynchronous data update method, device, electronic device and medium, which are described below respectively.

[0020] Figure 1 A schematic diagram of the structure of an embodiment of the asynchronous data update system based on WebSocket provided by the present invention is shown in FIG. Figure 1 As shown, the WebSocket-based asynchronous data update system 100 is applied to a WebSocket server, and is characterized by comprising: a first data processing module 101, a second data processing module 102, and an asynchronous data update module 103: The first data processing module 101 is used to receive an SQL statement sent by a client, convert the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data, and send the first data to the second data processing module; The second data processing module 102 is used to perform syntax analysis on the first data to obtain second data in the form of key-value pairs, and send the second data to the asynchronous update data module; The system update data module 103 is used to asynchronously update and save the second data to the local database.

[0021] It should be noted that the present invention is for a scenario where data in a remote database needs to be updated to a local database. It is understandable that the present invention is also applicable to a scenario where data in one database needs to be updated to another database.

[0022] Compared with the prior art, the present embodiment provides an asynchronous data update system based on WebSocket, which is applied to the WebSocket server. The system includes: a first data processing module, a second data processing module and an asynchronous data update module. The first data processing module is used to receive the SQL statement sent by the client, and convert the SQL statement and the type of the SQL statement into the format of a key-value pair to obtain the first data, and send the first data to the second data processing module. The first data is in the form of a key-value pair, thereby reducing the transmission of data objects and jdbc connection objects. The second data processing module is used to perform syntax analysis on the first data to obtain the second data in the form of a key-value pair, and send the second data to the asynchronous update data module. The second data is in the form of a key-value pair, which also reduces the transmission of data objects and jdbc connection objects. The asynchronous update data module is used to asynchronously update and save the second data to a local database. The present invention converts the SQL statement into the first data of the key-value pair, parses the first data to obtain the second data of the key-value pair, reduces the transmission of parameters, and improves the execution efficiency.

[0023] It should be noted that the embodiment provided by the present invention is based on springboot, WebSocket is integrated in springboot, and classes in java are used to perform multi-threaded asynchronous processing. It can be understood that other programming languages ​​can also be used to integrate WebSocket for multi-threaded asynchronous data processing, or multi-threaded asynchronous data processing can be performed directly based on WebSocket without using a framework.

[0024] In a specific embodiment of the present invention, firstly, the jar package dependency of websocket is introduced in the project pom.xml, and the code is as follows: <dependency> <groupid> org.springframework.boot< / groupid> <artifactid> spring-boot-starter-websocket< / artifactid> < / dependency> Configure the Config class and use the @Configuration annotation for global configuration. For example: @Configuration public class WebSocketConfiguration implements OrderStartup,DisposableBean { @Bean @Conditional(value = ServerEndpointExporterCondition.class) public ServerEndpointExporter serverEndpointExporter() { return new ServerEndpointExporter(); } @Bean public WebSocketConfigurator webSocketConfigurator() { return new WebSocketConfigurator(); } ...... } WebSocket communication server-side implementation class, use @Component annotation to register the implementation class as a global component (can be directly referenced in the class class of the same service). Then use annotations such as @OnOpen, @OnMessage to rewrite methods to implement connection, message processing, etc. Simple code example: @Component @ServerEndpoint(value = " / websocket / {userKey}") public class WebSocket { @OnOpen public void onOpen(Session session, @PathParam(value = "userKey")String userKey) {} @OnClose public void onClose() {} @OnMessage public void onMessage(String message) {} @OnError public void onError(Throwable throwable) {} public static void pushMessage(String userKey,String message) {} public static void pushLocal(String type,String message) {} } It can be understood that the Websocket message processing of this embodiment is executed in the pushLocal method.

[0025] It should be noted that after integrating Websocket in springboot, the SQL statement can be processed. In some embodiments of the present invention, the first data processing module is also used to execute the SQL statement asynchronously based on a thread and update it to a remote database.

[0026] In some embodiments of the present invention, the format of the key-value pairs is JSON format.

[0027] In some embodiments of the present invention, the data structure of the first data includes: SQL data type and SQL statement string, the data structure of the second data includes: update type and parsed data, the parsed data includes: operation type, database name, database type and table field. For example, the structure of the first data is: { type:"sql_syntax", sql: "create table test( \"id\" BIGINT IDENTITY(1, 1) NOT NULL, \"name\" VARCHAR(88) );" } For example, the structure of the second data is: { type:"refreash_local", data: { operation: "create", name: "test", type: "table", columns:{ ["id","BIGINT"], ["name","VARCHAR"], } } } On the other hand, in some embodiments of the present invention, Figure 2 As shown, a WebSocket-based asynchronous data update method is also provided. The WebSocket-based asynchronous data update method is based on any one of the WebSocket-based asynchronous data update systems described above, and is characterized in that it includes: S201, receiving an SQL statement sent by a client, and converting the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data; S202, parsing the first data to obtain second data in the form of key-value pairs; S203: Asynchronously update and save the second data to a local database.

[0028] In some embodiments of the present invention, the data structure of the first data includes: a string of SQL data types and SQL statements, and the data structure of the second data includes: an update type and parsed data, and the parsed data includes: an operation type, a database name, a database type, and a table field.

[0029] In some embodiments of the present invention, the step of parsing the first data to obtain the second data in the form of a key-value pair includes: The character string of the SQL statement in the first data is parsed according to the SQL data type in the first data based on the thread class in the programming language to obtain the second data.

[0030] In some embodiments of the present invention, the step of asynchronously updating and saving the second data to the local database includes: Based on the thread class in the programming language, the corresponding business method is called according to the update type in the second data to asynchronously update the database object and save it in the local database.

[0031] In some embodiments of the present invention, the thread class in the programming language is the Thread class in Java.

[0032] In a specific embodiment of the present invention, a table creation statement is executed in the client SQL editor, and the table creation statement is as follows: create table test( "id" BIGINT IDENTITY(1, 1) NOT NULL, "name" VARCHAR(88) ); The client sends the SQL statement to the server through WebSocket. The server executes the SQL statement asynchronously through the thread ExecuteStatementsThread, thereby creating a table named test in the remote database and returning the result to the client, telling the client that the test table has been created on the remote database. Further, the first data processing module 101 in the server side converts the message type and table information of the SQL statement into JSON data format. The converted JSON data format is as follows: { type:"sql_syntax", sql: "create table test( \"id\" BIGINT IDENTITY(1, 1) NOT NULL, \"name\" VARCHAR(88) );" } The converted JSON data format, i.e., the first data, is sent to the second data processing module 102. The second data processing module 102 is used to parse the first data. The specific code of the parsing is as follows: public void refreshCacheResourcesByExecute(SyntaxStatementFormVOsyntaxStatement) { / / After executing the DCL statement, update the resource and save it in the database try { List <sqlstatement>sqlStatements = DCL_STATEMENTS.get(syntaxStatement.getConnectId()); if (sqlStatements == null || sqlStatements.isEmpty()) { StaticLog.debug("No DCL statement, no need to update cache"); return; } executorPools.execute(new RefreshResourcesByExecuteThread(driverDynamicManager, syntaxStatement, sqlStatements,resourcesCacheService)); } catch (Exception e) { StaticLog.error(e, "refreshCacheResources error:"); } } After the second data processing module 102 parses, the second data is obtained. The second data is also in JSON data format. The second data is a further refinement of the SQL statement based on the first data. The specific format of the second data is as follows: { type:"refreash_local", data: { operation: "create", name: "test", type: "table", columns:{ ["id","BIGINT"], ["name","VARCHAR"], } } } After receiving the second data, the asynchronous data update module 103 calls the update thread RefreshResourcesByExecuteThread to asynchronously update the local database according to the update type of the second data, i.e., the message type, for example, refreash_local. For example, a new test table is added, and the table fields are id and name. The code for calling the update thread RefreshResourcesByExecuteThread to asynchronously update the local database is as follows: private void saveResources(List <resourcescache>data) { / / The list is too big, so process it in segments if (data.size()>SPLIT_NUMBER) { List <List <resourcescache>>lists = DataSetUtil.splitList(data, SPLIT_LENGTH); if (lists != null) { for (List <resourcescache>splitList : lists) { try { if (splitList == null || splitList.isEmpty()) { StaticLog.debug("saveResources splitList is empty"); return; } resourcesCacheService.insertBatch(dbTable, splitList); } catch (Exception e) { StaticLog.error(e, "saveResources splitList error:"); } } } } else { if (data.isEmpty()) { StaticLog.debug("saveResources list is empty"); return; } resourcesCacheService.insertBatch(dbTable, data); } / / Clean up after use if (CollUtil.isNotEmpty(data)) { data.clear(); } } In summary, the present invention converts SQL statements into first data of key-value pairs based on websocket, parses the first data to obtain second data of key-value pairs. Both the first data and the second data are in the form of key-value pairs, which reduces the transmission of parameters and improves execution efficiency.

[0033] On the other hand, in some embodiments of the present invention, a server is further provided, comprising the WebSocket-based asynchronous data update system as described in any one of the above.

[0034] It should be noted that the WebSocket-based asynchronous data update system of this embodiment runs on the server, and the server may also include other systems. Each system runs independently on the server.

[0035] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0036] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.< / resourcescache> < / resourcescache> < / resourcescache> < / sqlstatement>

Claims

1. A WebSocket-based asynchronous data update system, applied to a WebSocket server, characterized in that: include: The first data processing module, the second data processing module and the asynchronous data update module: A first data processing module is used to receive an SQL statement sent by a client, convert the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data, and send the first data to a second data processing module; A second data processing module is used to perform syntax analysis on the first data to obtain second data in the form of key-value pairs, and send the second data to the asynchronous update data module; The asynchronous data update module is used to asynchronously update and save the second data to the local database.

2. The WebSocket-based asynchronous data update system according to claim 1, characterized in that: The first data processing module is also used to asynchronously execute the SQL statement based on a thread and update it into a remote database.

3. The WebSocket-based asynchronous data update system according to claim 1, characterized in that: The format of the key-value pair is JSON format.

4. The WebSocket-based asynchronous data update system according to claim 3, characterized in that: The data structure of the first data includes: a string of SQL data type and SQL statement, and the data structure of the second data includes: an update type and parsed data, and the parsed data includes: an operation type, a database name, a database type and a table field.

5. A WebSocket-based asynchronous data update method, the WebSocket-based asynchronous data update method is based on the WebSocket-based asynchronous data update system according to any one of claims 1 to 4, characterized in that: include: Receive an SQL statement sent by a client, and convert the SQL statement and the type of the SQL statement into a key-value pair format to obtain first data; Parsing the first data to obtain second data in the form of key-value pairs; Asynchronously update and save the second data to the local database.

6. The WebSocket-based asynchronous data update method according to claim 5, characterized in that: The data structure of the first data includes: a string of SQL data type and SQL statement, and the data structure of the second data includes: an update type and parsed data, and the parsed data includes: an operation type, a database name, a database type and a table field.

7. The WebSocket-based asynchronous data update method according to claim 5, characterized in that: The step of parsing the first data to obtain the second data in the form of a key-value pair includes: The character string of the SQL statement in the first data is parsed according to the SQL data type in the first data based on the thread class in the programming language to obtain the second data.

8. The WebSocket-based asynchronous data updating method according to claim 5, characterized in that: The step of asynchronously updating and saving the second data to the local database includes: Based on the thread class in the programming language, the corresponding business method is called according to the update type in the second data to asynchronously update the database object and save it in the local database.

9. The WebSocket-based asynchronous data updating method according to claim 7 or 8, characterized in that: The thread class in the programming language is the Thread class in java.

10. A server, characterized in that: It includes an asynchronous data update system based on WebSocket as described in any one of claims 1 to 4.

Images