Incremental data processing method and device, computer equipment and readable storage medium

Abstract

Embodiments of the invention disclose an incremental data processing method and apparatus, a computer device and a readable storage medium. The method comprises the steps of constructing an uninterrupted communication connection between a client and a server; performing streaming processing on the incremental data at the server to obtain a data stream of the incremental data; and transmitting the data stream of the incremental data to the client for incremental rendering. According to the method and the device, the availability, the speed and the response capability of the data can be improved, the pressure of network bandwidth can be reduced, the limitation of application scenes is avoided, and the method and the device are particularly suitable for transmission of large files or real-time data streams.

Description

Technical Field

[0001] The present application relates to the field of incremental rendering technology, and in particular to an incremental data processing method, apparatus, computer equipment and readable storage medium. Background Art

[0002] Web applications usually involve application scenarios that present massive amounts of data, such as visualization of large amounts of images, tables, real-time monitoring systems, etc. Large-scale data sets usually contain tens of millions of records or more. If all the data is loaded at once and visualized, it will inevitably lead to very slow loading, and the huge amount of data will also cause page freezes, reducing the user experience.

[0003] At present, for the above application scenarios, incremental rendering is usually used to solve the problem. Incremental rendering refers to loading data in batches, gradually adding items in subsequent loading, and dynamically updating the display results according to the new data. Common incremental renderings include lazy loading, virtual scroll loading, etc. Lazy loading, also known as delayed loading, is a delayed loading technology, which means that when the page scrolls to the visible area, it requests to load the content of the area, instead of loading all the contents immediately when the system starts or loads; virtual scroll loading is a technology that optimizes the performance of scrolling lists. It can avoid page jams in the case of large amounts of data. Specifically, when scrolling to the last existing element, it automatically requests more data from the server and renders it to display the page content.

[0004] Both lazy loading and virtual scroll loading rely on front-end JavaScript and other technologies, requiring asynchronous requests to obtain data and loading the content that needs to be displayed on the website or application into the page in blocks. Although both can improve website speed and performance, they are usually used to optimize large data lists and mobile applications, and have limited application scenarios, especially those that require real-time updates of displayed content. Summary of the invention

[0005] In response to the shortcomings of the prior art, the present application provides an incremental data processing method, apparatus, computer device and readable storage medium, which can not only improve the availability, speed and responsiveness of data, but also reduce the pressure on network bandwidth. It is not limited by application scenarios and is particularly suitable for the transmission of large files or real-time data streams.

[0006] To solve the above problems, in a first aspect, an embodiment of the present application provides an incremental data processing method, which includes:

[0007] Establish an uninterrupted communication connection between the client and the server;

[0008] Performing streaming processing on the incremental data at the server to obtain a data stream of the incremental data;

[0009] The data stream of the incremental data is transmitted to the client for incremental rendering.

[0010] In a second aspect, an embodiment of the present application further provides an incremental data processing device, which includes:

[0011] A first construction unit is used to establish an uninterrupted communication connection between the client and the server;

[0012] A first processing unit, configured to perform stream processing on the incremental data at the server to obtain a data stream of the incremental data;

[0013] The first transmission unit is used to transmit the data stream of the incremental data to the client for incremental rendering.

[0014] In a third aspect, an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the incremental data processing method as described in the first aspect above is implemented.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the incremental data processing method described in the first aspect above.

[0016] The embodiments of the present application provide an incremental data processing method, apparatus, computer equipment and readable storage medium. The method pre-builds an uninterrupted communication connection between the client and the server, which can reduce the number of requests sent by the client to the server and reduce the burden on the server. When the client needs to update the displayed content in real time, the server can respond quickly and transmit the corresponding incremental data to the client in the form of a data stream, thereby realizing a data set that supports unlimited size, dynamic growth and real-time updates. It can not only improve the availability, speed and responsiveness of the data, but also reduce the pressure on the network bandwidth. It is not limited by application scenarios and is particularly suitable for the transmission of large files or real-time data streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 An architectural diagram of an incremental data processing method provided in an embodiment of the present application;

[0019] Figure 2 A flowchart of an incremental data processing method provided in an embodiment of the present application;

[0020] Figure 3 Another schematic diagram of the incremental data processing method provided in the embodiment of the present application;

[0021] Figure 4 Another schematic diagram of the incremental data processing method provided in the embodiment of the present application;

[0022] Figure 5 Another schematic diagram of the incremental data processing method provided in the embodiment of the present application;

[0023] Figure 6 Another schematic diagram of the incremental data processing method provided in the embodiment of the present application;

[0024] Figure 7 A schematic block diagram of an incremental data processing device provided in an embodiment of the present application;

[0025] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0028] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0029] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] See also Figure 1 , Figure 1 This is an architecture diagram of the incremental data processing method provided in the embodiment of the present application. Figure 1 As shown, the incremental data processing method provided in the embodiment of the present application is applied to the management server 10. The management server 10 can pre-build a communication connection between the server 20 and the client 30, and when the client 30 needs to update the display content in real time, the server 20 can respond quickly and transmit the corresponding incremental data to the client 30 in the form of a data stream.

[0031] It should be noted that the application scenario of the above embodiment is only an example. The services and scenarios described in the embodiment of this application are intended to more clearly illustrate the technical solution of the embodiment of this application, and do not constitute a limitation on the technical solution provided by the embodiment of this application. It is known to those skilled in the art that with the evolution of the system and the emergence of new business scenarios, the technical solution provided by the embodiment of this application is also applicable to similar technical problems. The following is a detailed description.

[0032] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments. The incremental data processing method is described in detail below.

[0033] See also Figure 2 , Figure 2 A flowchart of an incremental data processing method provided in an embodiment of the present application.

[0034] It should be noted that if there are substantially the same results, the method of the present application is not limited to Figure 2 The process sequence shown is limited to the following steps:

[0035] like Figure 2 As shown, the method includes the following steps S110 to S130.

[0036] S110. Establish an uninterrupted communication connection between the client and the server.

[0037] Specifically, the present application may pre-use EventSource technology to build an uninterrupted communication connection between the client and the server. Among them, EventSource, also known as Server-Sent Events, or SSE for short, is a technology that enables the client to automatically receive updates from the server through an HTTP connection. EventSource can be used as an API interface built into the client, and establish a persistent connection between the client and the server through the HTTP communication protocol, through which the server can push data to the client at any time. EventSource is implemented based on the HTTP protocol, which establishes a persistent connection between the client and the server, and implements the function of the server pushing event data to the client. The EventSource at the client can initiate a connection with the server through a URL. After the connection is successful, the server can send event data to the client, and then the EventSource object can be used on the client to register an event processing function to receive event data from the server.

[0038] This application uses EventSource technology to build a communication connection between the client and the server, so that a persistent connection can be established between the client and the server, which can reduce the number of requests sent by the client to the server and reduce the burden on the server. At the same time, after the connection receives a push message from the server, the client can capture the new or modified data sent from the server, thereby realizing local updates and displays in the client.

[0039] In some embodiments, Figure 3 The method further includes the following steps before step S110:

[0040] S210: construct the server.

[0041] In this embodiment, the present application may use Node.js to construct the server, that is, an HTTP server may be created as the server using the HTTP.createServer function in Node.js, and RequestListener may be used as the listening function for the request event.

[0042] Among them, Node.js is a JavaScript toolkit for writing high-performance network servers. It is an open source and cross-platform JavaScript runtime environment built on the Chrome V8 engine. It allows developers to write command-line tools and server-side scripts outside the browser.

[0043] S220. Create a readable stream on the server side according to the storage method of the incremental data.

[0044] Specifically, after completing the creation of the HTTP server, this application also needs to determine the storage method of the incremental data transmitted from the server to the client, and create a readable stream in the server according to its storage method to facilitate the configuration of the created HTTP server. Among them, a readable stream is a stream that produces data for program consumption. Common data production methods include reading disk files, reading network request content, etc.

[0045] S230: Set a response header of the server to respond to the client.

[0046] In this embodiment, after the server creates a readable stream, it is necessary to set the response header of the server's response to the client to further configure the created HTTP server.

[0047] Specifically, the server's response header can be set using the res.writeHead method, including setting the server's response header content type Content-Type, cache control Cache-Control, and connection mode Connection. Content type Content-Type is used to define the type of network files and the encoding of web pages, and determines the form and encoding of the file recipient to read the file; cache control Cache-Control is used to specify the instructions that all cache mechanisms must obey in the entire request / response chain. These instructions specify the behavior used to prevent the cache from causing adverse interference to the request or response; the connection mode Connection is used to determine whether the network connection will be closed after the current transaction is completed.

[0048] Among them, the content type of the response header can be set to 'Content-Type':'text / event-stream', the cache control of the response header can be set to 'Cache-Control':'no-cache', and the connection mode of the response header can be set to 'Connection':'keep-alive'.

[0049] S120: Stream-process the incremental data at the server to obtain a data stream of the incremental data.

[0050] Specifically, the incremental data is the data required for incremental rendering on the client side, and streaming processing refers to dividing the incremental data into multiple data blocks so that the incremental data can be transmitted between the server and the client in the form of data streams. The server side can continuously transmit data blocks, and the server side can continuously receive data blocks transmitted by the server side, so as to continuously send new or modified data from the server to the client side, thereby improving the availability, speed and responsiveness of the data, and reducing the pressure on the network bandwidth, and it can also be applied to the transmission of large files or real-time data streams. Among them, the server side can use Stream technology to implement streaming processing of incremental data. Stream is an element queue from a data source and supports aggregation operations.

[0051] In some embodiments, Figure 4 As shown, step S120 includes steps S121, S122 and S123.

[0052] S121. Determine a first data volume of the incremental data and a second data volume of a data block of the incremental data; wherein the first data volume includes at least one of the second data volumes, and the incremental data includes at least one of the data blocks.

[0053] In this embodiment, the first data amount is used to characterize the size of the incremental data, and the second data amount is used to characterize the size of each data block in the incremental data. The size of each data block can be the same or different. The size of the data block can be selected according to the actual application, and this application does not make a specific limitation.

[0054] S122. Slice the incremental data according to the first data amount and the second data amount to obtain data blocks of the incremental data.

[0055] Specifically, after determining the first data volume of the incremental data and the second data volume of the data block of the incremental data, the size of the incremental data and the size of the multiple data blocks into which the incremental data is divided can be determined, and then the incremental data can be sharded to be divided into multiple data blocks, so as to facilitate the transmission of the incremental data from the server to the client in the form of a data stream. The incremental data sharding process can be performed by sharding in average or by sharding in a configurable fixed size.

[0056] S123. Convert the data block of the incremental data into the data stream of the incremental data.

[0057] In this embodiment, the data blocks obtained after the incremental data segmentation cannot be directly transmitted from the server to the client. They need to be converted into a data stream of incremental data in advance before they can be transmitted from the server to the client in the form of a data stream. Specifically, the present application can write the data blocks of incremental data into a readable stream pre-created on the server, and then the data blocks of incremental data can be converted into a data stream of incremental data, so that the incremental data can be transmitted from the server to the client in the form of a data stream.

[0058] In other inventive embodiments, Figure 5 As shown, step S123 includes the following steps:

[0059] S1231. Set identification information for each data block of the incremental data.

[0060] In this embodiment, each data block of the incremental data can be identified by corresponding identification information, and the identification information includes information such as the size of the data block, the ID of the data block, and the incremental update identification, wherein the identification information can be understood as metadata information, and the ID of the data block can be characterized by a timestamp. After the incremental data is fragmented, an identification information can be added to the starting position of each data block of the incremental data. After the identification information is set for each data block of the incremental data, the data block can be encoded to compress each data block, thereby reducing the network traffic between the server and the client and improving the response speed of the client.

[0061] S1232: Encode the data block provided with the identification information to obtain an encoded data block.

[0062] In this embodiment, after the incremental data is segmented, an identification information can be added to the starting position of each data block of the incremental data, and the data block can be encoded to compress each data block, thereby reducing the network traffic between the server and the client and improving the client's response speed.

[0063] Among them, the compression of data blocks can be achieved by using the Gzip compression method. Specifically, the res.writeHead method is used in advance on the server to configure the content encoding Content-Encoding of the response header. Content-Encoding is a pair of header fields used in HTTP to agree on which encoding format to use to transmit the body. The purpose of Content-Encoding is to optimize the size of the transmitted content. In layman's terms, it is to compress as much as possible. Generally, the response text compressed by the Gzip compression method is only one-fourth of the original size. The content encoding Content-Encoding of the response header can be specifically set to 'Content-Encoding':'gzip'.

[0064] S1233. Convert the encoded data block into a data stream of the incremental data.

[0065] Specifically, after setting the identification information and encoding, each data block of the incremental data can be written into a readable stream created in advance on the server, thereby converting the data block of the incremental data into a data stream of the incremental data, so that the incremental data can be transmitted from the server to the client in the form of a data stream.

[0066] In some embodiments, after step S1231, the step of updating identification information of each data block of the incremental data according to a request message sent by the client to the server is also included.

[0067] In this embodiment, the identification information of the data block includes the ID of the data block and the incremental update identifier, wherein the ID of the data block is used to determine whether the data block has been transmitted to the client through the server, and the incremental update identifier is used to determine whether the data needs to be incrementally rendered again in the client. Each time the server sends a response message of the data block of incremental data to the client, the client needs to return a message to the server, and the message needs to be attached with the identification information of the data block to confirm to the server that the data block has been received. Furthermore, it is also possible to determine in advance whether the data block to be transmitted needs to be incrementally rendered in the client through the identification information of the data block in the message returned by the client, thereby avoiding the client from loading the data block again for incremental rendering. At the same time, it can also prevent the data block from being lost at the TCP connection level during transmission, resulting in the client not receiving it. At this time, the communication connection between the client and the server is not interrupted, and the server will continue to transmit data to the client. If the client does not confirm to the server, it will cause data rendering dislocation in the client.

[0068] Specifically, the server obtains the ID of the previous data block through the lastReceivedId in the message sent by the client, and then compares the ID of the current data block with the ID of the previous data block. If the ID of the current data block is greater than the lastReceivedId, it is determined that the current data block has not been transmitted. After receiving the current data block, the client can perform incremental rendering normally; if the ID of the current data block is less than the lastReceivedId, it is determined that the current data has been transmitted. At this time, it is necessary to determine whether the client needs to perform incremental rendering through the current data block through the incremental update identifier in the message sent by the client. If the incremental update identifier is a first value, such as 1, it indicates that the current data block contains new data, and the client needs to perform incremental rendering based on the current data block; if the incremental update identifier is a second value, such as 0, it indicates that the current data does not contain new data, thereby avoiding the client using the current data block to repeatedly perform incremental rendering.

[0069] Among them, lastReceivedId is a variable in the client, which is used to record the ID of the data most recently received by the client. lastReceivedId needs to be transmitted to the server as a parameter. The client determines whether an exception is encountered by listening to the Onerror event and needs to re-establish the communication connection between the client and the server. At this time, lastReceivedId can be used to determine the ID of the data block last received before the exception.

[0070] At the same time, during the client construction process, you need to set the initial value of lastReceivedId, such as setting the initial value to -1.

[0071] S130: Transmit the data stream of the incremental data to the client for incremental rendering.

[0072] Specifically, the client can use EventSource to establish a long connection with the specified URL (Uniform Resource Locator) for receiving event data from the server, and receive the incremental data transmitted by the server through the response message by listening to the Onmessage event. After listening to the incremental data transmitted by the server through the response message, it is necessary to replace lastReceivedId with the ID of the received incremental data, and detect the response header of the response message to determine whether the response text has been compressed and the compression method of the response text. If it is determined that the response text has been compressed and the compression method before transmission, it is necessary to decode it according to the compression method to realize the parsing of the response message, and then obtain the ID of the incremental data from the parsed response message to determine whether to load the incremental data to realize incremental rendering.

[0073] In some embodiments, Figure 6 As shown, step S130 includes the following steps:

[0074] S131. Transmit the data stream of the incremental data to the client for decoding to obtain identification information of the data block of the incremental data.

[0075] Specifically, before the data stream of the incremental data is transmitted to the client, it is encoded because it needs to be compressed. Therefore, after the data stream of the incremental data is transmitted to the client, it needs to be decoded in order to obtain the identification information of the data block of the incremental data, and then the incremental update identification information of the client can be determined by the identification information of the data block of the incremental data. Among them, the data block can be compressed using zlib.createGzip before transmission. After it is transmitted to the client in the form of a data stream through a response message, the content encoding Content-Encoding in the response header of the response message can be detected. Specifically, it can be detected that res.headers['content-encoding'] === 'gzip', which can indicate that the data is compressed by the Gzip compression method. At this time, the Zlib.gunzip decompression method can be used to decode it to obtain the identification information of the data block of the incremental data.

[0076] S132: Determine an incremental update identifier of the client according to identification information of the data block of the incremental data.

[0077] Specifically, the incremental update identifier is used to determine whether the data needs to be incrementally rendered again in the client. Each time the server sends a response message of a data block of incremental data to the client, the client needs to return a message to the server. The message needs to be attached with the identification information of the data block to confirm to the server that the data block has been received. Furthermore, it is also possible to determine in advance whether the data block to be transmitted needs to be incrementally rendered in the client through the identification information of the data block in the message returned by the client, thereby avoiding the client from loading the data block again for incremental rendering. At the same time, it can also prevent the data block from being lost at the TCP connection level during transmission, resulting in the client not receiving it. At this time, the communication connection between the client and the server is not interrupted, and the server will continue to transmit data to the client. If the client does not confirm to the server, it will cause data rendering misalignment in the client.

[0078] S133: Load the incremental data in the client according to the incremental update identifier to perform incremental rendering in the client.

[0079] In this embodiment, after the incremental update identifier of the data block is determined by the identification information of the data block, it can be determined whether the data block needs to be incrementally rendered in the client through the incremental update identifier, thereby avoiding the client from repeatedly performing incremental rendering using the current data block. Specifically, if the incremental update identifier is a first value, such as 1, it means that the current data block contains new data, and the client needs to perform incremental rendering based on the current data block; if the incremental update identifier is a second value, such as 0, it means that the current data does not contain new data, and the client does not need to repeatedly perform incremental rendering using the current data block.

[0080] In some embodiments, after step S131, the method further includes: generating a response message from the client to the server according to the identification information of the data block.

[0081] In this embodiment, the identification information of the data block includes the ID of the data block, and the ID of the data block is used to determine whether the data block has been transmitted to the client through the server. The client returns a message to the server with the identification information of the data block attached. This is specifically to prevent the data block from being lost at the TCP connection level during transmission, resulting in the client not receiving it. At this time, the communication connection between the client and the server is not interrupted, and the server will continue to transmit data to the client. If the client does not confirm with the server, it will cause data rendering dislocation in the client.

[0082] Specifically, the server sets an ID for each data block transmitted, that is, the ID of the data block. After receiving the data block, the client also needs to maintain the ID of the received data block, and confirm to the server that the data block has been successfully received, and attach the identification information of the data block. After the server receives a message returned by the client, it can generate a log based on the message, and then if it does not receive the client request confirmation of the data block within a certain period of time (such as the client confirmation timeout set by the server), it can directly re-transmit the data block to the client through the log.

[0083] In the incremental data processing method provided in the embodiment of the present application, the communication connection between the client and the server is constructed in advance by using EventSource technology; the incremental data at the server is streamed to obtain the data stream of the incremental data; and the data stream of the incremental data is transmitted to the client for incremental rendering. The present application pre-constructs an uninterrupted communication connection between the client and the server, which can reduce the number of requests sent by the client to the server and reduce the burden on the server. When the client needs to update the display content in real time, it can respond quickly at the server and stream the corresponding incremental data, and transmit it to the client in the form of a data stream, realizing a data set that supports unlimited size, dynamic growth and real-time updates. It can not only improve the availability, speed and responsiveness of the data, but also reduce the pressure on the network bandwidth. It is not limited by the application scenario and is particularly suitable for the transmission of large files or real-time data streams.

[0084] The embodiment of the present application also provides an incremental data processing device 100, which is used to execute any embodiment of the aforementioned incremental data processing method.

[0085] Specifically, see Figure 7 , Figure 7 It is a schematic block diagram of the incremental data processing device 100 provided in an embodiment of the present application.

[0086] like Figure 7 As shown, the incremental data processing device 100 includes: a first construction unit 110, a first processing unit 120 and a first transmission unit 130.

[0087] The first building unit 110 is used to build an uninterrupted communication connection between the client and the server.

[0088] In some embodiments, the incremental data processing apparatus 100 further includes: a second construction unit, a creation unit, and a first setting unit.

[0089] The second construction unit is used to construct the server.

[0090] A creation unit is used to create a readable stream on the server side according to the storage method of the incremental data.

[0091] The first setting unit is used to set a response header of the server to respond to the client.

[0092] The first processing unit 120 is used to perform streaming processing on the incremental data at the server to obtain a data stream of the incremental data.

[0093] In some embodiments, the first processing unit 120 includes: a first determining unit, a second processing unit, and a first converting unit.

[0094] The first determination unit is used to determine a first data volume of the incremental data and a second data volume of a data block of the incremental data; wherein the first data volume includes at least one of the second data volumes, and the incremental data includes at least one of the data blocks.

[0095] The second processing unit is used to perform segmentation processing on the incremental data according to the first data amount and the second data amount to obtain data blocks of the incremental data.

[0096] The first conversion unit is used to convert the data block of the incremental data into the data stream of the incremental data.

[0097] In some embodiments, the first conversion unit includes: a second setting unit, an encoding unit, and a second conversion unit.

[0098] The second setting unit is used to set identification information for each data block of the incremental data.

[0099] In some embodiments, the first processing unit 120 further includes an updating unit.

[0100] An updating unit is used to update the identification information of each data block of the incremental data according to the request message sent by the client to the server.

[0101] The encoding unit is used to encode the data block provided with the identification information to obtain an encoded data block.

[0102] The second conversion unit is used to convert the encoded data block into a data stream of the incremental data.

[0103] The first transmission unit 130 is used to transmit the data stream of the incremental data to the client for incremental rendering.

[0104] In some embodiments, the second transmission unit includes: a decoding unit, a second determination unit, and a rendering unit.

[0105] A decoding unit is used to transmit the data stream of the incremental data to the client for decoding to obtain identification information of the data block of the incremental data.

[0106] The second determining unit is used to determine the incremental update identifier of the client according to the identification information of the data block of the incremental data.

[0107] A rendering unit is used to load the incremental data in the client according to the incremental update identifier to perform incremental rendering in the client.

[0108] In some embodiments, the first transmission unit 130 further includes a generation unit.

[0109] A generating unit is used to generate a response message from the client to the server according to the identification information of the data block.

[0110] The incremental data processing device provided in the embodiment of the present application is used to execute the above-mentioned construction of an uninterrupted communication connection between the client and the server; streaming the incremental data at the server to obtain the data stream of the incremental data; transmitting the data stream of the incremental data to the client for incremental rendering.

[0111] The incremental data processing device can be implemented in the form of a computer program. Figure 8 Runs on the computer device shown.

[0112] See also Figure 8 , Figure 8 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal, wherein the terminal may be a computer device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device.

[0113] See also Figure 8 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0114] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute an incremental data processing method.

[0115] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0116] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an incremental data processing method.

[0117] The network interface 505 is used to communicate with other devices over the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] Among them, the processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps: establish an uninterrupted communication connection between the client and the server; stream the incremental data at the server to obtain the data stream of the incremental data; transmit the data stream of the incremental data to the client for incremental rendering.

[0119] In one embodiment, before implementing the construction of the communication connection between the client and the server, the processor 502 specifically implements the following steps: constructing the server; creating a readable stream on the server according to the storage method of the incremental data; and setting the response header of the server to respond to the client.

[0120] In one embodiment, when the processor 502 implements the streaming processing of the incremental data at the server to obtain the data stream of the incremental data, it specifically implements the following steps: determining a first data amount of the incremental data and a second data amount of a data block of the incremental data; wherein the first data amount includes at least one of the second data amounts, and the incremental data includes at least one of the data blocks; slicing the incremental data according to the first data amount and the second data amount to obtain the data blocks of the incremental data; and converting the data blocks of the incremental data into the data stream of the incremental data.

[0121] In one embodiment, when the processor 502 implements the conversion of the data block of the incremental data into the data stream of the incremental data, it specifically implements the following steps: setting identification information for each data block of the incremental data; encoding the data block with the identification information to obtain the encoded data block; and converting the encoded data block into the data stream of the incremental data.

[0122] In one embodiment, after setting identification information for each data block of the incremental data, the processor 502 specifically implements the following steps: updating the identification information of each data block of the incremental data according to the request message sent by the client to the server.

[0123] In one embodiment, when the processor 502 implements the transmission of the data stream of the incremental data to the client for incremental rendering, it specifically implements the following steps: transmitting the data stream of the incremental data to the client for decoding to obtain identification information of the data block of the incremental data; determining the incremental update identifier of the client based on the identification information of the data block of the incremental data; loading the incremental data in the client based on the incremental update identifier to perform incremental rendering in the client.

[0124] In one embodiment, after the processor 502 transmits the data stream of the incremental data to the client for decoding and obtains the identification information of the data block of the incremental data, it specifically implements the following steps: generating a response message from the client to the server based on the identification information of the data block.

[0125] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0126] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0127] Therefore, the present application also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor performs the following steps: establishing an uninterrupted communication connection between a client and a server; performing streaming processing on the incremental data at the server to obtain a data stream of the incremental data; and transmitting the data stream of the incremental data to the client for incremental rendering.

[0128] In one embodiment, before executing the program instructions to realize the construction of the communication connection between the client and the server, the processor specifically implements the following steps: constructing the server; creating a readable stream on the server according to the storage method of the incremental data; setting the response header of the server to respond to the client.

[0129] In one embodiment, when the processor executes the program instructions to implement the streaming processing of the incremental data at the server to obtain the data stream of the incremental data, it specifically implements the following steps: determining a first data amount of the incremental data and a second data amount of a data block of the incremental data; wherein the first data amount includes at least one of the second data amounts, and the incremental data includes at least one of the data blocks; slicing the incremental data according to the first data amount and the second data amount to obtain the data blocks of the incremental data; and converting the data blocks of the incremental data into the data stream of the incremental data.

[0130] In one embodiment, when the processor executes the program instructions to implement the conversion of the data blocks of the incremental data into the data stream of the incremental data, it specifically implements the following steps: setting identification information for each data block of the incremental data; encoding the data block with the identification information to obtain the encoded data block; and converting the encoded data block into the data stream of the incremental data.

[0131] In one embodiment, after executing the program instructions to implement the setting of identification information for each data block of the incremental data, the processor specifically implements the following steps: updating the identification information of each data block of the incremental data according to the request message sent by the client to the server.

[0132] In one embodiment, when the processor executes the program instructions to implement the transmission of the data stream of the incremental data to the client for incremental rendering, the following steps are specifically implemented: the data stream of the incremental data is transmitted to the client for decoding to obtain identification information of the data block of the incremental data; the incremental update identifier of the client is determined based on the identification information of the data block of the incremental data; and the incremental data is loaded in the client based on the incremental update identifier to perform incremental rendering in the client.

[0133] In one embodiment, after the processor executes the program instructions to transmit the data stream of the incremental data to the client for decoding and obtains the identification information of the data block of the incremental data, it specifically implements the following steps: generating a response message from the client to the server based on the identification information of the data block.

[0134] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0135] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0136] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0137] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0138] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0139] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for processing incremental data, characterized in that: include: Establish an uninterrupted communication connection between the client and the server; Performing streaming processing on the incremental data at the server to obtain a data stream of the incremental data; The data stream of the incremental data is transmitted to the client for incremental rendering.

2. The incremental data processing method according to claim 1, characterized in that: Before establishing an uninterrupted communication connection between the client and the server, it also includes: Constructing the server; Creating a readable stream on the server side according to the storage method of the incremental data; Set the response header of the server to respond to the client.

3. The incremental data processing method according to claim 1, characterized in that: The step of performing streaming processing on the incremental data at the server to obtain a data stream of the incremental data includes: Determine a first data volume of the incremental data and a second data volume of a data block of the incremental data; wherein the first data volume includes at least one of the second data volumes, and the incremental data includes at least one of the data blocks; Slice the incremental data according to the first data volume and the second data volume to obtain data blocks of the incremental data; The data blocks of the incremental data are converted into a data stream of the incremental data.

4. The incremental data processing method according to claim 3, characterized in that: The converting the data block of the incremental data into the data stream of the incremental data includes: Setting identification information for each data block of the incremental data; Encoding the data block provided with the identification information to obtain an encoded data block; The encoded data block is converted into a data stream of the incremental data.

5. The incremental data processing method according to claim 4, characterized in that: After setting identification information for each data block of the incremental data, the method further includes: The identification information of each data block of the incremental data is updated according to the request message sent by the client to the server.

6. The incremental data processing method according to claim 3, characterized in that: The step of transmitting the data stream of the incremental data to the client for incremental rendering includes: Transmitting the data stream of the incremental data to the client for decoding to obtain identification information of the data block of the incremental data; Determining the incremental update identifier of the client according to the identification information of the data block of the incremental data; The incremental data is loaded in the client according to the incremental update identifier to perform incremental rendering in the client.

7. The incremental data processing method according to claim 6, characterized in that: After transmitting the data stream of the incremental data to the client for decoding to obtain identification information of the data block of the incremental data, the method further includes: A response message from the client to the server is generated according to the identification information of the data block.

8. A method for processing incremental data, characterized in that: include: A first construction unit is used to establish an uninterrupted communication connection between the client and the server; A first processing unit, configured to perform stream processing on the incremental data at the server to obtain a data stream of the incremental data; The first transmission unit is used to transmit the data stream of the incremental data to the client for incremental rendering.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the incremental data processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to perform the incremental data processing method according to any one of claims 1 to 7.

Images