Collaborative note data processing method and device, storage medium and computer program product
By binding client information on the server side and performing data merging and aggregation processing, the problems of inaccurate update operation traceability and content expansion in collaborative note technology are solved, and safe and reliable collaborative note data management is achieved.
Patent Information
- Application Number
- CN202510055810.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-14
- Publication Date
- 2025-05-13
AI Technical Summary
The existing collaborative notes technology is easily maliciously forged when tracing client update operations, and as editing continues, the content of collaborative notes is prone to over-expanding.
Receive the client's collaborative notes update message on the server side, bind the client information with the update data, generate the second update data, and periodically save and delete data records through merging and aggregation processing of the buffer queue and database to prevent forgery and bloat.
It realizes accurate traceability of client update operations, while preventing malicious forgery, and reducing the database space occupied by collaborative notes, avoiding excessive content bloat.
Smart Images
Figure CN119988393A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet technology, and in particular to a method for processing collaborative note data, a device for processing collaborative note data, a computer-readable storage medium and a computer program product. Background Art
[0002] With the development of the Internet, collaborative office has become a new development trend. Among them, documents completed by multiple people are called collaborative notes. Collaborative notes allow multiple users to edit, update and comment on the same document, and update the content synchronously in real time, greatly improving work efficiency and the convenience of collaboration.
[0003] The existing technologies for implementing collaborative notes are usually operational transformation (OT) and conflict-free replicated data types (CRDT). Due to the decentralized nature of CRDT, which reduces dependence on servers, its performance surpasses traditional OT technology and has become the mainstream development model.
[0004] In the process of implementing CRDT collaborative notes, users usually require to trace the editing history of notes. In order to achieve the purpose of tracing, the method of periodically storing copies of the entire note can be adopted, but the tracing effect cannot be accurate to the operation of each user at any time. It is also possible to add a field in the note content to record the user's operation, but it is easy to be maliciously forged and becomes too bloated as the editing of the note content continues. Summary of the invention
[0005] In response to the above-mentioned prior art, an embodiment of the present invention discloses a method for processing collaborative note data, which can overcome the defects that client update operations are easily maliciously forged when traced and that collaborative notes expand excessively as editing extends, thereby achieving the purpose of preventing forgery and expansion while accurately tracing.
[0006] In view of this, an embodiment of the present application proposes a method for processing collaborative note data, the method comprising:
[0007] The server receives a collaborative note update message sent by the client through a WebSocket connection, where the collaborative note update message includes first update data of the collaborative note updated by the client;
[0008] The server binds the client information corresponding to the client with the first update data to generate second update data, and stores the second update data in a buffer queue;
[0009] The server periodically merges all the second update data in the buffer queue into one original data record according to a first preset time interval, and stores the original data record in sequence in the database;
[0010] The server periodically aggregates all the original data records in the database according to a second preset time interval to generate an aggregated data record, and deletes the original data records involved in the aggregated processing.
[0011] An embodiment of the present invention discloses a collaborative note data processing device that can overcome the defects of easy malicious forgery when client update operations are traced and excessive expansion of collaborative notes as editing extends, thereby achieving the purpose of preventing forgery and expansion while accurately tracing.
[0012] In view of this, an embodiment of the present application proposes a collaborative note data processing device, the device comprising: a transceiver module, a cache module, and a database operation module;
[0013] The transceiver module is used to receive a collaborative note update message sent by a client through a WebSocket connection, wherein the collaborative note update message includes first update data for the collaborative note updated by the client, and add the client information corresponding to the client to the first update data to generate second update data;
[0014] The cache module is used to cache the second update data;
[0015] The database operation module is used to periodically merge all the second updated data in the buffer queue into an original data record according to a first preset time interval, and save them in sequence in the database; periodically aggregate all the original data records in the database according to a second preset time interval to generate an aggregated data record, and delete the original data records involved in the aggregation processing.
[0016] The embodiments of the present invention disclose a computer-readable storage medium, which can overcome the defects that client update operations are easily maliciously forged when traced and that collaborative notes expand excessively as editing extends, thereby achieving the purpose of preventing forgery and expansion while accurately tracing.
[0017] A computer-readable storage medium stores computer instructions, which, when executed by a processor, can implement the steps of the above-mentioned collaborative note data processing method.
[0018] The embodiment of the present invention discloses a computer program product, which can overcome the defects that the client update operation is easy to be maliciously forged when being traced and the collaborative notes are over-expanded as the editing extends, so as to achieve the purpose of preventing forgery and expansion while accurately tracing.
[0019] A computer program product comprises computer instructions, which when executed by a processor implement the collaborative note data processing method as described in any one of the above.
[0020] In summary, the embodiments of the present application disclose a method, device, storage medium and computer program product for processing collaborative note data, which binds client information to update data on the server side, and the client cannot add it by itself, so that each update data can be accurately traced and malicious forgery by the client is avoided. In addition, since the client information is not added to the collaborative note content and is merged and aggregated regularly, the expansion of the collaborative note itself is prevented. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0022] Figure 1 This is the application scenario for the implementation of this application solution.
[0023] Figure 2 This is a flowchart of Example 1 of the method for processing collaborative note data implemented in this application.
[0024] Figure 3 It is a flowchart of the second embodiment of the method of the present application, in which the server periodically merges all the second updated data in the buffer queue into one original data record according to the first preset time interval, and stores them in sequence in the database.
[0025] Figure 4 In the third embodiment of the method of the present application, the server periodically aggregates all the original data records in the database according to the second preset time interval to generate a flow chart of aggregated data records.
[0026] Figure 5 This is a flow chart of the method for generating milestone data records in Example 4 of the present application.
[0027] Figure 6 This is a flow chart of the method for processing collaborative note data implemented in Example 5 of the present application.
[0028] Figure 7 This is a schematic diagram of server-side storage situation 1 in Example 5 of the present application.
[0029] Figure 8 This is a schematic diagram of server-side storage situation 2 in Example 5 of the present application.
[0030] Fig. 9 This is a schematic diagram of server-side storage situation 3 in Example 5 of the present application.
[0031] Fig.10 This is a schematic diagram of server-side storage situation 4 in Example 5 of the present application.
[0032] Fig.11 It is a schematic diagram of the internal structure of the first embodiment of the processing device for collaborative note data implemented in the present application. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0034] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0035] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0036] In view of the defects of existing collaborative note-taking technology in that the content of notes is easy to be maliciously forged and the content of notes is bloated when accurately traced, the embodiment of the present application proposes a method for processing collaborative note data, advocating binding the client information with the update data on the server side, so that each update data operated on the client can be accurately traced. This method avoids the situation where the client adds fields on its own and causes malicious forgery, and there is no need to increase the content of the collaborative notes themselves. It also further prevents the collaborative notes from expanding as the editing continues by means of merging and aggregation. The content of the collaborative notes themselves described in the embodiment of the present application refers to the document content collaboratively edited by multiple clients, which may include text, pictures, tables, videos and other content, and is usually presented to the user through the client browser.
[0037] Figure 1 This is the application scenario for the implementation of this application solution. Figure 1 As shown, the scenario includes N clients 101, where N is a natural number. Each client 101 can be connected to the server 102 via WebSocket, and the updated data of the collaborative note content will also be sent to the server 102 via the WebSocket connection. The server 102 receives the updated data from the client 101, processes and saves it. Any client 101 can also view the content of the collaborative note through the server 102.
[0038] Figure 2 This is a flow chart of the first embodiment of the method for processing collaborative note data implemented in this application. Figure 2 As shown, the method includes:
[0039] Step 201: The server receives a collaborative note update message sent by the client through a WebSocket connection, where the collaborative note update message includes first update data for the collaborative note updated by the client.
[0040] In the scenario of collaborative note-taking, multiple clients can edit and update the same collaborative note at the same time. The update data generated by each edit by any client is sent to the server through the WebSocket connection. In order to distinguish it from the data in subsequent steps, the data edited and updated by the client is referred to as the first update data, which is carried in the collaborative note update message and sent to the server. In addition, the "update" described in the embodiments of the present application can be understood as the editing of collaborative notes, which can include operations such as modification, addition, and deletion. These operations are called updates to the notes, and the data generated are called update data.
[0041] Step 202: The server binds the client information corresponding to the client with the first update data, generates second update data, and stores the second update data in a buffer queue.
[0042] Since each client is connected to the server through WebSocket, the server has client information to distinguish each client, and can also identify which client the received update data comes from. When the server receives the first update data from a client, the client information of the client saved in advance can be bound to the first update data. In order to distinguish it from the original first update data, the update data bound to the client information is referred to as the second update data. It should be noted that here only the client information is bound to the first update data, and no information is added to the first update data, and the collaborative note content itself will not be increased.
[0043] In addition, in order to reduce the frequency of operations on the database, the second update data is first stored in the buffer queue of the server.
[0044] Step 203: The server periodically merges all the second update data in the buffer queue into one original data record according to the first preset time interval, and stores it in the database in sequence.
[0045] The first preset time interval can be set according to the actual situation. If the first preset time interval is short, the client update status can be recorded in the database more accurately; if the first preset time interval is long, database space and database operation time can be saved. For example, the first preset time interval is set to 2 seconds. Every 2 seconds, the server merges all the second update data in the cache queue and saves it in the database as an original data record. If there are 200 second update data in the buffer queue at this time, only one original data record will be generated in the database after merging, which greatly reduces the occupancy of database space.
[0046] Step 204: the server periodically aggregates all original data records in the database according to a second preset time interval to generate an aggregated data record, and deletes the original data records involved in the aggregated processing.
[0047] The second preset time interval can be set according to actual conditions. Aggregating all the original data records in the database can further reduce the space occupied by the database. For example, the records from 101 to 200 in the database are all original data records, that is, the original data records generated by the above step 203. These 100 original data records can be further aggregated into an aggregated data record, and the original data records involved in the aggregation processing are deleted. At this time, the 101st record in the database will no longer be an original data record, but a new aggregated data record, which preserves the data content in the original 101st to 200th original data records.
[0048] By applying the embodiment of the present application, the client information is bound to the updated data so that each updated data can be accurately traced. Since the embodiment of the present application binds the client information by the server and does not add it to the collaborative note content, malicious forgery by the client and the expansion of the collaborative note itself are avoided. At the same time, the server further reduces the space occupied by the collaborative notes in the database by means of data merging and aggregation.
[0049] The database in the embodiment of the present application can be a relational database, such as a MySQL database. The MySQL database is mainly used to store structured data, supports complex query processing and multiple data types, and is suitable for complex business logic and data relationships. Of course, other database types can also be used in practical applications, which does not limit the scope of protection of the embodiment of the present application.
[0050] In an embodiment of the present application, a data primary key field, a record type field, a user operation record field, and a user operation record identification field and a field for the full data of the notes before the update can be designed for the MySQL database. A new record is created in the database, and the record can contain each of the above fields, and the record is referred to as a database record in an embodiment of the present application. Among them, the data primary key field represents the identification of the database record, the record type field represents the type of the database record, the user operation record field represents the content of the user's update operation on the collaborative notes, the user operation record identification field represents the identification of the user operation record, and the field for the full data of the notes before the update represents the overall content of the collaborative notes before the user's update operation. In actual applications, other fields can also be designed according to actual needs, which are not listed one by one here.
[0051] Figure 3 This is a flowchart of the server in the second embodiment of the present invention periodically merging all the second updated data in the buffer queue into an original data record according to the first preset time interval and sequentially storing them in the database, that is, a specific implementation of the above step 203. Figure 3 As shown, the method includes:
[0052] Step 301: When the first preset time interval arrives, the server merges all the second update data in the buffer queue to obtain a merge result.
[0053] The first preset time interval can be set according to the situation, such as 2 seconds.
[0054] Assume that the buffer queue includes 100 second update data UPDATA1~UPDATA100, and each second update data includes two pieces of information, one is the first update data update sent by the client, and the other is the corresponding client information userInfo, and the two are bound. This step merges UPDATA1~UPDATA100 to obtain the merged result "UPDATA1+UPDATA2+...+UPDATA100". It should be noted that the merger described in this step is not the final update result of the collaborative note update, but saves each update at the same time. The merger here can also be understood as splicing, that is, the merged result will not overwrite any second update data.
[0055] Step 302: The server generates a database record in the database as the first database record. The database is a relational database, which includes at least a data primary key field, a record type field, and a user operation record field. The data primary key field indicates the identifier of the database record, the record type field indicates the type of the database record, and the user operation record field indicates the content of the user's update operation on the collaborative notes.
[0056] Step 303: The server adds the merge result to the user operation record field of the first database record, sets the record type field of the first database record to the original type, and automatically generates the value of the data primary key field for the first database record in sequence, and the first database record is regarded as an original data record.
[0057] At this time, a new first database record is added to the database, and the first database record includes at least a data primary key field, a record type field, and a user operation record field. Among them, the data primary key field is automatically generated by the database in ascending order, the record type field is set to the original type, and the merge result "UPDATA1+UPDATA 2+...+UPDATA100" in the above example is added to the user operation record field. Since the record type of the first database record in this step is the original type, it is also called the original data record in the embodiment of the present application.
[0058] In the embodiment of the present application, the record type field can be divided into three types: the first is the original type, which means that the data in this record is directly derived from the buffer queue, and its data content is the original data content generated by the client; the second is the aggregate type, which means that the data in this record has been aggregated, and its data content is aggregated from multiple database records; the third is the milestone type, which means that the data in this record has not only been aggregated, but also carries the full amount of note data before the update, so you can see the full picture of the collaborative note content, which is a milestone. This step only involves the original type, and the aggregate type and milestone type will be introduced in subsequent embodiments.
[0059] Step 304: Clear the buffer queue.
[0060] Since all the second update data in the buffer queue are stored in the database according to the mode designed in this embodiment, the buffer queue can be cleared here to facilitate processing of the collaborative note update message subsequently sent by the client.
[0061] The embodiment of the present application merges the second updated data in the buffer queue and saves it in the database, and sets it to the original type. This not only comprehensively saves the client's update status of collaborative notes, but also reduces the frequency of database operations and the occupancy of database space.
[0062] Figure 4 The server in the third embodiment of the present invention periodically aggregates all the original data records in the database according to the second preset time interval to generate a flowchart of aggregated data records, that is, the specific implementation of the above step 204. Figure 4 As shown, the method includes:
[0063] Step 401: When the second preset time interval arrives, the server detects that there is an original data record in the database.
[0064] The second preset time interval can be set according to actual conditions, for example, to 60 seconds.
[0065] Assume that 50 original data records are detected in the database, and the record type fields of the 101st to 150th records in the database are all original types. In actual application, if no original data records in the database are detected when the second preset time interval arrives, no processing is performed, and subsequent steps 402 to 404 are omitted and the detection of the next cycle is continued.
[0066] Step 402: Aggregate all detected original data records, concatenate the contents of the user operation record fields in all original data records, and generate an aggregated user operation record value.
[0067] Taking the case where 50 original data records are detected, this step aggregates the 101st to 150th database records. The aggregation mentioned here can be understood as concatenating the contents of the user operation record fields in the above 50 original data records, and the concatenation result is used as the aggregated user operation record value. It should be noted that the concatenation described in this step will not overwrite the contents of the user operation record fields of any original data record participating in the aggregation.
[0068] Step 403: The server generates a database record in the database as the second database record, determines the user operation record field of the second database record according to the aggregated user operation record value, sets the record type field of the second database record to the aggregate type, and determines the value of the data primary key field for the second database record in sequence, and the second database record is treated as an aggregated data record.
[0069] In order to better save database resources, this step can also further analyze the aggregated user operation record value. If the aggregated user operation record value is too large and occupies more database resources, it is not suitable to continue to be saved in the database. Otherwise, it should still be saved in the database. Specifically, this step can determine the user operation record field of the second database record according to the aggregated user operation record value by: first serializing the aggregated user operation record value to obtain the corresponding string; then calculating the length of the string; and then determining the user operation record field according to the length of the string. In other words, this step uses the method of serializing the aggregated user operation record value into a string to measure its size. There are two ways to determine the user operation record field according to the length of the string:
[0070] The first type: when the length of the string exceeds the preset length, the aggregated user operation record value is saved in the cloud storage file in text form, and the key value corresponding to the cloud storage file is added to the user operation record identification field of the second database record, and the user operation record field of the second database record itself is set to empty. The database further includes a user operation record identification field.
[0071] That is to say, if the length of the string is too long, the aggregated user operation record value is too large to be stored in the database. It can be stored in a cloud storage file outside the database. Cloud storage files are also called object storage service (OSS) files, such as Alibaba Cloud OSS storage files. OSS storage mainly stores unstructured data and metadata objects, and has the characteristics of scalability, persistence, and accessibility. Since the aggregated user operation record value has been saved in the cloud storage file, the corresponding user operation record field itself can be set to empty and no information is saved. At this time, since the aggregated user operation record value is saved in the cloud storage file, only its corresponding key value is stored in the user operation record identification field, which greatly reduces the space occupied by the database.
[0072] The second method: if the length of the string does not exceed the preset length, the aggregated user operation record value is directly added to the user operation record field of the second database record. In other words, if the length of the string is not too long, the aggregated user operation record value can continue to be directly saved in the database, that is, saved in the user operation record field.
[0073] Step 404: Delete the original data records involved in the aggregation process.
[0074] Since the aggregated data record has been generated in step 403, in which the data contents of all original data records are saved, it is not necessary to retain them here and they can be deleted.
[0075] The embodiment of the present application aggregates the original data records in the database so that multiple original data records are aggregated into one aggregated data record, reducing the occupation of database resources. In addition, when the aggregated user operation record value is too large, the aggregated user operation record value is saved in a cloud storage file, further reducing the occupation of database resources.
[0076] In the above embodiment, the updated data in the buffer queue is merged into the original data record by merging, and then the original data record in the database is further aggregated into the aggregated data record by aggregation. After aggregation, the aggregated data record in the database only contains the aggregated user operation record value or its corresponding key value. The essence of the aggregated user operation record value is the update data of the client's update operation on the collaborative note, which cannot reflect the overall content of the collaborative note. If the overall content of the collaborative note needs to be obtained, it is necessary to accumulate the updated data in the aggregated data record one by one in sequence on the basis of the full amount of note data before the update. The full amount of note data before the update here refers to the overall content of the collaborative note before the user's update operation. Assuming that the full amount of note data before the update is recorded as A, the user updates it five times, namely B1, B2, B3, B4, and B5, then A+B1+B2+B3+B4+B5 can get the overall content of the collaborative note after five updates. If the number of aggregated data records is quite large, it will take more time to obtain the overall content of the collaborative note. In order to quickly obtain the overall content of the collaborative note later, the embodiment of the present application also provides a method for generating milestone data records. In this way, assuming that the overall content of the collaborative note is calculated after the third update, the overall content of the new collaborative note can be obtained by accumulating only two update data. In other words, if after the third update, the overall content of the collaborative note is calculated to be A+B1+B2+B3, recorded as A1, then the overall content of the new collaborative note can be calculated by A1+B4+B5, thereby speeding up the acquisition of the overall content of the collaborative note. Since A1 has a milestone-like meaning, this method is called milestone data recording in the embodiment of the present application.
[0077] Figure 5 This is a flow chart of a method for generating milestone data records in Example 4 of the present application. The method includes:
[0078] Step 501: generating an aggregated data record in an aggregated data record as a current aggregated data record.
[0079] The aggregated data record in this step refers to the aggregated data record generated in step 204 and step 403. This aggregated data record is newly generated, and for the convenience of description, it is referred to as the current aggregated data record here.
[0080] Step 502: trace back from the current aggregate data record, query the database record whose previous record type field is a milestone type and record it as the third database record, the third database record is a milestone data record, and when no record is found, an empty database record is used as the third database record.
[0081] Database records use the data primary key field as their identifier, usually increasing in order from small to large. The backtracking mentioned here refers to querying from the current aggregate data record to the direction with smaller data primary key fields. The milestone data record closest to the current aggregate data record is queried as the third database record. If it is not queried, it means that there is no milestone data record in the database, and the current aggregate data record may become the first milestone data record. When an empty database record participates in the calculation, the content of each field can be regarded as a default initial value such as empty or 0.
[0082] It should be noted that the database records in the embodiments of the present application have the same essential structure. For the convenience of description, the record type field with the original type is called the original data record, the record type field with the aggregate type is called the aggregate data record, and the record type field with the milestone type is called the milestone data record. Similarly, for the convenience of describing the database operation, the original data record stored in the database after the buffer queue is merged is called the first database record, the aggregated data record stored in the database after aggregation is called the second database record, and the record marked as a milestone data record is called the third database record.
[0083] Step 503: Calculate the sequence distance between the current aggregate data record and the third database record, where the sequence distance represents the distance between the data primary key fields of the two database records.
[0084] This step actually calculates the distance between the current aggregate data record and the previous milestone data record. If the distance is large, it means that the milestone data record has not been marked for a long time, and it will take a long time to trace back to obtain the overall content of the collaborative note. If the distance is small, it means that the milestone data record has been marked in a short time, and the overall content of the collaborative note can be quickly traced back.
[0085] Step 504: When the sequence distance exceeds the preset distance, calculation is performed based on the full data field of the note before update recorded in the third database, the aggregated user operation record value of all aggregated data records between the current aggregated data record and the third database record, and the calculation result is used as the new full data of the note before update. The database further includes a full data field of the note before update, and the full data field of the note before update represents the overall content of the collaborative note before the user's update operation.
[0086] The preset distance can be set according to the actual situation, such as 100. Since the queried third database record is a milestone data record, the full data field of the note before update saves the overall content of the collaborative note, which is used as the basis for calculation here. The steps for calculating based on the full data field of the note before update recorded in the third database, the aggregated user operation record value of all aggregated data records between the current aggregated data record and the third database record are as follows: based on the content of the full data field of the note before update recorded in the third database, the aggregated user operation record values of all aggregated data records between the current aggregated data record and the third database record are added one by one in sequence to the content of the full data field of the note before update recorded in the third database, and new full data of the note before update is generated. In this way, the new full data of the note before update is obtained, that is, the overall content of the collaborative note corresponding to the current aggregated data record. For example: based on the above example, a new update B6 is generated, that is, the distance between B6 and A1 reaches the preset distance 3. Then, based on A1, all updated data B4, B5, and B6 are added to A1, A1+B4+B5+B6, and the new pre-update note data A2=A1+B4+B5+B6 is calculated.
[0087] Step 505: Add the new full data of the notes before update to the full data field of the notes before update of the current aggregate data record, update the record type field of the current aggregate data record to the milestone type, and change the current aggregate data record to a milestone data record as a new third database record.
[0088] If the sequence distance between the current aggregated data record and the third database record exceeds the preset distance, and the new full data of the notes before the update has been calculated, it can be added to the full data field of the notes before the update of the current aggregated data record. In the above example, the new full data of the notes before the update A2 = A1 + B4 + B5 + B6 is calculated. At this time, A2 can be added to the current aggregated record to become data records such as A, B1, B2, A1, B4, B5, A2. Here A, A1, and A2 are milestone data records, while B1, B2, B4, and B5 are ordinary aggregated data records. Among them, the full data field of the notes before the update in the milestone data record contains the overall content of the collaborative notes. For example, A1 contains the contents of A, B1, B2, and B3, and A2 contains the contents of A1, B4, B5, and B6. Since the current aggregated data record is changed to a milestone data record, it can become the basis for other subsequent data records.
[0089] Of course, if the sequence distance does not exceed the preset distance, the above steps 504 and 505 will not be executed and can be omitted.
[0090] By applying the embodiments of the present application, milestone data records are set in the database according to a certain distance, and the overall content of the collaborative notes can be quickly obtained on this basis, meeting the user's needs to view the collaborative notes at any time.
[0091] In another embodiment, the above-mentioned CRDT collaborative notes can be implemented based on the yjs open source framework. Yjs can realize real-time collaborative editing functions, provide a series of CDRT data types, and can realize collaborative editing (addition, deletion, modification, etc.) between different users to ensure that they finally converge to a consistent state. The embodiment includes multiple browser note editors and a centralized yjs CRDT data server. The browser note editor here is the client in the above embodiment, and the centralized yjs CRDT data server is the above server, which can be placed in an all-in-one collaborative office platform. The all-in-one collaborative office platform can be any office platform, such as the office platform of a certain enterprise or scientific research unit. The client browser is connected to the back end (centralized yjs CRDT data server) via HTTP and WebSocket, and its scientific research data and editing operations on the notes need to be traceable to ensure the rigor of the scientific research process. The database of the embodiment of the present application is a MySQL database, and the fields set include: data primary key field (id), record type field (historyType), full data field of notes before update (pre), user operation record field (ops), user operation record identification field (opsKey), and its specific situation is shown in Table 1:
[0092]
[0093] Table 1
[0094] Figure 6 This is a flowchart of a method for processing collaborative note data implemented in Embodiment 5 of the present application. Figure 6 As shown, the method includes:
[0095] Step 601: The server receives a collaborative note update message sent by the client through a WebSocket connection. The collaborative note update message includes first update data for the collaborative note updated by the client.
[0096] Step 602: The server binds the client information corresponding to the client with the first update data, generates second update data, and stores the second update data in a buffer queue.
[0097] The above steps 601 to 602 are the same as steps 201 to 202 in the first method embodiment.
[0098] Figure 7 This is a schematic diagram of the server-side storage situation 1 in the fifth embodiment of the present application. Figure 7 As shown, the server side (centralized yjs CRDT data server) has a buffer queue and a database, where the database is a MySQL database, which can be inside the server or a third party outside the server. Assume that at a certain moment, the server side receives a collaborative note update message from a client (browser note editor) through WebSocket, which includes the first update data update. At this time, the server side binds the client information userinfo corresponding to the client to the first update data update, generates the second update data updateData100, and pushes it into the buffer queue.
[0099] Step 603: When the first preset time interval arrives, the server merges all the second update data in the buffer queue to obtain a merge result.
[0100] Step 604: The server generates a database record in the database and records it as the first database record.
[0101] Step 605: The server adds the merge result to the user operation record field of the first database record, sets the record type field of the first database record to the original type, and automatically generates the value of the data primary key field for the first database record in sequence, and the first database record is regarded as an original data record.
[0102] Step 606: Clear the buffer queue.
[0103] The above steps 603 to 606 are the same as steps 301 to 304 in the second method embodiment.
[0104] Figure 8 This is a schematic diagram of the server-side storage situation 2 in the fifth embodiment of the present application. Figure 8 As shown, the server periodically merges all updateData in the buffer queue into a database record and saves it in the database, which is recorded as the first database record, also called the original data record. Assuming that at a certain moment, the first preset time interval (for example, 2 seconds) is reached, the server merges all the second update data updateData1~updateData100 in the buffer queue to generate a database record record499, that is, a database record with the data primary key field (id) as 499, and the record type field (historyType) of the database record is set to "RAW", and the user operation record field (ops) is the merged result of updateData1~updateData100. After generating this database record, the buffer queue will also be cleared.
[0105] Step 607: When the second preset time interval arrives, the server detects that there is an original data record in the database.
[0106] Step 608: Aggregate all detected original data records, concatenate the contents of the user operation record fields in all original data records, and generate an aggregated user operation record value.
[0107] Step 609: The server generates a database record in the database as the second database record, determines the user operation record field of the second database record according to the aggregated user operation record value, sets the record type field of the second database record to the aggregate type, and determines the value of the data primary key field for the second database record in sequence. The second database record is treated as an aggregated data record.
[0108] Step 610: Delete the original data records involved in the aggregation process.
[0109] The above steps 607 to 610 are the same as steps 401 to 404 in the third method embodiment.
[0110] Fig. 9 This is a schematic diagram of the server-side storage situation 3 in the fifth embodiment of the present application. Fig. 9As shown, the server periodically aggregates the original data records in the database into one database record, recorded as the second database record, also called the aggregated data record. Assuming that at a certain moment, the second preset time interval (for example, 60 seconds) is reached, the server aggregates all the original data records RAW400 to RAW499 queried in the database into one database record record400, that is, a database record with the data primary key field (id) as 400, and sets the record type field (historyType) of the database record to "MERGED", and aggregates the user operation record value mergedOps = ops400 + ... + ops499. In actual applications, js can be used to implement it as follows:
[0111] const mergedOps=datas.map(x=>x.ops).flatten();
[0112] Similarly, the method for determining the user operation record field of the second database record based on the aggregated user operation record value can be: first serialize the aggregated user operation record value mergedOps to obtain the corresponding string; then calculate the length of the string; and then determine the user operation record field based on the length of the string.
[0113] In actual applications, the JSON.stringify method can be used for serialization. There are two ways to determine the user operation record field based on the length of the string: The first is that if the length of the string exceeds the preset length, the aggregated user operation record value is saved in the cloud storage file in text form, and the key value corresponding to the cloud storage file is added to the user operation record identification field of the second database record, and the user operation record field of the second database record itself is set to empty, and the database further includes a user operation record identification field. The second is that if the length of the string does not exceed the preset length, the aggregated user operation record value is directly added to the user operation record field of the second database record. In actual applications, the preset length can be set by yourself, for example, to 256*10 bytes.
[0114] Fig. 9 Two cases are shown. When the length of the string exceeds the preset length, the aggregated user operation record value mergedOps is saved in the OSS cloud storage file in text format, and the key value corresponding to the cloud storage file is added to the user operation record identification field (opsKey) of record400, recorded as "key", and the user operation record field (ops) of record400 is empty. If the length of the string does not exceed the preset length, the aggregated user operation record value mergedOps is directly added to the user operation record field (ops) of record400.
[0115] Step 611: generating an aggregated data record in an aggregated data record as a current aggregated data record.
[0116] Step 612: trace back from the current aggregate data record, query the database record whose previous record type field is a milestone type and record it as the third database record, the third database record is a milestone data record, and when not found, an empty database record is used as the third database record.
[0117] Step 613: Calculate the sequence distance between the current aggregated data record and the third database record, where the sequence distance represents the distance between the data primary key fields of the two database records.
[0118] Step 614: When the sequence distance exceeds the preset distance, calculation is performed based on the full data field of the note before update recorded in the third database, the aggregated user operation record value of all aggregated data records between the current aggregated data record and the third database record, and the calculation result is used as the new full data of the note before update.
[0119] Step 615: Add the new full data of the notes before update to the full data field of the notes before update of the current aggregate data record, update the record type field of the current aggregate data record to the milestone type, and change the current aggregate data record to a milestone data record as a new third database record.
[0120] The above steps 611 to 615 are the same as steps 501 to 505 in the fourth method embodiment.
[0121] Fig.10 This is a schematic diagram of the server-side storage situation 4 in the fifth embodiment of the present application. Fig. 9 As shown, when the server generates an aggregate data record, it will further analyze whether it can become a milestone data record. If the sequence distance between the current aggregate data record and the third database record exceeds the preset distance, the new pre-update note full data newPre is calculated and added to the pre-update note full data field (pre) of record400, and the record type field (historyType) of record400 is modified to "MILESTONE". At this time, record400 becomes a milestone data record. In actual applications, the preset distance can be set according to actual conditions, such as 100.
[0122] Assume that the full data field (pre) of the third database record before the update is pre, and all the user operation record fields (ops) between the current aggregated data record and the third database record are ops1…ops100, and are added one by one to the pre of the third database record to obtain the new full data of the notes before the update newPre. Of course, if the user operation record field (ops) is empty, the corresponding user operation record identification field (opsKey) can be used to obtain it from the cloud storage file. This method uses the following to calculate newPre:
[0123] const newPre=Y.applyUpdate(pre,Y.mergeUpdates(allOps.map(x=>x.update)));
[0124] In particular, if the database record whose record type field is a milestone type is not queried, an empty database record is used as the third database record, and newPre is calculated using the following method:
[0125] const nowPre=Y.applyUpdate(new Y.Doc(),Y.mergeUpdates(allOps.map(x=>x.update)));
[0126] Applying the embodiment scheme of the present application, CRDT collaborative notes are implemented based on the yjs open source framework, and the client information is bound to the updated data so that each updated data can be accurately traced. The client information is bound by the server and is not added to the collaborative note content, which avoids malicious forgery by the client and prevents the expansion of the collaborative note itself. The server further reduces the space occupied by the collaborative notes in the database by merging and aggregating data. At the same time, by setting milestone data records in the database, when the overall content of the collaborative notes needs to be obtained, it is necessary to traverse too many historical data records, and the overall content of the collaborative notes can be quickly restored.
[0127] An embodiment of the present application also provides a collaborative note data processing device, which is implemented on the server side. Fig.11 This is a schematic diagram of the internal structure of the first embodiment of the collaborative note data processing device implemented in this application. Fig.11 As shown, the device includes: a transceiver module 1101, a cache module 1102, and a database operation module 1103. Among them:
[0128] The transceiver module 1101 is used to receive a collaborative note update message sent by the client through a WebSocket connection. The collaborative note update message includes first update data for the collaborative note updated by the client, and adds the client information corresponding to the client to the first update data to generate second update data.
[0129] The cache module 1102 is used to cache the second update data.
[0130] The database operation module 1103 is used to periodically merge all the second updated data in the buffer queue into an original data record according to a first preset time interval, and save them in sequence in the database, and periodically aggregate all the original data records in the database according to a second preset time interval to generate an aggregated data record, and delete the original data records involved in the aggregation processing.
[0131] That is, the transceiver module 1101 receives the collaborative note update message sent by the client through the WebSocket connection, adds the client information corresponding to the client to the first update data, and generates the second update data; the cache module 1102 caches the second update data; the database operation module 1103 periodically merges all the second update data in the buffer queue into an original data record according to the first preset time interval, and saves them in sequence in the database, and periodically aggregates all the original data records in the database according to the second preset time interval to generate an aggregated data record, and deletes the original data records involved in the aggregation processing.
[0132] By applying the embodiment of the present application, the client information is bound to the updated data so that each updated data can be accurately traced. Since the client information is bound by the server and is not added to the collaborative note content, malicious forgery by the client and the expansion of the collaborative note itself are avoided. At the same time, the server further reduces the space occupied by the collaborative notes in the database by merging and aggregating the data.
[0133] In actual applications, the database operation module 1103 may also be implemented in the manner of the above-mentioned method embodiments, which will not be described in detail here.
[0134] The embodiment of the present application also provides a computer-readable medium, the computer-readable storage medium stores instructions, and the instructions can execute the steps in the collaborative note data processing method described above when executed by a processor. In practical applications, the computer-readable medium can be included in the device / apparatus / system described in the above embodiment, or it can exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the collaborative note data processing method described in the above embodiments can be implemented. According to the embodiment disclosed in the present application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, for example, it can include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, but it is not used to limit the scope of protection of the present application. In the embodiment disclosed in the present application, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0135] An embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.
[0136] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0137] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in the present application.
[0138] Specific embodiments are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation methods and application scopes according to the ideas, spirits and principles of the present invention, and any updates, equivalent replacements, improvements, etc. made by them should be included in the scope of protection of this application.
Claims
1. A method for processing collaborative note data, characterized in that: The method includes: The server receives a collaborative note update message sent by the client through a WebSocket connection, where the collaborative note update message includes first update data of the collaborative note updated by the client; The server binds the client information corresponding to the client with the first update data to generate second update data, and stores the second update data in a buffer queue; The server periodically merges all the second update data in the buffer queue into one original data record according to a first preset time interval, and stores the original data record in sequence in the database; The server periodically aggregates all the original data records in the database according to a second preset time interval to generate an aggregated data record, and deletes the original data records involved in the aggregated processing.
2. The method according to claim 1, characterized in that The server periodically merges all the second update data in the buffer queue into one original data record according to a first preset time interval, and sequentially stores the data in the database, comprising: When the first preset time interval arrives, the server merges all the second update data in the buffer queue to obtain a merge result; The server generates a database record in the database as a first database record, and the database is a relational database, which at least includes a data primary key field, a record type field, and a user operation record field; the data primary key field indicates an identifier of the database record, the record type field indicates a type of the database record, and the user operation record field indicates the update operation content of the user on the collaborative note; The server adds the merge result to the user operation record field of the first database record, sets the record type field of the first database record to the original type, and automatically generates the value of the data primary key field for the first database record in sequence, and the first database record is used as one of the original data records; The buffer queue is cleared.
3. The method according to claim 2, characterized in that The server periodically aggregates all the original data records in the database according to a second preset time interval to generate an aggregated data record, comprising: When the second preset time interval arrives, the server detects that the original data record exists in the database; Aggregate all the detected original data records, concatenate the contents of the user operation record fields in all the original data records, and generate an aggregated user operation record value; The server generates a database record in the database as a second database record, determines the user operation record field of the second database record according to the aggregated user operation record value, sets the record type field of the second database record to the aggregate type, and sequentially determines the value of the data primary key field for the second database record, and the second database record serves as the aggregated data record.
4. The method according to claim 3, characterized in that The step of determining the user operation record field of the second database record according to the aggregated user operation record value comprises: Serializing the aggregated user operation record value to obtain a corresponding character string; Calculate the length of the string; The user operation record field is determined according to the length of the character string.
5. The method according to claim 4, characterized in that The step of determining the user operation record field according to the length of the character string comprises: If the length of the string exceeds a preset length, the aggregated user operation record value is saved in a cloud storage file in text form, and the key value corresponding to the cloud storage file is added to the user operation record identification field of the second database record, and the user operation record field of the second database record itself is set to empty, and the database further includes the user operation record identification field.
6. The method according to claim 4, characterized in that The step of determining the user operation record field according to the length of the character string comprises: If the length of the character string does not exceed a preset length, the aggregated user operation record value is directly added to the user operation record field of the second database record.
7. The method according to any one of claims 1 to 6, characterized in that: After the step of generating an aggregated data record and deleting the original data record involved in the aggregation process, the method further comprises: Using the aggregated data record in the generating of an aggregated data record as a current aggregated data record; Tracing back from the current aggregated data record, querying the database record whose record type field is a milestone type and recording it as a third database record, the third database record being a milestone data record, and when no milestone data record is found, taking an empty database record as the third database record; Calculating a sequence distance between the current aggregated data record and the third database record, wherein the sequence distance represents a distance between data primary key fields of the two database records; When the sequence distance exceeds the preset distance, a calculation is performed based on the full data field of the note before update recorded in the third database, the aggregated user operation record value of all the aggregated data records between the current aggregated data record and the third database record, and the calculation result is used as the new full data of the note before update, and the database further includes the full data field of the note before update, and the full data field of the note before update represents the overall content of the collaborative note before the user update operation; Add the new full data of the notes before update to the full data field of the notes before update of the current aggregate data record, update the record type field of the current aggregate data record to the milestone type, and change the current aggregate data record to the milestone data record as a new third database record.
8. The method according to claim 7, characterized in that The step of calculating the aggregated user operation record value of all the aggregated data records between the current aggregated data record and the third database record according to the full data field of the note before update recorded in the third database includes: Based on the content of the full data field of the notes before update recorded in the third database, the aggregated user operation record values of all the aggregated data records between the current aggregated data record and the third database record are added one by one to the content of the full data field of the notes before update recorded in the third database record in order to generate new full data of the notes before update.
9. A collaborative note data processing device, characterized in that: The device comprises: a transceiver module, a cache module, and a database operation module; The transceiver module is used to receive a collaborative note update message sent by a client through a WebSocket connection, wherein the collaborative note update message includes first update data for the collaborative note updated by the client, and add the client information corresponding to the client to the first update data to generate second update data; The cache module is used to cache the second update data; The database operation module is used to periodically merge all the second updated data in the buffer queue into an original data record according to a first preset time interval, and save them in sequence in the database; periodically aggregate all the original data records in the database according to a second preset time interval to generate an aggregated data record, and delete the original data records involved in the aggregation processing.
10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by the processor, the steps of the collaborative note data processing method described in any one of claims 1 to 8 can be implemented.
11. A computer program product, comprising computer instructions, which, when executed by a processor, implement the collaborative note data processing method as described in any one of claims 1 to 8.