A Consistency Maintenance Method in Collaborative Annotation of Accessible Information

Abstract

This invention provides a consistency maintenance method for collaborative accessibility information annotation in shared maps, specifically addressing collaborative scenarios involving accessibility information annotation. The invention includes: designing a copy synchronization protocol based on scalar timestamps, enabling rapid determination of causal or concurrent relationships between operations with the aid of scalar timestamps; designing a method for partially copying accessibility information in shared map pairs, significantly reducing resource consumption such as memory usage and network resource requirements on mobile devices; improving traditional address space translation algorithms to maintain data consistency in collaboratively edited documents by multiple users in accessibility information annotation scenarios; and proposing a rule-based address space translation algorithm, ensuring that client operations not only follow rules established by various operations to guarantee semantic consistency between accessibility information but also ensure content consistency between copies. This invention can meet the needs of real-time collaborative annotation of accessibility information by multiple users in shared maps.

Description

Technical Field

[0001] This invention belongs to the field of computer-supported collaborative work technology, and specifically relates to a consistency maintenance method in collaborative annotation of accessible information. Background Technology

[0002] (1) Urban Spatial Information Acquisition Technology

[0003] Urban accessibility is often inadequate for vulnerable groups such as people with disabilities and the elderly. Despite efforts to improve urban accessibility for all residents, the urban built environment remains one of the most practical reasons why accessibility is difficult for people with disabilities. Furthermore, the lack of information about the urban environment and its accessibility is itself a significant barrier preventing people with disabilities from navigating outside known areas. Therefore, collecting and utilizing this accessibility information is crucial for improving accessibility. In the current social environment and with current technology, obtaining georeferenced data on urban accessibility generally comes from three sources: First, utilizing various sensors: data generated by users moving through the urban environment. Smartphone users, equipped with gyroscopes, accelerometers, and GPS, can run an application to perceive urban accessibility data. While data from a single user may not be entirely accurate, multiple perceptions of the same barrier / facility can enhance the data's effectiveness. Second, using crowdsourcing: users interested in urban accessibility can collect data through mobile applications. Such applications can collect both textual and multimedia data, and in this case, multiple data points enhance the effectiveness of the collected information. Thirdly, data can be obtained from official sources or relevant organizations: Many authoritative institutions and organizations conduct official surveys on indoor and outdoor accessibility environments. However, the most mainstream data collection method today is to use a crowdsourcing model to collect urban spatial information.

[0004] With the continuous development of modern communication and internet technologies, an increasing number of studies are dedicated to marking accessible or obstructed geographical information on GPS satellite maps to demonstrate the accessibility of these areas. Most annotation tasks are distributed through crowdsourcing, which not only reduces annotation costs but also significantly increases the number of annotations. User participation is a key factor distinguishing these applications from others, determining their success or failure; therefore, much research is focused on improving user engagement. However, these methods only concentrate individual efforts, and accessibility annotation is not a one-time solution. Independent crowdsourced annotation tasks have certain limitations: the power of a single contributor is insufficient to perfect the annotation content and make it highly valuable. Furthermore, methods that pool collective wisdom and involve multiple users collaboratively annotating accessibility facilities are not currently widely researched.

[0005] (2) Consistency Maintenance Algorithm AST

[0006] Address Space Transformation (AST) was initially proposed as a concurrency control and data consistency maintenance method for real-time collaborative editing applications. In typical real-time collaborative editing applications, shared documents are completely copied across all clients. AST technology allows users to perform unrestricted editing operations on the document, thus providing users with a fast local response experience. When a set of identical operations is executed on all collaborating sites, all copies will be in an eventually consistent state. As a lock-free concurrency control method, AST technology is well-suited for maintaining data consistency in interactive applications. The basic idea of ​​AST technology is: before executing a remote operation, the document state (i.e., address space) is backtracked to the state at the time the operation occurred, so that the operation can be executed in its original form. After the operation is executed, the document state is restored to the latest state. To achieve address space transformation, i.e., to achieve the backtracking and restoration of document state, executed historical operations are recorded and stored in the corresponding target node; deleted nodes are not removed from the address space, but are instead set to an invalid state. However, it's worth noting that AST technology was proposed for a full data replication architecture, where address space translation is achieved by traversing the entire address space and setting valid state flags for all nodes. Determining causal and concurrency relationships between operations is fundamental to maintaining data consistency in multi-user interactive applications, and vector timestamp-based operation relationship determination methods are among the most common.

[0007] As more and more tasks require multi-person collaboration to complete smoothly, the application scenarios of Address Space Translation (AST) technology for maintaining consistency in collaborative editing are constantly expanding. At the same time, the difficulties and challenges of using AST algorithms for collaborative editing in different scenarios are also increasing. Therefore, researchers need to improve and update existing technologies to adapt to the new characteristics and requirements of different collaborative scenarios. For collaborative scenarios involving barrier-free information annotation, not only are the data models different in collaborative editing, but new copy synchronization protocols are also needed for message communication between multiple users, as well as to solve practical difficulties encountered in specific applications. Therefore, address space translation algorithms need to be improved and adjusted according to different characteristics and requirements when solving the consistency maintenance problem of collaborative editing in specific application scenarios. Traditional AST solutions cannot solve the needs and difficulties in various collaborative scenarios. Summary of the Invention

[0008] The purpose of this invention is to propose a consistency maintenance method in collaborative annotation of accessibility information.

[0009] This invention proposes a consistency maintenance method in collaborative annotation of accessibility information, the specific steps of which are as follows:

[0010] (1) Design a replica synchronization protocol based on scalar timestamps to realize the message communication protocol for real-time collaborative annotation by multiple users. With the assistance of scalar timestamps, the causal or concurrent relationship between operations can be quickly determined.

[0011] (2) Design a partial copying method for synchronizing and operating accessibility information in a shared map pair, which completes the local copy synchronization and remote operation synchronization process while greatly reducing the consumption of various resources on the mobile terminal.

[0012] (3) Adjustments and improvements were made to the traditional address space translation algorithm so that it could maintain the consistency of data content in the data model of the barrier-free information collaborative annotation scenario and realize real-time collaborative editing by multiple users;

[0013] (4) A rule-based address space conversion algorithm is proposed. While ensuring the consistency of collaborative annotation document content, various rules are set to ensure the semantic consistency of accessibility information in the final operation effect of client operations. Ultimately, a collaborative annotation method that supports real-time collaborative annotation of accessibility information by multiple users can be realized.

[0014] In this invention, the message communication protocol based on scalar timestamps described in step (1) is specifically as follows:

[0015] To better determine the causal relationships between user-generated operations, participating client sites append a timestamp to each operation before sending the operation sequence to the central server. On the central server, each received operation is assigned an auto-incrementing sequence number, called the "Server Mark Receive Operation Number" (MRN). All operation sequences received by the central server are stored in the maker sequence history buffers (SHBs) of the corresponding Marker object node MN, arranged chronologically by execution time (in ascending order of the MRNs), and are referred to as MSHBs. On the client side, locally generated operations are recorded in the local history sequence (LHBs) of the corresponding Marker object node MN, while received remote operations are recorded in the remote history sequence (RHBs). The data structure of the scalar timestamp (TS) appended to each operation is as follows:

[0016] TS=<Client,Ocpnt,LastUpdateMRN,MRN,LMS>

[0017] Wherein: Client is the client user initiating this operation; Opcnt is the number of all operation sequences within the local site LHB when this operation occurs; LastUpdateMRN is the sequence number MRN of the most recent operation synchronized with the server by the client site that initiated this operation when this operation occurs, which can be understood as the largest MRN value among all operations within LHB and RHB when the operation occurs; the MRN value is an incrementing sequence number assigned by the server to each operation; the specific meaning of LMS will be given in the next part of this invention;

[0018] When an operation O is executed on the client site S that generated it, the timestamp TS attached to operation O can determine the following data: the client user Client that generated operation O; the size of the local operation history sequence LHB (Opcnt) at the time this operation was generated; and the LastUpdateMRN of the most recent operation synchronized with the server by site S, i.e., the largest MRN value among LHB and RHB. Since the server-marked operation sequence number MRN for each operation is an auto-incrementing value assigned by the server, operation O is not synchronized with the server after its local execution, and its MRN value is temporarily empty. At this time, since all operations in LHB and RHB have been completed when O was generated, all operations in LHB are causally preceded by O.

[0019] When a client synchronizes its local operation O and its attached timestamp TS to the central server, the central server assigns a globally incrementing sequence number to each operation whose MRN value is empty in the timestamp information and returns a response to the collaborating client site that generated the operation. The response carries the sequence number MRN value assigned to this operation. After site S receives this message, the MRN value in the timestamp information of operation O is no longer empty. Simultaneously, as the server sends the response to client site S, operation O is also broadcast to other participating client sites for synchronization. After operation O is transmitted to other sites and executed, it is stored in the local site's RHB. The operation sequences stored in the RHB must be arranged in ascending order according to their MRN values.

[0020] When operation O is synchronized to another collaborating site P, at this time, in the remote operation sequence RHB of the corresponding node, any one of operation O... r The causal relationship with O can be determined in the following ways:

[0021] ●O r →O requires one of the following two conditions to be met:

[0022] (1)O r .MRN≤O.LastUpdateMRN

[0023] (2)O r When .MRN>O.LastUpdateMRN,

[0024] O r .Openid = O.Openid && O r Opcnt < O.Opcnt

[0025] ●O r When ||O, it must satisfy the condition that there is no O. r →O, and there is no O→O r

[0026] At this point, in the local operation sequence LHB of p, any operation O l The causal relationship with O can be determined in a manner similar to that in a remote operation sequence, but since the MRN value of an operation in LHB may be null, some improvements are needed:

[0027] ●O l →O, the following conditions must be met:

[0028] O l .MRN≠null&&O l.MRN≤O.LastUpdateMRN

[0029] ●O l When ||O, one of the following conditions must be met:

[0030] (1)O l .MRN=null

[0031] (2)O l When MRN ≠ null, O l .MRN>O.LastUpdateMRN

[0032] Based on the above method for determining operational relationships, we can summarize a method for determining the causal and concurrent relationships between any two operations using the timestamps attached to each operation. The methods for determining the causal relationship between any two timestamps TS1 and TS2 are summarized as follows:

[0033] ●When TS1→TS2, one of the following conditions must be met:

[0034] (1)TS1.MRN≠null&&TS1.MRN<TS2.LastUpdateMRN

[0035] (2) When TS1.Openid=TS2.Openid, TS1.Opcnt<TS2.Opcnt

[0036] ●When TS1||TS2, the following condition must be met: neither TS1→TS2 nor TS2→TS1.

[0037] In this invention, the method for partially copying accessibility information in step (2) is as follows:

[0038] In an accessible information collaborative annotation environment, user collaboration primarily operates in two modes: The first is map editing mode, mainly focused on collaborative annotation of accessible information within a shared map. This mode includes five operations: adding markers, deleting markers, updating basic marker information, connecting two markers with directed lines to form a route, and deleting the route between two markers. The second is content editing mode, which mainly involves inserting and deleting character nodes and modifying specific content of accessible information of corresponding types. The specific data structure for accessible information annotation is as follows:

[0039] Marker: <index, mId, lat, lng, type, side, contId>

[0040] Wherein: Marker represents an accessibility information marker in the shared map; index is the serial number of this marker in the collaborative document; mId is the unique identifier of the marker; lat and lng are the latitude and longitude of the marker on the map, respectively; type is the type of this marker, and different types of markers will have different editable content; side represents the region to which the marker belongs; contId records the ID of the content corresponding to this marker, which can establish a mapping relationship with the content and serve the content editing function.

[0041] Route: <index, rId, preMarker, nextMarker, side, contId>

[0042] Where: Route represents a directed path in the shared map; index is the sequence number of this marker in the collaborative document; Rid is the unique identifier of the path; preMarker represents the starting marker of a path; nextMarker represents the ending marker of the path connection. Connecting the two will form a directed line segment, representing a directed path; type is the type of this marker. Different types of markers will have different editable content; contId records the ID of the content corresponding to this marker, which can establish a mapping relationship with the content and serve the content editing function.

[0043] Side: <sideId, locationName, markers>

[0044] In this context, "Side" represents a region on the shared map, which can be a park, a building, a school, or other facilities. During collaborative annotation, various accessible markers are assigned to different regions, which are defined and selected by the user. `sideId` is a unique identifier for each region; `locationName` is the name of this region's area, such as Liao Kaiyuan Law Building, Intersection Building 2, Peace Park, etc.; `markers` represents the collection of all markers within this region (Route paths are also a type of marker). Essentially, a Side is a collection of Marker objects, but unlike the two types mentioned above, Side is defined and selected by the user and then automatically generated.

[0045] During collaborative annotation, various accessibility information is divided into different areas. These areas might be a park, a building, a school, etc., defined and selected by the user. However, each time an accessibility marker is added, the accessibility information within that area is automatically generated. Each area is defined as a "Side," and each Side contains all the marker objects annotated within that area. A Side is essentially a collection of marker objects. Each accessibility marker belongs to its own Side. The partial replication architecture in the collaborative annotation scenario is based on these Sides, synchronizing marker objects across all Sides within the shared map area for the client user. Using the client user's geographical location as a reference, various accessibility information within a 5-kilometer radius of the shared map's center point is replicated and synchronized. These Sides belong to the client user's active range, and the markers within them are active operation objects. Synchronization is performed based on these active Sides.

[0046] To ensure the proper functioning of a collaborative editing system under a partial replication architecture, the following data structures need to be introduced to optimize the collaborative editing algorithm.

[0047] (1) Add data structure to the server side:

[0048] (a) UserActiveSides: Records each client user and its corresponding active Side. It's an array set of SideIds and their corresponding active users, recording all active Sides in the collaborative system and their corresponding client users. By taking the intersection of these arrays corresponding to SideIds, overlapping collaboration areas among collaborative client users can be identified, and then operations and data synchronization can be performed with the corresponding clients.

[0049] (b) UserSidesLastUpdateMRN: Records the MRN value of the most recently synchronized operation with the server in each active side of each client user. Using this data, when the client and server synchronize again, the server can quickly filter out operations in each client user's corresponding active side that have not yet been synchronized locally, thus completing the operation and data synchronization between the client and server more quickly.

[0050] (c) SideLastRecieveMRN: Records the latest operation received from client users and synchronized to the server within each active Side. This is the largest MRN value among all operations sent by client users to synchronize to this active Side. Comparing this data with the previous data allows for quick determination of whether a client user currently synchronizing with the server has any new operations on this Side that need to be synchronized.

[0051] (2) Add data structure to the client:

[0052] (a) SideLastUpdateMRN: Records the MRN value of the last synchronized update operation performed with the server within each active Side of the client user. In a partial replication architecture, client users only synchronize operation information of the currently active Side with the server. Recording this data allows for quick access to the operation information of the new active Side when the client user moves or manually moves the shared map center point, and then synchronizes its historical operations, ensuring the system continues to function normally. This value allows for quick identification of the operations that each currently active Side of the client needs to synchronize with the server.

[0053] (b)SideOperationList: Records the operation sequence within each Side to improve retrieval speed.

[0054] The processing algorithm of the server after receiving the client's synchronization operation request is as follows: The server will receive all the operations transmitted synchronously by the client users. As mentioned above, the server will assign a globally auto-incrementing and unique server marker operation sequence number (MRN) to each operation, store it in MSHB, and broadcast the operation and its carried MRN to each client. Operations with empty MRNs in the client will no longer be empty.

[0055]

[0056] After processing various annotation operations, the central server retrieves client users who are collaborating with the client user who performed the operation to edit the same area. As shown in the following method, the central server performs a series of comparative analyses to select the operations that need to be synchronized with each participating client, and then immediately pushes them to the selected clients for synchronization. Unlike pulling a model, the server does not need to wait for the operation synchronization mechanism to be triggered before synchronization can begin. It can push the corresponding remote operations to each collaborating client in real time without unnecessary waiting, thus ensuring that server performance is not wasted due to waiting, and making the client copy synchronization process more real-time and smooth.

[0057]

[0058]

[0059] In this invention, the consistency maintenance algorithm AST in step (3) is as follows:

[0060] When remote operations are executed on the client site, the backtracking algorithm ensures that the operation's effect is executed within the correct address space. Its main idea is to determine which nodes in the current document are valid and which are invalid at the time the operation occurred based on the timestamps associated with the operation in each node of the document. Valid nodes are marked, and the operation's effect is then executed within the scope of those valid nodes.

[0061] A valid node is defined as follows: For a remote operation O, (1) the insertion operation of the node is causally preceded by operation O; (2) in the operation sequence of the node, there is only one insertion operation or all of its deletion operations are concurrent with O. Then the node is valid.

[0062] Therefore, we can identify all valid nodes for operation O based on the operation sequence of each node in the document, and then perform backtracking. The main steps of backtracking are as follows:

[0063] (1) Traverse all operation sequences of each node and determine whether the node is a valid node.

[0064] (2) Mark the valid nodes and denote them as valid nodes.

[0065] For collaborative annotation systems, due to the complexity of their operations, the data structure and its timestamp data structure have been modified, thus requiring adjustments to the backtracking algorithm. First, the entire document's address space is traversed. Because the collaborative editing operations are complex, they need to be categorized. This mainly involves traversing all operation sequences in each Marker to determine the validity of the Marker, Route, and Content character nodes. Although these are divided into three data structures, they are interrelated; all operation sequences are stored in their corresponding Markers or Routes. Therefore, they are traversed. Since operations on content must be performed on the premise that the Marker (MN) is valid, operations on Content and Route are equivalent to Add operations on Markers. The timestamp information attached to each Marker operation sequence is checked; if there is an Add or Update operation, it is valid; otherwise, it is invalid.

[0066]

[0067] The operation execution algorithm ensures that when backtracking to the state at which the operation occurred, it is executed within its correct address space. During operation execution, some operations can be directly executed at the corresponding node in the backtracked address space. However, some operations require determining their true insertion position due to the presence of invalid nodes. The address space translation algorithm defines a `TOrder` function to determine the total order position among all nodes, thus finding the true address space where the operation should be executed. `TOrder` is a transitive total order relation, meaning that any two nodes are comparable. AST uses a range-scan algorithm to determine the specific insertion position of an operation. The principle of range-scan lies in defining the `TOrder` function to determine the total order relation between any two operations (operations can be directly compared).

[0068] The TOrder is a total order relation and is transitive, meaning that any two object nodes are comparable. Traditional address space translation algorithms use vector timestamps for timestamps, and the TOrder is calculated based on these vector timestamps. Consider two elements CN. a and CN b His corresponding operations were generated at site a and site b respectively, and the timestamps were SV. a and SV b There is a TOrder (CN) a ) < TOrder(CN b If and only if:

[0069] (1)sum(SV a ) < sum(SV b )

[0070] (2)sum(SV a ) = sum(SV b And a < b

[0071] In this invention, scalar timestamps are used, therefore the Torder cannot be calculated using traditional summation methods. To ensure the range-scan algorithm can still execute correctly to complete the AST algorithm flow, the calculation method for Torder must be modified. The scalar timestamps used in this invention are also calculated as Total Order Relations (TOrders) that can be compared. As mentioned above, the MRN value in the scalar timestamps used in this invention is itself a value with a total order relation. It is a globally incrementing and unique sequence number assigned by the server. Therefore, the MRN value of the timestamp TS for each operation is an ordered sequence with a total order relation that can be compared pairwise. The MRN value can be directly converted into the TOrder for comparison to determine the total order relation of each object node. Therefore, the calculation method for TOrder is as follows:

[0072] TOrder(TS) = TS.MRN

[0073] For any two operations, the resulting object node MN a and MN b The timestamp of the operation is TS a and TS b If there is a Torder(MN) a ) <TOrder(MN b The total order relation of the entire replica can be calculated as follows:

[0074]

[0075] However, in the client, the MRN value in the timestamp TS of some operations is empty. In this case, the total order relation of the object nodes cannot be calculated and compared using the above method. The mechanism for determining the total order relation when the MRN is empty is as follows:

[0076] TOrder(TS a ) < TOrder(TS b When ), there are exactly three conditions that are satisfied:

[0077] (1)TS a ·MRN≠null&&TS a .MRN≠null&&TS a .MRN<TSb.MRN

[0078] (2)TS a .Openid=TS b .Openid&&TS a Opcnt < TS b .Opcnt

[0079] (3)TS a .MRN≠null&&TS b .MRN=null

[0080] By improving the calculation method of the Torder function and the method of judging the total order relationship of node objects, the range-scan algorithm can still be executed normally under the scalar timestamp of the collaborative annotation of accessibility information.

[0081] In this invention, the rule-based address space translation algorithm described in step (4)

[0082] Collaborative work frameworks in accessible information collaborative annotation scenarios, such as Figure 3 As shown, however, editing accessibility information doesn't allow for arbitrary operations. To ensure the system operates normally and efficiently, and to prevent user operations from compromising the integrity and rationality of other people's data, user actions must adhere to certain rules. These rules not only include restrictions arising from local copying and initialization, but also new restrictions created by remote synchronous operations. A key research issue is how to ensure that synchronous remote operations comply with these rules, or how to handle rule-violating operations that occur during synchronization. Rule constraints are a set of constraints on operations. Many operations contain a lot of information or modify certain information, possessing semantic meaning, and different accessibility information may have certain connections. Therefore, operations on accessibility information may require rule constraints to ensure semantic consistency among various types of accessibility information. Thus, a mechanism is needed to ensure that user operations are standardized and reasonable, and to correct errors if unreasonable operations violate rule constraints, allowing the system to continue correctly and efficiently. This situation is common in accessibility information; let's illustrate with an example. For example, a user sees an accessible toilet and adds its information to the map. However, feeling the location is not accurate, they move it to edit the location information. But this movement might be an error, moving the toilet too far. For the same facility, such a large-scale movement is clearly inappropriate. Therefore, distance rules are established to ensure that facilities added by users are either moved within a reasonable range to their true location, or that the facility information is completely incorrect and can be deleted and re-added. Many scenarios in accessibility information labeling require rule constraints to ensure the rationality of user actions and the accuracy of accessibility information. Therefore, some elements need to be added to the consistency maintenance algorithm (AST).

[0083] Since the various rules in the collaborative annotation of accessibility information are generated and established after the user's operation is executed, these rules are related to the operation. The main targets of the rules are (1) the restricted operation content, (2) the user who performed the operation, and (3) the target object of the operation. The establishment of each rule comes from the user's historical operation. Therefore, this invention associates the establishment of rules with the execution of operations to improve the rule set. Under this mechanism, the central server is still the end that forwards and synchronizes various operations. The rules established by the operation constrain various operations on the client. Therefore, the server does not need to add specific data structures. The client needs to add corresponding data structures to complete the rule constraints. As mentioned above, each operation of the user will be attached with a scalar timestamp. Therefore, some content can be added to the timestamp to assist in the establishment of rules. For each operation's attached timestamp, a new limit set (Limit Marker Set) LMS{Client, OpMes, MN} needs to be added. LMS is a triple added to the timestamp TS. Client represents the user to which this rule applies, OpMes represents the information of the operation content restricted by this rule, and MN represents the object node to which the operation restricted by the rule applies. User actions also need to be flagged, with a value of true indicating a valid action and a value of false indicating an invalid action, to distinguish whether the action can be executed. Simultaneously, a rule set data structure (Rule LimitSet), called RLS, needs to be added to the client, with the following structure: RLS{OpMes, MN}. RLS is a set that stores all the rules that the user must follow. The content of this set is a tuple. Since the rules definitely apply to this client user, the user's actions do not need to be recorded; it only needs to record the restriction operation content OpMes and the target operation object MN for each rule. After recording the operation rules followed by each object node on the client, it can be used to check whether the operation violates the rules. The algorithm for rule construction and verification is as follows.

[0084]

[0085] When an operation is executed, it only needs to be checked whether the target node MN and the operation content exist in all rule sets of the RLS. If a violation occurs, the flag for this operation is set to false, and the operation will not be executed. If the operation is under the rule constraint, then the operation is executed, and in this case, the operation does not violate any rules in the RLS. The process of establishing rules on the client involves traversing the operation sequence within the corresponding range and extracting the operation rules for this client user from the LMS within its timestamp. First, when a user requests to retrieve a partial copy of the content for the first time, historical operations are retrieved, all historical operations are traversed, and an initial RLS is established based on the LMS in the operation timestamp. Subsequent operations generated locally will be executed under the constraints of this rule set. Synchronous remote operations also need to be executed under rule constraints. Operations from a remote location will necessarily comply with the rules at the site where they originate, but when remotely synchronized to the local site, new local operations may create new restrictive rules that prevent execution, or the rules generated by this remote operation may constrain operations that have already been executed locally. Therefore, it is necessary to backtrack, transform the content of the rule set back to the state at the time the operation occurred, and then execute various operations, and then backtrack to the current timestamp. Another possibility is that this operation was originally restricted, but this restriction wasn't sent to its site. Therefore, the rule constraint set restricts this operation and sets it to false. Similar to AST, rule verification also requires two backtracking steps. The first backtracking executes the remote operation under the rule state at the time of its occurrence; if a rule is violated, it is invalidated. The second backtracking returns to the state when the remote operation arrived. Since the remote operation generates new rule constraints, this backtracking allows the new rules to filter out operations that are causally related to it and violate the new rules, invalidating them. Finally, the entire replica ensures consistency and runs operations under the correct rules. The control flow of the entire AST algorithm is shown below.

[0086]

[0087] The beneficial effects of this invention are as follows: This invention designs a replica synchronization protocol based on scalar timestamps, enabling synchronization operations to quickly determine causal or concurrent relationships between operations with the assistance of scalar timestamps; it designs a method for partially replicating accessibility information in shared map pairs, which can significantly reduce resource consumption such as memory usage and network resource requirements of mobile devices; this invention improves upon traditional address space translation methods, enabling them to maintain data consistency in accessibility information collaborative annotation scenarios; and it proposes an address space translation algorithm that integrates semantic rules, ensuring that client operations not only follow the rules established by various operations to guarantee semantic consistency between accessibility information but also ensure consistency between replicas. Attached Figure Description

[0088] Figure 1 This is a diagram illustrating the message communication process of the present invention.

[0089] Figure 2 This is an illustration of the accessibility labeling types of this invention. Wherein: (a) Marker, (b) Side, (c) Route.

[0090] Figure 3 This is a diagram illustrating the collaborative annotation framework of the present invention.

[0091] Figure 4 This is a diagram of the accessibility information collaborative annotation system of the present invention.

[0092] Figure 5 This is a diagram illustrating the core implementation of the AST module of this invention.

[0093] Figure 6 This is a diagram of Embodiment 1 of the present invention.

[0094] Figure 7 This is a diagram of Embodiment 2 of the present invention.

[0095] Figure 8 This is a diagram of Embodiment 3 of the present invention. Detailed Implementation

[0096] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0097] Example 1:

[0098] Figure 1In the message communication process of the barrier-free information collaborative annotation system, participating client sites append a timestamp to each operation before sending the operation sequence to the central server. On the central server, each received operation is assigned an auto-incrementing sequence number, called the "Server Mark Receive Operation Number" (MRN). All operation sequences received by the central server are stored in the maker sequence history buffers (SHBs) of the corresponding Marker object node MN, arranged chronologically by execution time (in ascending order of the MRNs), and are referred to as MSHBs. On the client side, locally generated operations are recorded in the local history sequence (LHB) of the corresponding Marker object node MN, while received remote operations are recorded in the remote history sequence (RHB). When a client synchronizes its local operation O and its associated timestamp TS to the central server, the central server assigns a globally incrementing sequence number to each operation with an empty MRN value in its timestamp information and returns a response to the collaborating client site that generated the operation. This response carries the assigned MRN value. Upon receiving this message, the MRN value in the timestamp information of operation O will no longer be empty at site S. Simultaneously, as the server sends the response to client site S, operation O is also broadcast to other participating client sites for synchronization. After operation O is transmitted to other sites and executed, it is stored in the local site's RHB (Resource Hub). The operation sequences stored in the RHB must be arranged in ascending order according to their MRN values.

[0099] Figure 2In this invention's accessibility annotation type, a Marker represents an accessibility information marker point in the shared map, a Route represents a directed path in the shared map, and a Side represents a region in the shared map, which can be a park, a building, a school, or other facilities. During collaborative annotation, various accessibility marker points are divided into different regions, which are defined and selected by the user. `sideId` is a unique identifier for each region; `locationName` is the name of this region, such as Liao Kaiyuan Law Building, Intersection Building No. 2, Peace Park, etc.; `markers` represents the collection of all marker points within this region (Route paths are also a type of marker point). A Side is essentially a collection of Marker objects, but unlike the two types mentioned above, a Side is defined and selected by the user and then automatically generated. During collaborative annotation, various accessibility information is divided into different regions, which may be a park, a building, a school, etc., defined and selected by the user. However, each time an accessibility information Marker is added, the accessibility information within that region is automatically generated. Each region is defined as a Side, and each Side contains all the Marker objects annotated within that region. A Side is essentially a collection of Marker objects. Each accessibility marker has its own associated area (Side). The partial replication architecture in collaborative accessibility labeling scenarios is based on these Sides, synchronizing Marker objects across all Sides within the shared map area for client users.

[0100] Figure 3 In the collaborative annotation framework of this invention, participating clients synchronize their locally executed operations O to the central server. The central server then uniformly schedules and processes all received operations. With the support of a consistency maintenance algorithm, remote operations sent to other collaborative clients can correctly execute their operational effects, thus avoiding inconsistencies in content.

[0101] Figure 4The architecture of the accessible information collaborative annotation system is divided into four layers: front-end interface, service layer, control layer, and working memory. The first layer is the front-end interface, which displays accessible information annotations on the shared map and defines various operations for users to annotate accessible information. Users can observe the annotation status and participate in collaborative annotation work through this interface. The second layer is the service layer, which encapsulates the implementation of various operations and other functions in accessible information annotation. When users perform operations on the front-end interface to achieve certain functions, they only need to call the various services encapsulated in the service layer. The third layer is the control layer, which contains the specific implementation steps of the system's various functional services. The core implementation module in the control layer is the AST module. The AST algorithm in this paper is mainly implemented in this module and then called to maintain the consistency of collaboratively annotated document content. The specific implementation of the encapsulated services in the service layer is also completed by calling the control layer. Specifically, the various capabilities related to consistency maintenance are accomplished by calling the AST module in the control layer. Because the AST algorithm implementation is integrated into a single module, this module is pluggable and can be replaced with other AST algorithm implementations without affecting system operation. Therefore, it's possible to compare the performance of different AST algorithms within the system. The final layer on the client side is the working memory space of each mobile device. Collaborative annotation documents copied locally by the client, along with local and remote historical operation sequences, are all stored in this working memory. The execution of various user-generated operations produces concrete effects in the local document. Finally, the collaborative annotation document in the working memory is rendered to the front-end interface, allowing users to continue viewing or editing.

[0102] Figure 5 This is the core implementation of the AST module. The AST module in the control layer is the core of the entire client-side consistency maintenance algorithm. The AST module consists of three parts: the operation history buffer, the AST core implementation (AST-core), and the message queue. Client user-defined operations are implemented through service layer calls. The service layer transforms user operations into operation models that can run in the AST module and stores them in the local operation history buffer. Operations in the operation history buffer are then sequentially executed in the AST-core using the core consistency maintenance algorithm, ensuring correct execution. The message queue is responsible for sending and receiving user-generated operation sequences, enabling message communication between the client and server.

[0103] Adopting a centralized architecture, with clients consisting of several mobile devices and servers deployed on Linux systems running on personal PC virtual machines, and using WeChat developer tools as the development platform and JavaScript programming, this invention presents a method for maintaining consistency in collaborative annotation of accessibility information. The resulting collaborative annotation system supports real-time collaborative annotation and updating of various types of accessibility information by multiple users.

[0104] like Figure 6 As shown, a user can initiate the addition of accessibility information in a shared map, while another user might simultaneously initiate the same action in the same area for collaborative annotation. This concurrent operation could lead to errors in the copied content, affecting normal user operation and collaboration. With the support of this invention, all concurrent user operations can ultimately execute successfully without affecting the consistency of the collaboratively annotated document content. Other similar concurrent operations can also execute successfully and ensure content consistency between copies. Multiple users can complete real-time interactive collaboration in a shared map without considering the impact of concurrent operations.

[0105] Example 2: Figure 7 As shown, when accessible entrances / exits in a building area have already been marked, another user marks the same accessible entrances / exits within that building area. This action would cause inconsistencies and confusion in the original accessible information. However, under the constraints of semantic rules, this user action is unsuccessful; the original accessible information is not affected by this action, preventing confusion or errors that could compromise semantic consistency between the markings. This is one constraint within the scope rules; various other rules exist to ensure semantic consistency between accessible information for other semantic relationships.

[0106] Example 3: As Figure 8 As shown, when two users simultaneously edit the same accessibility information, each user's real-time editing results are reflected in their respective collaborative annotation documents as they change the description of the information. In the description, each user's insertion, deletion, and modification operations are transmitted in real time, and they can perceive each other's editing content, enabling them to collaboratively edit the same document content in real time.

Claims

1. A method for maintaining consistency in collaborative annotation of accessibility information, characterized in that... The specific steps are as follows: (1) Design a replica synchronization protocol based on scalar timestamps to realize the message communication protocol for real-time collaborative annotation by multiple users. With the assistance of scalar timestamps, the synchronization operation can quickly determine the causal or concurrent relationship between operations. Each participating client site, upon generating an operation, first appends a timestamp to the operation and then sends the operation sequence to the central server. On the central server, each received operation is assigned an auto-incrementing sequence number. All operation sequences received by the central server are stored in the sequence history buffer of the corresponding Marker object node MN in chronological order of execution. On the client side, locally generated operations are recorded in the local operation history sequence LHB of the corresponding Marker object node MN, and received remote operations are recorded in the remote operation history sequence RHB. The data structure of the scalar timestamp TS appended to each operation is as follows: TS =<Client,Ocpnt,LastUpdateMRN,MRN,LMS> Where: Client is the client user initiating this operation; Opcnt is the number of all operation sequences within the local site's LHB when this operation occurs; LastUpdateMRN is the sequence number (MRN) of the most recent operation synchronized with the server by the client site that initiated this operation when this operation occurs, which can be understood as the largest MRN value among all operations within LHB and RHB when the operation occurs; the MRN value is an incrementing sequence number assigned by the server to each operation; LMS{Client,OpMes,MN}; LMS is a triple added to the timestamp TS, where Client represents the user subject to this rule, and OpMes represents the information of the operation content restricted by this rule; When a client synchronizes its local operation O and its attached timestamp TS to the central server, the central server assigns a globally incrementing sequence number to each operation whose timestamp information has an empty MRN value, and returns a response to the collaborating client site that generated the operation. The response carries the sequence number MRN value assigned to this operation. After site S receives this message, the MRN value in the timestamp information of operation O will no longer be empty. At the same time that the server sends the response to client site S, operation O is also broadcast to other participating client sites for synchronization. After operation O is transmitted to other sites and completed, it is stored in the local site's RHB. The operation sequences stored in the RHB must be arranged in ascending order according to the size of their MRN values. When operation O is synchronized to another collaborating site P, in the remote operation sequence RHB of the corresponding node, any operation O r The causal relationship with O is determined in the following way: ●O r → When O, one of the following two conditions must be met: (1)O r .MRN≤O.LastUpdateMRN (2)O r When .MRN>O.LastUpdateMRN, O r .Openid=O.Openid&&O r .Opcnt<O.Opcnt ●O r When ||O, it must satisfy the condition that there is no O. r →O, and there is no O→O r The method for determining the causal and concurrency relationship between any two operations using the timestamps attached to each operation is as follows: The causal relationship between any two timestamps TS1 and TS2 is determined as follows: ●When TS1→TS2, one of the following conditions must be met: (1)TS1.MRN≠null&&TS1.MRN <TS2.LastUpdateMRN (2) When TS1.Openid = TS2.Openid, TS1.Opcnt <TS2.Opcnt ●When TS1||TS2, the following condition must be met: neither TS1→TS2 nor TS2→TS1; (2) Design a partial copying method for synchronizing and operating accessibility information in a shared map pair, which completes the local copy synchronization and remote operation synchronization process while greatly reducing the consumption of various resources on the mobile terminal. In the accessible information collaborative annotation environment, user collaboration is divided into two modes: the first is the map editing mode, which includes: adding markers, deleting markers, updating the basic information of markers, connecting two markers with a directed line segment to form a directed path Route, and deleting the directed path Route between two markers. A marker represents an accessible information annotation point in the shared map, and a route represents a directed path in the shared map. The second type is the content editing mode, which allows for the insertion and deletion of character nodes, as well as the modification of certain specific content of the corresponding type of accessibility information. During collaborative annotation, various accessibility information is divided into different areas, which are defined and selected by the user. However, each time an accessibility information marker is added, the accessibility status within that area is automatically generated. Each area is defined as a "Side," and each Side contains all the marker objects annotated within that area. A Side is essentially a collection of marker objects. Each accessibility marker has its own associated Side. The partial replication architecture in the collaborative annotation scenario is based on these Sides, synchronizing the marker objects in all Sides within the shared map area of ​​the client user. Based on the client user's geographical location, various accessibility information in the Sides within a 5-kilometer radius of the shared map's center point is replicated and synchronized. These Sides belong to the client user's active range, and the markers within them are active operation objects. Synchronization is based on these active Sides. To ensure the proper functioning of a collaborative editing system under a partial replication architecture, the following data structures need to be introduced to optimize the collaborative editing algorithm. (1) Add data structure to the server side: (a)UserActiveSides: Records each client user and its corresponding active Side; (b)UserSidesLastUpdateMRN: Records the MRN value of the most recent operation synchronized with the server within each active Side of each client user; (c)SideLastRecieveMRN: Records the latest operation received from the client user and synchronized to the server within each active Side, which is the largest MRN value among all operations sent by the client user to synchronize to this active Side; (2) Add data structure to the client: (a)SideLastUpdateMRN: Records the MRN value of the last operation that was synchronized with the server within each active Side of the client user; (b)SideOperationList: Records the operation sequence within each Side to improve retrieval speed; After processing various annotation operations, the central server will retrieve client users who are collaborating with the client user who performed the operation to edit the same area. The central server will then conduct a series of comparative analyses to select the operations that need to be synchronized with each participating client, and then immediately push them to the selected clients in real time for operation synchronization. (3) Adjustments and improvements are made to the traditional address space conversion algorithm so that it can maintain the consistency of data content in the data model of the barrier-free information collaborative annotation scenario and realize real-time collaborative editing by multiple users; when remote operation is executed on the client site, the backtracking algorithm ensures that the operation effect is executed in the correct address space; based on the timestamp attached to the operation in each node in the document, it is determined which nodes in the current document are valid and which nodes are invalid when the operation occurs, the valid nodes are marked, and then the operation effect is executed within the scope of the valid nodes; The definition of a valid node is as follows: For remote operation O, (1) the insertion operation of the node is causally preceded by operation O; (2) there is only one insertion operation in the operation sequence of the node or all of its deletion operations are concurrent with O; then the node is valid. Identify all valid nodes for operation O based on the operation sequence of each node in the document, and then perform backtracking; the backtracking steps are as follows: (1) Traverse all operation sequences of each node and determine whether the node is a valid node. (2) Mark the valid nodes as valid nodes; For collaborative annotation systems, the first step is to traverse the address space of the entire document. This mainly involves traversing all operation sequences in each Marker to determine the validity of the Marker, the Route, and the character nodes of the Content. The timestamp information attached to each Marker operation sequence is checked. If there is an Add or Update operation, it is valid; otherwise, it is invalid. The operation execution algorithm ensures that when backtracking to the state at which the operation occurred, it is executed in its correct address space. During execution, some operations can be directly executed at the corresponding node in the backtracked address space. However, for others, due to the presence of invalid nodes, it is necessary to determine their true insertion position. The address space translation algorithm defines the TOrder function to determine the total order position among all nodes, thus finding the address space where the operation should be executed. TOrder is a total order relation and is transitive, meaning that any two nodes are comparable. AST uses the range-scan algorithm to determine the specific insertion position of the operation. The principle of range-scan lies in defining the TOrder function to determine the total order relation between each pair of operations. TOrder is calculated based on vector timestamps; consider two elements CN. a and CN b The corresponding operations are generated at site a and site b respectively, and the timestamps are SV. a and SV b ; There is a TOrder (CN) a ) <TOrder(CN b If and only if: (1) sum(SV a )<sum(SV b ) (2) sum(SV a ) = sum(SV b ) and a < b It uses scalar timestamps, directly converting MRN values ​​into TOrder for comparison to determine the total order relationship of each object node; therefore, TOrder is calculated as follows: TOrder(TS) = TS.MRN For any two operations, the resulting object node MN a and MN b The timestamp of the operation is TS a and TS b If there is a Torder(MN) a ) <TOrder(MN b The total order relation for the entire replica is calculated as follows: In the client, the MRN value in the timestamp TS of some operations is empty. In this case, the total order relation of object nodes cannot be calculated and compared using the above method. When the MRN is empty, the mechanism to determine the total order relation is as follows: TOrder(TS) a ) <TOrder(TS b When ), there are exactly three conditions that are satisfied: (1)TS a .MRN≠null&&TS b .MRN≠null&&TS a .MRN<TS b .MRN (2)TS a .Openid=TS b .Openid&&TS a .Opcnt<TS b .Opcnt (3)TS a .MRN≠null&&TS b .MRN=null By improving the calculation method of the Torder function and the method of judging the total order relationship of node objects, the range-scan algorithm can still be executed normally under the scalar timestamp of the collaborative annotation of accessibility information; (4) A rule-based address space conversion algorithm is proposed. While ensuring the consistency of collaborative annotation document content, various rules are set to ensure that the operation effect of client operations can ultimately ensure the semantic consistency between accessibility information. This ultimately realizes a collaborative annotation method that supports real-time collaborative annotation of accessibility information by multiple users.

2. The consistency maintenance method in collaborative annotation of accessibility information according to claim 1, characterized in that... The message communication protocol based on scalar timestamps described in step (1) is as follows: When an operation O is executed on the client site S that generated it, the following data is determined in the timestamp TS attached to the operation O: the client user Client that generated the operation O; the size of the local operation history sequence LHB at the time the operation was generated (Opcnt); Site S recently synchronized its operations with the server using LastUpdateMRN, which is the largest MRN value between LHB and RHB. Since the server-marked operation sequence number MRN for each operation is an auto-incrementing value assigned by the server, operation O was not synchronized with the server after it was executed locally, and its MRN value is temporarily empty. At this time, since all operations in LHB and RHB have been completed when O was generated, all of these operations are causally preceded by O. When operation O is synchronized to another collaborating site P, at this time, in the remote operation sequence RHB of the corresponding node, any one of operation O... r The causal relationship with O is determined in the following way: ●O r → When O, one of the following two conditions must be met: (1)O r .MRN≤O.LastUpdateMRN (2)O r When .MRN>O.LastUpdateMRN, O r .Openid=O.Openid&&O r .Opcnt<O.Opcnt ●O r When ||O, it must satisfy the condition that there is no O. r →O, and there is no O→O r At this point, in the local operation sequence LHB of p, any operation O l The causal relationship with O is determined in a manner similar to that in a remote operation sequence, but since the MRN value of an operation in LHB may be null, some improvements are needed: ●O l →O, the following conditions must be met: O l .MRN≠null&&O l .MRN≤O.LastUpdateMRN ●O l When ||O, one of the following conditions must be met: (1)O l .MRN=null (2)O l .When MRN ≠ null, O l .MRN > O.LastUpdateMRN Based on the above method for determining operational relationships, the method of using the timestamp attached to each operation to determine the causal and concurrent relationships between any two operations, and the way to determine the causal relationship between any two timestamps TS1 and TS2, is summarized as follows: ●When TS1→TS2, one of the following conditions must be met: (1)TS1.MRN≠null&&TS1.MRN <TS2.LastUpdateMRN (2) When TS1.Openid = TS2.Openid, TS1.Opcnt <TS2.Opcnt ●When TS1||TS2, the following condition must be met: neither TS1→TS2 nor TS2→TS1.

3. The consistency maintenance method in collaborative annotation of accessibility information according to claim 1, characterized in that... The method for partially copying accessibility information described in step (2) is as follows: The specific data structure for accessibility information labeling is as follows: Marker:<index,mId,lat,lng,type,side,contId> Where: Marker represents an accessibility information marker in the shared map; index is the serial number of this marker in the collaborative document; mId is the unique identifier of the marker; lat and lng are the latitude and longitude of the marker on the map, respectively; type is the type of this marker, and different types of markers will have different editable content; side represents the area to which the marker belongs; contId records the ID of the content corresponding to this marker, establishes a mapping relationship with the content, and serves the content editing function; Route:<index,rId,preMarker,nextMarker,side,contId> Where: Route represents a directed path in the shared map; index is the sequence number of this marker in the collaborative document; Rid is the unique identifier of the path; preMarker represents the starting marker of a path; nextMarker represents the ending marker of the path connection; the connection between the two forms a directed line segment, representing a directed path; type is the type of this marker; different types of markers will have different editable content; contId records the ID of the content corresponding to this marker, establishing a mapping relationship with the content and serving the content editing function; Side∶<sideId,locationName,markers> Wherein: Side represents a region in the shared map. During collaborative annotation, various accessibility markers are divided into different regions. This region is defined and selected by the user. sideId is a unique identifier for each region. locationName is the name of this region. markers represent the collection of all markers within this region. Route is also a type of marker. Side is essentially a collection of Marker objects. Side is defined and selected by the user and then automatically generated. To ensure the proper functioning of a collaborative editing system under a partial replication architecture, the following data structures need to be introduced to optimize the collaborative editing algorithm. (1) Add data structure to the server side: (a)UserActiveSides: An array set of SideId and its corresponding active users, recording all active Sides in the collaborative system and the client users corresponding to these Sides; by taking the intersection of the arrays corresponding to these SideIds, the collaborative scope that overlaps between the collaborative client users is filtered out, and then operations and data synchronization are performed with the corresponding clients; (b)UserSidesLastUpdateMRN: With this data, when the client and server synchronize again, the server can quickly filter out the operations in the active Side of each client user that have not yet been synchronized locally, and complete the operation and data synchronization between the client and the server more quickly. (c)SideLastRecieveMRN: By comparing this data with the previous data, it can be quickly determined whether the client user who is currently synchronizing with the server has any new operations on this side that need to be synchronized. (2) Add data structure to the client: (a)SideLastUpdateMRN: In a partial replication architecture, client users only synchronize the operation information of the currently active Side with the server. This data is recorded. When a client user moves or manually moves the center point of the shared map, the operation information of the new active Side is quickly obtained, and its historical operations are synchronized so that the system can continue to run normally. This value can be used to quickly determine the operations that each currently active Side of the client needs to synchronize with the server. (b)SideOperationList: The processing algorithm for the server after receiving client synchronization operation requests is as follows: The server receives all operations transmitted from client users. The server assigns a globally auto-incrementing and unique server-marked operation sequence number (MRN) to each operation, stores it in MSHB, and broadcasts the operation and its MRN to each client. Operations with empty MRNs in the client will no longer be empty. After processing various marked operations, the central server retrieves client users who are collaborating with the client user who generated the operation to edit the same area. The central server performs a series of comparative analyses to select the operations that need to be synchronized with each participating client, and then immediately pushes them to the selected clients for operation synchronization in real time.

Images