Method and system for assisting manual text annotation

Abstract

Provided is a method and system for assisting manual text annotation. The method comprises: displaying a text annotation page, wherein the text annotation page comprises a text area; detecting an annotation operation of a user in the text area, wherein the annotation operation comprises at least one of annotating a node and annotating a relationship; in response to detecting that the user is annotating a node in the text area, displaying annotation information of the node being annotated by the user in the text area, and displaying annotation information of the marked nodes and the marked relationships with a lower visual conspicuity; in response to detecting that the user is annotating a relationship in the text area, displaying annotation information of the relationship being annotated by the user in the text area, and displaying annotation information of the marked nodes and the marked relationships with a lower visual conspicuity.

Description

Technical Field

[0001] The present disclosure generally relates to text annotation technology, and more particularly, to a method and system for assisting manual text annotation. Background Art

[0002] The manual annotation process can be summarized as manually extracting entities from text and classifying or labeling each entity. For supervised machine learning, labeled data is required as prior experience. The quantity and quality of annotations determine the final effect of machine learning to a certain extent.

[0003] Annotations can be generally divided into two categories: annotated nodes and annotated relationships. Taking the knowledge graph as an example, text annotation is mainly concentrated in the cold start stage (the machine has not yet started learning). If the machine needs to learn what a "company" is, it is necessary to manually annotate the content that conforms to the semantics of "company" in multiple texts. Then the model uses this as input to gradually master the text features of "company" to achieve automatic detection and automatic recognition. This is called annotated nodes. Similarly, if the machine needs to learn a complete relationship such as "founder-establishment-company", it is necessary to first specify the type of relationship currently being annotated ("founder-establishment-company"), and mark the preceding node ("founder") and the succeeding node ("company") in turn. This is called annotated relationships.

[0004] In existing manual annotation methods, annotating nodes and annotating relationships are divided into two pages. That is, it is not possible to annotate both nodes and relationships in the same text area, which increases the user's cognitive cost and annotation workload. Summary of the invention

[0005] Exemplary embodiments of the present disclosure may solve at least the above problems, or may not solve the above problems.

[0006] According to one aspect of the present disclosure, a method for assisting manual text annotation is provided, comprising: displaying a text annotation page, wherein the text annotation page includes a text area; detecting a user's annotation operation in the text area, wherein the annotation operation includes at least one of annotating nodes and annotating relationships; in response to detecting that the user is annotating nodes in the text area, displaying annotation information of the nodes being annotated by the user in the text area, and displaying annotation information of the marked nodes and marked relationships with a lower visual conspicuity; in response to detecting that the user is annotating relationships in the text area, displaying annotation information of the relationships being annotated by the user in the text area, and displaying annotation information of the marked nodes and marked relationships with a lower visual conspicuity.

[0007] Optionally, the display state of the annotation information of the relationship may include a first relationship display state and a second relationship display state, and the visual conspicuity of the first relationship display state and the second relationship display state decreases in sequence; the display state of the annotation information of the node may include a first node display state and a second node display state, and the visual conspicuity of the first node display state and the second node display state decreases in sequence; the step of displaying the annotation information of the relationship being annotated by the user in the text area in response to detecting that the user is annotating a relationship in the text area, and displaying the annotation information of the marked nodes and the marked relationship with a lower visual conspicuity may include: displaying the annotation information of the relationship being annotated by the user in the first relationship display state; displaying the annotation information of the marked relationship in the second relationship display state; displaying the annotation information of the marked node in the second node display state; the step of displaying the annotation information of the node being annotated by the user in the text area in response to detecting that the user is annotating a node in the text area includes: displaying the annotation information of the node being annotated by the user in the first node display state.

[0008] Optionally, the second node display state may include a first sub-state and a second sub-state, and the visual prominence of the first sub-state and the second sub-state decreases successively; the step of displaying the annotation information of the marked nodes in the second node display state may include: displaying the annotation information of the first node in the marked nodes in the first sub-state; displaying the annotation information of the second node in the marked nodes in the second sub-state; wherein the first node is a node of a node type related to the relationship being annotated by the user, and the second node is a node that does not belong to a node type related to the relationship being annotated by the user.

[0009] Optionally, the display state of the annotation information of the node may include a first node display state and a second node display state, and the visual prominence of the first node display state and the second node display state decreases in sequence; the display state of the annotation information of the relationship may include at least one of the first relationship display state and the second relationship display state, and the visual prominence of the first relationship display state and the second relationship display state decreases in sequence; in response to detecting that the user is annotating a node in the text area, the step of displaying the annotation information of the node being annotated by the user in the text area, and displaying the annotation information of the marked node and the marked relationship with a lower visual prominence may include: displaying the annotation information of the node being annotated by the user in the first node display state; displaying the annotation information of the marked node in the second node display state; and displaying the annotation information of the marked relationship in the second relationship display state;

[0010] In response to detecting that the user is marking a relationship in the text area, displaying in the text area the marking information of the relationship being marked by the user includes:

[0011] The annotation information of the relationship being annotated by the user is displayed in a first relationship display state.

[0012] Optionally, in the text area, the annotation information of the nodes and the annotation information of the relationships may be displayed in different colors; in the text area, the annotation information of all nodes may be displayed in the same color, and the annotation information of all relationships may be displayed in the same color.

[0013] Optionally, the method may further include: in response to detecting that a marked node is also marked with a relationship, displaying the marking information of the node in the color of the marking information of the relationship.

[0014] Optionally, the method may further include: in response to detecting that a node corresponding to a marked relationship is marked with a new node type, displaying the marking information of the node in the color of the marking information of the node.

[0015] Optionally, in response to detecting that the user is marking a relationship in the text area, the step of displaying annotation information of the relationship being marked by the user in the text area may include: in response to detecting that the user selects the preceding node corresponding to the relationship being marked by the user and then selects the succeeding node across the screen, displaying a blurred form of the preceding node in a predetermined area of ​​the current screen during the process of the user selecting the succeeding node.

[0016] Optionally, the text annotation page may also include an annotation type selection area; wherein the step of detecting the user's annotation operation in the text area may include: in response to detecting that the user selects a node type in a node type list in the annotation type selection area, selecting a text area in the text area to determine that the user is annotating a node in the text area.

[0017] Optionally, the text annotation page may also include an annotation type selection area; wherein the step of detecting the user's operation in the text area may include: in response to detecting that the user selects a relationship type in a relationship type list in the annotation type selection area, selecting a text area in the text area to determine that the user is annotating a relationship in the text area.

[0018] Optionally, the text annotation page may also include a switch for all relationship connections; wherein the method may also include: detecting a user's operation on the relationship connection switch; in response to detecting that the user turns on the switch for all relationship connections, displaying the relationship connections of all marked relationships in the text area; in response to detecting that the user turns off the switch for all relationship connections, not displaying the relationship connections of all marked relationships in the text area.

[0019] Optionally, the annotation information of the node may include: at least one of a node selection box and a node type.

[0020] Optionally, the annotation information of the relationship may include: a predecessor node selection box corresponding to the relationship, a predecessor node type corresponding to the relationship, a successor node selection box corresponding to the relationship, a successor node type corresponding to the relationship, a relationship line connecting the predecessor node and the successor node, and at least one of the relationship types.

[0021] Optionally, in response to detecting that the user is marking a relationship in the text area, the step of displaying annotation information of the relationship being marked by the user in the text area may include: displaying in the text area the annotation information of the preceding node, the annotation information of the succeeding node, and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being marked by the user.

[0022] Optionally, the step of displaying in the text area the annotation information of the preceding node, the annotation information of the succeeding node, and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being annotated by the user may include: in response to detecting that the user selects the preceding node in the text area and releases it, detecting the movement trajectory of the input tool used by the user for annotation, and displaying in the text area the relationship line starting from the preceding node and moving along the detected movement trajectory until it is detected that the user selects the succeeding node in the text area.

[0023] Optionally, the step of displaying the relationship line in the text area may include: displaying the relationship line using a Bezier curve.

[0024] Optionally, in the first relationship display state, the annotation information of the relationship may include the relationship type and the relationship type is displayed near the relationship line, and in the second relationship display state, the annotation information of the relationship may not include the relationship type.

[0025] According to another aspect of the present disclosure, a system for assisting manual text annotation is provided, including: an annotation page display unit, configured to: display a text annotation page, wherein the text annotation page includes a text area; a user operation detection unit, configured to: detect a user's annotation operation in the text area, wherein the annotation operation includes at least one of annotating nodes and annotating relationships; an annotation information display unit, configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the node being annotated by the user in the text area, and display the annotation information of the marked nodes and the marked relationships with a lower visual conspicuity; in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display the annotation information of the relationship being annotated by the user in the text area, and display the annotation information of the marked nodes and the marked relationships with a lower visual conspicuity.

[0026] Optionally, the display state of the annotation information of the relationship may include a first relationship display state and a second relationship display state, and the visual prominence of the first relationship display state and the second relationship display state decreases in sequence; the display state of the annotation information of the node may include a first node display state and a second node display state, and the visual prominence of the first node display state and the second node display state decreases in sequence; the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display the annotation information of the relationship being annotated by the user in the first relationship display state; display the annotation information of the marked relationship in the second relationship display state; display the annotation information of the marked node in the second node display state; the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the node being annotated by the user in the first node display state.

[0027] Optionally, the second node display state may include a first sub-state and a second sub-state, and the visual prominence of the first sub-state and the second sub-state decreases successively; the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display the annotation information of the first node among the marked nodes in the first sub-state; and display the annotation information of the second node among the marked nodes in the second sub-state; wherein the first node is a node of a node type related to the relationship being annotated by the user, and the second node is a node that does not belong to a node type related to the relationship being annotated by the user.

[0028] Optionally, the display state of the annotation information of the node may include a first node display state and a second node display state, and the visual prominence of the first node display state and the second node display state decreases in sequence; the display state of the annotation information of the relationship may include at least one of the first relationship display state and the second relationship display state, and the visual prominence of the first relationship display state and the second relationship display state decreases in sequence. The annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the node being annotated by the user in the first node display state; display the annotation information of the marked node in the second node display state; display the annotation information of the marked relationship in the second relationship display state; the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the relationship being annotated by the user in the first relationship display state.

[0029] Optionally, the annotation information display unit can also be configured to: in the text area, display the annotation information of the node and the annotation information of the relationship in different colors; in the text area, display the annotation information of all nodes in the same color, and display the annotation information of all relationships in the same color.

[0030] Optionally, the annotation information display unit may be further configured to: in response to the user operation detection unit detecting that a marked node is also annotated with a relationship, display the annotation information of the node in the color of the annotation information of the relationship.

[0031] Optionally, the annotation information display unit may be further configured to: in response to the user operation detection unit detecting that the node corresponding to the marked relationship is labeled with a new node type, display the annotation information of the node in the color of the node's annotation information.

[0032] Optionally, the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user has selected the preceding node corresponding to the relationship that the user is annotating and then selects the succeeding node across the screen, display the preceding node in a blurred form in a predetermined area of ​​the current screen during the process of the user selecting the succeeding node.

[0033] Optionally, the text annotation page may also include an annotation type selection area; wherein the user operation detection unit may be configured to: in response to detecting that the user selects a node type in the node type list in the annotation type selection area, select a text area in the text area to determine that the user is annotating a node in the text area.

[0034] Optionally, the text annotation page may also include an annotation type selection area; wherein the user operation detection unit may be configured to: in response to detecting that the user selects a relationship type from the relationship type list in the annotation type selection area, select a text area in the text area, and determine that the user is annotating a relationship in the text area.

[0035] Optionally, the text annotation page may further include a switch for all relationship connections; wherein the user operation detection unit may be configured to: detect the user's operation on the relationship connection switch; the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user turns on the switch for all relationship connections, display the relationship connections of all marked relationships in the text area; in response to detecting that the user turns off the switch for all relationship connections, not display the relationship connections of all marked relationships in the text area.

[0036] Optionally, the annotation information of the node may include: at least one of a node selection box and a node type.

[0037] Optionally, the annotation information of the relationship may include: a predecessor node selection box corresponding to the relationship, a predecessor node type corresponding to the relationship, a successor node selection box corresponding to the relationship, a successor node type corresponding to the relationship, a relationship line connecting the predecessor node and the successor node, and at least one of the relationship types.

[0038] Optionally, the annotation information display unit may be configured to: in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display in the text area the annotation information of the preceding node, the annotation information of the succeeding node, and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being annotated by the user.

[0039] Optionally, the user operation detection unit may be configured to: in response to the user operation detection unit detecting that the user selects the preceding node in the text area and releases it, detect the movement trajectory of the input tool used by the user for annotation; the annotation information display unit may be configured to: display the relationship line in the text area starting from the preceding node and following the detected movement trajectory until it is detected that the user selects the succeeding node in the text area.

[0040] Optionally, the annotation information display unit may be configured to: use a Bezier curve to display the relationship line.

[0041] Optionally, in the first relationship display state, the annotation information of the relationship may include the relationship type and the relationship type is displayed near the relationship line, and in the second relationship display state, the annotation information of the relationship may not include the relationship type.

[0042] According to another aspect of the present disclosure, a computer-readable storage medium storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the method for assisting manual text annotation according to the present disclosure.

[0043] According to another aspect of the present disclosure, a system is provided, comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to perform the method of assisting manual text annotation according to the present disclosure.

[0044] According to the method and system for assisting manual text annotation disclosed in the present invention, allowing users to annotate both nodes and relationships in the same text area can reduce the user's cognitive cost and annotation workload, which has a great auxiliary effect on users.

[0045] According to the method and system for assisting manual text annotation disclosed in the present invention, it is set to display the marked nodes or marked relationships, marked nodes and marked relationships with different visual conspicuities in different annotation states, so that users can reduce the visual burden brought by annotating both nodes and relationships in the same text area, help users concentrate on their own tasks, and thus better assist users in performing text annotation.

[0046] According to the method and system for assisting manual text annotation disclosed in the present invention, when a user annotates a relationship, the annotation information of different marked nodes is displayed in different sub-states, which enables the user to more easily distinguish which nodes are related to a certain relationship type and which nodes are not related to the relationship type when annotating a certain relationship type, thereby helping the user to concentrate, annotate relationships more quickly, and improve the user experience.

[0047] According to the method and system for assisting manual text annotation disclosed in the present invention, when a user annotates a relationship, an auxiliary annotation method in which the relationship line follows the movement of the input tool can help the user focus more, be more beautiful, and improve the user experience.

[0048] According to the method and system for assisting manual text annotation disclosed in the present invention, Bezier curves are used to display relationship lines, so that the annotated page is more beautiful and more readable, the visual burden of the user is reduced, and the user experience is improved.

[0049] According to the method and system for assisting manual text annotation disclosed in the present invention, when a user annotates a relationship, setting a virtual form of the front node when the front and rear nodes cross the screen can help the user focus on the relationship content being annotated, which is conducive to better assisting the user in annotating the relationship and improving the user experience.

[0050] According to the method and system for assisting manual text annotation disclosed in the present invention, a switch for connecting all relationships is set, allowing users to freely choose whether to view all marked relationships at any time, which is convenient for user operation, helps better assist users in annotating relationships, and improves user experience.

[0051] According to the method and system for assisting manual text annotation disclosed in the present invention, the annotation information of nodes and the annotation information of relationships are displayed in different colors, so that users can quickly distinguish the marked nodes and the marked relationships on the same page, reducing the visual burden of users.

[0052] According to the method and system for assisting manual text annotation disclosed in the present invention, the color of the node type and the node annotation information are unified, and the color of the relationship type and the node information of the relationship are unified, so that the annotation page is more beautiful and easier to read, which can better help users concentrate and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] These and / or other aspects and advantages of the present disclosure will become clear and more easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0054] Figure 1 is a flowchart illustrating a method of assisting manual text annotation according to an exemplary embodiment of the present disclosure.

[0055] Figure 2a and Figure 2b is a schematic diagram illustrating a text annotation page according to an exemplary embodiment of the present disclosure.

[0056] Figure 3a and Figure 3b is a schematic diagram illustrating a relationship display state according to an exemplary embodiment of the present disclosure.

[0057] Figure 4a , Figure 4b and Figure 4c is a schematic diagram illustrating a node display state according to an exemplary embodiment of the present disclosure.

[0058] Figure 5 is a schematic diagram illustrating a labeling relationship according to an exemplary embodiment of the present disclosure.

[0059] Figure 6 is a schematic diagram showing the cross-screen of previous and next nodes of annotated relationships according to an exemplary embodiment of the present disclosure.

[0060] Figure 7a and Figure 7b is a schematic diagram showing a switch for turning on / off all relationship connections according to an exemplary embodiment of the present disclosure.

[0061] Figure 8 Detailed description is a block diagram illustrating a system for assisting manual text annotation according to an exemplary embodiment of the present disclosure.

[0062] Fig. 9 Detailed description is given of an implementation scenario of manual text annotation according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0063] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the embodiments of the present disclosure as defined by the claims and their equivalents. Various specific details are included to assist in understanding, but these details are considered to be exemplary only. Therefore, one of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures are omitted for clarity and brevity.

[0064] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0065] In existing manual annotation methods, annotating nodes and annotating relationships are divided into two pages for the sake of clarity. That is to say, users cannot annotate both nodes and relationships in the same text area. However, if users can annotate both nodes and relationships in the same text area, it can reduce users' cognitive costs and annotation workload, which greatly assists users. Therefore, the present disclosure proposes an auxiliary method that allows users to annotate both nodes and relationships in the same text area. In order to reduce the visual burden of annotating both nodes and relationships in the same text area and help users focus on their tasks, the auxiliary method is configured to display the marked nodes or the marked relationships, the marked nodes, and the marked relationships with different visual conspicuities in different annotation states, so as to better assist users in performing text annotation. Below, reference will be made to Figures 1 to 9 A method and system for assisting manual text annotation according to exemplary embodiments of the present disclosure are described in detail.

[0066] Figure 1 is a flowchart illustrating a method of assisting manual text annotation according to an exemplary embodiment of the present disclosure.

[0067] Reference Figure 1, in step 101, a text annotation page is displayed. The text annotation page includes a text area. In the text area, the text to be annotated can be displayed. The user can use an input tool to annotate. The input tool may include a mouse, a stylus, a user's body part, etc. The user can select an entity to be annotated in the text to be annotated. The annotation may include annotating an entity (e.g., Hexing Baihua Oil) as a node of a certain type (e.g., medicine), or may include annotating two entities (e.g., Hexing Baihua Oil, mosquito bites) as a relationship of a certain type (medicine-applicable to-disease). When the user annotates a node or a relationship for an entity in the text to be annotated, the relevant annotation information may be displayed in the text to be annotated in the text area. According to an exemplary embodiment of the present disclosure, the annotation information of the node may include at least one of a node selection box and a node type. According to an exemplary embodiment of the present disclosure, the annotation information of the relationship may include at least one of a preceding node selection box corresponding to the relationship, a preceding node type corresponding to the relationship, a following node selection box corresponding to the relationship, a following node type corresponding to the relationship, a relationship line connecting the preceding node and the following node, and a relationship type.

[0068] According to an exemplary embodiment of the present disclosure, the text annotation page may further include an annotation type selection area. For example, the annotation type selection area may include a node type list and a relationship type list. The user may add or select the node type to be annotated in the node type list, and the user may add or select the relationship type to be annotated in the relationship type list. For another example, the annotation type selection area may include a switch button and a node type list or a relationship type list. That is, the annotation type selection area may currently only display a node type list or a relationship type list, and when the user selects the switch button, it may switch from the node type list to the relationship type list or from the relationship type list to the node type list. Therefore, the user may select the desired node type list / relationship type list by switching the button, and add or select the node type / relationship type to be annotated in the desired node type list / relationship type list. Of course, the display mode of the annotation type selection area may also include any other possible modes, and the present disclosure does not limit this.

[0069] According to an exemplary embodiment of the present disclosure, the text annotation page may further include an annotation result area. The annotation result area may display the results that the user has marked. When it is detected that the user has completed the annotation (for example, the user releases the input tool), the user's annotation results are synchronously displayed in the annotation result area. For example, the node types that have been marked may be displayed in the annotation result area, and under each marked node type, the nodes belonging to the node type in the marked nodes may be displayed. For another example, the relationship types that have been marked may be displayed in the annotation result area, and under each marked relationship type, the relationships belonging to the relationship type in the marked relationships may be displayed. Of course, the display method of the annotation result area may also include any other possible methods, and the present disclosure does not limit this.

[0070] Figure 2a and Figure 2b is a schematic diagram illustrating a text annotation page according to an exemplary embodiment of the present disclosure.

[0071] like Figure 2a and Figure 2b As shown, in the left area 201, a label type selection area is displayed, and the label type selection area includes a node type list and a relationship type list. The user can add or select the node type that he wants to label in the node type list displayed in the area 201. The user can add or select the node type that he wants to label in the relationship type list displayed in the area 201.

[0072] In the middle area 202, a text area is displayed for displaying the text to be annotated.

[0073] like Figure 2a As shown, when the user selects a node type (e.g., disease) in the annotation type selection area, and selects a text area in the text to be annotated (i.e., an area where a certain entity is located, for example, an area where "travel sickness" is located), the annotation information of the node can be displayed near the text area. For example, a node selection box can be displayed around the text area (e.g., travel sickness), and the marked node type (e.g., disease) can be displayed in a smaller font on the right side of the text area in the node selection box. Of course, the display mode of the node annotation information can also be any other possible display mode, for example, the node selection box and / or the node type can be not displayed, etc., and the present disclosure does not limit this.

[0074] like Figure 2bAs shown, when the user selects a relationship type (e.g., medicine-applicable to-disease) in the annotation type selection area, and selects two text areas (e.g., Hexing Baihua Oil and mosquito bites) in the text to be annotated, the annotation information of the relationship can be displayed near the two text areas. For example, when the user selects a text area (e.g., Hexing Baihua Oil) as a preceding node in the text to be annotated, a node selection box can be displayed around the text area (e.g., Hexing Baihua Oil), and the marked node type (e.g., medicine) is displayed in a smaller font on the right side of the text area in the node selection box. Subsequently, when the user selects a text area (e.g., mosquito bites) as a following node in the text to be annotated, a node selection box can be displayed around the text area (e.g., mosquito bites), and the marked node type (e.g., disease) is displayed in a smaller font on the right side of the text area in the node selection box, and a relationship line connecting the preceding node and the following node is displayed, and the relationship type (also called relationship name, e.g., applicable to) can be displayed near the relationship line. Of course, the display method of the relationship annotation information can also be any other possible display method, for example, the selection boxes of the preceding node and the succeeding node may not be displayed, and / or the node type may not be displayed, and / or the connection line or the vicinity of the relationship connection line may not be displayed, and / or the relationship type or relationship name may not be displayed, etc. The present disclosure does not impose any restrictions on this.

[0075] like Figure 2a and Figure 2b As shown, in the right area 203, the annotation result area is displayed. In the annotation result area, the marked node types are displayed, and under each marked node type, the nodes belonging to the marked node type are displayed; the marked relationship types are displayed, and under each marked relationship type, the relationships belonging to the marked relationship type are displayed.

[0076] Return to reference Figure 1 In step 102, a user's annotation operation in the text area is detected, wherein the annotation operation includes at least one of annotating a node and annotating a relationship.

[0077] According to an exemplary embodiment of the present disclosure, in response to detecting that a user selects a node type in a node type list in a marking type selection area and then selects a text area in a text area, it is determined that the user is marking a node in the text area. That is, when it is detected that the most recent selection in the marking type selection area before the user selects a text area in the text area is a selection of a certain node type, it can be determined that the user is marking a node in the text area.

[0078] According to an exemplary embodiment of the present disclosure, in response to detecting that a user selects a relationship type in a relationship type list in a marking type selection area and then selects a text area in a text area, it is determined that the user is marking a relationship in the text area. That is, when it is detected that the most recent selection in the marking type selection area before the user selects a text area in the text area is a selection of a certain relationship type, it can be determined that the user is marking a relationship in the text area. For example, when it is detected that the user clicks to select a relationship type in the marking type selection area and then sequentially swipes across a first text area and a second text area in the text area, it can be determined that the user has completed marking the relationship. For another example, when it is detected that the user clicks to select a relationship type in the marking type selection area and then sequentially clicks on two marked nodes in the text area, it can be determined that the user has completed marking the relationship.

[0079] In step 103, in response to detecting that the user is marking a node in the text area, the marking information of the node being marked by the user is displayed in the text area, and the marking information of the marked nodes and marked relationships is displayed with a lower visual prominence.

[0080] In step 104 , in response to detecting that the user is marking a relationship in the text area, the marking information of the relationship being marked by the user is displayed in the text area, and the marking information of the marked nodes and the marked relationships are displayed with a lower visual prominence.

[0081] According to an exemplary embodiment of the present disclosure, in order to reduce the visual burden on users and focus on the current annotation task when annotating and displaying both the annotation information of nodes and the annotation information of relationships in the same text area, the display state of the annotation information of nodes can be set to two states with different visual conspicuities, for example, a highlighted state and a non-highlighted state, and the display state of the annotation information of relationships can be set to two states with different visual conspicuities, for example, a highlighted state and a non-highlighted state. The visual conspicuity of the highlighted state is higher than that of the non-highlighted state. Below, it will be specifically described that when a user is annotating nodes / relationships, the annotation information of the nodes / relationships being annotated, the annotation information of the marked nodes, and the annotation information of the annotated relationships are displayed with different visual conspicuities.

[0082] According to an exemplary embodiment of the present disclosure, the display state of the annotation information of the relationship includes a first relationship display state (e.g., a highlighted state) and a second relationship display state (e.g., a non-highlighted state), and the visual conspicuity of the first relationship display state and the second relationship display state decreases in sequence. The display state of the annotation information of the node includes a first node display state (e.g., a highlighted state) and a second node display state (e.g., a non-highlighted state), and the visual conspicuity of the first node display state and the second node display state decreases in sequence. According to an exemplary embodiment of the present disclosure, the second node display state may also include two sub-states, namely, a first sub-state (e.g., a normal state) and a second sub-state (e.g., an off state), and the visual conspicuity of the first sub-state and the second sub-state decreases in sequence.

[0083] Figure 3a and Figure 3b is a schematic diagram illustrating a relationship display state according to an exemplary embodiment of the present disclosure.

[0084] like Figure 3a As shown, the annotation information of the relationship in the first relationship display state is shown, wherein the border color and fill color of the selection box of the preceding node and the succeeding node corresponding to the relationship are brighter, and the color of the relationship line and the relationship name are brighter (for example, the border color, fill color, relationship line and relationship name color are all green). Figure 3b As shown, the annotation information of the relationship in the second relationship display state is shown, wherein the border color and fill color of the selection boxes of the predecessor node and the successor node corresponding to the relationship are darker, and the color of the relationship line and the relationship name are darker (for example, the border color, fill color, the color of the relationship line and the relationship name are all gray) or the relationship name may not be displayed.

[0085] For example, when it is detected that the user is marking a relationship or the user has just finished marking a relationship and has not yet performed other operations, the marking information of the relationship being marked may be displayed in the first relationship display state.

[0086] For another example, when it is detected that a user selects a relationship line of a marked relationship (for example, it is detected that the user clicks on the relationship line using an input tool or it is detected that the user hovers over the relationship line using an input tool), the annotation information of the preceding node and the succeeding node of the marked relationship is changed to be displayed in the first relationship display state.

[0087] For another example, when it is detected that a user selects a node corresponding to a marked relationship (for example, it is detected that a user uses an input tool to click on a node corresponding to a marked relationship or it is detected that a user uses an input tool to hover over a node corresponding to a marked relationship), the annotation information of the marked relationship is changed to be displayed in a first relationship display state.

[0088] For another example, when it is detected that the user selects an area other than the area related to the relationship displayed in the first relationship display state (for example, it is detected that the user clicks on a blank space in the text area, marks a new node or a new relationship, clicks on other areas such as the annotation type selection area, the annotation result area, etc.), the relationship displayed in the first relationship display state is changed to be displayed in the second relationship display state.

[0089] For example, Figure 3a As shown, the annotation information of the relationship can also display a delete icon (e.g., a cross circle symbol in the upper left corner of the selection box) in the first relationship display state. For example, the delete icon can be displayed on at least one of the selection box of the preceding node, the selection box of the succeeding node, and the relationship line. When it is detected that the user selects the delete icon, the annotation information of the corresponding relationship can be deleted. The style and position of the delete icon are not limited to this, and can be displayed in any possible style and position, and the present disclosure does not limit this.

[0090] Figure 4a , Figure 4b and Figure 4c is a schematic diagram illustrating a node display state according to an exemplary embodiment of the present disclosure.

[0091] like Figure 4a As shown, the annotation information of the node in the first node display state is shown, wherein the border color and fill color of the node selection box are brighter (for example, the border color and fill color are both blue). Figure 4b As shown, the annotation information of the node in the first sub-state in the second node display state is shown, wherein the border color of the node selection box is brighter and the filling color is darker (for example, the border color is a blue with a higher saturation and the filling color is a blue with a lower saturation). Figure 4c As shown, the annotation information of the node in the second sub-state in the second node display state is shown, wherein the border color and the filling color of the selection box of the node are darker (for example, the border color and the filling color are both gray).

[0092] For example, when it is detected that the user is marking a node or the user has just marked a node and has not yet performed other operations, the marking information of the node being marked may be displayed in the first node display state.

[0093] For another example, when it is detected that a user selects a marked node (for example, it is detected that the user uses an input tool to click on the marked node or it is detected that the user uses the input tool to hover over the marked node, or it is detected that the user uses the input tool to click on the selection box border of the marked node or it is detected that the user uses the input tool to hover over the selection box border of the marked node), the annotation information of the marked node is changed to be displayed in the first relationship display state.

[0094] For another example, when it is detected that the user selects an area other than an area related to the node displayed in the first node display state (for example, it is detected that the user clicks a blank space in the text area, marks a new node or a new relationship, clicks other areas such as the marking type selection area, the marking result area, etc.), the node displayed in the first node display state is changed to be displayed in the second node display state. For example, in the case where the second node display state includes a first sub-state and a second sub-state, the node displayed in the first node display state is changed to be displayed in the first sub-state or the second sub-state of the second node display state.

[0095] For example, Figure 4a As shown, the annotation information of the node can also display a delete icon (e.g., a cross circle symbol in the upper left corner of the selection box) in the first node display state. When it is detected that the user selects the delete icon, the annotation information of the corresponding node can be deleted. The style and position of the delete icon are not limited to this, and can be displayed in any possible style and position, and the present disclosure does not limit this.

[0096] Of course, the display state of the annotation information of the relationship and the node is not limited to the display state described above, and any other display state with different visual prominence can be used, and the present disclosure does not limit this.

[0097] According to an exemplary embodiment of the present disclosure, when it is detected that a user is marking a relationship in a text area, the marking information of the relationship being marked by the user can be displayed in a first relationship display state; the marking information of the marked relationship can be displayed in a second relationship display state; and the marking information of the marked node can be displayed in a second node display state. Figure 2b As shown, when the user selects a relationship type (e.g., drug-applicable to-disease) in the annotation type selection area, and selects a preceding node (e.g., Hexing Baihua Oil) and a succeeding node (e.g., mosquito bites) in the text area, the annotation information of the relationship that the user is annotating can be displayed in a first relationship display state (e.g., highlighted state) (e.g., "Hexing Baihua Oil" selection box and node type (drug), "Mosquito Bites" selection box and node type (disease), relationship connection and relationship name (applicable to)); the annotation information of the marked relationship can be displayed in a second relationship display state (e.g., non-highlighted state); the annotation information of the marked nodes can be displayed in a second node display state (e.g., non-highlighted state) (e.g., "Baihua Oil" selection box and node type (drug), "Wintergreen Oil" selection box and node type (ingredient), "Dizziness and Headache" selection box and node type (disease), "Rash" selection box and node type (symptom), etc.).

[0098] According to an exemplary embodiment of the present disclosure, when it is detected that a user is marking a relationship in a text area, the marking information of the first node in the marked nodes can be displayed in a first sub-state; the marking information of the second node in the marked nodes can be displayed in a second sub-state; wherein the first node is a node of a node type related to the relationship being marked by the user, and the second node is a node that does not belong to a node type related to the relationship being marked by the user. For example, Figure 2b As shown, the node type (drug) of "white flower oil" and the node type (disease) of "dizziness and headache" in the marked nodes belong to the node type involved in the selected relationship type (for example, drug-applicable to-disease), so the annotation information of "white flower oil" and "dizziness and headache" is displayed in the first sub-state (for example, normal state). The node type (ingredient) of "wintergreen oil" and the node type (symptom) of "rash" in the marked nodes do not belong to the node type involved in the selected relationship type (for example, drug-applicable to-disease), so the annotation information of "wintergreen oil" and "rash" is displayed in the second sub-state (for example, extinguished state). Displaying the annotation information of nodes in different sub-states can make it easier for users to distinguish which nodes are related to a certain relationship type and which nodes are not related to the relationship type when annotating a certain relationship type, helping users to concentrate their attention, annotate relationships more quickly, and improve user experience.

[0099] According to an exemplary embodiment of the present disclosure, when it is detected that a user is marking a node in a text area, the marking information of the node being marked by the user can be displayed in a first node display state; the marking information of the marked node can be displayed in a second node display state; and the marking information of the marked relationship can be displayed in a second relationship display state. Figure 2a As shown, when the user selects a node type (disease) in the annotation type selection area and then selects a node (e.g., motion sickness) in the text area, the annotation information of the node that the user is annotating is displayed in the first node display state (e.g., the selection box and node type (disease) of "motion sickness"); the annotation information of the marked nodes is displayed in the second node display state (e.g., the selection box and node type (drug) of "Hexing Baihua Oil", the selection box and node type (ingredient) of "Wintergreen Oil", etc.); and the annotation information of the marked relationship is displayed in the second relationship display state.

[0100] According to an exemplary embodiment of the present disclosure, when it is detected that a user is marking a relationship in a text area, the marking information of the preceding node, the marking information of the succeeding node, and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being marked by the user may be displayed in the text area, or the marking information of the preceding node, the marking information of the succeeding node, the relationship line connecting the preceding node and the succeeding node, and the relationship name may be displayed in the text area.

[0101] According to an exemplary embodiment of the present disclosure, when a user is annotating a relationship in a text area, in response to detecting that the user selects a preceding node in the text area and releases the preceding node, the moving track of the input tool used by the user for annotation is detected, and the relationship line is displayed in the text area starting from the preceding node and moving along the detected moving track until the user is detected to select a succeeding node in the text area. When it is detected that the user has selected the succeeding node (for example, it is detected that the input tool is released after selecting the succeeding node), the relationship line is fixed. For example, Figure 5 Schematic diagram showing the annotation relationship according to an exemplary embodiment of the present disclosure. Figure 5 As shown in the figure, when the user selects the front node (for example, Hexing Baihua Oil), the relationship line can be moved from the front node to follow the mouse cursor used by the user for annotation, and the relationship name (for example, applicable to) can also be displayed near the relationship line, and the relationship name can also move with the relationship line. The auxiliary annotation method of using the relationship line to move with the input tool can make the user focus more, be more beautiful, and improve the user experience.

[0102] According to an exemplary embodiment of the present disclosure, a Bezier curve may be used to display a relationship line. Figure 2b , Figure 3a , Figure 3b , Figure 5 As shown in the figure, the relationship lines are displayed with Bezier curves. However, the traditional auxiliary annotation method uses horizontal lines to display relationship lines, which is difficult to distinguish when there is a cross-line situation, especially when the distance between the front node and the back node is too far (across the screen), which makes the distinction more difficult. Compared with the traditional auxiliary annotation method of using horizontal lines to display relationship lines, the use of Bezier curves can make the annotation page more beautiful and more readable, reduce the visual burden of users, and improve user experience.

[0103] According to an exemplary embodiment of the present disclosure, in the first relationship display state, the annotation information of the relationship includes the relationship type and the relationship type is displayed near the relationship line (such as Figure 3a In the second relationship display state, the relationship annotation information does not include the relationship type (such as Figure 3b shown).

[0104] According to an exemplary embodiment of the present disclosure, when a user is marking a relationship in a text area, in response to detecting that the user has selected a preceding node corresponding to the relationship being marked and then selected a succeeding node across the screen, a virtualized form of the preceding node is displayed in a predetermined area of ​​the current screen during the user's selection of the succeeding node. Specifically, the display position of the virtualized form of the preceding node may be related to the real-time position of the relationship line, for example, the vertical display position is the intersection position of the upper edge of the text area and the relationship line, and the horizontal position is the left / right side of the line. For example, Figure 6It is a schematic diagram showing the cross-screen of the front and rear nodes of the annotation relationship according to an exemplary embodiment of the present disclosure. When it is detected that the user has finished selecting the front node, when the screen is crossed in the process of selecting the rear node, the blurred form of the front node (for example, "xxxxx company" displayed in a semi-transparent form) is displayed in a predetermined area of ​​the current screen (for example, the blank space above). In addition, when it is detected that the user has finished selecting the rear node (for example, the user uses an input tool to select the rear node and releases it), the blurred form of the front node is not displayed. Setting the blurred form of the front node when the front and rear nodes cross the screen can help users focus on the relationship content being annotated, which is conducive to better assisting users in annotating relationships and improving user experience.

[0105] According to an exemplary embodiment of the present disclosure, the text annotation page may further include a switch for all relationship connections. The user's operation on the relationship connection switch may be detected; in response to detecting that the user turns on the switch for all relationship connections, the relationship connections of all marked relationships are displayed in the text area; in response to detecting that the user turns off the switch for all relationship connections, the relationship connections of all marked relationships are not displayed in the text area. It should be noted that after turning off the switch for all connection connections, the relationship connections currently being marked may still be displayed, and the relationship connections in a highlighted state may not be affected by the switch for all relationship connections. After turning off the switch for all connection connections, the relationship connections in a highlighted state may still remain highlighted. For example, Figure 7a and Figure 7b is a schematic diagram showing a switch for turning on / off all relationship connections according to an exemplary embodiment of the present disclosure. Figure 7a and Figure 7b As shown, the switch for all relationship connections can be displayed in the upper area of ​​the text annotation page. The display position and display form of the switch for all relationship connections are not limited thereto, and the present disclosure does not impose any restrictions on this. Figure 7a As shown, in response to detecting that the user turns on the switch for all relationship connections, the relationship connections of all marked relationships are displayed in the text area. In this case, the annotation information of the relationship being annotated by the user can be displayed in the first relationship display state, and the relationship connections of all marked relationships can be displayed in the second relationship display state, and the relationship names are not displayed near the relationship connections. Figure 7b As shown, in response to detecting that the user turns off the switch for all relationship connections, the relationship connections of all marked relationships are not displayed in the text area. In this case, the annotation information of the relationship being annotated by the user can be displayed in the first relationship display state, but for the marked relationship, the annotation information of the nodes corresponding to all marked relationships is displayed in the second relationship display state, but the relationship connections of the marked relationship are not displayed. Setting the switch for all relationship connections allows the user to freely choose whether to view all marked relationships at any time, which is convenient for user operation, and is conducive to better assisting users in annotating relationships and improving user experience.

[0106] According to an exemplary embodiment of the present disclosure, in order to enable users to quickly distinguish between marked nodes and marked relationships on the same page and reduce the visual burden on users, the annotation information of nodes and the annotation information of relationships can be displayed in different colors in the text area, and the annotation information of all nodes can be displayed in the same color, and the annotation information of all relationships can be displayed in the same color. For example, the annotation information of all nodes can be blue, and the annotation information of all relationships can be green. In comparison, traditional auxiliary annotation methods often use different colors (or different colors + descriptive text) to distinguish different node types and relationship types, and color duplication will occur, especially when the number is large, the display is more messy. The color display method according to the exemplary embodiment of the present disclosure solves the problems of traditional auxiliary annotation methods well.

[0107] According to an exemplary embodiment of the present disclosure, the color of the node type in the node type list in the annotation type selection area can be displayed as the same as the color of the annotation information of the node in the text area, and the color of the relationship type in the relationship type list in the annotation type selection area can be displayed as the same as the color of the annotation information of the relationship in the text area. For example, the colors of the node type and the annotation information of the node in the node type list are unified into blue, and the colors of the relationship type and the node information of the relationship in the relationship type list are unified into green. Unifying the colors of the node type and the annotation information of the node, and unifying the colors of the relationship type and the node information of the relationship, makes the annotation page more beautiful and more readable, helps users concentrate, and improves the user experience.

[0108] According to an exemplary embodiment of the present disclosure, in response to detecting that a marked node is also marked with a relationship, the marking information of the node is displayed in the color of the marking information of the relationship. For example, assuming that the color of the marking information of the node is blue and the color of the marking information of the relationship is green, when it is detected that a marked node is also marked with a relationship, the color of the marking information of the marked node is changed from blue to green.

[0109] According to an exemplary embodiment of the present disclosure, in response to detecting that a node corresponding to a marked relationship is marked with a new node type, the annotation information of the node is displayed in the color of the node's annotation information. For example, assuming that the color of the node's annotation information is blue and the color of the relationship's annotation information is green, when it is detected that a node corresponding to the marked relationship is marked with a new node type, the color of the node's annotation information is changed from green to blue.

[0110] Figure 8 Detailed description is a block diagram illustrating a system for assisting manual text annotation according to an exemplary embodiment of the present disclosure.

[0111] Reference Figure 8According to an exemplary embodiment of the present disclosure, a system 800 for assisting manual text annotation may include: an annotation page display unit 801 , a user operation detection unit 802 , and an annotation information display unit 803 .

[0112] The annotation page display unit 801 can display a text annotation page. Among them, the text annotation page includes a text area. In the text area, the text to be annotated can be displayed. The user can use an input tool to annotate. The input tool may include a mouse, a stylus, a user's body part, etc. The user can select the entity to be annotated in the text to be annotated. The annotation may include annotating an entity (e.g., Hexing Baihua Oil) as a node of a certain type (e.g., medicine), or may include annotating two entities (e.g., Hexing Baihua Oil, mosquito bites) as a relationship of a certain type (medicine-applicable to-disease). When the user annotates a node or a relationship for an entity in the text to be annotated, the relevant annotation information may be displayed in the text to be annotated in the text area. According to an exemplary embodiment of the present disclosure, the annotation information of the node may include at least one of a node selection box and a node type. According to an exemplary embodiment of the present disclosure, the annotation information of the relationship may include at least one of a preceding node selection box corresponding to the relationship, a preceding node type corresponding to the relationship, a following node selection box corresponding to the relationship, a following node type corresponding to the relationship, a relationship line connecting the preceding node and the following node, and a relationship type.

[0113] According to an exemplary embodiment of the present disclosure, the text annotation page may further include an annotation type selection area. For example, the annotation type selection area may include a node type list and a relationship type list. The user may select the node type to be annotated in the node type list, and the user may select the relationship type to be annotated in the relationship type list. For another example, the annotation type selection area may include a switch button and a node type list or a relationship type list. That is, the annotation type selection area may currently only display a node type list or a relationship type list, and when the user selects the switch button, it may switch from the node type list to the relationship type list or from the relationship type list to the node type list. Therefore, the user may select the desired node type list / relationship type list by switching the button, and select the node type / relationship type to be annotated in the desired node type list / relationship type list. Of course, the display mode of the annotation type selection area may also include any other possible modes, and the present disclosure does not limit this.

[0114] According to an exemplary embodiment of the present disclosure, the text annotation page may further include an annotation result area. The annotation result area may display the results that the user has marked. When it is detected that the user has completed the annotation (for example, the user releases the input tool), the user's annotation results are synchronously displayed in the annotation result area. For example, the node types that have been marked may be displayed in the annotation result area, and under each marked node type, the nodes belonging to the node type in the marked nodes may be displayed. For another example, the relationship types that have been marked may be displayed in the annotation result area, and under each marked relationship type, the relationships belonging to the relationship type in the marked relationships may be displayed. Of course, the display method of the annotation result area may also include any other possible methods, and the present disclosure does not limit this.

[0115] The user operation detection unit 802 can detect the user's annotation operation in the text area, wherein the annotation operation includes at least one of annotating a node and annotating a relationship.

[0116] According to an exemplary embodiment of the present disclosure, the user operation detection unit 802 determines that the user is marking a node in the text area in response to detecting that the user selects a node type in the node type list in the annotation type selection area and then selects a text area in the text area. That is, when the user operation detection unit 802 detects that the most recent selection in the annotation type selection area before the user selects the text area in the text area is to select a certain node type, the user operation detection unit 802 can determine that the user is marking a node in the text area.

[0117] According to an exemplary embodiment of the present disclosure, the user operation detection unit 802 determines that the user is marking a relationship in the text area in response to detecting that the user selects a relationship type in the relationship type list in the annotation type selection area and then selects a text area in the text area. That is, when the user operation detection unit 802 detects that the most recent selection in the annotation type selection area before the user selects a text area in the text area is to select a certain relationship type, the user operation detection unit 802 can determine that the user is marking a relationship in the text area. For example, when the user operation detection unit 802 detects that the user clicks to select a relationship type in the annotation type selection area and then sequentially crosses the first text area and the second text area in the text area, the user operation detection unit 802 can determine that the user has completed marking the relationship. For another example, when the user operation detection unit 802 detects that the user clicks to select a relationship type in the annotation type selection area and then sequentially clicks on two marked nodes in the text area, the user operation detection unit 802 can determine that the user has completed marking the relationship.

[0118] In response to the user operation detection unit 802 detecting that the user is marking a node in the text area, the marking information display unit 803 can display the marking information of the node being marked by the user in the text area, and display the marking information of the marked nodes and marked relationships with a lower visual conspicuity.

[0119] In response to the user operation detection unit 802 detecting that the user is marking a relationship in the text area, the annotation information display unit 803 can display the annotation information of the relationship being marked by the user in the text area, and display the annotation information of the marked nodes and marked relationships with a lower visual conspicuity.

[0120] According to an exemplary embodiment of the present disclosure, in order to reduce the visual burden on users and focus on the current annotation task when annotating and displaying both the annotation information of nodes and the annotation information of relationships in the same text area, the display state of the annotation information of nodes can be set to two states with different visual conspicuities, for example, a highlighted state and a non-highlighted state, and the display state of the annotation information of relationships can be set to two states with different visual conspicuities, for example, a highlighted state and a non-highlighted state. The visual conspicuity of the highlighted state is higher than that of the non-highlighted state. Below, it will be specifically described that when a user is annotating nodes / relationships, the annotation information of the nodes / relationships being annotated, the annotation information of the marked nodes, and the annotation information of the annotated relationships are displayed with different visual conspicuities.

[0121] According to an exemplary embodiment of the present disclosure, the display state of the annotation information of the relationship includes a first relationship display state (e.g., a highlighted state) and a second relationship display state (e.g., a non-highlighted state), and the visual conspicuity of the first relationship display state and the second relationship display state decreases in sequence. The display state of the annotation information of the node includes a first node display state (e.g., a highlighted state) and a second node display state (e.g., a non-highlighted state), and the visual conspicuity of the first node display state and the second node display state decreases in sequence. According to an exemplary embodiment of the present disclosure, the second node display state may also include two sub-states, namely, a first sub-state (e.g., a normal state) and a second sub-state (e.g., an off state), and the visual conspicuity of the first sub-state and the second sub-state decreases in sequence.

[0122] like Figure 3a As shown, the annotation information of the relationship in the first relationship display state is shown, wherein the border color and fill color of the selection box of the preceding node and the succeeding node corresponding to the relationship are brighter, and the color of the relationship line and the relationship name are brighter (for example, the border color, fill color, relationship line and relationship name color are all green). Figure 3bAs shown, the annotation information of the relationship in the second relationship display state is shown, wherein the border color and fill color of the selection boxes of the predecessor node and the successor node corresponding to the relationship are darker, and the color of the relationship line and the relationship name are darker (for example, the border color, fill color, the color of the relationship line and the relationship name are all gray) or the relationship name may not be displayed.

[0123] For example, when the user operation detection unit 802 detects that the user is marking a relationship or the user has just marked a relationship and has not performed other operations, the marking information display unit 803 may display the marking information of the relationship being marked in the first relationship display state.

[0124] For another example, when the user operation detection unit 802 detects that the user selects a relationship line of a marked relationship (for example, detects that the user uses an input tool to click on the relationship line or detects that the user uses an input tool to hover over the relationship line), the annotation information display unit 803 may change the annotation information of the preceding node and the succeeding node of the annotated relationship to be displayed in the first relationship display state.

[0125] For another example, when the user operation detection unit 802 detects that the user selects a node corresponding to a marked relationship (for example, detects that the user uses an input tool to click on the node corresponding to the marked relationship or detects that the user uses an input tool to hover over the node corresponding to the marked relationship), the annotation information display unit 803 may change the annotation information of the marked relationship to be displayed in the first relationship display state.

[0126] For another example, when the user operation detection unit 802 detects that the user selects an area other than the area related to the relationship displayed in the first relationship display state (for example, it detects that the user clicks on a blank space in the text area, marks a new node or a new relationship, clicks on other areas such as the annotation type selection area, the annotation result area, etc.), the annotation information display unit 803 may change the relationship displayed in the first relationship display state to be displayed in the second relationship display state.

[0127] For example, Figure 3a As shown, the annotation information of the relationship can also display a delete icon (e.g., a cross circle symbol in the upper left corner of the selection box) in the first relationship display state. For example, the delete icon can be displayed on at least one of the selection box of the preceding node, the selection box of the succeeding node, and the relationship line. When the user operation detection unit 802 detects that the user selects the delete icon, the annotation information display unit 803 can delete the annotation information of the corresponding relationship. The style and position of the delete icon are not limited to this, and can be displayed in any possible style and position, and the present disclosure does not limit this.

[0128] like Figure 4aAs shown, the annotation information of the node in the first node display state is shown, wherein the border color and fill color of the node selection box are brighter (for example, the border color and fill color are both blue). Figure 4b As shown, the annotation information of the node in the first sub-state in the second node display state is shown, wherein the border color of the node selection box is brighter and the filling color is darker (for example, the border color is a blue with a higher saturation and the filling color is a blue with a lower saturation). Figure 4c As shown, the annotation information of the node in the second sub-state in the second node display state is shown, wherein the border color and the filling color of the selection box of the node are darker (for example, the border color and the filling color are both gray).

[0129] For example, when the user operation detection unit 802 detects that the user is marking a node or the user has just marked a node and has not performed other operations, the marking information display unit 803 may display the marking information of the node being marked in the first node display state.

[0130] For another example, when the user operation detection unit 802 detects that the user selects a marked node (for example, it is detected that the user uses an input tool to click on the marked node or it is detected that the user uses the input tool to hover over the marked node, or it is detected that the user uses the input tool to click on the selection box border of the marked node or it is detected that the user uses the input tool to hover over the selection box border of the marked node), the annotation information display unit 803 can change the annotation information of the marked node to be displayed in the first relationship display state.

[0131] For another example, when the user operation detection unit 802 detects that the user selects an area other than an area related to the node displayed in the first node display state (for example, it is detected that the user clicks a blank space in the text area, annotates a new node or a new relationship, clicks other areas such as the annotation type selection area, the annotation result area, etc.), the annotation information display unit 803 may change the node displayed in the first node display state to be displayed in the second node display state. For example, in the case where the second node display state includes a first sub-state and a second sub-state, the annotation information display unit 803 may change the node displayed in the first node display state to be displayed in the first sub-state or the second sub-state in the second node display state.

[0132] For example, Figure 4a As shown, the annotation information of the node can also display a delete icon (e.g., a cross circle symbol in the upper left corner of the selection box) in the first node display state. When the user operation detection unit 802 detects that the user selects the delete icon, the annotation information display unit 803 can delete the annotation information of the corresponding node. The style and position of the delete icon are not limited to this, and can be displayed in any possible style and position, and the present disclosure does not limit this.

[0133] Of course, the display state of the annotation information of the relationship and the node is not limited to the display state described above, and any other display state with different visual prominence can be used, and the present disclosure does not limit this.

[0134] According to an exemplary embodiment of the present disclosure, when the user operation detection unit 802 detects that the user is marking a relationship in the text area, the annotation information display unit 803 can display the annotation information of the relationship that the user is marking in a first relationship display state; display the annotation information of the marked relationship in a second relationship display state; and display the annotation information of the marked node in a second node display state.

[0135] According to an exemplary embodiment of the present disclosure, when the user operation detection unit 802 detects that the user is marking a relationship in the text area, the marking information display unit 803 can display the marking information of the first node in the marked nodes in the first sub-state; and display the marking information of the second node in the marked nodes in the second sub-state; wherein the first node is a node of a node type related to the relationship being marked by the user, and the second node is a node that does not belong to a node type related to the relationship being marked by the user. Displaying the marking information of the nodes in different sub-states can make it easier for the user to distinguish which nodes are related to the relationship type and which nodes are not related to the relationship type when marking a certain relationship type, thereby helping the user to concentrate, mark the relationship more quickly, and improve the user experience.

[0136] According to an exemplary embodiment of the present disclosure, when the user operation detection unit 802 detects that the user is marking a node in the text area, the annotation information display unit 803 can display the annotation information of the node being marked by the user in a first node display state; display the annotation information of the marked node in a second node display state; and display the annotation information of the marked relationship in a second relationship display state.

[0137] According to an exemplary embodiment of the present disclosure, when the user operation detection unit 802 detects that the user is marking a relationship in the text area, the annotation information display unit 803 may display in the text area the annotation information of the preceding node, the annotation information of the succeeding node, and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being marked by the user, or may display in the text area the annotation information of the preceding node, the annotation information of the succeeding node, the relationship line connecting the preceding node and the succeeding node, and the relationship name corresponding to the relationship being marked by the user.

[0138] According to an exemplary embodiment of the present disclosure, when a user is annotating a relationship in a text area, the user operation detection unit 802 can detect the movement trajectory of the input tool used by the user for annotation in response to detecting that the user selects a preceding node in the text area and releases it. The annotation information display unit 803 can display in the text area that the relationship line moves from the preceding node following the detected movement trajectory until it is detected that the user selects a succeeding node in the text area. When the user operation detection unit 802 detects that the user has selected the succeeding node (for example, it detects that the input tool is released after selecting the succeeding node), the annotation information display unit 803 can fix the relationship line. For example, Figure 5 As shown in the figure, when the user selects the front node (for example, Hexing Baihua Oil), the relationship line can be moved from the front node to follow the mouse cursor used by the user for annotation, and the relationship name (for example, applicable to) can also be displayed near the relationship line, and the relationship name can also move with the relationship line. The auxiliary annotation method of using the relationship line to move with the input tool can make the user focus more, be more beautiful, and improve the user experience.

[0139] According to an exemplary embodiment of the present disclosure, the annotation information display unit 803 may use a Bezier curve to display the relationship line. Figure 2b , Figure 3a , Figure 3b , Figure 5 As shown in the figure, the relationship lines are displayed with Bezier curves. However, the traditional auxiliary annotation method uses horizontal lines to display relationship lines, which is difficult to distinguish when there is a cross-line situation, especially when the distance between the front node and the back node is too far (across the screen), which makes the distinction more difficult. Compared with the traditional auxiliary annotation method of using horizontal lines to display relationship lines, the use of Bezier curves can make the annotation page more beautiful and more readable, reduce the visual burden of users, and improve user experience.

[0140] According to an exemplary embodiment of the present disclosure, in the first relationship display state, the annotation information of the relationship includes the relationship type and the relationship type is displayed near the relationship line (such as Figure 3a In the second relationship display state, the relationship annotation information does not include the relationship type (such as Figure 3b shown).

[0141] According to an exemplary embodiment of the present disclosure, when a user is annotating a relationship in a text area, in response to the user operation detection unit 802 detecting that the user has selected the preceding node corresponding to the relationship being annotated and then selects the succeeding node across the screen, the annotation information display unit 803 may display the blurred form of the preceding node in a predetermined area of ​​the current screen during the user's selection of the succeeding node. In addition, when the user operation detection unit 802 detects that the user has selected the succeeding node (for example, the user uses an input tool to select the succeeding node and releases it), the annotation information display unit 803 may not display the blurred form of the preceding node. Setting the blurred form of the preceding node when the preceding and succeeding nodes are across the screen can help the user focus on the content of the relationship being annotated, which is conducive to better assisting the user in annotating the relationship and improving the user experience.

[0142] According to an exemplary embodiment of the present disclosure, the text annotation page may further include a switch for all relationship connections. The user operation detection unit 802 may detect the user's operation on the relationship connection switch. In response to the user operation detection unit 802 detecting that the user turns on the switch for all relationship connections, the annotation information display unit 803 may display the relationship connections of all marked relationships in the text area. In response to the user operation detection unit 802 detecting that the user turns off the switch for all relationship connections, the annotation information display unit 803 may not display the relationship connections of all marked relationships in the text area. Figure 7a and Figure 7b As shown, the switch for all relationship connections can be displayed in the upper area of ​​the text annotation page. The display position and display form of the switch for all relationship connections are not limited thereto, and the present disclosure does not impose any restrictions on this. Figure 7a As shown, in response to the user operation detection unit 802 detecting that the user turns on the switch for all relationship connections, the annotation information display unit 803 can display the relationship connections of all marked relationships in the text area. In this case, the annotation information display unit 803 can display the annotation information of the relationship being annotated by the user in the first relationship display state, and display the relationship connections of all marked relationships in the second relationship display state, and display the relationship names without displaying them near the relationship connections. Figure 7b As shown, in response to the user operation detection unit 802 detecting that the user turns off the switch for all relationship connections, the annotation information display unit 803 may not display the relationship connections of all marked relationships in the text area. In this case, the annotation information display unit 803 may display the annotation information of the relationship that the user is annotating in the first relationship display state, but for the marked relationship, the annotation information of the nodes corresponding to all marked relationships is displayed in the second relationship display state, but the relationship connections of the marked relationship are not displayed. Setting the switch for all relationship connections allows the user to freely choose whether to view all marked relationships at any time, which is convenient for user operation, and is conducive to better assisting users in annotating relationships and improving user experience.

[0143] According to an exemplary embodiment of the present disclosure, in order to enable users to quickly distinguish between marked nodes and marked relationships on the same page and reduce the visual burden of users, the annotation information display unit 803 can display the annotation information of nodes and the annotation information of relationships in different colors in the text area, and display the annotation information of all nodes in the same color, and display the annotation information of all relationships in the same color. For example, the annotation information of all nodes can be blue, and the annotation information of all relationships can be green. In comparison, traditional auxiliary annotation methods often use different colors (or different colors + description text) to distinguish different node types and relationship types, and color duplication will occur, especially when the number is large, the display is more messy. The color display method according to the exemplary embodiment of the present disclosure solves the problems of traditional auxiliary annotation methods well.

[0144] According to an exemplary embodiment of the present disclosure, the annotation information display unit 803 may display the color of the node type in the node type list in the annotation type selection area as the same as the color of the annotation information of the node in the text area, and may display the color of the relationship type in the relationship type list in the annotation type selection area as the same as the color of the annotation information of the relationship in the text area. For example, the annotation information display unit 803 may unify the colors of the node type and the annotation information of the node in the node type list into blue, and unify the colors of the relationship type and the node information of the relationship in the relationship type list into green. Unifying the colors of the node type and the annotation information of the node, and unifying the colors of the relationship type and the node information of the relationship, makes the annotation page more beautiful and more readable, helps users concentrate, and improves the user experience.

[0145] According to an exemplary embodiment of the present disclosure, in response to the user operation detection unit 802 detecting that a marked node is also marked with a relationship, the annotation information display unit 803 can display the annotation information of the node in the color of the annotation information of the relationship. For example, assuming that the color of the annotation information of the node is blue and the color of the annotation information of the relationship is green, when the user operation detection unit 802 detects that a certain marked node is also marked with a relationship, the annotation information display unit 803 changes the color of the annotation information of the marked node from blue to green.

[0146] According to an exemplary embodiment of the present disclosure, in response to the user operation detection unit 802 detecting that the node corresponding to the marked relationship is marked with a new node type, the annotation information display unit 803 can display the annotation information of the node in the color of the node's annotation information. For example, assuming that the color of the node's annotation information is blue and the color of the relationship's annotation information is green, when the user operation detection unit 802 detects that the node corresponding to the marked relationship is marked with a new node type, the annotation information display unit 803 changes the color of the node's annotation information from green to blue.

[0147] Fig. 9 1 is an implementation scenario diagram of manual text annotation according to an exemplary embodiment of the present disclosure. In this implementation scenario, at least one electronic device 101 and at least one server 102 may be included. The at least one electronic device 101 may include various types of electronic devices that can be used to perform manual text annotation, such as computer devices, smart phones, tablet computers, and laptop computers. A page for performing manual text annotation may be displayed on the screen of the electronic device, and the user may be assisted in performing annotation in the page by the method and system for assisting manual text annotation according to the exemplary embodiment of the present disclosure. At least one electronic device 101 and at least one server 102 may communicate via a wired or wireless network. At least one electronic device 101 may send manual text annotation results to at least one server 102, and at least one server 102 may perform machine learning training using the collected manual text annotation results. In addition, at least one electronic device 101 may also perform machine learning training on the local machine based on the manual text annotation results.

[0148] According to the method and system for assisting manual text annotation disclosed in the present invention, allowing users to annotate both nodes and relationships in the same text area can reduce the user's cognitive cost and annotation workload, which has a great auxiliary effect on users.

[0149] According to the method and system for assisting manual text annotation disclosed in the present invention, it is set to display the marked nodes or marked relationships, marked nodes and marked relationships with different visual conspicuities in different annotation states, so that users can reduce the visual burden brought by annotating both nodes and relationships in the same text area, help users concentrate on their own tasks, and thus better assist users in performing text annotation.

[0150] According to the method and system for assisting manual text annotation disclosed in the present invention, when a user annotates a relationship, the annotation information of different marked nodes is displayed in different sub-states, which enables the user to more easily distinguish which nodes are related to a certain relationship type and which nodes are not related to the relationship type when annotating a certain relationship type, thereby helping the user to concentrate, annotate relationships more quickly, and improve the user experience.

[0151] According to the method and system for assisting manual text annotation disclosed in the present invention, when a user annotates a relationship, an auxiliary annotation method in which the relationship line follows the movement of the input tool can help the user focus more, be more beautiful, and improve the user experience.

[0152] According to the method and system for assisting manual text annotation disclosed in the present invention, Bezier curves are used to display relationship lines, so that the annotated page is more beautiful and more readable, the visual burden of the user is reduced, and the user experience is improved.

[0153] According to the method and system for assisting manual text annotation disclosed in the present invention, when a user annotates a relationship, setting a virtual form of the front node when the front and rear nodes cross the screen can help the user focus on the relationship content being annotated, which is conducive to better assisting the user in annotating the relationship and improving the user experience.

[0154] According to the method and system for assisting manual text annotation disclosed in the present invention, a switch for connecting all relationships is set, allowing users to freely choose whether to view all marked relationships at any time, which is convenient for user operation, helps better assist users in annotating relationships, and improves user experience.

[0155] According to the method and system for assisting manual text annotation disclosed in the present invention, the annotation information of nodes and the annotation information of relationships are displayed in different colors, so that users can quickly distinguish the marked nodes and the marked relationships on the same page, reducing the visual burden of users.

[0156] According to the method and system for assisting manual text annotation disclosed in the present invention, the color of the node type and the node annotation information are unified, and the color of the relationship type and the node information of the relationship are unified, so that the annotation page is more beautiful and easier to read, which can better help users concentrate and improve user experience.

[0157] The above has been referred to Figures 1 to 9 A method and system for assisting manual text annotation according to exemplary embodiments of the present disclosure are described.

[0158] Figure 8 Each unit in the system for assisting manual text annotation shown can be configured as software, hardware, firmware, or any combination of the above items to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, or to a pure software code, or to a module combining software and hardware. In addition, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, a client, or a server, etc.).

[0159] In addition, refer to Figure 1 The method for assisting manual text annotation described can be implemented by a program (or instruction) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions can be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the method for assisting manual text annotation according to the present disclosure.

[0160] The computer program in the computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the computer program can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been described in reference to Figure 1 It is mentioned in the description of the related method, so it will not be repeated here to avoid repetition.

[0161] It should be noted that the various units in the system for assisting manual text annotation according to the exemplary embodiment of the present disclosure can completely rely on the operation of the computer program to realize the corresponding functions, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (for example, lib library) to realize the corresponding function.

[0162] on the other hand, Figure 8 The various units shown may also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segment for performing the corresponding operation may be stored in a computer-readable medium such as a storage medium, so that the processor may perform the corresponding operation by reading and running the corresponding program code or code segment.

[0163] For example, an exemplary embodiment of the present disclosure may also be implemented as a computing device, which includes a storage component and a processor, wherein a set of computer-executable instructions is stored in the storage component, and when the set of computer-executable instructions is executed by the processor, the method of assisting manual text annotation according to the exemplary embodiment of the present disclosure is executed.

[0164] Specifically, the computing device can be deployed in a server or client, or can be deployed on a node device in a distributed network environment. In addition, the computing device can be a PC, a tablet device, a personal digital assistant, a smart phone, a web application, or other device capable of executing the above instruction set.

[0165] Here, the computing device is not necessarily a single computing device, but may also be any device or circuit collection that can execute the above instructions (or instruction sets) individually or jointly. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0166] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0167] According to the exemplary embodiment of the present disclosure, some operations described in the method for assisting manual text annotation may be implemented by software, some operations may be implemented by hardware, and furthermore, these operations may be implemented by a combination of software and hardware.

[0168] The processor may execute instructions or codes stored in one of the storage components, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0169] The storage component may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage component may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage component and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the storage component.

[0170] In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the computing device may be connected to each other via a bus and / or a network.

[0171] The method for assisting manual text annotation according to an exemplary embodiment of the present disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logical device or operate according to non-exact boundaries.

[0172] Therefore, refer to Figure 1 The described method of assisting manual text annotation may be implemented by a system including at least one computing device and at least one storage device storing instructions.

[0173] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a method for assisting manual text annotation according to an exemplary embodiment of the present disclosure, and a computer executable instruction set is stored in the storage device. When the computer executable instruction set is executed by the at least one computing device, the execution reference Figure 1 The described method for assisting human text annotation.

[0174] The above describes various exemplary embodiments of the present disclosure, and it should be understood that the above description is only exemplary and not exhaustive, and the present disclosure is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present disclosure, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present disclosure should be based on the scope of the claims.

Claims

1. A method for assisting manual text annotation, comprising: Displaying a text annotation page, wherein the text annotation page includes a text area; Detecting a user's annotation operation in the text area, wherein the annotation operation includes at least one of annotating a node and annotating a relationship; In response to detecting that the user is marking a node in the text area, displaying in the text area the marking information of the node being marked by the user, and displaying the marking information of the marked nodes and marked relationships with a lower visual prominence; In response to detecting that the user is marking a relationship in the text area, the marking information of the relationship being marked by the user is displayed in the text area, and the marking information of the marked nodes and the marked relationships are displayed with a lower visual prominence.

2. The method of claim 1, wherein: The display state of the annotation information of the relationship includes a first relationship display state and a second relationship display state, and the visual obviousness of the first relationship display state and the second relationship display state decreases in sequence; The display state of the node annotation information includes a first node display state and a second node display state, and the visual obviousness of the first node display state and the second node display state decreases in sequence; In response to detecting that the user is marking a relationship in the text area, the step of displaying the marking information of the relationship being marked by the user in the text area and displaying the marking information of the marked nodes and the marked relationship with a lower visual conspicuity includes: Displaying the annotation information of the relationship being annotated by the user in a first relationship display state; Displaying the annotation information of the marked relationship in the second relationship display state; Displaying the annotation information of the marked node in the second node display state; In response to detecting that the user is marking a node in the text area, the step of displaying the marking information of the node being marked by the user in the text area comprises: The annotation information of the node being annotated by the user is displayed in the first node display state.

3. The method of claim 2, wherein: The second node display state includes a first sub-state and a second sub-state, and the visual prominence of the first sub-state and the second sub-state decreases in sequence; The step of displaying the annotation information of the marked node in the second node display state comprises: Displaying the labeling information of the first node among the labeled nodes in the first sub-state; Displaying the marking information of the second node in the marked nodes in the second sub-state; The first node is a node of a node type related to the relationship being annotated by the user, and the second node is a node of a node type not related to the relationship being annotated by the user.

4. The method of claim 1, wherein: The display state of the node annotation information includes a first node display state and a second node display state, and the visual obviousness of the first node display state and the second node display state decreases in sequence; The display state of the annotation information of the relationship includes at least one of a first relationship display state and a second relationship display state, and the visual obviousness of the first relationship display state and the second relationship display state decreases in sequence; In response to detecting that the user is marking a node in the text area, the step of displaying the marking information of the node being marked by the user in the text area, and displaying the marking information of the marked nodes and marked relationships with a lower visual prominence includes: Displaying the annotation information of the node being annotated by the user in the first node display state; Displaying the annotation information of the marked node in the second node display state; Displaying the annotation information of the marked relationship in the second relationship display state; In response to detecting that the user is marking a relationship in the text area, displaying in the text area the marking information of the relationship being marked by the user includes: The annotation information of the relationship being annotated by the user is displayed in a first relationship display state.

5. The method of claim 1, wherein: In the text area, the annotation information of the node and the annotation information of the relationship are displayed in different colors; In the text area, the annotation information of all nodes is displayed in the same color, and the annotation information of all relationships is displayed in the same color.

6. The method of claim 5, further comprising: In response to detecting that a marked node is also marked with a relationship, the marking information of the node is displayed in the color of the marking information of the relationship.

7. The method of claim 5, further comprising: In response to detecting that a node corresponding to a marked relationship is marked with a new node type, the marking information of the node is displayed in the color of the marking information of the node.

8. The method of claim 1, wherein: In response to detecting that the user is marking a relationship in the text area, the step of displaying the marking information of the relationship being marked by the user in the text area comprises: In response to detecting that the user selects a subsequent node across the screen after selecting a preceding node corresponding to the relationship being annotated by the user, a virtualized form of the preceding node is displayed in a predetermined area of ​​the current screen during the process of the user selecting the subsequent node.

9. The method of claim 1, wherein: The text annotation page also includes an annotation type selection area; The step of detecting the user's annotation operation in the text area includes: In response to detecting that the user selects a node type in a node type list in the annotation type selection area and then selects a text area in the text area, it is determined that the user is annotating a node in the text area.

10. The method of claim 1, wherein: The text annotation page also includes an annotation type selection area; The step of detecting the user's operation in the text area includes: In response to detecting that the user selects a relationship type from a relationship type list in the annotation type selection area and then selects a text area in the text area, it is determined that the user is annotating a relationship in the text area.

11. The method according to any one of claims 1 to 10, wherein: The text annotation page also includes a switch for all relationship connections; Wherein, the method further comprises: Detecting the user's operation on the relationship connection switch; in response to detecting that the user turns on all the relationship connection switches, displaying the relationship connections of all marked relationships in the text area; in response to detecting that the user turns off all the relationship connection switches, not displaying the relationship connections of all marked relationships in the text area.

12. The method according to any one of claims 1 to 10, wherein: The annotation information of the node includes: at least one of a node selection box and a node type.

13. The method according to any one of claims 1 to 10, wherein: The annotation information of the relationship includes: a preceding node selection box corresponding to the relationship, a preceding node type corresponding to the relationship, a succeeding node selection box corresponding to the relationship, a succeeding node type corresponding to the relationship, a relationship line connecting the preceding node and the succeeding node, and at least one of the relationship types.

14. The method of claim 13, wherein: In response to detecting that the user is marking a relationship in the text area, the step of displaying the marking information of the relationship being marked by the user in the text area comprises: The annotation information of the preceding node, the annotation information of the succeeding node, and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being annotated by the user are displayed in the text area.

15. The method of claim 14, wherein: The step of displaying the annotation information of the preceding node, the annotation information of the succeeding node and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being annotated by the user in the text area comprises: In response to detecting that the user selects the preceding node in the text area and releases it, the movement trajectory of the input tool used by the user to mark is detected, and the relationship line is displayed in the text area starting from the preceding node and moving along the detected movement trajectory until it is detected that the user selects the succeeding node in the text area.

16. The method of claim 14, wherein: The step of displaying the relationship line in the text area includes: The relationship line is displayed using a Bezier curve.

17. The method of claim 13, wherein: In the first relationship display state, the annotation information of the relationship includes the relationship type and the relationship type is displayed near the relationship line. In the second relationship display state, the annotation information of the relationship does not include the relationship type.

18. A system for assisting manual text annotation, comprising: The annotation page display unit is configured to: display a text annotation page, wherein the text annotation page includes a text area; A user operation detection unit is configured to: detect a user's annotation operation in the text area, wherein the annotation operation includes at least one of annotating a node and annotating a relationship; The annotation information display unit is configured as follows: In response to the user operation detection unit detecting that the user is marking a node in the text area, displaying the marking information of the node being marked by the user in the text area, and displaying the marking information of the marked nodes and marked relations with a lower visual conspicuity; In response to the user operation detection unit detecting that the user is marking a relationship in the text area, the marking information of the relationship being marked by the user is displayed in the text area, and the marking information of the marked nodes and the marked relationships are displayed with a lower visual prominence.

19. The system of claim 18, wherein: The display state of the annotation information of the relationship includes a first relationship display state and a second relationship display state, and the visual obviousness of the first relationship display state and the second relationship display state decreases in sequence; The display state of the node annotation information includes a first node display state and a second node display state, and the visual obviousness of the first node display state and the second node display state decreases in sequence; The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display the annotation information of the relationship being annotated by the user in a first relationship display state; Displaying the annotation information of the marked relationship in the second relationship display state; Displaying the annotation information of the marked node in the second node display state; The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the node being annotated by the user in a first node display state.

20. The system of claim 19, wherein: The second node display state includes a first sub-state and a second sub-state, and the visual prominence of the first sub-state and the second sub-state decreases in sequence; The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display the annotation information of a first node in the marked nodes in a first sub-state; and display the annotation information of a second node in the marked nodes in a second sub-state; The first node is a node of a node type related to the relationship being annotated by the user, and the second node is a node of a node type not related to the relationship being annotated by the user.

21. The system of claim 18, wherein: The display state of the node annotation information includes a first node display state and a second node display state, and the visual obviousness of the first node display state and the second node display state decreases in sequence; The display state of the annotation information of the relationship includes at least one of a first relationship display state and a second relationship display state, and the visual obviousness of the first relationship display state and the second relationship display state decreases in sequence; The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the node being annotated by the user in a first node display state; Displaying the annotation information of the marked node in the second node display state; Displaying the annotation information of the marked relationship in the second relationship display state; The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user is annotating a node in the text area, display the annotation information of the relationship being annotated by the user in a first relationship display state.

22. The system of claim 18, wherein: The annotation information display unit is further configured to: display the annotation information of the node and the annotation information of the relationship in different colors in the text area; In the text area, the annotation information of all nodes is displayed in the same color, and the annotation information of all relationships is displayed in the same color.

23. The system of claim 22, wherein: The annotation information display unit is further configured to: in response to the user operation detection unit detecting that a marked node is also labeled with a relationship, display the annotation information of the node in the color of the annotation information of the relationship.

24. The system of claim 22, wherein: The annotation information display unit is further configured to: in response to the user operation detection unit detecting that the node corresponding to the marked relationship is marked with a new node type, display the annotation information of the node in the color of the node's annotation information.

25. The system of claim 18, wherein: The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user has selected the preceding node corresponding to the relationship being annotated and then selects the succeeding node across the screen, display the preceding node in a blurred form in a predetermined area of ​​the current screen during the user selecting the succeeding node.

26. The system of claim 18, wherein: The text annotation page also includes an annotation type selection area; Wherein, the user operation detection unit is configured as follows: In response to detecting that the user selects a node type in a node type list in the annotation type selection area and then selects a text area in the text area, it is determined that the user is annotating a node in the text area.

27. The system of claim 18, wherein: The text annotation page also includes an annotation type selection area; Wherein, the user operation detection unit is configured as follows: In response to detecting that the user selects a relationship type from a relationship type list in the annotation type selection area and then selects a text area in the text area, it is determined that the user is annotating a relationship in the text area.

28. A system as claimed in any one of claims 18 to 27, wherein: The text annotation page also includes a switch for all relationship connections; Wherein, the user operation detection unit is configured to: detect the user's operation on the relationship connection switch; The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user turns on the switch for all relationship connections, display the relationship connections of all marked relationships in the text area; in response to detecting that the user turns off the switch for all relationship connections, not display the relationship connections of all marked relationships in the text area.

29. The system of any one of claims 18 to 27, wherein: The annotation information of the node includes: at least one of a node selection box and a node type.

30. The system of any one of claims 18 to 27, wherein: The annotation information of the relationship includes: a preceding node selection box corresponding to the relationship, a preceding node type corresponding to the relationship, a succeeding node selection box corresponding to the relationship, a succeeding node type corresponding to the relationship, a relationship line connecting the preceding node and the succeeding node, and at least one of the relationship types.

31. The system of claim 30, wherein: The annotation information display unit is configured to: in response to the user operation detection unit detecting that the user is annotating a relationship in the text area, display the annotation information of the preceding node, the annotation information of the succeeding node and the relationship line connecting the preceding node and the succeeding node corresponding to the relationship being annotated by the user in the text area.

32. The system of claim 31, wherein: The user operation detection unit is configured to: in response to the user operation detection unit detecting that the user selects the preceding node in the text area and releases it, detect a moving track of an input tool used by the user for annotation; The annotation information display unit is configured to: display in the text area that the relationship line moves from the preceding node along the detected movement trajectory until it is detected that the user selects the succeeding node in the text area.

33. The system of claim 31, wherein: The annotation information display unit is configured to: use a Bezier curve to display the relationship line.

34. The system of claim 30, wherein: In the first relationship display state, the annotation information of the relationship includes the relationship type and the relationship type is displayed near the relationship line. In the second relationship display state, the annotation information of the relationship does not include the relationship type.

35. A computer-readable storage medium storing instructions, wherein: When the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the method for assisting manual text annotation as claimed in any one of claims 1 to 17.

36. A system comprising at least one computing device and at least one storage device storing instructions, wherein: When the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the method for assisting manual text annotation as claimed in any one of claims 1 to 17.

Images