Crowdsourcing translation task cutting method and device
By calculating the semantic similarity and spacing distance between sentences, a relationship graph is constructed and the minimum cutting algorithm of the graph is used to cut the translation task, which solves the problems of slow processing speed and high cost of translation task, and realizes efficient and low-cost translation task cutting, and retains The correlation information between sentences is improved to improve the accuracy of translation results.
Patent Information
- Application Number
- CN202111547723.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-16
AI Technical Summary
The existing technology cannot effectively cut large-scale translation tasks, resulting in slow processing speed and high cost of translation tasks, and may lose inter-sentence information after subdivision, affecting the accuracy of translation results.
By calculating the semantic similarity and spacing distance between sentences, a relationship graph is constructed, and the minimum cutting algorithm of the graph is used to cut the task into a subtask, and the correlation information between sentences is preserved as much as possible. The edge addition method is used to span the tree, and the depth-first traversal and path are used to prioritize traversal and Cut, find the set of edges with the smallest weight to cut.
It realizes high-efficiency, high-quality and low-cost translation task cutting, reduces the loss of related information between sentences, and improves the processing speed and accuracy of translation tasks.
Smart Images

Figure CN114331062B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of task segmentation in crowdsourcing and the field of translation technology, and in particular to a method and device for segmenting crowdsourcing translation tasks. Background Art
[0002] In recent years, with the rapid development of the Internet, the communication between countries has become increasingly close, which has brought a lot of demand to the translation industry. In this case, the crowdsourcing translation model that uses Internet technology to outsource translation tasks to various translators on the Internet has emerged. Some translation tasks have large tasks and need to be delivered quickly. In this case, a single translator obviously cannot complete the task on time. Moreover, since most of the people who handle these translation tasks on the Internet are not professionals and handle these tasks in a small amount of spare time, they sometimes translate the tasks for a long time and need time to review the context information, which reduces the processing speed of the task.
[0003] The traditional method of handling large-scale translation tasks is to let translators with high ability levels divide the translation tasks into several subtasks, and then assign the subtasks to other translators to realize translation task processing in the form of secondary crowdsourcing. Obviously, the cost of manually dividing translation tasks is high, and it is impossible to further subdivide the tasks in real time. Therefore, the present invention provides a crowdsourcing translation task cutting method and device, which can automatically cut crowdsourcing translation tasks under the premise of losing the least correlation information between sentences. Summary of the invention
[0004] In order to solve the problem that crowdsourcing translation tasks take a long time to complete individually and that subdivision may cause inaccurate translation results due to the loss of correlation information between sentences, the present invention provides a crowdsourcing translation task segmentation method and device, which can complete crowdsourcing translation tasks with relatively high efficiency, high quality and low cost.
[0005] The objective of the present invention is achieved through the following technical solutions:
[0006] On the one hand, the present application proposes a crowdsourcing translation task segmentation method, which comprises the following steps:
[0007] (1) Receive crowdsourcing translation tasks posted by users, calculate the semantic similarity between sentences in the crowdsourcing translation tasks, and characterize the correlation between sentences by combining the semantic similarity and the distance between sentences in the crowdsourcing translation tasks;
[0008] (2) Construct a relationship graph between sentences in the crowdsourcing translation task through association relationships;
[0009] (3) The crowdsourcing translation task is divided into multiple subtasks by finding the minimum cut of the graph, including:
[0010] (3-1) Based on the relationship graph, a tree is generated using the edge addition method;
[0011] (3-2) Select an edge on the tree. After removing the edge, the tree is divided into two subtrees. Find the edges whose two endpoints belong to the two subtrees in the edge set of the relationship graph, and record the sum of the weights of these edges and edges;
[0012] (3-3) Use depth-first traversal to cut the tree into a set of paths;
[0013] (3-4) Take two edges in the path with more than 2 vertices. The two edges divide the tree into three subtrees. Find the edges in the edge set of the relationship graph whose two endpoints belong to the subtree in the middle of the two edges and the other two subtrees respectively. Record the sum of the weights of these edges and edges.
[0014] (3-5) Find the edge set with the smallest sum of weights, cut the relationship graph into two subgraphs, and determine whether the number of vertices in the subgraph is greater than the threshold. If so, continue to cut the subgraph, otherwise output the subtask set corresponding to the subgraph.
[0015] Furthermore, the step (1) includes the following sub-steps:
[0016] (1-1) Receive crowdsourcing translation tasks posted by users;
[0017] (1-2) Sentence the crowdsourcing translation task, obtain n sentences, and use the set V{S0,S1,S2,...,S n}Record;
[0018] (1-3) Segment each sentence and remove stop words to obtain a set of n words N{W0,W1,...,W n};
[0019] (1-4) For any two word sets in the set N, use the union of the two word sets to calculate the word frequency of the two word sets respectively, vectorize the word frequency to obtain the word vector, and calculate the cosine similarity based on the word vector;
[0020] (1-5) The cosine similarity is used as the semantic similarity between sentences and stored in an n×n matrix β. The element value of the i-th row and j-th column is defined as s ij ;
[0021] (1-6) Calculate the weights corresponding to the associations between sentences and store them using the n×n matrix δ. Define the value of the element in the i-th row and j-th column as r ij , weight r ij The calculation formula is as follows:
[0022] r ij =s ij ×k (|i-j-1|)
[0023] where s ij represents the semantic similarity between sentences, and k is the attenuation coefficient.
[0024] Furthermore, the step (2) is specifically as follows: let the relationship graph G = (V, E), where V is a set of points, representing each sentence; E is an undirected edge connecting two points, representing the association relationship between the two sentences, and each element in E is a triple (S i ,S j ,r ij ), S i and S j Represents two sentences, r ij Represents the weight corresponding to the relationship between two sentences.
[0025] Furthermore, the step (3-1) is specifically as follows: for the relationship graph G, take the starting sentence of the crowdsourcing translation task, that is, S0 in the set V as the starting point, and use the edge addition method to generate a tree T(V, E t ), E t is the edge set of tree T, that is, set set A as the set of visited points, set B as the set of unvisited points, put the starting point S0 into set A, and the other points into set B, find the edge with the largest weight in edge set E, one end of which belongs to set A and the other end belongs to set B, and record it in set E t And add the point of set B corresponding to the edge to set A and delete it from set B, and keep adding points to set A until there are no elements in set B.
[0026] Furthermore, the step (3-2) is specifically as follows: select a tree T(V,E t ), after removing this edge, the tree is divided into two subtrees T i (V i ,E i ), T j (V j ,E j ), find an edge R in the edge set E of the relationship graph x , it connects the points that belong to V i , and the other belongs to V j , record these edges C{R x ,R y ,...} and the sum w of the edge weights; traverse each edge in the tree according to the above operation and aggregate the results into the set O{{C1,w1},{C2,w2},...}.
[0027] Furthermore, in step (3-3), the tree T(V,E t ) is cut into a set of paths P{p0(V p0 ,E p0),p1(V p1 ,E p1 ),...},include:
[0028] (a) Taking the starting sentence as the starting point, perform a depth-first traversal of the tree T to obtain a point sequence and record the leaf nodes;
[0029] (b) Traverse the point sequence sequentially, record each point, and if a leaf node is encountered, group all the previously recorded points and leaf nodes into a path p i (V pi ,E pi ), where V pi For path p i The set of points on E pi For path p i The set of edges between two adjacent points;
[0030] (c) Set up a path set P and record these paths.
[0031] Furthermore, the step (3-4) is specifically as follows: take a path p in the set P whose number of vertices is greater than 2 i (V pi ,E pi ), V pi is a set of path points, E pi is the set of path edges, take the set E pi There are two edges in the tree T(V,E t ) is divided into three subtrees, and the vertices of the subtrees in the middle of the two sides are taken as the vertex set V i , take the vertices of the remaining two subtrees as the vertex set V j ; Find an edge R in the edge set E of the relationship graph x , it connects the points that belong to V i , and the other belongs to V j , record these edges C{R x ,R y ...} and the sum of the edge weights w; perform the above operation to obtain E pi The combination of all two edges in , aggregates the results into the set O{{C1,w1},{C2,w2},...}.
[0032] Furthermore, the step (3-5) is specifically as follows: looping step (3-4) until all paths with a vertex number greater than 2 in the path set P are obtained, finding the element with the smallest w in the set O, taking out the edge set C corresponding to the element, deleting all edges of the edge set C in the relationship graph G(V,E), thereby cutting the relationship graph G(V,E) into two subgraphs G1(V1,E1) and G2(V2,E2), and respectively determining whether the number of elements in the point set V in the subgraphs G1(V1,E1) and G2(V2,E2) is greater than the threshold value vmax , if it is greater than , continue to cut the subgraph using the method in steps (3-1) to (3-5), otherwise output the subtask set corresponding to the subgraph.
[0033] On the other hand, the present application proposes a crowdsourcing translation task cutting device, which includes the following modules:
[0034] Task receiving module: receiving crowdsourcing translation tasks posted by users;
[0035] Sentence semantic similarity calculation module: calculates the semantic similarity between sentences in the crowdsourcing translation task;
[0036] Sentence correlation characterization module: Combines semantic similarity and the distance between sentences in crowdsourcing translation tasks to characterize the correlation between sentences;
[0037] Relationship graph construction module: constructs the relationship graph between sentences in the crowdsourcing translation task through the association relationship between sentences;
[0038] Subtask cutting module: The crowdsourcing translation task is divided into multiple subtasks by finding the minimum cut of the graph, including:
[0039] Based on the relationship graph, a tree is generated using the edge addition method; an edge on the tree is selected, and after removing the edge, the tree is divided into two subtrees. In the edge set of the relationship graph, the edges whose two endpoints belong to the two subtrees respectively are found, and the sum of the weights of these edges and edges is recorded;
[0040] Use depth-first traversal to cut the tree into a set of paths; take two edges in the path with more than 2 vertices, and divide the tree into three subtrees on both sides. Find the edges in the edge set of the relationship graph whose two endpoints belong to the subtree in the middle of the two sides and the other two subtrees respectively, and record the sum of the weights of these edges and edges;
[0041] Find the edge set with the smallest sum of weights, cut the relationship graph into two subgraphs, and determine whether the number of subgraph vertices is greater than the threshold. If so, continue to cut the subgraph, otherwise output the subtask set corresponding to the subgraph.
[0042] In addition, the present application also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps in the above-mentioned crowdsourcing translation task cutting method.
[0043] The present application also proposes a storage medium storing computer-readable instructions, which, when executed by one or more processors, enables the one or more processors to execute the steps in the above-mentioned crowdsourcing translation task segmentation method.
[0044] The beneficial effects of the present invention are as follows: compared with traditional methods, the present invention takes into account the correlation between sentences in crowdsourcing translation tasks, and cuts the crowdsourcing translation tasks with less loss of correlation information by finding the minimum cut of the graph, thereby completing the translation tasks with relatively high efficiency, high quality and low cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A flow chart of a crowdsourcing translation task segmentation method provided by an embodiment of the present invention;
[0046] Figure 2 A flowchart of dividing a crowdsourcing translation task into subtasks by finding a minimum graph cut is provided for an embodiment of the present invention. DETAILED DESCRIPTION
[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0049] like Figure 1 As shown, the embodiment of the present application provides a crowdsourcing translation task segmentation method, the method comprising the following steps:
[0050] (1) Receive crowdsourcing translation tasks posted by users, calculate the semantic similarity between sentences in the crowdsourcing translation tasks, and characterize the correlation between sentences by combining the semantic similarity and the interval distance between sentences in the crowdsourcing translation tasks; specifically:
[0051] (1-1) Receive crowdsourcing translation tasks posted by users. Crowdsourcing translation tasks are usually long texts in any language.
[0052] (1-2) Use the NLTK natural language processing toolkit to segment the crowdsourcing translation task into sentences, obtain n sentences, and use the set V{S0,S1,S2,...,S n}Record.
[0053] (1-3) Segment each sentence and remove stop words to obtain a set of n words N{W0,W1,...,W n}.
[0054] (1-4) For any two word sets W in set N i , W j , find the union U of the two word sets, and use the set U to calculate Wi , W j Calculate the word frequency and vectorize it to get the word vectors a and b. Substitute them into the cosine formula to find the cosine similarity sim(a,b):
[0055]
[0056] (1-5) The cosine similarity is used as the semantic similarity between sentences and stored in an n×n matrix β. The element value of the i-th row and j-th column is defined as s ij .
[0057] (1-6) Calculate the weights corresponding to the associations between sentences and store them using the n×n matrix δ. Define the value of the element in the i-th row and j-th column as r ij , weight r ij The calculation formula is as follows:
[0058] r ij =s ij ×k (|i-j-1|)
[0059] where s ij represents the semantic similarity between sentences, i and j represent the labels of the two sentences, and k is the attenuation coefficient, usually k=0.1.
[0060] (2) Construct a relationship graph between sentences in the crowdsourcing translation task through association relationships, specifically:
[0061] Let the relationship graph G = (V, E), where V is a set of points representing each sentence, that is, V{S0,S1,S2,...,S n}; E is an undirected edge connecting two points, representing the relationship between two sentences, that is, E{R0,R1,...}, each element in E is a triple (S i ,S j ,r ij ), S i and S j Represents two sentences, r ij Represents the weight corresponding to the association relationship between the above two sentences.
[0062] The generation method of set E is: traverse the matrix δ, if r ij If it is not 0, r ij The corresponding triple (S i ,S j ,r ij ) is stored in set E, otherwise it is not stored in set E.
[0063] (3) By finding the minimum cut of the graph, the crowdsourcing translation task is divided into multiple subtasks with strong correlations between sentences, including: based on the relationship graph, using the edge addition method to generate a tree; selecting an edge on the tree, removing the edge and dividing the tree into two subtrees, finding the edges whose two endpoints belong to the two subtrees in the edge set of the relationship graph, and recording the sum of the weights of these edges and the edges; using depth-first traversal to cut the tree into a set of paths; taking two edges in the path with more than 2 vertices, both of which can divide the tree into three subtrees, finding the edges whose two endpoints belong to the subtree in the middle of the two edges and the other two subtrees in the edge set of the relationship graph, and recording the sum of the weights of these edges and the edges; finding the edge set with the smallest sum of weights, cutting the relationship graph into two subgraphs, and judging whether the number of vertices in the subgraph is greater than a threshold. If so, continue to cut the subgraph, otherwise, output the subtask set corresponding to the subgraph.
[0064] like Figure 2 As shown, this step specifically includes the following sub-steps:
[0065] (3-1) For the relationship graph G, take the starting sentence of the crowdsourcing translation task, that is, S0 in the set V, as the starting point, and use the edge addition method to generate a tree T(V,E t ), E t is the edge set of tree T, that is, set set A as the set of visited points, set B as the set of unvisited points, put the starting point S0 into set A, and the other points into set B, find the edge with the largest weight in edge set E, one end of which belongs to set A and the other end belongs to set B, and record it in set E t And add the point of set B corresponding to the edge to set A and delete it from set B, and keep adding points to set A until there are no elements in set B.
[0066] (3-2) Select the tree T(V,E t ), after removing this edge, the tree is divided into two subtrees T i (V i ,E i ), T j (V j ,E j ), find an edge R in the edge set E x , it connects the points that belong to V i , and the other belongs to V j , record these edges C{R x ,R y ,...} and the sum of the edge weights w. Traverse each edge in the tree according to the above operation and aggregate the results into the set O{{C1,w1},{C2,w2},...}.
[0067] (3-3) Use depth-first traversal to transform the tree T(V,E t ) is cut into a set of paths P{p0(Vp0 ,E p0 ),p1(V p1 ,E p1 ),...}, specifically:
[0068] (a) Taking the initiation sentence as the starting point, perform a depth-first traversal of the tree T to obtain a point sequence and record the leaf nodes.
[0069] (b) Traverse the point sequence sequentially, record each point, and if a leaf node is encountered, group all the previously recorded points and leaf nodes into a path p i (V pi ,E pi ), where V pi For path p i The set of points on E pi For path p i The set of edges between two adjacent points.
[0070] (c) Set up a set P to record these paths.
[0071] (3-4) Take a path p in the set P whose number of vertices is greater than 2 i (V pi ,E pi ), V pi is a set of path points, E pi is the set of path edges, take the set E pi There are two edges in the middle, both of which can connect the tree T(V,E t ) is divided into three subtrees, and the vertices of the subtrees in the middle of the two sides are taken as the vertex set V i , take the vertices of the remaining two subtrees as the vertex set V j . Find an edge R in the edge set E x , it connects the points that belong to V i , and the other belongs to V j , record these edges C{R x ,R y ...} and the sum of the edge weights w. Follow the above operation to get E pi The combination of all two edges in , aggregates the results into the set O{{C1,w1},{C2,w2},...}.
[0072] (3-5) Loop through step (3-4) until all paths with vertices greater than 2 in set P are taken, find the element with the smallest w in set O, take out the edge set C corresponding to the element, delete all edges of edge set C in the relationship graph G(V,E), and thus cut the relationship graph G(V,E) into two subgraphs G1(V1,E1) and G2(V2,E2), and respectively determine whether the number of elements in the point set V in the subgraphs G1(V1,E1) and G2(V2,E2) is greater than the threshold vmax If it is greater than, continue to cut the subgraph using the method in steps (3-1) to (3-5). Otherwise, output the subtask set corresponding to the subgraph. max You can set it according to the amount of translation tasks that your staff can accept.
[0073] The present application also proposes a crowdsourcing translation task cutting device, which includes the following modules:
[0074] Task receiving module: receiving crowdsourcing translation tasks posted by users;
[0075] Sentence semantic similarity calculation module: calculates the semantic similarity between sentences in the crowdsourcing translation task;
[0076] Sentence correlation characterization module: Combines semantic similarity and the distance between sentences in crowdsourcing translation tasks to characterize the correlation between sentences;
[0077] Relationship graph construction module: constructs the relationship graph between sentences in the crowdsourcing translation task through the association relationship between sentences;
[0078] Subtask cutting module: The crowdsourcing translation task is divided into multiple subtasks by finding the minimum cut of the graph, including:
[0079] Based on the relationship graph, a tree is generated using the edge addition method; an edge on the tree is selected, and after removing the edge, the tree is divided into two subtrees. In the edge set of the relationship graph, the edges whose two endpoints belong to the two subtrees respectively are found, and the sum of the weights of these edges and edges is recorded;
[0080] Use depth-first traversal to cut the tree into a set of paths; take two edges in the path with more than 2 vertices, and divide the tree into three subtrees on both sides. Find the edges in the edge set of the relationship graph whose two endpoints belong to the subtree in the middle of the two sides and the other two subtrees respectively, and record the sum of the weights of these edges and edges;
[0081] Find the edge set with the smallest sum of weights, cut the relationship graph into two subgraphs, and determine whether the number of subgraph vertices is greater than the threshold. If so, continue to cut the subgraph, otherwise output the subtask set corresponding to the subgraph.
[0082] The present application also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps in the above-mentioned crowdsourcing translation task segmentation method.
[0083] The present application also proposes a storage medium storing computer-readable instructions, which, when executed by one or more processors, enables the one or more processors to execute the steps in the above-mentioned crowdsourcing translation task segmentation method.
[0084] The above is only a preferred embodiment of the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention by using the above disclosed methods and technical contents without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.
Claims
1. A crowdsourcing translation task segmentation method, characterized in that: The method comprises the following steps: (1) Receive crowdsourcing translation tasks posted by users, calculate the semantic similarity between sentences in the crowdsourcing translation tasks, and characterize the correlation between sentences by combining the semantic similarity and the distance between sentences in the crowdsourcing translation tasks; (2) Construct a relationship graph between sentences in the crowdsourcing translation task through association relationships; (3) The crowdsourcing translation task is divided into multiple subtasks by finding the minimum cut of the graph, including: (3-1) Based on the relationship graph, a tree is generated using the edge addition method; (3-2) Select an edge on the tree. After removing the edge, the tree is divided into two subtrees. Find the edges whose two endpoints belong to the two subtrees in the edge set of the relationship graph, and record the sum of the weights of these edges and edges; (3-3) Use depth-first traversal to cut the tree into a set of paths; (3-4) Take two edges in the path with more than 2 vertices. The two edges divide the tree into three subtrees. Find the edges in the edge set of the relationship graph whose two endpoints belong to the subtree in the middle of the two edges and the other two subtrees respectively. Record the sum of the weights of these edges and edges. (3-5) Find the edge set with the smallest sum of weights, cut the relationship graph into two subgraphs, and determine whether the number of vertices in the subgraph is greater than the threshold. If so, continue to cut the subgraph, otherwise output the subtask set corresponding to the subgraph.
2. A crowdsourcing translation task segmentation method according to claim 1, characterized in that: The step (1) comprises the following sub-steps: (1-1) Receive crowdsourcing translation tasks posted by users; (1-2) Sentence the crowdsourcing translation task, obtain n sentences, and use the set V{S0,S1,S2,...,S n }Record; (1-3) Segment each sentence and remove stop words to obtain a set of n words N{W0,W1,...,W n }; (1-4) For any two word sets in the set N, use the union of the two word sets to calculate the word frequency of the two word sets respectively, vectorize the word frequency to obtain the word vector, and calculate the cosine similarity based on the word vector; (1-5) The cosine similarity is used as the semantic similarity between sentences and stored in an n×n matrix β. The element value of the i-th row and j-th column is defined as s ij ; (1-6) Calculate the weights corresponding to the associations between sentences and store them using the n×n matrix δ. Define the value of the element in the i-th row and j-th column as r ij , weight r ij The calculation formula is as follows: r ij =s ij ×k (|i-j-1|) where s ij represents the semantic similarity between sentences, and k is the attenuation coefficient.
3. A crowdsourcing translation task segmentation method according to claim 1 or 2, characterized in that: The step (2) is specifically as follows: let the relationship graph G = (V, E), where V is a set of points, representing each sentence; E is an undirected edge connecting two points, representing the association relationship between the two sentences, and each element in E is a triple (S i ,S j ,r ij ), S i and S j Represents two sentences, r ij Represents the weight corresponding to the relationship between two sentences.
4. A crowdsourcing translation task segmentation method according to claim 3, characterized in that: The step (3-1) is specifically as follows: for the relationship graph G, take the starting sentence of the crowdsourcing translation task, i.e., S0 in the set V, as the starting point, and generate a tree T(V, E t ), E t is the edge set of tree T, that is, set set A as the set of visited points, set B as the set of unvisited points, put the starting point S0 into set A, and the other points into set B, find the edge with the largest weight in edge set E, one end of which belongs to set A and the other end belongs to set B, and record it in set E t And add the point of set B corresponding to the edge to set A and delete it from set B, and keep adding points to set A until there are no elements in set B.
5. A crowdsourcing translation task segmentation method according to claim 4, characterized in that: The step (3-2) is specifically as follows: select a tree T(V,E t ), after removing this edge, the tree is divided into two subtrees T i (V i ,E i ), T j (V j ,E j ), find an edge R in the edge set E of the relationship graph x , it connects the points that belong to V i , and the other belongs to V j , record these edges C{R x ,R y ,...} and the sum w of the edge weights; traverse each edge in the tree according to the above operation and aggregate the results into the set O{{C1,w1},{C2,w2},...}.
6. A crowdsourcing translation task segmentation method according to claim 5, characterized in that: In step (3-3), the tree T(V,E t ) is cut into a set of paths P{p0(V p0 ,E p0 ),p1(V p1 ,E p1 ),...},include: (a) Taking the starting sentence as the starting point, perform a depth-first traversal of the tree T to obtain a point sequence and record the leaf nodes; (b) Traverse the point sequence sequentially, record each point, and if a leaf node is encountered, group all the previously recorded points and leaf nodes into a path p i (V pi ,E pi ), where V pi For path p i The set of points on E pi For path p i The set of edges between two adjacent points; (c) Set a path set P and record these paths.
7. A crowdsourcing translation task segmentation method according to claim 6, characterized in that: The specific steps of step (3-4) are: take a path p in the set P whose number of vertices is greater than 2 i (V pi ,E pi ), V pi is a set of path points, E pi is the set of path edges, take the set E pi There are two edges in the tree T(V,E t ) is divided into three subtrees, and the vertices of the subtrees in the middle of the two sides are taken as the vertex set V i , take the vertices of the remaining two subtrees as the vertex set V j ; Find an edge R in the edge set E of the relationship graph x , it connects the points that belong to V i , and the other belongs to V j , record these edges C{R x ,R y ...} and the sum of the edge weights w; perform the above operation to obtain E pi The combination of all two edges in , aggregates the results into the set O{{C1,w1},{C2,w2},...}.
8. A crowdsourcing translation task segmentation method according to claim 7, characterized in that: The step (3-5) is specifically as follows: looping through step (3-4) until all paths with a vertex number greater than 2 in the path set P are obtained, finding the element with the smallest w in the set O, taking out the edge set C corresponding to the element, deleting all edges of the edge set C in the relationship graph G(V,E), thereby cutting the relationship graph G(V,E) into two subgraphs G1(V1,E1) and G2(V2,E2), and respectively determining whether the number of elements in the point set V in the subgraphs G1(V1,E1) and G2(V2,E2) is greater than a threshold value v max , if it is greater than , continue to cut the subgraph using the method in steps (3-1) to (3-5), otherwise output the subtask set corresponding to the subgraph.
9. A crowdsourcing translation task cutting device, characterized in that: The device includes the following modules: Task receiving module: receiving crowdsourcing translation tasks posted by users; Sentence semantic similarity calculation module: calculates the semantic similarity between sentences in the crowdsourcing translation task; Sentence correlation characterization module: Combines semantic similarity and the distance between sentences in crowdsourcing translation tasks to characterize the correlation between sentences; Relationship graph construction module: constructs the relationship graph between sentences in the crowdsourcing translation task through the association relationship between sentences; Subtask cutting module: The crowdsourcing translation task is divided into multiple subtasks by finding the minimum cut of the graph, including: Based on the relationship graph, a tree is generated using the edge addition method; Select an edge on the tree. After removing the edge, the tree is divided into two subtrees. Find the edges whose two endpoints belong to the two subtrees in the edge set of the relationship graph, and record the sum of the weights of these edges and edges. Use depth-first traversal to cut the tree into a set of paths; take two edges in the path with more than 2 vertices, and divide the tree into three subtrees on both sides. Find the edges in the edge set of the relationship graph whose two endpoints belong to the subtree in the middle of the two sides and the other two subtrees respectively, and record the sum of the weights of these edges and edges; Find the edge set with the smallest sum of weights, cut the relationship graph into two subgraphs, and determine whether the number of subgraph vertices is greater than the threshold. If so, continue to cut the subgraph, otherwise output the subtask set corresponding to the subgraph.
10. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps in the crowdsourcing translation task segmentation method as described in any one of claims 1 to 8.
