Method for checking dependency of sentence components and lexical arrangement in english text

Abstract

The application provides a method for checking the similarity of sentence component dependency and vocabulary arrangement of English texts, which comprises a checking model composed of an English text sentence component dependency similarity checking module and an English text vocabulary arrangement similarity checking module. After two English texts are checked by the model, the similarity of sentence component dependency and vocabulary arrangement of the two English texts can be obtained, and the problem that other models cannot check whether the sentence component dependency and vocabulary arrangement of English texts are similar is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a method for checking similarity of English text sentence component dependency and word arrangement. BACKGROUND

[0002] The existing method for checking similarity of English text adopts an edit distance method to analyze the similarity degree of words and phrases of English text. This method for checking similarity of English text can only check whether the words and phrases of English text are similar, and cannot check whether the sentence component dependency and word arrangement of English text are similar. The present application provides a method for checking similarity of English text sentence component dependency and word arrangement, which can check the similarity degree of sentence component dependency and word arrangement of English text. SUMMARY

[0003] The method for checking similarity of English text sentence component dependency and word arrangement comprises an English text sentence component dependency similarity checking module and an English text word arrangement similarity checking module.

[0004] The processing flow of the English text sentence component dependency similarity checking module is as follows: first, input the English text to be checked, and perform word segmentation and sentence segmentation processing, delete special characters, and convert word capitalization to word lowercase on the English text; second, construct a triple of core words, modifier words and word dependency relationship according to the core words and modifier words in the English text sentence component dependency and the dependency relationship between two words; third, perform vectorization processing on the words in the constructed triple; fourth, generate a core word similarity matrix, a modifier word similarity matrix and a word dependency relationship similarity matrix by calculating the core word similarity, the modifier word similarity and the word dependency relationship similarity in the constructed triple; fifth, generate a triple similarity matrix according to the three generated similarity matrices, calculate the value of each triple similarity matrix by using the calculation formula of the triple similarity of the core words, modifier words and word dependency relationship, select the most similar triple in each row of the generated triple matrix to represent the final triple similarity value; sixth, assign a corresponding weight factor to the word dependency relationship, combine the sentences in the English text in pairs to obtain the combination of all sentence pairs in the English text, calculate the combination of all sentence pairs to obtain the component dependency similarity of all sentence pairs, construct a sentence component dependency similarity matrix of the English text, and calculate each value in the sentence component dependency similarity matrix by using the calculation formula of the component dependency similarity of the English sentence pair; seventh, select the value corresponding to the position of the most similar sentence pair in each row of the sentence component dependency similarity matrix to represent the final component dependency similarity of the sentence pair, and store the coordinates of the position in an index file.

[0005] The processing flow of the English text word arrangement similarity checking module is as follows: inputting the English text to be checked, performing word segmentation and sentence segmentation processing, deleting special characters, and converting word capitalization to word lowercase on the English text; secondly, reading the sentence index file, performing stop word deletion processing on the sentence pair in the English text, and obtaining the joint word set corresponding to the two sentences respectively; thirdly, performing set union operation on the joint word set between the two sentences to obtain the joint word set of the two sentences; fourthly, calculating the word arrangement vector corresponding to the sentence pair according to the joint word set, and each value in the word arrangement vector is the index number of the corresponding position of the word in the joint word set; fifthly, calculating the two word arrangement vectors respectively by using the word arrangement similarity calculation formula, and finally calculating the word arrangement similarity of the English text sentence pair by using the English sentence pair word arrangement similarity calculation formula.

[0006] The calculation formula of the application is defined as follows:

[0007] 1. Triplet similarity calculation formula composed of core word, modifier word and word dependency relationship

[0008]

[0009] In formula (1), the core word similarity refers to the similarity degree of the core word in the English text sentence pair, the modifier word similarity refers to the similarity degree of the modifier word in the English text sentence pair, and the word dependency relationship similarity refers to the similarity degree of the dependency relationship between the core word and the modifier word in the English text sentence pair.

[0010] 2. English sentence pair component dependency similarity calculation formula

[0011]

[0012] In formula (2), the English sentence pair component dependency similarity refers to the similarity degree of all triplet similarities in the two English sentences, the English sentence component refers to the core word, the modifier word, and the word dependency relationship between the core word and the modifier word, m represents the number of triplets in the English text sentence pair, and i represents the i th triplet in the English text sentence pair.

[0013] 3. English sentence word arrangement vector calculation formula

[0014] English sentence word arrangement vector = [x1, x2, …, x n ] (3)

[0015] In formula (3), x n represents the value of each dimension of the word arrangement vector in the English sentence, the value of each dimension of the English sentence word arrangement vector, and the index number of the corresponding position of the word in the joint word set, and the joint word set is obtained by performing set union operation on the joint word set between the two sentences.

[0016] 4. The formula for calculating the lexical arrangement similarity of English sentence pairs

[0017]

[0018] In formula (4), the symbol "||" represents the square root of the sum of the square of the difference in the vector dimension of the sentence pair, the lexical arrangement vector of the first sentence represents the lexical arrangement vector of the first sentence in the English text sentence pair, the lexical arrangement vector of the second sentence represents the lexical arrangement vector of the second sentence in the English text sentence pair, and the lexical arrangement vector is obtained from formula (3).

[0019] The overall processing flow, the English text sentence component dependency similarity checking module, and the English text lexical arrangement similarity checking module processing flow of the present application are as follows.

[0020] As shown in Figure 1 , the overall processing flow is as follows:

[0021] P101 starts;

[0022] P102 executes the English text sentence component dependency similarity checking module (see Figure 2 for details);

[0023] P103 English text lexical arrangement similarity checking module (see Figure 3 for details);

[0024] P104 ends;

[0025] As shown in Figure 2 , the English text sentence component dependency similarity checking module processing flow is as follows:

[0026] P201 starts;

[0027] P202 reads English text respectively;

[0028] P203 performs sentence processing and special character deletion operations on the English text, and outputs the preprocessed English text sentence results;

[0029] P204 reads the sentence results processed by P203 on the English text respectively;

[0030] P205 constructs a triple according to the core word and the modifier word in the text sentence, as well as the dependency relationship between the two words;

[0031] P206 vectorizes the core word and the modifier word in the triple;

[0032] P207 Calculate the core word similarity and modifier word similarity matrix according to the output result of P206, and then calculate the word dependency relationship similarity matrix;

[0033] P208 According to the output result of P207, the triplet similarity is obtained by using the triplet similarity calculation formula (1) composed of core word, modifier word and word dependency relationship, and then the triplet matrix is obtained;

[0034] P209 According to the output result of P208, traverse each row of the triplet matrix, select the most similar triplet in each row to represent the final triplet similarity value;

[0035] P210 Assign a corresponding weight factor to the word dependency relationship in the triplet;

[0036] P211 Combine the sentences in two texts two by two to obtain a plurality of sentence pairs;

[0037] P212 According to the component dependency similarity calculation formula (2) of English sentence pair, the component dependency similarity of the sentence pair is obtained;

[0038] P213 According to the output result of P212, the component dependency similarity matrix of all sentence pairs combined in the text is obtained;

[0039] P214 According to the output result of P213, traverse each row of the component dependency similarity matrix, select the value corresponding to the most similar sentence pair in the position to represent the final component dependency similarity of the sentence pair, and store the coordinates of the position into the index file;

[0040] P215 According to the output result of P214, the position coordinates corresponding to the most similar sentence pair are stored in the sentence index file;

[0041] P216 End.

[0042] As shown in Figure 3 The English text vocabulary arrangement similarity checking module processing flow is as follows:

[0043] P301 Start;

[0044] P302 Read English text respectively;

[0045] P303 Process the English text, delete special characters, and convert word capitalization to word lowercase, output the preprocessed English text sentence result;

[0046] P304 Traverse the sentence index file to obtain a sentence pair;

[0047] P305 respectively do the stop word deletion processing to the sentence pair in the English text, and get two joint word sets;

[0048] P306 does the union set to the two joint word sets, and gets the joint word set;

[0049] P307 respectively calculates the two lexical arrangement vectors according to the output result of P306 step, and adopts the lexical arrangement similarity calculation formula (3);

[0050] P308 finally gets the lexical arrangement similarity of the English text sentence pair according to the output result of P307 step, and brings the lexical arrangement vector into the English sentence pair lexical arrangement similarity calculation formula (4);

[0051] P309 ends. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is the overall processing flowchart of the method of the present application;

[0053] Figure 2 is the English text sentence component dependency similarity checking module processing flowchart of the method of the present application;

[0054] Figure 3 is the English text lexical arrangement similarity checking module processing flowchart of the method of the present application; DETAILED DESCRIPTION

[0055] The specific embodiment of the English text sentence component dependency and lexical arrangement similarity checking method of the present application is divided into the following steps:

[0056] The first step: execute the "English text sentence component dependency similarity checking module"

[0057] The English text sentence component dependency similarity checking module in the embodiment of the present application first performs the sentence processing, deletes the special characters and the like to the English text to be checked, then constructs the core word triple according to the core word and the modifier word in the sentence dependency relationship in the English text, and the dependency relationship between the two words, and gets the triple vector by taking the core word and the modifier word in the triple, finally gets the triple similarity by calculation, and gets the English sentence component dependency similarity according to the triple similarity. Finally, the English sentence component dependency similarity and the sentence index information are stored in the file. The input topic of the English text in the embodiment of the present application is "Importance of Education", and the English text is shown as follows:

[0058] English text to be checked

[0059] As a school student, I know that education is the cornerstone of personal growth and social progress. Firstly, it equips us with the knowledge and skills necessary to thrive in a competitive society. From specialized courses to real-life lessons, education prepares us to face challenges with confidence and competence.

[0060] Secondly, education also develops critical thinking and analytical skills. It encourages us to question assumptions, challenge norms and explore new ideas. Through education, we develop the ability to think independently and make informed decisions that contribute positively to our personal and professional lives.

[0061] Finally, education is a key way to cultivate talents, providing strong support for the country's economic, scientific, technological and cultural development. By raising the people's level of education, China is able to cultivate more high-quality talents, it promotes the overall progress and civilized development of Chinese society.

[0062] In short, education is not just about acquiring knowledge, but about enabling individuals to realize their full potential and make a contribution to their country. We must actively pursue educational opportunities and recognize the transformative power of education.

[0063] Original English text

[0064] Education is very important in today's society. Firstly, education is the knowledge and skills necessary for us to thrive in a competitive society. Education prepares us to face challenges with confidence and competence at all times, from professional courses to life lessons.

[0065] In addition, education equips us with critical thinking and analytical skills. Through education, we can be equipped to think independently and make decisions that contribute positively to our personal and professional lives.

[0066] In another way, education is an important support for China's development, and it can provide strong support for the country's economic development, scientific and technological level and cultural development. The long-term prosperity of the country and the well-being of its people can be realized only through improving the quality of education.

[0067] In conclusion, education does not only enable individuals to realize their full potential and contribute to national development. Therefore, as a student it is important to recognize the importance of education for our personal and national development.

[0068] (1) For sentence processing of the English text, the results are as follows:

[0069] The English text to be checked

[0070] 1 - [As a school student, I know that education is the cornerstone of personal growth and social progress.]

[0071] 2 - [Firstly, it equips us with the knowledge and skills necessary to thrive in a competitive society.]

[0072] 3-[From specialized courses to real-life lessons,education preparesus to face challenges with confidence and competence.]

[0073] 4-[Secondly,education also develops critical thinking and analyticalskills.]5-[It encourages us to question assumptions,challenge norms andexplore new ideas.]

[0074] 6-[Through education,we develop the ability to think independentlyand make informed decisions that contribute positively to our personal andprofessional lives.]

[0075] 7-[Finally,education is a key way to cultivate talents,providingstrong support for the country's economic,scientific,technological andcultural development.]

[0076] 8-[By raising the people's level of education,China is able tocultivate more high-quality talents,it promotes the overall progress andcivilized development of Chinese society.]

[0077] 9 - [简而言之,教育不仅仅是获取知识,更是让个人能够充分发挥潜力并为国家做出贡献。]

[0078] Original English text

[0079] 1 - [教育在当今社会非常重要。]

[0080] 2 - [首先,教育是我们在竞争激烈的社会中茁壮成长所需的知识和技能。]

[0081] 3 - [教育使我们随时能够自信且有能力面对各种挑战,从专业课程到生活经验。]

[0082] 4 - [此外,教育还使我们具备批判性思维和分析能力。]

[0083] 5 - 通过教育,我们能够具备独立思考的能力,并做出对我们的个人生活和职业生活有积极贡献的决策。6 - 另一方面,教育是中国发展的重要支撑,它能为国家的经济发展、科技水平和文化发展提供有力支持。

[0084] 7 - 只有通过提高教育质量,才能实现国家的长期繁荣和人民的幸福安康。

[0085] 8 - 总之,教育不仅能使个人实现自身的全部潜力,还能为国家发展做出贡献。

[0086] 9 - 因此,作为一名学生,认识到教育对我们个人和国家发展的重要性是很重要的。

[0087] (2) Triplet construction is performed on the sentences in the two English texts, and the results are as follows:

[0088] The English text to be checked

[0089] Sentence 1 (student, As, case), (student, a, det), (student, school, compound), (know, student, obl: as), (know, I, nsubj), (ROOT, know, root), (cornerstone, that, mark), (cornerstone, education, nsubj), (cornerstone, is, cop), (cornerstone, the, det), (know, cornerstone, ccomp), (growth, of, case), (growth, personal, amod), (cornerstone, growth, nmod: of), (progress, and, cc), (progress, social, amod), (cornerstone, progress, nmod: of), (growth, progress, conj: and)

[0090] ……

[0091] Sentence 10 (pursue, We, nsubj), (recognize, We, nsubj), (pursue, must, aux),

[0092] (pursue, actively, advmod), (ROOT, pursue, root), (opportunities,

[0093] educational, amod), (pursue, opportunities, obj), (recognize, and, cc),

[0094] (pursue, recognize, conj: and), (power, the, det), (power, transformative,

[0095] amod), (recognize, power, obj), (education, of, case), (power, education, nmod: of)

[0096] Original English text sentence 1 (important,Education,nsubj),(important,is,cop),(important,very,advmod),(ROOT,important,root),(society,in,case),(society,today,nmod:poss),(today,'s,case),(important,society,obl:in)

[0097] ...

[0098] Sentence 9(important,Therefore,advmod),(student,as,case),(student,a,det),(important,student,obl:as),(important,it,nsubj),(recognize,it,ns ubj:xsubj),(important,is,cop),(ROOT,important,root),(recognize,to,mark),(important,recognize,xcomp),(importance,the,det),(recognize ,importance,obj),(education,of,case),(importance,education,nmod:of),(development,for,case),(development,our,nmod:poss),(development,personal,compound),(national,and,cc),(personal,national,conj:and),(development,national,compound),(recognize,development,obl:for)

[0099] (3) Vectorize the core words and modifiers of the triplet in the English text. The word vectors of the core words and modifiers are 300-dimensional. The results are shown below:

[0100] English text to be checked

[0101] The word vector of the core word in the first triplet of sentence 1: [-0.17354 -0.24319 0.053452 -0.26109 0.38268 -0.29653 0.083852 -0.32169 0.15131 -0.9557 -0.066864 0.6069 0.76895 -0.03473 0.22002 0.054201 0.60046 0.21231 -0.19465 -0.42065 -0.10255 0.13275 -0.21647 0.19864 -0.40713 0.053763 0.12418 0.15331] 0.028677 -0.19328 0.10747 -0.57035 -0.06532 0.36929 -0.91206 -0.40628 -0.15784 -0.04464 -0.087597 0.090843 0.53778 -0.56325 -0.42146 0.24712 -0.102 -0.045638 -0.060849 0.75258 -0.20409 -0.61832 0.27005 -0.067166 0.0018298 0.1711 0.1786 -0.13522 0.42749 -0.040237 -0.11687 -0.36497 0.30304 0.1239 -0.0042274 0.47082 -0.60705 -0.2746 0.17079 -0.14009 -0.27486 0.19345 -0.22945 -0.074551 -0.28176 0.17424 -0.29497 0.23893 0.029792 0.14545 0.18294 0.15164 0.064472 -0.3498 0.051402 -0.018188 0.041326 -0.1018 0.039224 -0.28511 -0.61756 0.094616 -0.259730.014945 -0.17069 -0.084753 0.7178 0.11422 0.28718 0.077269 -0.16699 0.045772-0.359 -0.21323 0.14497-0.16233 0.38281 -0.022212 0.47206 0.20043 0.27197 -0.28235 -0.50042-0.29787 0.59838 0.037282 0.18367 0.18232 -0.029056 -0.0264040.05982 0.146 -0.0030843 1.1628 -0.10191 0.38915 0.43375 -0.14014 0.13268 -0.63488 0.51159 -0.17917 -0.35081 -0.052044 -0.1112 -0.70654 0.376460.052922-0.42105 0.29974 -0.23137 0.29782 0.60306 0.26989 0.048716 -0.252380.35802 0.11701 -0.12656 -0.20669 0.25375 0.096585 0.89743 0.069477 0.704490.6464 0.19253 -0.018205 -0.09417 0.12029 -0.51626 -0.57525 0.29842 -0.167590.1996 -0.4289 -0.40478 -0.68878 -0.39256 0.23887 0.33337 0.13379 0.39734-0.17772 -0.74419 0.049038 0.21368 0.052148 -0.19793 -0.60364 0.13647 0.285070.22165 -0.72391 -0.39423 -0.21487 0.077877 0.18741 -0.69936-0.028105 0.329320.57378 -0.2227 -0.026564 -0.29643 -0.12938 -0.11859 0.51787 -0.00548170.0054538 -0.087312 -0.44221 0.41763 0.075215 -0.10789 0.11641 -0.26596 -0.59762 -0.28858 0.43134 0.14175 0.078326 0.653790.015019 0.0082632 -0.087615-0.23133 0.15133 0.042002 -0.10898 0.19038-0.1205 -0.32398 0.72856 -0.242510.2977 -0.0034756 0.42557 -0.29279-0.71382 -0.39535 0.38058 0.21173 0.0617820.077676 -0.45748 0.14287 0.18937 0.203 0.33633 0.053148 -0.16261 0.956030.6669 -0.24884 -0.18995 -0.9227 0.21424 -0.0027574 0.17988 -0.28669 0.283010.69782 -0.033009 -0.33567-0.10929 0.74921 0.011132 0.2532 0.37947 -0.18872 -0.45535 -0.46355 -0.178110.48092 -0.04634 -0.2911 -0.21017 -0.18733 0.51647 -0.53055 0.76003-0.31745 0.064124 0.19352 -0.30535 0.12628 -0.22487 -2.1313 -0.26715 0.27908 0.3218 -0.22084 -0.029111 0.099611 -0.20906 0.0014877 0.77484-0.21874 0.4923 -0.32626 -0.22296 0.20883 -0.55722 -0.21543 -0.12474 0.504950.081237 0.53363 0.040683 -0.6322 0.10496].

[0102] The word vectors of the modifiers in the first triplet of sentence 1 are: [-0.056826 0.23863 0.44515 -0.014863 0.1713 -0.10457 0.04575 0.32209 -0.28611 -1.7106 0.22951 0.22666 -0.15038 0.048056 0.33896 0.24517 -0.57485 -0.25071 -0.0033446 -0.061135 -0.005179 0.26721 0.065713 0.2751 -0.11415 0.076964 -0.11396 -0.17367] 0.2507 0.379 -0.10846 0.086291 -0.15533 0.25446 -0.94704 0.10617 0.084343 0.22894 -0.18593 -0.091148 -0.12637 -0.0066005 -0.14456 0.042843 0.18968 0.067754 -0.019222 0.34123 -0.078346 0.12396 0.080519 -0.15997 0.29014 0.03687 -0.036792 -0.0076505 -0.21335 0.17402 0.32853 -0.013976 0.23121 0.44286 0.54756 0.031942 0.050643 -0.2723 0.11318 0.22192 0.24858 -0.0076112 -0.20666 0.17354 -0.10419 0.056558 -0.037414 -0.1826 0.41065 0.6852 -0.18628 0.13912 -0.048764 -0.069608 -0.064254 -0.023668 0.30091 0.083493 -0.39237 0.46004 0.202340.01315 -0.20761 -0.0022161 0.051497 -0.22715 -0.50697 0.23306 -0.3201 -0.014049 0.21677 -0.17386 0.20849 0.050146 0.11978 0.25419 0.37301 0.14686 -0.23652 0.52242 0.069126 0.17529 -0.002298 0.12054 -0.18213 -0.1632 0.243380.30967 0.35451 -0.34301 0.16931 -0.18414 0.11462-0.23398 -0.13362 0.285480.042403 0.20324 -0.15008 0.1497 0.35743 -0.14366 -0.14729 0.29442 0.36739 -0.019627 0.021403 -0.30437 0.15316 0.017475 0.11758 0.46037 0.24902 0.225150.19306 0.51874 -0.24188 0.0067025 0.510480.16012 -0.52281 -0.24828 0.38047 -0.12651 0.0043307 0.15807 0.18793-0.15368 -0.27258 -0.14053 0.36902 -0.0535790.19715 -0.28491 0.26238-0.0011838 -0.22213 0.039658 -0.32987 0.22877 0.474820.37566 0.37278 0.044077 -0.5171 0.38193 -0.033861 -0.28934 -0.140130.0096003 -0.42021 0.6281 0.256 0.31528 0.28145 0.17193 0.13255 -0.1203 -0.47558 -0.056726 0.10113 0.22485 0.17682 0.46901 -0.0721 -0.19534 -0.19161 -0.21283 -0.13168 -0.26071 0.19202 -0.1396 1.0061 -0.046612 0.062724 -0.0336780.10796 0.13363 0.089588 -0.06322 0.038797 0.16555 0.23608 -0.049322 0.15457-0.12405 0.2511 0.21012 0.013695 0.10874 -0.16885 -0.12889 0.094318 0.10308-0.33216 0.11497 0.10646 0.32881 0.035044 0.20747 -0.30386 -4.4732E-4-0.0483790.0254 -0.039434 0.047665 0.12588 0.37809 0.31197 -0.010997-0.45973 0.41432 -0.18112 0.12212 -0.10869 0.176 -0.55822 0.14679 0.67731-0.34697 0.0676240.14096 0.44492 -0.13261 -0.32253 -0.13906 0.4692 -0.3191 -0.28152 -0.0532590.11532 0.22467 -0.168 -0.41067 0.38376 0.25146 -0.27562 -0.2707 -0.33162 -0.0064134 0.19679 0.13543 0.02376 0.026737 -0.058935 0.050393 -0.0695840.041622 -1.9014 -0.34915 0.38343 0.16815 -0.23634 0.055377 0.15216 0.165160.25894 0.35041 3.3606E-4 0.26671 -0.05825 0.020595 -0.037092 0.02902 -0.19494 -0.19114 -0.022407 0.27512 -0.31988-0.31911 -0.2283 0.15058].

[0103] ……

[0104] The word vector of the core word in the first triplet of the original English text sentence 1: [0.23742 0.10227 0.020129 -0.4598 0.38031 -0.02312 -0.41138 0.17419 -0.45459 -2.1113 -0.18164 -0.30143 -0.13739 0.36605 -0.12323 -0.53453 -0.50244 -0.30108 -0.21171 0.026677 -0.11552 0.14041 0.27622 0.15262 -0.33663 -0.25601 -0.35266 -0.23018 -0.19201] 0.31146 0.42691 0.46049 -1.0289 0.16542 -0.74515 0.41644 0.36312 0.37067 -0.23914 -0.38753 0.29871 -0.55714 -0.13163 -0.34564 -0.23977 0.36676 0.2849 0.4954 -0.28397 0.60126 0.19726 0.14862 -0.13933 -0.1375 -0.3742 0.34422 -0.27666 0.048394 0.47699 0.41375 0.35223 -0.069964 0.27557 0.11858 0.13685 0.28332 -0.028147 0.40988 -0.17754 0.25614 -0.38936 0.074637 -0.23105 -0.019972 -0.13213 -0.24211 -0.0065713 0.12045 -0.15536 ​​-0.15892 -0.515 -0.26252 0.28131 -0.21372 0.33389 0.54999 -0.11584 0.1017 -0.22609 0.11208 -0.00314 0.078501 -0.37652 -0.031265 0.2147 0.037163 0.031461 -0.47669 0.10733 0.14666 0.017993 0.033742 -0.049264 -0.19895 0.069923 -0.14071 -0.094304 0.15721 -0.099324 0.19255 0.059596 -0.046276 -0.21658 -0.096115 -0.25588 -0.040946 0.52519 -0.019395 0.14144 -0.0096384 -0.12283 0.0294480.24035 -0.23854 0.094082 -0.033102 0.060435 0.10865 -0.11544 -0.196490.49286 -0.24943 0.045552 -0.28061 -0.31221 -0.36973 0.33045 -0.18573 0.392320.47127 -0.066649 -0.14111 0.50598 -0.1716 -0.13737 -0.22247 0.20968 0.33599-0.75281 0.26426 -0.2187 0.10926 -0.20494 -0.12364 0.16171-0.19322 0.10140.049202 0.043313 -0.10402 -0.11171 0.092624 0.1933-0.24142 -0.0895860.026791 0.084296 -0.0031712 0.10945 1.001 0.46118-0.49048 -0.61932 0.211880.52683 -0.033891 0.04489 -0.21321 -0.46244-0.024046 -0.049367 0.0307950.048561 -0.15195 0.099095 0.16178 -0.36694 0.16123 -0.33765 0.0982240.043577 0.27112 0.39864 0.12706 0.51089 -0.42379 0.3467 -0.33506 0.046156 -0.34733 0.7826 -0.074445 -0.28826 -0.40227 0.17408 0.13641 0.14192 -0.123510.037849 0.28567 0.05465 -0.014481 0.37123 0.2626 0.029205 0.11122 -0.187240.25086 0.38875 0.21976 -0.12042 -0.50688 -0.19816 0.30553 0.3486 0.046491 -0.17698 0.53191 -0.2653 0.21164 -0.234710.1755 0.20213 0.4005 0.0561060.41016 0.32028 -0.11679 0.07194 0.19591-0.28286 0.49895 0.56229 -0.047331 -0.88056 0.15936 -0.29381 0.48074 0.12642 -0.11697 -0.040651 -0.47003 -0.32601-0.15169 -0.0016554 0.20392 -0.083253 -0.32426 0.050106 -0.049847 0.11416 -0.13306 0.61666 0.10717 -0.08601-0.073811 -0.44645 -0.076805 0.35575 0.173720.41841 0.03138 -0.061756 0.42202 -0.0020804 0.017583 -1.8913 -0.156420.94532 -0.08332 -0.3585 0.10848 0.18192 -0.29692 0.018156 -0.29673 0.335230.20154 0.093419 0.131430.062971 -0.149 -0.013108 5.63E-4 -0.053977 0.17796 -0.23668 -0.40023-0.58028 0.16608].

[0105] The word vectors of the modifiers in the first triplet of sentence 1 are: [-0.15599 -0.48863 -0.095793 0.33088 -0.058741 0.19546 0.14079 -0.23351 -0.22091 -1.809 0.23999 0.23155 -0.2614 0.58763 0.14984 0.039441 0.26857 -0.35678 0.33978 -0.086884 -0.31331 0.083792 -0.45803 0.48424 -0.014717 0.53933 0.088042 0.13115] 0.18257 0.107 0.48577 -0.029373 0.18012 0.47554 -0.87488 0.39149 0.53745 -0.0063546 0.060698 -0.46311 0.55727 -0.58281 -0.20869 -0.45452 -0.13888 -0.083076 -0.45609 0.88219 0.18462 -0.84753 -0.2523 0.2111 0.037397 -0.36724 0.10139 -0.11059 0.52339 0.13964 -0.012866 0.2447 0.24951 -0.17792 0.19481 -0.13235 -0.1326 0.0090331 0.11366 -0.22846 -0.55384 -0.19502 -0.69034 -0.60809 0.19883 -0.19056 0.51211 -0.11389 -0.32236 0.4768 -0.068785 0.42006 0.46125 -0.25753 0.24404 -0.24938 -0.088638 -0.050059 0.12597 -0.41955 -0.32934 -0.093084 -0.37752 -0.042846 -0.41694 0.44503 0.40834 -0.078672 -0.19077 0.10634 -0.45438 0.10417 -0.091169 -0.25406 -0.050085 -0.15111 0.19377 -0.48826 -0.32771 -0.013199 -0.12172 -0.082718 0.22685 -1.1618 0.26638 -0.095021 -0.051916 -0.0094577 0.17666 -0.10423 0.11075 0.31282 0.080793 0.9009 -0.149880.040196 0.087837 -0.19295 -0.17704 -0.36456 -0.10088-0.041492 0.053511 -0.58331 0.7997 -0.16211 -0.14938 -0.37602 -0.3355 0.40658 -0.22852 0.567020.45796 -0.17536 0.24725 -0.20521 0.33738 -0.20325 0.46125 -0.096619 0.355030.458 1.1045 -0.06066 0.72095 0.26253 -0.78934 0.19634 -0.18822 -0.59450.1074 -0.21854 0.45644 0.26283 -0.12558 -0.5837-0.35181 -0.77446 -0.057786 -0.45396 0.021407 0.40866 -0.19257 -0.39938-1.1346 0.53859 0.2992 0.18479 -0.32065 -0.92896 -0.11041 0.43687 -0.22184 0.17882 -0.29285 -0.21025 0.0199580.11018 -0.28076 0.091363 -0.043574 0.2452 -0.12323 0.24119 0.082295 0.0633770.052041 0.67751 0.058919 0.19593 0.45511 -0.16609 0.24755 0.26625 0.164690.098158 -0.40091 -0.12506 -0.3074 -0.19394 0.1118 0.29548 -0.0029244 -0.37724 -0.50793 -0.37191 0.2317 0.44089 -0.25758 0.27855 0.54572 -0.14578 -0.10627 0.46546 -0.74192-0.21909 0.21883 -0.18607 0.30635 -0.27689 0.160280.15592 1.0438E-4 0.40641 -0.11154 -0.067444 -0.21263 -0.054281 0.223080.1675 -0.10388 0.21529 0.44342 0.0782 0.74798 -0.36624 -1.197 0.660830.10179 -0.27821 0.11598 0.17297 0.061977 0.15777 -2.5323E-4 0.10331 0.18671-0.0088602 0.26155-0.12531 -0.28656 0.095677 -0.21804 -0.41513 0.49224 -0.18266 -0.064651 -0.15015 -0.35053 0.27396 -0.17054 0.45922 -0.54717 -0.079053 0.047745 -0.15922 -0.42948 -0.092585 -1.9926 0.20223 0.88295 -0.11956 -0.054063 -0.45556 -0.29706 -0.34774 0.10337 0.48005 -0.10413 0.26968 0.04454 -0.12409 -0.30056 -0.086034 -0.33568 0.018582 0.14153 0.50129 -0.24236 -0.71039 -0.44195 0.032832].

[0106] ...

[0107] (4) The triplet similarity is finally obtained by calculating the similarity of core words, modifiers, and dependencies. The results are shown below:

[0108] The similarity matrix between each triplet of sentence 1 and each triplet of sentence 2 in the first sentence pair of the English text is [[0.0 0.057095595 0.0 0.0 0.41404375 0.0 0.28465596 0.0],[0.0 0.00.025545206 0.01911017 0.0 0.0 0.0 0.0],[0.0 0.07467574 0.018738296 0.00.045000542 0.0 0.029802712 0.0],[0.0 0.0 0.0 0.0 0.0 0.0 0.011929799] 0.00.12884846],[0.3348724 0.0 0.0 0.0 0.0 0.019990483 0.0 0.025264373],[0.0 0.00.0 0.7014274 0.0 0.0 0.0 0.0],[0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0],[0.548343240.0 0.0 0.0 0.0 0.014447474 0.0 0.029227814],[0.0 0.548343240.0 0.0 0.03760140.0 0.027060948 0.0],[0.0 0.0 0.030642169 0.028392496 0.0 0.0 0.0 0.0],[0.00.073628105 0.0 0.0 0.021248048 0.0 0.03112107 0.0],[0.0 0.064363725 0.0 0.00.33271304 0.0 0.36920875 0.0],[0.0 0.0 0.11692736 0.0 0.0 0.0 0.0 0.0],[0.030846745 0.0 0.0 0.0 [0.0 0.067208506 0.0 0.020415977],[0.0 0.1587669 0.00.0 0.10124568 0.0 0.095587716 0.0],[0.0 0.0 0.12755804 0.0 0.0 0.0 0.0 0.0],[0.031866513 0.0 0.0 0.0 0.0 0.07407688 0.0 0.024619287],[0.0 0.122906215 0.030746583 0.0 0.031657495 0.0 0.03536927] 0.0]]

[0109] ...

[0110] (5) The component dependency similarity of sentence pairs in the English text was finally obtained based on the triplet similarity, and the results are as follows: 0.21761236, 0.57566303, 0.7988905, 0.6234212, 0.46078095, 0.8264, 0.93661336, 0.3163639, 0.8330549, 0.2316.

[0111] Step 2: Execute the "English Text Vocabulary Alignment Similarity Check Module"

[0112] The steps of the English text vocabulary arrangement similarity checking module are as follows: First, the index file is read, and stop words are removed from the sentence pairs in the text to obtain a joint word set for each sentence pair. Then, a union operation is performed on the two joint word sets to obtain a new joint word set. Based on this joint word set, a vocabulary arrangement vector can be obtained. Finally, the vocabulary arrangement similarity of the sentence pairs is calculated. The analysis results of this implementation are as follows:

[0113] (1) After processing the sentence pairs by removing stop words, the combined word set of the sentence pairs in the text is obtained, as shown below: Combined word set of sentence 1 in text 1 [school,student,,education,cornerstone,personal,growth,social,progress].

[0114] The set of conjunctions for sentence 1 in text 2: [[education, today's, society.]]

[0115] ...

[0116] (2) The union set is obtained by performing a union operation on the union set, as shown below:

[0117] The first sentence contains the following set of conjunctions: [school, student, education, cornerstone, personal, growth, social, progress, today's, society].

[0118] ...

[0119] (3) The word permutation vector is obtained from the joint word set, and the result is shown below:

[0120] The word permutation vector of sentence 1 in text 1 is [1,2,3,4,5,6,7,8,0,0].

[0121] The word permutation vector of sentence 1 in text 2 is [0,0,1,0,0,0,0,0,2,3].

[0122] ...

[0123] (4) The word arrangement similarity can be obtained through the word arrangement vector, and the results are shown below:

[0124] 0.0476, 0.9286, 0.24, 0.857, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0482.

Claims

1. A method for checking the dependency of English text sentence components and the similarity of lexical arrangement, characterized in that: It comprises an English text sentence component dependency similarity checking module and an English text vocabulary arrangement similarity checking module, and its checking method comprises the following processing flow: (1) The processing flow of the English text sentence component dependency similarity checking module is as follows: first, input the English text to be checked, and perform word segmentation and sentence segmentation processing, delete special characters, and convert word capitalization to word lowercase on the English text; second, construct a triple of core words, modifier words and word dependency relationship according to the core words and modifier words in the English text sentence component dependency and the dependency relationship between two words; third, perform vectorization processing on the words in the constructed triple; fourth, generate a core word similarity matrix, a modifier word similarity matrix and a word dependency relationship similarity matrix by calculating the core word similarity, the modifier word similarity and the word dependency relationship similarity in the constructed triple; fifth, generate a triple similarity matrix according to the three generated similarity matrices, and calculate each value in the triple similarity matrix using the triple similarity calculation formula composed of the core words, the modifier words and the word dependency relationship; sixth, traverse each row of the generated triple matrix, select the most similar triple in each row to represent the final triple similarity value; seventh, assign a corresponding weight factor to the word dependency relationship, combine the sentences in the English text in pairs to obtain all sentence pairs of the English text, calculate the component dependency similarity of all sentence pairs to construct a sentence component dependency similarity matrix of the English text, and calculate each value in the sentence component dependency similarity matrix using the English sentence pair component dependency similarity calculation formula; eighth, traverse each row of the sentence component dependency similarity matrix, select the value corresponding to the most similar sentence pair in the position to represent the final component dependency similarity of the sentence pair, and store the coordinates of the position in an index file. (2) The processing flow of the English text vocabulary arrangement similarity checking module is as follows: input the English text to be checked, and perform word segmentation and sentence segmentation processing, delete special characters, and convert word capitalization to word lowercase on the English text; second, traverse the sentence index file to delete stop words in the sentence pairs of the English text and obtain the joint word set corresponding to the two sentences respectively; third, perform set operation on the joint word sets between the two sentences to obtain the joint word set of the two sentences; fourth, calculate the vocabulary arrangement vector corresponding to the sentence pair according to the joint word set, and each value in the vocabulary arrangement vector is the index number of the corresponding position of the word in the joint word set; fifth, calculate the vocabulary arrangement similarity of the English text sentence pair according to the vocabulary arrangement vector of the sentence pair and the English sentence pair vocabulary arrangement similarity calculation formula. The calculation formula of the English text sentence component dependency similarity checking module is defined as follows: The triple similarity calculation formula composed of the core words, the modifier words and the word dependency relationship is as follows: ​ ​ ​ 2. The method of claim 1, wherein the similarity of the constituent dependency and the lexical arrangement of the English text sentence is checked by: ​ ​ ​ In formula (1), the core word similarity refers to the similarity of core words in the English text sentence pair, the modifier similarity refers to the similarity of modifiers in the English text sentence pair, and the word dependency relationship similarity refers to the similarity of the dependency relationship between the core word and the modifier in the English text sentence pair; The component dependency similarity calculation formula of the English sentence pair is as follows: In formula (2), the component dependency similarity of the English sentence pair refers to the similarity of all the triplets in the two English sentences, the English sentence component refers to the core word, the modifier, and the word dependency relationship between the core word and the modifier, m represents the number of triplets in the English text sentence pair, and i represents the i th triplet in the English text sentence pair.

3. The method of claim 2, wherein the similarity of the dependency of the components of the English text sentence and the arrangement of the vocabulary is checked by: The processing steps of the English text sentence component dependency similarity checking module are as follows: ​ P201 starts; P202 reads the English text respectively; P203 performs sentence processing and special character deletion operations on the English text, and outputs the preprocessed English text sentence result; P204 reads the sentence result processed by P203 on the English text respectively; P205 constructs triplets according to the core word and the modifier in the text sentence, and the dependency relationship between the two words; P206 performs vectorization processing on the core word and the modifier in the triplet; P207 calculates the core word similarity and modifier similarity matrix according to the output result of P206, and then calculates the word dependency relationship similarity matrix; P208 obtains the triplet similarity according to the triplet similarity calculation formula (1) composed of the core word, the modifier, and the word dependency relationship according to the output result of P207, and obtains the triplet matrix; P209 traverses each row of the triplet matrix according to the output result of P208, selects the most similar triplet in each row, and represents the final triplet similarity value; P210 assigns a corresponding weight factor to the word dependency relationship in the triplet; P211 combines the sentences in two texts to obtain a plurality of sentence pairs; P212 obtains the component dependency similarity of the sentence pair according to the English sentence pair component dependency similarity calculation formula (2); P213 obtains the component dependency similarity matrix of all sentence pairs combined in the text according to the output result of P212; P214 traverses each row of the component dependency similarity matrix according to the output result of P213, selects the value corresponding to the most similar sentence pair in the position, represents the final component dependency similarity of the sentence pair, and stores the position coordinates in the index file; P215 stores the position coordinates corresponding to the most similar sentence pair in the sentence index file according to the output result of P214; P216 ends. The calculation formula of the English text sentence component dependency and vocabulary arrangement similarity checking module is defined as follows: The English sentence vocabulary arrangement vector calculation formula is as follows: The English text sentence pair vocabulary arrangement similarity calculation formula is as follows: ​ ​ ​ ​ 4. The method for checking the dependency of sentence components and similarity of word arrangement in English text according to claim 3, characterized in that: ​ ​ English sentence vocabulary arrangement vector = [x1, x2,..., x n ] (3) In formula (3) x n represents the value of each dimension of the vocabulary arrangement vector of the English sentence, the value of each dimension of the vocabulary arrangement vector of the English sentence, represents the index number of the corresponding position of the word in the joint vocabulary set, and the joint vocabulary set is obtained by performing a set operation on the joint vocabulary sets between two sentences; ​ In formula (4), the symbol "||" represents the square root of the square sum of the difference in vector dimension of the sentences, the lexical arrangement vector of the first sentence represents the lexical arrangement vector of the first sentence in the sentence pair of the English text, the lexical arrangement vector of the second sentence represents the lexical arrangement vector of the second sentence in the sentence pair of the English text, and the lexical arrangement vector is obtained from formula (3).

5. The method of claim 4, wherein the checking of the dependency of the components of the English text sentence on the arrangement of the vocabulary is characterized by: The processing steps of the English text sentence component dependency and lexical arrangement similarity checking module are as follows: P301 starts; P302 reads the English text respectively; P303 performs sentence processing, special character deletion, and word capitalization conversion to word lowercase operation on the English text, outputs the preprocessed English text sentence processing result; P304 traverses the sentence index file to obtain a sentence pair; P305 performs stop word deletion processing on the sentence pair in the English text respectively to obtain two joint word sets; P306 performs set union on the two joint word sets to obtain a joint word set; P307 calculates the two lexical arrangement vectors respectively according to the output result of step P306 by using the lexical arrangement similarity calculation formula (3); P308 inputs the lexical arrangement vector into the English sentence pair lexical arrangement similarity calculation formula (4) according to the output result of step P307, and finally obtains the English text sentence pair lexical arrangement similarity; P309 ends. ​

Images