Input method evaluation method, related device, equipment and storage medium
By calculating the semantic similarity between candidate statements and input statements in the input method and fusion, the problems of low efficiency and poor objectivity of the entire sentence evaluation of the input method in the prior art are solved, and efficient and objective automatic evaluation is achieved.
Patent Information
- Application Number
- CN202411410789.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-02-25
AI Technical Summary
The prior art is difficult to effectively evaluate the efficiency and objectivity of the entire sentence association of the input method, which mainly relies on manual evaluation, and is inefficient and difficult to obtain objective results.
By obtaining several input statements of the target input method and candidate statements obtained by associating the target statement, the semantic similarity between the candidate statement and the target statement and the input statement are calculated, the first weight factor and the second weight factor are obtained, and the first weight factor is obtained, and the second weight factor is obtained, and the It is fused to generate evaluation results.
It realizes automatic evaluation of the entire sentence association of the input method, improves the evaluation efficiency and objectivity, and can comprehensively evaluate the association effect of candidate sentences in both local and global scope.
Smart Images
Figure CN119512901A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of input methods, and in particular to an input method evaluation method and related devices, equipment and storage media. Background Art
[0002] With the development of artificial intelligence technology, integrating functions such as whole-sentence association into input methods to improve input efficiency has gradually become popular.
[0003] At present, the whole sentence association effect is mainly evaluated manually, which makes it difficult to give an objective and comprehensive evaluation result for the whole sentence association, and the evaluation efficiency is not high.
[0004] Therefore, how to improve the evaluation efficiency and objectivity of whole sentence association of input methods has become an urgent problem to be solved. Summary of the invention
[0005] The main technical problem solved by the present application is to provide an input method evaluation method and related devices, equipment and storage media, which can improve the evaluation efficiency and objectivity of the input method whole sentence association.
[0006] In order to solve the above-mentioned technical problems, the first aspect of the present application provides an input method evaluation method, which includes: obtaining a number of input sentences of the target input method and candidate sentences associated with the target sentences; wherein the target sentence includes at least the last input sentence; based on the candidate sentences and the target sentence, obtaining a first weight factor of the importance of the target sentence when evaluating the candidate sentences, and based on the candidate sentence and a number of input sentences, obtaining a second weight factor of the importance of each input sentence when evaluating the candidate sentences; based on the first weight factor and the second weight factor of the importance of the target sentence when evaluating the candidate sentences, obtaining a fused weight factor of the importance of the target sentence when evaluating the candidate sentences; based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, obtaining an evaluation result of the candidate sentence associated with the target input method.
[0007] In order to solve the above-mentioned technical problems, the second aspect of the present application provides an input method evaluation device, which includes an acquisition module, a processing module, a fusion module and an evaluation module; the acquisition module is used to acquire several input sentences of the target input method and candidate sentences associated with the target sentence; wherein the target sentence includes at least the last input sentence; the processing module is used to obtain a first weight factor of the importance of the target sentence when evaluating the candidate sentence based on the candidate sentence and the target sentence, and to obtain a second weight factor of the importance of each input sentence when evaluating the candidate sentence based on the candidate sentence and the several input sentences; the fusion module is used to fuse the first weight factor and the second weight factor of the importance of the target sentence when evaluating the candidate sentence, and obtain a fused weight factor of the importance of the target sentence when evaluating the candidate sentence; the evaluation module is used to obtain the evaluation result of the candidate sentence associated with the target input method based on the fused weight factor of the input sentence as the target sentence and the second weight factor of the input sentence that is not the target sentence.
[0008] In order to solve the above technical problem, the third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the processor is used to execute program instructions stored in the memory to implement the input method evaluation method in the above first aspect.
[0009] In order to solve the above technical problem, the fourth aspect of the present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the input method evaluation method in the above first aspect is implemented.
[0010] The above scheme obtains several input sentences of the target input method and candidate sentences obtained by association with the target sentences; wherein the target sentences at least include the last input sentence; based on the candidate sentences and the target sentences, a first weight factor of the importance of the target sentence when evaluating the candidate sentences is obtained, and based on the candidate sentences and several input sentences, a second weight factor of the importance of each input sentence when evaluating the candidate sentences is obtained; based on the first weight factor and the second weight factor of the importance of the target sentence when evaluating the candidate sentences, a fused weight factor of the importance of the target sentence when evaluating the candidate sentences is obtained; based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, an evaluation result of the candidate sentence associated with the target input method is obtained. On the one hand, by performing data processing on several input sentences and candidate sentences of the target input method to obtain the evaluation results of the candidate sentences associated with the target input method, it is possible to realize automatic evaluation of the target input method and improve the evaluation efficiency of the whole sentence association of the input method; on the other hand, since the target sentence includes at least the last input sentence, the candidate sentences can be evaluated in a local scope through the input sentence as the target sentence, and the candidate sentences can be evaluated in a global scope through each input sentence. Combining the local evaluation with the global evaluation, the whole sentence association of the target input method can be evaluated as comprehensively as possible, and the objectivity of the whole sentence association of the target input method can be improved. Therefore, the evaluation efficiency and objectivity of the whole sentence association of the input method can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 A flowchart of an embodiment of an input method evaluation method of the present application; Figure 2 This is an application diagram of an embodiment of the target input method of the present application; Figure 3 A schematic diagram of a framework of an embodiment of an evaluation device for an input method of the present application; Figure 4 A schematic diagram of a framework of an embodiment of an electronic device of the present application; Figure 5 A schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0012] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0013] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0014] The term "multiple" herein means two or more than two. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, and may mean including any one or more elements selected from the set consisting of A, B, and C.
[0015] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of an input method evaluation method of the present application, specifically, comprising the following steps: Step S11: obtaining a number of input sentences of the target input method and candidate sentences associated with the target sentence; wherein the target sentence at least includes the last input sentence.
[0016] In an implementation scenario, the target input method may include, but is not limited to: Chinese Pinyin input method, Wubi input method, or input method of other languages, etc. The language and form of the target input method are not specifically limited here.
[0017] In one implementation scenario, see Figure 2 , Figure 2 The application diagram of the target input method of the present application is shown in the device interface of the target input method, including an input box for accommodating inputted sentences and a pop-up window for displaying candidate sentences. The device of the target input method may include but is not limited to: a smart phone, a tablet computer, a desktop computer, a learning machine, a translation pen and other devices. Exemplarily, the inputted sentence in the input box may be "The weather is good today", and "The weather is good today" may be used as the target sentence to associate candidate sentences such as "The sun is out", "It's sunny and bright", "It's neither cold nor hot", etc., and when the selection instruction of "The sun is out" is detected, "The sun is out" is imported into the input box as a new target sentence, and then new candidate sentences such as "It's hot", "It's not cold", "It's not cold anymore" and the like may be associated with "The sun is out".
[0018] In an implementation scenario, the target sentence may include a last input sentence among several input sentences, or may include a plurality of last input sentences among several input sentences. The number of target sentences is not limited here.
[0019] Step S12: Based on the candidate sentence and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentence is obtained, and based on the candidate sentence and several input sentences, a second weight factor of the importance of each input sentence in evaluating the candidate sentence is obtained.
[0020] In an implementation scenario, before obtaining the first weight factor and the second weight factor, the sentence embedding features of the input sentence and the candidate sentence can be obtained through a Sentence-BERT model, so as to convert the text into a high-dimensional vector representation, thereby obtaining the first weight factor and the second weight factor in a manner that evaluates the semantic relevance between sentences using the relationship between the vectors, which can improve the efficiency of obtaining the first weight factor or the second weight factor. It should be noted that the specific process of obtaining sentence embedding features using the Sentence-BERT model can refer to the relevant technology, which will not be repeated here. Of course, extracting sentence embedding features through the Sentence-BERT model is only a possible example in the actual application process, and the specific method of extracting sentence embedding features is not limited here. For example, sentence embedding features can also be extracted through models such as SimCSE, Transformer Encoder, etc., and examples are not given one by one here.
[0021] For example, the input sentences include "The weather is nice today" and "The sun is out", where "The sun is out" is the target sentence and the candidate sentences include "It's hot" and "It's not cold". The sentence embedding features of each input sentence and candidate sentence can be expressed as: ; ; ; .
[0022] Of course, the above examples are only possible scenarios involving two input sentences and two candidate sentences. In actual applications, the specific number of input sentences can also be three, four, five, etc., and the specific number of candidate sentences can also be three, four, five, etc. The specific number and specific content of the input sentences and candidate sentences are not limited here. When the specific number and specific content of the input sentences and candidate sentences are other situations, the method of obtaining the embedding features of each sentence can be deduced by analogy, and no examples are given here one by one.
[0023] In one implementation scenario, in order to obtain the first weight factor, at least the first semantic similarity between the target sentence and each candidate sentence can be statistically analyzed to obtain a first similarity statistical value, so that the first semantic similarity corresponding to the candidate sentence and the first similarity statistical value can be used to obtain a first weight factor of the importance of the target sentence in evaluating the candidate sentence. Therefore, based on the semantic similarity information between the target sentence and the candidate sentence, the contextual information of the candidate sentence in a local range can be taken into account, which can improve the accuracy of the input method evaluation compared to only considering a single candidate sentence.
[0024] In a specific implementation scenario, based on the statistics of the first semantic similarity between the target sentence and each candidate sentence, the first semantic similarity of the target sentence itself can be further counted to obtain the first similarity statistic, which helps to improve the accuracy of obtaining the first weight factor. As a possible example, each first semantic similarity can be first mapped to a numerical range not less than 0 (for example, it can be mapped by an exponential function with a base of e), and then the first semantic similarities after mapping are summed as the first similarity statistic. Of course, the above example is only a possible calculation method of the first similarity statistic in actual application, and the calculation method of the first similarity statistic is not limited here.
[0025] In a specific implementation scenario, a fixed-length context window containing the target sentence and the candidate sentences can be defined, and the first semantic similarities between the sentences and the sentences themselves can be obtained in the context window by combining the local attention mechanism. Then, the first semantic similarities are counted to obtain the first weight factor of each sentence in the context window relative to itself and other sentences. The first weight factor The calculation formula can be expressed as:
[0026] in, , Respectively represent the sentence embedding features of the i-th and j-th sentences in the context window, represents the set of sentences in the context window, cos() represents the cosine similarity function, exp() represents the exponential function with base e, represents the first similarity statistic, Represents the first semantic similarity between the i-th sentence and the j-th sentence after mapping. Therefore, the context window is used to capture the contextual information before and after the target sentence, and the local attention mechanism is used to capture the importance of the target sentence in the local scope when evaluating the candidate sentence. Thus, the first semantic similarity between sentences is obtained in the context window of fixed length through the local attention mechanism, which can more accurately evaluate the correlation between sentences, especially reflect the correlation between sentences in the current local scope.
[0027] As a possible example, the sentence embedding features of the input sentence and the candidate sentence are obtained as mentioned above: , , , Based on this, we define a context window that can accommodate the length of two statements: , so that the target sentence can be obtained through the local attention mechanism Relative to the candidate sentence The first weight factor :
[0028] And, we can get the target sentence Relative to the candidate sentence The first weight factor :
[0029] in, Indicates that a statement has been entered The first semantic similarity of itself, Represents the target sentence With candidate sentences The first semantic similarity between Represents the target sentence With candidate sentences The first semantic similarity between them, and the denominator represents the first similarity statistical value.
[0030] As another possible example, when there are two or more target sentences, a context window that can accommodate all target sentences and one of the candidate sentences can be defined accordingly, so that the first weight factor of each target sentence when evaluating the candidate sentences in the same context window can be obtained separately.
[0031] In one implementation scenario, in order to obtain the second weight factor, at least the second semantic similarities between the input sentence and other input sentences and each candidate sentence can be counted to obtain a second similarity statistical value, so that the second weight factor of the importance of the input sentence in evaluating the candidate sentence can be obtained according to the second semantic similarity corresponding to the candidate sentence and the second similarity statistical value. Therefore, by obtaining the second weight factor of the importance of all input sentences including the target sentence in evaluating the candidate sentence, the overall structure of the sentence and the semantic relationship between the sentences can be considered from a global perspective, thereby improving the accuracy of the input method evaluation.
[0032] In a specific implementation scenario, on the basis of counting the second semantic similarities between the input sentence and other input sentences and each candidate sentence, the second semantic similarity of the input sentence itself can be further counted to obtain a second similarity statistical value, which helps to improve the accuracy of obtaining the second weight factor.
[0033] In a specific implementation scenario, a context window that can include all input sentences and one of the candidate sentences can be defined, and the second semantic similarities between the sentences and each sentence itself can be obtained in the context window in combination with the global attention mechanism, and each second semantic similarity is counted to obtain the second weight factor of each sentence in the context window relative to itself and other sentences. The calculation formula of the second weight factor can be expressed as:
[0034] in, , They represent the sentence embedding features of the i-th and j-th sentences in the context window, respectively. N represents the total number of sentences in the context window. cos() represents the cosine similarity function. exp() represents the exponential function with base e. represents the second similarity statistic, represents the second semantic similarity between the i-th sentence and the j-th sentence after mapping. Therefore, by obtaining the second semantic similarity between sentences in the context window through the global attention mechanism, the relevance between sentences can be evaluated as comprehensively as possible.
[0035] As a possible example, the sentence embedding features of the input sentence and the candidate sentence are obtained as mentioned above: , , , Based on , the context windows corresponding to each candidate sentence are , so that through the global attention mechanism, the input sentences that are not the target sentences can be obtained Relative to the candidate sentence The second weight factor :
[0036] And get the input sentence that is not the target sentence Relative to the candidate sentence The second weight factor :
[0037] in, Indicates that a statement has been entered The second semantic similarity of itself, Indicates that a statement has been entered With the target sentence The second semantic similarity between Indicates that a statement has been entered With candidate sentences The second semantic similarity between Indicates that a statement has been entered With candidate sentences The second semantic similarity between them, and the denominator represents the second similarity statistical value.
[0038] And get the input sentence as the target sentence Relative to the candidate sentence The second weight factor :
[0039] And get the input sentence as the target sentence Relative to the candidate sentence The second weight factor :
[0040] in, Indicates that a statement has been entered With the target sentence The second semantic similarity between Represents the target sentence The second semantic similarity of itself, Represents the target sentence With candidate sentences The second semantic similarity between Represents the target sentence With candidate sentences The second semantic similarity between them, and the denominator represents the second similarity statistical value.
[0041] Step S13: The first weight factor and the second weight factor based on the importance of the target sentence in evaluating the candidate sentences are fused to obtain a fused weight factor of the importance of the target sentence in evaluating the candidate sentences.
[0042] In one implementation scenario, a fusion weight factor is obtained by obtaining a first weighting coefficient of a first weighting factor and a second weighting coefficient of a second weighting factor, wherein the first weighting coefficient is negatively correlated with the second weighting coefficient, and the first weighting factor is weighted by the first weighting coefficient and the second weighting factor is weighted by the second weighting coefficient. Therefore, by combining the importance of the target sentence when evaluating the candidate sentence in the local scope and the global scope respectively, and setting different weighting coefficients in the local scope and the global scope for fusion, the weighting coefficients of different scopes can be adjusted according to different application scenarios, thereby improving the flexibility of input method evaluation.
[0043] In a specific implementation scenario, the first weighting coefficient is negatively correlated with the second weighting coefficient, and the sum of the first weighting coefficient and the second weighting coefficient is 1. The first weighting coefficient is expressed as In the case of ), thus, the fusion weight factor It can be expressed as:
[0044] As a possible example, =0.5, the first weight factor obtained under the local attention mechanism as mentioned above And the second weight factor obtained under the global attention mechanism Based on this, we can get the target sentence In evaluating candidate sentences The fusion weight factor of the importance :
[0045] As another possible example, =0.5, the first weight factor obtained under the local attention mechanism as mentioned above And the second weight factor obtained under the global attention mechanism Based on this, we can get the target sentence In evaluating candidate sentences The fusion weight factor of the importance :
[0046] Step S14: Based on the fusion weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, the evaluation result of the target input method association candidate sentence is obtained.
[0047] In one implementation scenario, the semantic similarity between the target sentence and the candidate sentence is weighted by a fusion weight factor to obtain a first evaluation result, and the semantic similarity between the input sentence that is not the target sentence and the candidate sentence is weighted by a second weight factor to obtain a second evaluation result, so that based on the fusion of the first evaluation result and the second evaluation result, the evaluation result of the candidate sentence associated with the target input method can be obtained. Therefore, from a horizontal level, based on the first evaluation result of the candidate sentence evaluated as the target sentence and the second evaluation result of the candidate sentence evaluated as the input sentence that is not the target sentence, the semantic relationship between each input sentence and the sentence can be considered as comprehensively as possible; from a vertical level, the candidate sentence evaluated as the target sentence can fuse the semantic information of the local scope and the global scope. Therefore, combining the above two levels can improve the accuracy and objectivity of the input method evaluation.
[0048] In a specific implementation scenario, the evaluation result of the candidate sentences of the target input method association may be in the form of a score, specifically, a MOS score:
[0049] in, represents the fusion weight factor or the second weight factor, represents the cosine similarity between sentence embedding features. As a possible example, the second weight factor is obtained as described above. and fusion weight factor Based on this, we can get candidate sentences MOS scores:
[0050] The second weight factor is obtained as described above. and fusion weight factor Based on this, we can get candidate sentences MOS scores:
[0051] In another specific implementation scenario, the evaluation results of the target input method's candidate association sentences can also be in text form. In this case, the evaluation results can include text describing the effects of the target input method's candidate association sentences, such as "the current candidate sentence has a poor association effect" or "the current candidate sentence has a good association effect" and so on.
[0052] In an implementation scenario, the evaluation results of the candidate sentences associated with the target input method are predicted by the evaluation model, and the evaluation model is trained based on several sample input sentences of the target input method and sample candidate sentences associated with the sample target sentences, the sample target sentences at least include the last sample input sentences, and the sample candidate sentences are marked with real evaluation results, and the training steps of the evaluation model include: processing the sample input sentences and the sample candidate sentences based on the evaluation model to obtain the predicted evaluation results of the sample candidate sentences, and adjusting the network parameters of the evaluation model based on the difference between the predicted evaluation results and the real evaluation results, wherein the network parameters of the evaluation model at least include the network parameters for fusing the first weight factor and the second weight factor. Therefore, obtaining the evaluation results of the candidate sentences associated with the target input method through the evaluation model can further improve the evaluation efficiency of the input method. In addition, in the training process of the evaluation model, by gradually reducing the difference between the predicted evaluation results and the real evaluation results, the final trained evaluation model is obtained, which can improve the accuracy of input method evaluation using the evaluation model. It should be noted that the method for obtaining sample input sentences, sample target sentences, and sample candidate sentences used to train the evaluation model is similar to the method for obtaining the aforementioned input sentences, target sentences, and candidate sentences, and will not be repeated here.
[0053] In a specific implementation scenario, the actual evaluation results can be obtained by annotators based on the sample input sentences and sample candidate sentences.
[0054] In a specific implementation scenario, the network parameters of the evaluation model include a first weighting coefficient of a first weight factor and a second weighting coefficient of a second weight factor, and the first weighting coefficient is negatively correlated with the second weighting coefficient. The first weight factor is weighted by the first weighting coefficient and the second weight factor is weighted by the second weighting coefficient to obtain a fused weight factor, thereby achieving the fusion of the first weight factor and the second weight factor.
[0055] In a specific implementation scenario, the evaluation model may be a natural language model that uses a local attention mechanism and a global attention mechanism, and the natural language model may include but is not limited to: Sentence-BERT model, large language model, etc. Among them, the natural language model is at least used to obtain sentence embedding features of sentences (such as input sentences, candidate sentences, sample input sentences, sample candidate sentences, etc.), the local attention mechanism is used to evaluate the semantic relationship between sentences within a local context, and the global attention mechanism is used to evaluate the semantic relationship between sentences within a global context.
[0056] In one implementation scenario, after obtaining the evaluation results of the candidate sentences associated with the target input method based on the fusion weight factor of the input sentence as the target sentence and the second weight factor of the input sentence that is not the target sentence, and in the case of detecting a selection instruction for any candidate sentence, the selected candidate sentence can be used as a new target sentence, and a new candidate sentence associated by the target input method based on the new target sentence is obtained, and steps S12, S13, and S14 are re-executed in sequence to obtain the evaluation results of the new candidate sentences associated with the target input method. Therefore, by implementing the effect evaluation of the candidate sentences associated with the target input method for multiple rounds, it is possible to implement the automatic evaluation of the text fragments or text paragraphs obtained in combination with the association of the target input method, which helps to further improve the evaluation efficiency of the input method.
[0057] In a specific implementation scenario, the final evaluation result of the target input method can be obtained by taking the mean or weighted average of the evaluation results of the candidate sentences associated with the target input method in each round, which can improve the granularity of the input method evaluation and thus improve the accuracy of the input method evaluation.
[0058] The above scheme obtains several input sentences of the target input method and candidate sentences obtained by association with the target sentences; wherein the target sentences at least include the last input sentence; based on the candidate sentences and the target sentences, a first weight factor of the importance of the target sentence when evaluating the candidate sentences is obtained, and based on the candidate sentences and several input sentences, a second weight factor of the importance of each input sentence when evaluating the candidate sentences is obtained; based on the first weight factor and the second weight factor of the importance of the target sentence when evaluating the candidate sentences, a fused weight factor of the importance of the target sentence when evaluating the candidate sentences is obtained; based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, an evaluation result of the candidate sentence associated with the target input method is obtained. On the one hand, by performing data processing on several input sentences and candidate sentences of the target input method to obtain the evaluation results of the candidate sentences associated with the target input method, it is possible to realize automatic evaluation of the target input method and improve the evaluation efficiency of the whole sentence association of the input method; on the other hand, since the target sentence includes at least the last input sentence, the candidate sentences can be evaluated in a local scope through the input sentence as the target sentence, and the candidate sentences can be evaluated in a global scope through each input sentence. Combining the local evaluation with the global evaluation, the whole sentence association of the target input method can be evaluated as comprehensively as possible, and the objectivity of the whole sentence association of the target input method can be improved. Therefore, the evaluation efficiency and objectivity of the whole sentence association of the input method can be improved.
[0059] See also Figure 3 , Figure 3The present invention is a schematic diagram of a framework of an embodiment of an evaluation device for an input method of the present application, wherein the evaluation device 30 for an input method comprises an acquisition module 31, a processing module 32, a fusion module 33 and an evaluation module 34; the acquisition module 31 is used to acquire a number of input sentences of a target input method and candidate sentences obtained by association with the target sentences; wherein the target sentence at least includes the last input sentence; the processing module 32 is used to obtain a first weight factor of the importance of the target sentence in evaluating the candidate sentences based on the candidate sentences and the target sentence, and to obtain a second weight factor of the importance of each input sentence in evaluating the candidate sentences based on the candidate sentences and the number of input sentences; the fusion module 33 is used to fuse the first weight factor and the second weight factor of the importance of the target sentence in evaluating the candidate sentences to obtain a fused weight factor of the importance of the target sentence in evaluating the candidate sentences; the evaluation module 34 is used to obtain an evaluation result of the candidate sentences associated with the target input method based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence.
[0060] In the above scheme, the input method evaluation device 30 obtains several input sentences of the target input method and candidate sentences associated with the target sentences; wherein the target sentence includes at least the last input sentence; based on the candidate sentences and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentences is obtained, and based on the candidate sentence and several input sentences, a second weight factor of the importance of each input sentence in evaluating the candidate sentences is obtained; based on the first weight factor and the second weight factor of the importance of the target sentence in evaluating the candidate sentences, a fused weight factor of the importance of the target sentence in evaluating the candidate sentences is obtained; based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, an evaluation result of the candidate sentence associated with the target input method is obtained. On the one hand, by performing data processing on several input sentences and candidate sentences of the target input method to obtain the evaluation results of the candidate sentences associated with the target input method, it is possible to realize automatic evaluation of the target input method and improve the evaluation efficiency of the whole sentence association of the input method; on the other hand, since the target sentence includes at least the last input sentence, the candidate sentences can be evaluated in a local scope through the input sentence as the target sentence, and the candidate sentences can be evaluated in a global scope through each input sentence. Combining the local evaluation with the global evaluation, the whole sentence association of the target input method can be evaluated as comprehensively as possible, and the objectivity of the whole sentence association of the target input method can be improved. Therefore, the evaluation efficiency and objectivity of the whole sentence association of the input method can be improved.
[0061] In some disclosed embodiments, the processing module 32 includes a first processing submodule, which is used to obtain a first weight factor of the importance of the target sentence when evaluating the candidate sentences based on the candidate sentences and the target sentence, and is specifically used to: at least perform statistics based on the first semantic similarities between the target sentence and each candidate sentence to obtain a first similarity statistical value; obtain the first weight factor of the importance of the target sentence when evaluating the candidate sentences based on the first semantic similarities corresponding to the candidate sentences and the first similarity statistical value.
[0062] Therefore, based on the semantic similarity information between the target sentence and the candidate sentence, the contextual information of the candidate sentence in a local range can be taken into account, which can improve the accuracy of input method evaluation compared to only considering a single candidate sentence.
[0063] In some disclosed embodiments, the processing module 32 includes a second processing submodule. For any input sentence, the second processing submodule is used to obtain a second weight factor of the importance of each input sentence when evaluating the candidate sentence based on the candidate sentence and several input sentences, and is specifically used to: at least based on the second semantic similarity between the input sentence and other input sentences and each candidate sentence, to obtain a second similarity statistical value; based on the second semantic similarity corresponding to the candidate sentence and the second similarity statistical value, to obtain the second weight factor of the importance of the input sentence when evaluating the candidate sentence.
[0064] Therefore, by obtaining the second weight factor of the importance of all input sentences including the target sentence when evaluating the candidate sentence, the overall structure of the sentence and the semantic relationship between sentences can be considered globally, thereby improving the accuracy of input method evaluation.
[0065] In some disclosed embodiments, the fusion module 33 is specifically used to obtain a first weighting coefficient of a first weighting factor, and to obtain a second weighting coefficient of a second weighting factor; wherein the first weighting coefficient is negatively correlated with the second weighting coefficient; the first weighting factor is weighted based on the first weighting coefficient and the second weighting factor is weighted based on the second weighting coefficient to obtain a fusion weight factor.
[0066] Therefore, by combining the importance of the target sentence when evaluating the candidate sentence in the local scope and the global scope respectively, and setting different weighting coefficients in the local scope and the global scope for fusion, the weighting coefficients of different scopes can be adjusted according to different application scenarios, thereby improving the flexibility of input method evaluation.
[0067] In some disclosed embodiments, the evaluation module 34 is specifically used to weight the semantic similarity between the target sentence and the candidate sentence based on a fusion weight factor to obtain a first evaluation result, and to weight the semantic similarity between the input sentence that is not the target sentence and the candidate sentence based on a second weight factor to obtain a second evaluation result; based on the fusion of the first evaluation result and the second evaluation result, an evaluation result of the target input method association candidate sentence is obtained.
[0068] Therefore, from a horizontal perspective, based on the first evaluation result of the candidate sentence for evaluation of the target sentence and the second evaluation result of the candidate sentence for evaluation of the input sentence that is not the target sentence, the semantic relationship between each input sentence can be considered as comprehensively as possible; from a vertical perspective, the candidate sentence for evaluation of the input sentence that is the target sentence can integrate the semantic information of the local scope and the global scope. Therefore, combining the above two levels can improve the accuracy and objectivity of the input method evaluation.
[0069] In some disclosed embodiments, the evaluation results of the candidate sentences associated with the target input method are predicted by an evaluation model, and the evaluation model is trained based on several sample input sentences of the target input method and sample candidate sentences associated with the sample target sentences, the sample target sentences at least include the last sample input sentence, and the sample candidate sentences are marked with real evaluation results, and the input method evaluation device 30 also includes a training module, and the training module is used to train the evaluation model, specifically for: processing the sample input sentences and the sample candidate sentences based on the evaluation model to obtain predicted evaluation results of the sample candidate sentences; adjusting the network parameters of the evaluation model based on the difference between the predicted evaluation results and the real evaluation results; wherein the network parameters of the evaluation model at least include network parameters for fusing the first weight factor and the second weight factor.
[0070] Therefore, obtaining the evaluation results of the target input method's candidate association sentences through the evaluation model can further improve the input method's evaluation efficiency. In addition, during the training process of the evaluation model, by gradually reducing the difference between the predicted evaluation results and the actual evaluation results, the final trained evaluation model is obtained, which can improve the accuracy of input method evaluation using the evaluation model.
[0071] In some disclosed embodiments, the input method evaluation device 30 also includes a return module, which is used to, after obtaining the evaluation result of the target input method's associated candidate sentences based on the fusion weight factor of the input sentence that serves as the target sentence and the second weight factor of the input sentence that does not serve as the target sentence, in response to detecting a selection instruction for any candidate sentence, use the selected candidate sentence as the new target sentence, obtain a new candidate sentence associated by the target input method based on the new target sentence, and return the step of obtaining the first weight factor of the importance of the target sentence in evaluating the candidate sentence based on the candidate sentence and the target sentence, so as to obtain the evaluation result of the target input method's associated new candidate sentence.
[0072] Therefore, by evaluating the effectiveness of multiple rounds of candidate sentences associated with the target input method, it is possible to automatically evaluate the text fragments or text paragraphs associated with the target input method, which helps to further improve the evaluation efficiency of the input method.
[0073] See also Figure 4 , Figure 4 The electronic device 40 of the present application is a schematic diagram of a framework of an embodiment. The electronic device 40 includes a memory 41 and a processor 42 coupled to each other, and the processor 42 is used to execute program instructions stored in the memory 41 to implement the steps in the embodiment of the evaluation method of any input method described above. Specifically, the electronic device 40 may include but is not limited to: a microcomputer, a server, a laptop computer, a tablet computer, a smart phone, etc., which are not limited here.
[0074] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps of the embodiment of the evaluation method of any of the above-mentioned input methods. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.
[0075] In the above scheme, the electronic device 40 obtains several input sentences of the target input method and candidate sentences associated with the target sentences; wherein the target sentence includes at least the last input sentence; based on the candidate sentences and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentences is obtained, and based on the candidate sentences and several input sentences, a second weight factor of the importance of each input sentence in evaluating the candidate sentences is obtained; based on the first weight factor and the second weight factor of the importance of the target sentence in evaluating the candidate sentences, a fused weight factor of the importance of the target sentence in evaluating the candidate sentences is obtained; based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, an evaluation result of the candidate sentence associated with the target input method is obtained. On the one hand, by performing data processing on several input sentences and candidate sentences of the target input method to obtain the evaluation results of the candidate sentences associated with the target input method, it is possible to realize automatic evaluation of the target input method and improve the evaluation efficiency of the whole sentence association of the input method; on the other hand, since the target sentence includes at least the last input sentence, the candidate sentences can be evaluated in a local scope through the input sentence as the target sentence, and the candidate sentences can be evaluated in a global scope through each input sentence. Combining the local evaluation with the global evaluation, the whole sentence association of the target input method can be evaluated as comprehensively as possible, and the objectivity of the whole sentence association of the target input method can be improved. Therefore, the evaluation efficiency and objectivity of the whole sentence association of the input method can be improved.
[0076] See also Figure 5 , Figure 5 The schematic diagram of the framework of an embodiment of a computer-readable storage medium 50 of the present application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor, and the program instructions 51 are used to implement the steps of the embodiment of the input method evaluation method.
[0077] In the above scheme, the computer-readable storage medium 50 obtains several input sentences of the target input method and candidate sentences associated with the target sentences; wherein the target sentence includes at least the last input sentence; based on the candidate sentences and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentences is obtained, and based on the candidate sentences and several input sentences, a second weight factor of the importance of each input sentence in evaluating the candidate sentences is obtained; based on the first weight factor and the second weight factor of the importance of the target sentence in evaluating the candidate sentences, a fused weight factor of the importance of the target sentence in evaluating the candidate sentences is obtained; based on the fused weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, an evaluation result of the candidate sentence associated with the target input method is obtained. On the one hand, by performing data processing on several input sentences and candidate sentences of the target input method to obtain the evaluation results of the candidate sentences associated with the target input method, it is possible to realize automatic evaluation of the target input method and improve the evaluation efficiency of the whole sentence association of the input method; on the other hand, since the target sentence includes at least the last input sentence, the candidate sentences can be evaluated in a local scope through the input sentence as the target sentence, and the candidate sentences can be evaluated in a global scope through each input sentence. Combining the local evaluation with the global evaluation, the whole sentence association of the target input method can be evaluated as comprehensively as possible, and the objectivity of the whole sentence association of the target input method can be improved. Therefore, the evaluation efficiency and objectivity of the whole sentence association of the input method can be improved.
[0078] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0079] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0080] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0081] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0082] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0083] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.
Claims
1. A method for evaluating an input method, characterized in that: include: Acquire a number of input sentences of the target input method and candidate sentences associated with the target sentence; wherein the target sentence at least includes the last input sentence; Based on the candidate sentence and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentence is obtained, and based on the candidate sentence and the plurality of input sentences, a second weight factor of the importance of each of the input sentences in evaluating the candidate sentence is obtained; Based on the first weight factor and the second weight factor of the importance of the target sentence in evaluating the candidate sentence, a fusion weight factor of the importance of the target sentence in evaluating the candidate sentence is obtained; Based on the fusion weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, an evaluation result of the target input method associating the candidate sentence is obtained.
2. The method according to claim 1, characterized in that The obtaining, based on the candidate sentence and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentence includes: At least based on the first semantic similarity between the target sentence and each of the candidate sentences, a first similarity statistical value is obtained; Based on the first semantic similarity corresponding to the candidate sentence and the first similarity statistic, a first weight factor of the importance of the target sentence in evaluating the candidate sentence is obtained.
3. The method according to claim 1, characterized in that: For any of the inputted sentences, obtaining the second weight factor of the importance of each of the inputted sentences in evaluating the candidate sentences based on the candidate sentences and the plurality of inputted sentences includes: At least based on the second semantic similarities between the input sentence and other input sentences and each of the candidate sentences, a second similarity statistical value is obtained; Based on the second semantic similarity corresponding to the candidate sentence and the second similarity statistical value, a second weight factor of the importance of the input sentence in evaluating the candidate sentence is obtained.
4. The method according to claim 1, characterized in that: The first weight factor and the second weight factor based on the importance of the target sentence in evaluating the candidate sentence are fused to obtain a fused weight factor of the importance of the target sentence in evaluating the candidate sentence, including: Obtaining a first weighting coefficient of the first weighting factor, and obtaining a second weighting coefficient of the second weighting factor; wherein the first weighting coefficient is negatively correlated with the second weighting coefficient; The first weighting factor is weighted based on the first weighting coefficient and the second weighting factor is weighted based on the second weighting coefficient to obtain the fusion weighting factor.
5. The method according to claim 1, characterized in that: The step of obtaining the evaluation result of the target input method associating the candidate sentence based on the fusion weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence comprises: The semantic similarity between the target sentence and the candidate sentence is weighted based on the fusion weight factor to obtain a first evaluation result, and the semantic similarity between the input sentence that is not the target sentence and the candidate sentence is weighted based on the second weight factor to obtain a second evaluation result; Based on the fusion of the first evaluation result and the second evaluation result, an evaluation result of the target input method associating the candidate sentence is obtained.
6. The method according to claim 1, characterized in that The evaluation result of the candidate sentence associated with the target input method is predicted by an evaluation model, and the evaluation model is trained based on a number of sample input sentences of the target input method and sample candidate sentences associated with sample target sentences, wherein the sample target sentences at least include the last sample input sentence, and the sample candidate sentences are annotated with real evaluation results, and the training steps of the evaluation model include: Processing the sample input sentence and the sample candidate sentence based on the evaluation model to obtain a prediction evaluation result of the sample candidate sentence; Based on the difference between the predicted evaluation result and the actual evaluation result, the network parameters of the evaluation model are adjusted; wherein the network parameters of the evaluation model at least include network parameters for fusing the first weight factor and the second weight factor.
7. The method according to claim 1, characterized in that After obtaining the evaluation result of the target input method associating the candidate sentence based on the fusion weight factor of the input sentence that is the target sentence and the second weight factor of the input sentence that is not the target sentence, the method further includes: In response to detecting a selection instruction for any of the candidate sentences, the selected candidate sentence is used as a new target sentence, and a new candidate sentence associated with the new target sentence by the target input method is obtained, and the step of obtaining a first weight factor of the importance of the target sentence in evaluating the candidate sentence based on the candidate sentence and the target sentence is returned to obtain an evaluation result of the target input method associating the new candidate sentence.
8. An input method evaluation device, characterized in that: include: An acquisition module, used to acquire a number of input sentences of a target input method and candidate sentences associated with the target sentence; wherein the target sentence at least includes the last input sentence; A processing module, configured to obtain, based on the candidate sentence and the target sentence, a first weight factor of the importance of the target sentence in evaluating the candidate sentence, and, based on the candidate sentence and the plurality of input sentences, obtain a second weight factor of the importance of each input sentence in evaluating the candidate sentence; A fusion module, configured to fuse the first weight factor and the second weight factor of the importance of the target sentence when evaluating the candidate sentence, to obtain a fusion weight factor of the importance of the target sentence when evaluating the candidate sentence; The evaluation module is used to obtain the evaluation result of the candidate sentence associated with the target input method based on the fusion weight factor of the input sentence that serves as the target sentence and the second weight factor of the input sentence that does not serve as the target sentence.
9. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the input method evaluation method according to any one of claims 1 to 7.
10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the input method evaluation method according to any one of claims 1 to 7 is implemented.