Computer-implemented method for automatic question answering and related products

Abstract

Embodiments of the present application provide a computer-implemented automatic answering method and related products. Wherein, the method comprises: in response to obtaining a question image to be answered, determining the answering constraint information based on the question image; determining the answer information of the question image based on the answering constraint information and the first generative language model; and determining the answering result of the question image based on the answering constraint information, the answer information and the second generative language model. Through the technical scheme of the present application, the generative language model can be guided by the answering constraint information related to the question image, so as to ensure that the final answering result is related to the question. Therefore, not only the inertial thinking of the traditional question bank optimization dimension is broken through, but also the probability of mismatch between the answering result and the question is effectively reduced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of information processing, and more particularly, embodiments of the present application relate to a computer-implemented method for automatic answering, and an electronic device and a computer-readable storage medium for performing the foregoing method. BACKGROUND

[0002] This section is intended to provide background information to facilitate a better understanding of embodiments of the application recited in the claims. The description herein can include concepts that can be pursued, but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, the content in this section is not, and should not be interpreted as, a recognition or a suggestion that this content constitutes any form of prior art to the application described and claimed herein.

[0003] Users have the need for answering questions in the process of work or study, and some automatic answering technologies appear on the market. For example, the current mainstream automatic answering technology based on photographing searching for questions, which directly takes the similar questions searched from the question bank as the answering content. It can be seen that this automatic answering technology needs to rely on the coverage rate of the question bank, and once the coverage rate of the question bank is not complete, it is very likely to return some answers irrelevant to the question to the user. That is, the current automatic answering scheme on the market mainly focuses on the optimization dimension of expanding the question bank, and cannot ensure that the answering content for the user is related to the question itself, and obviously cannot meet the actual answering needs of the user. SUMMARY

[0004] It is very annoying to know that the effect of automatic answering is not ideal.

[0005] Therefore, there is a great need for an improved computer-implemented automatic answering scheme, which can effectively reduce the probability that the answering result is inconsistent with the question and improve the user experience.

[0006] In this context, embodiments of the present application aim to provide a computer-implemented automatic answering method and related products.

[0007] In a first aspect of the embodiments of the present application, a computer-implemented automatic answering method is provided, comprising: in response to obtaining a question image to be answered, determining answering constraint information based on the question image; determining answer information of the question image based on the answering constraint information and a first generative language model; and determining an answering result of the question image based on the answering constraint information, the answer information and a second generative language model.

[0008] In an embodiment of the present application, determining the question answering constraint information based on the question image comprises: performing page correction processing on the question image; and determining the question answering constraint information according to the corrected question image.

[0009] In another embodiment of the present application, determining the question answering constraint information according to the corrected question image comprises: determining question text and / or user answer information according to the corrected question image; and determining the question answering constraint information according to the question text and / or the user answer information.

[0010] In yet another embodiment of the present application, determining the question answering constraint information according to the question text and / or the user answer information comprises: obtaining similar questions of the question text; performing knowledge point prediction based on the question text and / or the similar questions to obtain knowledge point information; and determining the question answering constraint information according to the similar questions, the knowledge point information and / or the user answer information.

[0011] In still another embodiment of the present application, wherein the question answering constraint information comprises the similar questions and the knowledge point information, determining the answer information of the question image comprises: fine-tuning the first generative language model based on the similar questions and the knowledge point information; and calculating the answer information of the question image based on the fine-tuned first generative language model.

[0012] In an embodiment of the present application, the question text comprises a calculation type question text, and determining the answer information of the question image specifically comprises: generating executable code about the calculation type question text; obtaining an execution result of the executable code; obtaining an operation result of the fine-tuned first generative language model for the calculation type question text; and performing result fine-tuning using the execution result and the operation result to obtain the answer information of the calculation type question text.

[0013] In another embodiment of the present application, generating executable code about the calculation type question text comprises: outputting the executable code based on the fine-tuned first generative language model.

[0014] In yet another embodiment of the present application, wherein the question answering constraint information comprises the similar questions, the knowledge point information and / or the user answer information, determining the question answering result of the question image comprises: fine-tuning the second generative language model based on the similar questions, the knowledge point information, the answer information and / or the user answer information; and calculating the question answering result of the question image based on the fine-tuned second generative language model.

[0015] In a further embodiment of the present application, the method further comprises: detecting whether the stem in the question text satisfies a preset condition; and in response to determining that the stem in the question text satisfies the preset condition, performing the step of determining the answer constraint information according to the question text and / or user answer information.

[0016] In an embodiment of the present application, the method further comprises: detecting whether an additional question for the answer result of the question image is acquired; and

[0017] In response to acquiring the additional question, generating an answer result for the additional question based on the answer result of the question image.

[0018] In a second aspect of the embodiments of the present application, an electronic device is provided, comprising: a processor; and a memory storing computer instructions for automatic answering, which, when executed by the processor, cause the electronic device to perform the method according to the foregoing and the following embodiments.

[0019] In a third aspect of the embodiments of the present application, a computer readable storage medium is provided, containing program instructions for automatic answering, which, when executed by a processor, cause the implementation of the method according to the foregoing and the following embodiments.

[0020] According to the computer-implemented automatic answering method and related products of the embodiments of the present application, a generative language model can be constrained by the determined answer constraint information to obtain answer information, and then another generative language model can be constrained by the answer constraint information and the answer information to obtain an answer result. It can be seen that the scheme of the present application uses the answer constraint information related to the question image to guide the result of the generative language model, thereby ensuring that the final answer result can be related to the question. Thus, not only does the present application break through the inertial thinking of the traditional question bank optimization dimension, but also effectively reduces the probability that the answer result does not match the question.

[0021] In addition, in some embodiments of the present application, the additional question for the answer result of the question image can also generate an answer result for the additional question, thereby supporting multi-round answering to meet the diversified answering needs of users. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other objects, features and advantages of the exemplary embodiments of the present application will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A block diagram schematically illustrates an exemplary computing system 100 suitable for implementing embodiments of the present application;

[0024] Figure 2 A schematic flowchart of an automated question-answering method implemented by a computer according to an embodiment of the present invention is shown.

[0025] Figure 3 A schematic flowchart of a computer-implemented automatic question-answering method according to another embodiment of the present invention is shown.

[0026] Figure 4 A schematic flowchart of a computer-implemented automatic question-answering method according to another embodiment of the present invention is shown.

[0027] Figure 5 A schematic diagram illustrating automated question answering implemented by a computer according to yet another embodiment of the present invention is shown.

[0028] Figure 6 A schematic comparison diagram of page correction before and after according to an embodiment of the present invention is shown;

[0029] Figure 7 A schematic diagram illustrating similar questions retrieved according to an embodiment of the present invention is shown.

[0030] Figure 8 A schematic diagram illustrating answer information according to an embodiment of the present invention is shown; and

[0031] Figure 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention is shown.

[0032] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0033] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0034] Figure 1 A block diagram of an exemplary computing system 100 suitable for implementing embodiments of the present invention is shown. For example... Figure 1As shown, the computing system 100 can include a central processing unit (CPU) 101, a random access memory (RAM) 102, a read only memory (ROM) 103, a system bus 104, a hard disk controller 105, a keyboard controller 106, a serial interface controller 107, a parallel interface controller 108, a display controller 109, a hard disk 110, a keyboard 111, a serial peripheral 112, a parallel peripheral 113, and a display 114. Of these devices, the CPU 101, the RAM 102, the ROM 103, the hard disk controller 105, the keyboard controller 106, the serial controller 107, the parallel controller 108, and the display controller 109 are coupled to the system bus 104. The hard disk 110 is coupled to the hard disk controller 105, the keyboard 111 is coupled to the keyboard controller 106, the serial peripheral 112 is coupled to the serial interface controller 107, the parallel peripheral 113 is coupled to the parallel interface controller 108, and the display 114 is coupled to the display controller 109. It will be appreciated that Figure 1 The structural diagram described is for the purpose of example only and is not intended to limit the scope of the present application. In some cases, certain devices can be added or removed depending on the specific circumstances.

[0035] Those skilled in the art know that the embodiments of the present application can be implemented as a system, a method or a computer program product. Therefore, the present disclosure can be embodied in the form of entire hardware, entire software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to herein as "circuitry", "module", "unit" or "system". In addition, in some embodiments, the present application can also be implemented as a computer program product in one or more computer readable media, which contains computer readable program codes.

[0036] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any suitable combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0037] Computer readable signal media can include a propagated data signal with computer readable program code embodied therein. For example, a propagated signal can be an electromagnetic signal, an optical signal, and / or any other suitable type of signal. Such a propagated signal can carry computer readable program code in the form of electrical signals, optical signals, and / or magnetic

[0038] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0039] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). These network connections are

[0040] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0041] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0042] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0043] According to embodiments of the present application, a computer-implemented automatic answering method and related products are provided. Moreover, any number of elements in the drawings are used for illustration only and not limitation, and any naming is only for distinction and does not have any limiting meaning.

[0044] The principles and spirits of the present application will be explained in detail below with reference to several representative embodiments of the present application. SUMMARY

[0046] The inventors find that the existing artificial intelligence (AI) based automatic answering has unsatisfactory effect. Specifically, the mainstream automatic answering technology in the market involves automatic answering based on photograph searching and automatic answering based on computable language model. Among them, the automatic answering based on photograph searching usually uses the question text picture to combine the searching technology to search in the existing question bank, and returns the search result with the highest similarity to the user as the answering content. However, this operation will have the following problems:

[0047] 1. The returned result is inconsistent with the question taken by the user. Usually, the consistency of the answering result of this kind of scheme depends on two aspects: the coverage rate of the question bank and the related algorithm ability such as optical character recognition (OCR) and searching technology. If the coverage rate of the question bank is not complete, it is likely to return some similar questions to the user which are irrelevant to the question. In addition, the OCR technology and the searching technology will further affect the consistency.

[0048] 2. The analysis quality is uneven, and lacks a unified standard. The answer sources of the current scheme may include artificial intelligence algorithms, contributions of education experts or other non-professional users. Due to the diversity of answer sources in the question bank, the quality of answers may be uneven, and even some answers may be inaccurate or not clear.

[0049] 3. It does not support multi-round answering. Usually, the flow of this kind of scheme stops at searching for similar questions and feeding back to the user. If the user does not understand the details in the question, the existing scheme cannot support automatic and targeted answering for the user.

[0050] The automatic answering technology based on the computable language model is mainly for the automatic answering of the computable questions in English. The problems of this scheme are that the answerable field is limited and is constrained by the computable language model, and this scheme can only be applied to the answering of the mathematical calculation questions, and cannot be applied to the answering of the questions in other fields. In addition, this technical scheme cannot solve the problem that the answering content does not match the question.

[0051] The inventor further found that the current market solutions mainly focus on the expansion of the question bank, the optimization of the detection, identification and search question technology, and the essence of this inert thinking has limited innovation points, and still cannot solve the consistency problem of the answering. That is, the current scheme still cannot guarantee that the answering content for the user is related to the question itself, but not other questions.

[0052] Based on this, the inventor found that the traditional inert thinking can be broken through, the generative language model is used to assist in answering, and the generative language model is guided by the answering constraint information related to the question image, so as to ensure that the final answering result is related to the question.

[0053] After introducing the basic principle of the present application, various non-limiting embodiments of the present application will be specifically introduced below.

[0054] Exemplary method

[0055] The method of automatically answering by computer according to the exemplary embodiments of the present application will be described below with reference to Figure 2 It should be noted that the embodiments of the present application can be applied to any applicable scenario.

[0056] Figure 2 A flowchart of the method 200 of automatically answering by computer according to one embodiment of the present application is schematically shown.

[0057] As shown in Figure 2 At step S201, the answering constraint information can be determined based on the question image in response to obtaining the question image to be answered. The question image here can be understood as an image containing the question to be answered. In actual application, the question image can be obtained in various ways. For example, the question image collected by an external device can be obtained. For another example, the question image can also be collected by the device itself. That is, the specific acquisition process of the question image can be adjusted in combination with the configuration of the device itself, the design requirements of the application and the like, and the specific acquisition process of the question image is not limited by the scheme of the present application.

[0058] In addition, in some embodiments, the obtained question image can contain only one question or multiple questions. The number and type of questions contained in the question image are not limited here.

[0059] The answer constraint information in this embodiment can be understood as some information related to the question in the question image and capable of constraining and guiding the output result of the generative language model, which is obtained based on the question image. For example, in some implementation scenarios, the answer constraint information can include but is not limited to question type information, user answer information, knowledge points, answers and analyses of similar questions, and the like. It should be noted that the scheme of the present application does not limit the specific content of the answer constraint information, which can be set and adjusted according to specific application scenarios and design requirements.

[0060] At step S202, the answer information of the question image can be determined based on the answer constraint information and the first generative language model. In actual application, the first generative language model can output an answer based on the question image, and in this process, the answer constraint information related to the question image is used to guide and constrain the first generative language model to ensure that the answer information matches the question in the question image. The foregoing first generative language model can be a neural network model that is pre-trained and supports answer generation. In some embodiments, the first generative language model can be trained based on a large language model such as GPT, which has the advantages of strong text analysis and content generation capabilities. It should be noted that the specific network model of the generative language model is not displayed here and can be adjusted according to specific application requirements.

[0061] Finally, at step S203, the answer result of the question image can be determined based on the answer constraint information, the answer information, and the second generative language model. In some embodiments, the second generative language model can be trained based on a large language model such as GPT. In the process of outputting the answer result by the second generative language model, the answer constraint information and the answer information output based on the first generative language model are introduced to constrain and standardize the answer result, so that the answer result can match the question in the question image.

[0062] By using the answer constraint information related to the question image to guide the result of the generative language model, it is ensured that the final answer result can be related to the question. Thus, not only does it break the inertial thinking of the traditional question bank optimization dimension, but it also effectively reduces the probability that the answer result does not match the question. In addition, the answer result can be further standardized by means of the answer constraint information and the answer information, avoiding uneven quality of the answer content and improving the user experience.

[0063] Furthermore, the solution of the present invention does not limit the type of question, that is, the solution of the present invention can be applied to the automatic answering of various types of questions, including mathematical calculation questions (such as some logic evolution questions, subjective questions, graph questions, etc.), and has strong practicality and market prospects.

[0064] Figure 3 A schematic flowchart of a computer-implemented automated question-answering method 300 according to another embodiment of the present invention is shown. It will be understood that method 300 is a... Figure 2 Further limitations and / or extensions of Chinese method 200. Therefore, the foregoing is combined with Figure 2 The relevant detailed descriptions also apply to the following text.

[0065] like Figure 3 As shown, in step S301, page correction processing can be performed on the acquired question image. As mentioned earlier, the question image can be acquired by the device itself or by linking with other external devices (scanning pen, smart study lamp, or other devices with image scanning or photography functions). After obtaining the question image, page correction processing can be performed on the question image to improve the accuracy of subsequent OCR.

[0066] In practical applications, image page correction can be achieved in various ways. For example, this implementation combines whole-image correction and text region correction to correct the title image. Specifically, a pre-trained corner detection model can be used to obtain the four corner points of the image (e.g., top left, top right, bottom right, and bottom left). Then, after slicing the title image, each corner point is modified (e.g., using a lightweight semantic segmentation network based on convolution, ENet). Next, the 3D perspective of the image can be calculated based on the corrected corner points. Thus, by performing a pseudo-3D transformation on the ENet deformation field, the problem of no 3D modeling at the page edges is solved. Finally, text region correction is performed using, for example, a multi-scale transverse network based on convolution (TNet network). This achieves page correction processing for the title image. It should be noted that the detailed description of the page correction process here is merely illustrative, and the solution of this invention is not limited thereto.

[0067] Next, in step S302, the question text and / or user answer information can be determined based on the corrected question image. In some embodiments, the question image may contain the question and / or user-defined answer information. In this case, detection can be performed on the question area and the answer area in the image to obtain the question text and / or user answer information.

[0068] Then, at step S303, the question answering constraint information can be determined according to the aforementioned question text and / or user answering information. In some embodiments, if only the question text is obtained, the question answering constraint information can be determined based on the question text. Of course, when the user answering information is also obtained, the question answering constraint information can be determined based on the question text and the user answering information.

[0069] Specifically, similar questions of the question text can be obtained. Then, knowledge point prediction is performed based on the question text and / or the similar questions to obtain knowledge point information. Finally, the question answering constraint information is determined according to the similar questions, the knowledge point information and / or the user answering information.

[0070] Further, in some embodiments, the similar questions of the question text can be retrieved from a question bank. Specifically, the similar questions can be recalled by using, for example, an ES tool to obtain a question text candidate set. The ES tool is based on a BM25 algorithm to calculate similarity. Then, the similar questions returned by the ES are re-ranked by a longest common subsequence algorithm between the text recognized by the OCR and the question text candidate set returned by the ES. Finally, according to the application requirements, several similar questions with high ranking are selected as the similar questions of the question text. In addition, if the question contains image information (for example, image information such as picture list type), the ranking result will also be corrected in combination with the result of the picture search algorithm. It should be noted that the detailed description of the process of obtaining similar questions herein is only exemplary and the scheme of the present application is not limited thereto.

[0071] Further, in some embodiments, the knowledge point information can be predicted in various ways. For example, keyword extraction can be performed on the question text, and the matching knowledge point information can be found according to the extracted keywords. For another example, the knowledge point information corresponding to the similar questions can be obtained from the question bank. For yet another example, the knowledge point information obtained according to the question text and the knowledge point information obtained according to the similar questions can be integrated to obtain the final knowledge point information.

[0072] Further, the similar questions obtained are usually configured with analysis and answers. After obtaining the similar questions and the knowledge point information, the question answering constraint information can be determined according to the similar questions and the knowledge point information. Of course, when the user answering information is obtained, the question answering constraint information can be determined according to the similar questions, the knowledge point information and the user answering information, so that the question answering constraint information can be highly associated with the actual question answering needs of the user. It should be noted that the detailed description of the determination process of the question answering constraint information herein is only exemplary and can be adjusted in combination with the application scenario. For example, the question answering constraint information can be further adjusted in combination with some information highly associated with the question answering needs of the user (for example, the actual school age of the user, etc.).

[0073] After obtaining the answering constraint information, at step S304, the first generative language model can be fine-tuned based on the similar question and knowledge point information in the answering constraint information. And at step S305, the answer information of the question image is calculated based on the fine-tuned first generative language model. As described above, the first generative language model can be some large language model that is pre-trained and supports answer generation. During the generation of the answer information by the first generative language model, the output result of the first generative language model can be constrained based on the similar question and knowledge point information, so that the output answer information can match the question and cover the knowledge category, thereby avoiding the situation that the user cannot understand because the question is answered with out-of-class knowledge.

[0074] Further, in some embodiments, for the calculation type question text, executable code about the calculation type question text can be generated (for example, the executable code can be output based on the fine-tuned first generative language model, etc.). Then, the operation result of the fine-tuned first generative language model for the calculation type question text can be obtained. Then, the execution result and the operation result are fine-tuned to obtain the answer information of the calculation type question text. For example, when the execution result and the operation result are consistent, the execution result or the operation result can be selected as the answer information of the calculation type question text. For another example, when the execution result and the operation result are inconsistent, the execution code can be repeatedly executed and the operation result of the fine-tuned first generative language model can be requested multiple times. If the execution results of the execution code are consistent after multiple executions, the execution result is taken as the answer information of the calculation type question text. If the operation results obtained multiple times are consistent, the operation result is taken as the answer information of the calculation type question text. In this way, the accuracy of the answer information is improved.

[0075] After obtaining the answer information, at step S306, the second generative language model can be fine-tuned based on the similar question, knowledge point information, answer information, and / or user answering information. And at step S307, the answering result of the question image can be calculated based on the fine-tuned second generative language model. As described above, the second generative language model can be some large language model that is pre-trained and supports answer generation. During the generation of the answering result by the second generative language model, the output result of the second generative language model can be constrained based on the similar question, knowledge point information, answer information, or the similar question, knowledge point information, answer information, and user answering information, so that the output answering result can match the question and the answering content style is standardized, thereby avoiding the problem that the quality of the answering content is uneven.

[0076] Figure 4A flowchart of a method 400 of computer-implemented automatic answering of questions according to still another embodiment of the present application is schematically shown. It can be appreciated that the method 400 can be understood as a further supplement or extension of the method 200 or the method 300. Thus, the foregoing detailed description in connection with the related details in Figure 2 and Figure 3 also applies hereinafter.

[0077] As shown in Figure 4 , at step S401, the acquired question image can be subjected to page rectification processing. Then, at step S402, the rectified question image can be used to determine question answering constraint information. Then, at step S403, the question answering constraint information and a first generative language model can be used to determine answer information of the question image. And at step S404, the question answering constraint information, the answer information and a second generative language model can be used to determine a question answering result of the question image. The process of determining the question answering result at steps S401-S404 can refer to the foregoing detailed description in connection with the related details in Figure 2 and Figure 3 , which will not be repeated here.

[0078] After obtaining the question answering result, the question answering result can be shown to the user. If the user still has questions about the question answering result, the user can continue to ask questions about the question answering result. At step S405, an additional question input by the user manually or by voice, or the question answering result can support user-defined annotations, and the additional question can be determined according to the annotations of the user to the question answering result. The detailed description of the process of obtaining the additional question here is only exemplary, and the way of obtaining the additional question can be determined according to the specific human-computer interaction design.

[0079] Finally, at step S406, a question answering result about the additional question can be generated based on the question answering result of the question image. Specifically, the question answering constraint information can be generated based on the additional question, and the question answering result about the additional question can be obtained based on the question answering constraint information and the generative language model.

[0080] Thus, the present embodiment not only can guide the generative language model based on the question answering constraint information related to the question image to ensure that the final question answering result can be related to the question, but also can ensure that the question answering content is standardized in style. In addition, the present embodiment can also support the user to ask additional questions about the parsed content and multi-round question answering to meet the diversified question answering needs of the user.

[0081] Figure 5 A flowchart of a method 500 of computer-implemented automatic answering of questions according to still another embodiment of the present application is schematically shown. It can be appreciated that the method 500 can be understood as a specific technical implementation of the method 200, the method 300 and the method 400. Thus, the foregoing detailed description in connection with the related details inFigures 2 to 4 The relevant details described in the text also apply to the following text.

[0082] like Figure 5 As shown, in step S501, the questions to be answered, photographed by the user, can be acquired to obtain a question image. There are no restrictions on the type and number of questions in the question image.

[0083] Next, in step S502, page correction processing can be performed on the obtained title image. This page correction can involve full image correction and text area correction; the specific correction process can be found in the preceding text. Figure 3 The relevant details will not be repeated here. Page correction of the image can improve the accuracy of subsequent OCR. Figure 6 This shows a comparison of the title image before and after page correction. For example... Figure 6 As shown, the image before page correction is the question image uploaded by the user. After page correction, the text in the image is clearly clearer, which is beneficial for OCR recognition.

[0084] Next, in step S503, OCR processing is performed on the question and answer area in the question image. Specifically, the question in the question image can be detected to obtain question type information.

[0085] In step S504, text recognition technology can be used to parse the question text and user answer information from the question image. For example, for... Figure 6 The image of the problem shows the problem text: Xiao Pang's home is 840 meters from school. He walks from home to school, and after 6 minutes, he is still 390 meters away from school. What is Xiao Pang's walking speed? Answer: 840 ÷ 6 = 140 (meters / minute), so Xiao Pang's walking speed is 140 meters / minute.

[0086] In step S505, the parsed question text needs further inspection to determine whether the question stem meets preset conditions (preset conditions may include question stem compliance checks and whether it contains questions to be answered, etc., which can be adjusted according to the application scenario). Subsequent operations are only performed if the question stem meets the preset conditions (e.g., determining answer constraint information based on the question text and / or user answer information). If the question stem does not meet the preset conditions, further parsing of the question text is stopped. Furthermore, a prompt message can be output to promptly notify the user to make adjustments.

[0087] In step S506, similar questions can be retrieved by searching the question text in the question bank. These similar questions usually include explanations and answers. Figure 7 The retrieved similar questions are shown. It should be noted that...Figure 7 is obtained from the question image shown in Figure 6 The similar questions are only taken as an example, and the number of similar questions is not limited in actual use.

[0088] Meanwhile, at step S507, knowledge point information can be predicted according to the question text. For example, the knowledge point prediction is performed on the question text in the question image shown in Figure 6 , and the obtained knowledge point "speed and distance formula" and the like are obtained. It should be noted that the retrieval of similar questions and the prediction of knowledge points can refer to the related detailed description in the foregoing Figure 3 , and will not be described here in detail.

[0089] Then, at step S508, the answer information can be calculated. Specifically, the first generative language model (LLM) can be fine-tuned through similar questions and knowledge point information and the like. The fine-tuned LLM model calculates the correct answer according to the similar questions and the knowledge point information and the like, and returns. In some example scenarios, for calculation type questions, the LLM needs to output an executable code segment, and the result is fine-tuned through the code execution result and the LLM operation result to ensure the correctness of the answer. Figure 8 shows the answer information obtained from the question image shown in Figure 6 . In the answer information, the operation result of the first generative language model can be included, and the executable code in the dashed box can also be included.

[0090] Then, at step S509, another generative language model can be fine-tuned according to the calculated correct answer, the user's answer information, the similar questions and the knowledge point information to obtain the question answering result. The generative language model will generate personalized question answering results combined with the user's answer information. For example, the question answering result can include an evaluation of the user's answer information, correct answer content, and user-specific error information, and the like. For example, "Oh, it's a pity that the little friend made a mistake on this question! It's okay, let's take a look at where the problem is! According to the question, the speed of the fat boy walking is equal to the distance they walk divided by the time they walk. Let the speed of the fat boy walking be v, then the distance the fat boy walks is 840-390=450 meters, and the time the fat boy walks is 6 minutes. Therefore, the speed of the fat boy walking is 450 / 6=75 meters / minute. I found that the little friend made a mistake in calculating the distance the fat boy walked, so next time, please carefully observe the question before answering!".

[0091] Then, at step S510, the user-added follow-up question can be acquired, and at step S511, additional analysis can be generated for the follow-up question. After obtaining the answer result, it is returned to the user. If the user provides an additional question, a new answer result can be generated according to the previous answer result and returned to the user. For example, the user can continue to ask: Why is the walking distance of Xiaopang 450 meters? The new answer result can be: Xiaopang's home is 840 meters away from the school, but he walked for 6 minutes and is still 390 meters away from the school. Therefore, the distance he walked in 6 minutes is 840 meters minus 390 meters, that is, 450 meters. Of course, if the user does not respond for a long time or has no additional question, the current round of answering can be ended.

[0092] Therefore, the scheme of the present example not only greatly reduces the probability that the answer result does not match the user's question, but also makes the answer result more targeted, and can generate an answer result that is more in line with the user's actual situation according to the user's existing answer content. In addition, it can also support the user to ask additional questions about the answered content and support multiple rounds of answering.

[0093] Exemplary device

[0094] After introducing the method of the exemplary embodiment of the present application, next, with reference to Figure 9 The method of the computer-implemented automatic answering of the exemplary embodiment of the present application is described.

[0095] Figure 9 A schematic block diagram of an electronic device 900 according to an embodiment of the present application is schematically shown. As shown, the electronic device 900 can include a processor 901 and a memory 902. The memory 902 stores computer instructions for computer-implemented automatic answering, which, when executed by the processor 901, causes the electronic device 900 to perform the method according to the foregoing description in conjunction with Figure 9 For example, in some embodiments, the electronic device 900 can be used to acquire answer constraint information, obtain answer information using answer constraint information and generative language model, obtain answer result using answer information, answer constraint information and generative language model, etc. Based on this, the electronic device 900 can break through the inertia of the traditional question bank optimization dimension and effectively reduce the probability that the answer result does not match the question. Figures 2 to 5

[0096] In actual application, the electronic device 900 in the present embodiment can be a brand new product, or an electronic device obtained by expanding the functions of some existing electronic devices (such as smart learning tablets, smart learning table lamps, etc.).

[0097] ​It should be noted that, although several means or sub-means of the device are mentioned in the foregoing detailed description, such a division is merely not mandatory. Indeed, according to an embodiment of the application, the features and functionalities of two or more means described above can be embodied in one mean. Conversely, the features and functionalities of one mean described above can be further divided into means embodied by several means.

[0098] The use of the verb "comprise", "comprising" and conjugations thereof, in the application file, does not exclude the presence of elements other than those mentioned in the application file. The use of the article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0099] While the principles and spirit of the application have been described with reference to several specific embodiments, it is to be understood that the application is not limited to the disclosed specific embodiments, and that the division into aspects does not imply that features from these aspects cannot be combined to benefit, this division being merely for ease of presentation. The application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is the broadest interpretation of the scope to which they are fairly entitled under the law.

Claims

1. A computer-implemented method for automatic question answering, the method comprising: The method comprises: in response to obtaining a question image to be answered, determining answer constraint information based on the question image, wherein a similar question of the question text is obtained; based on the question text and / or the similar question, knowledge point prediction is performed to obtain knowledge point information; and determining the answer constraint information according to the similar question, the knowledge point information and / or user answer information; based on the answer constraint information and a first generative language model, determining answer information of the question image, wherein the answer constraint information comprises the similar question and the knowledge point information, the first generative language model is fine-tuned based on the similar question and the knowledge point information; and based on the fine-tuned first generative language model, calculating the answer information of the question image; and based on the answer constraint information, the answer information and a second generative language model, determining an answer result of the question image, wherein the answer constraint information comprises the similar question, the knowledge point information and / or the user answer information, the second generative language model is fine-tuned based on the similar question, the knowledge point information, the answer information and / or the user answer information; and based on the fine-tuned second generative language model, calculating the answer result of the question image.

2. The method of claim 1, wherein, Determining the answer constraint information based on the question image comprises: performing page correction processing on the question image; and determining the answer constraint information according to the corrected question image.

3. The method of claim 2, wherein, Determining the answer constraint information according to the corrected question image comprises: determining question text and / or user answer information according to the corrected question image; and determining the answer constraint information according to the question text and / or user answer information.

4. The method of claim 1, wherein, The question text comprises a calculation type question text, and determining the answer information of the question image specifically comprises: generating executable code about the calculation type question text; obtaining an execution result of the executable code; obtaining an operation result of the fine-tuned first generative language model for the calculation type question text; and performing result fine-tuning using the execution result and the operation result to obtain answer information of the calculation type question text.

5. The method of claim 4, wherein, Generating executable code about the calculation type question text comprises: outputting the executable code based on the fine-tuned first generative language model.

6. The method of claim 3, wherein, The method further comprises: detecting whether a stem in the question text meets a preset condition; and in response to determining that the stem in the question text meets the preset condition, performing the step of determining the answer constraint information according to the question text and / or user answer information.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: detecting whether an additional question for an answer result of the question image is obtained; and in response to obtaining the additional question, generating an answer result for the additional question based on the answer result of the question image.

8. An electronic device, comprising: The electronic device comprises: a processor; and a memory storing computer instructions for automatic answering, when the computer instructions are run by the processor, the electronic device executes the method according to any one of claims 1-7. ​ 9. A computer-readable storage medium, characterized in that, Program instructions comprising automatic answering, which when executed by a processor, cause the implementation of the method according to any one of claims 1-7.

Images