Test question recognition method, device, computer equipment and storage medium

Abstract

The present application relates to a test question recognition method, device, computer equipment and storage medium. The method includes: extracting features from the test question description text to obtain feature vectors corresponding to the test question segmentation, arranging the feature vectors according to the order of the test question segmentation in the test question description text, and obtaining a test question text feature sequence; extracting features from the target test question image to obtain test question image features; fusing the test question text feature sequence with the test question image features to obtain target fusion features; determining the feature attention strength corresponding to each feature vector in the test question text feature sequence based on the target fusion features; adjusting the feature vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; and identifying the target knowledge points corresponding to the target test question based on the target text feature sequence. Among them, feature extraction and feature fusion can be performed based on an artificial intelligence model, and the use of this method can effectively improve the accuracy of test question recognition results.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a test question recognition method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of computer and internet technologies, different types of information need to be categorized and labeled to help users quickly filter out the information that meets their needs. For example, personalized learning systems and other educational products use categorized knowledge points to enable users to quickly find the specific knowledge points they need.

[0003] However, the current test question recognition method usually classifies and labels the test question text information based on an artificial intelligence model. However, the current test question recognition method often has poor recognition effect, resulting in a low accuracy rate of the test question recognition results. Summary of the Invention

[0004] Based on this, it is necessary to provide a test question recognition method, device, computer equipment and storage medium that can effectively improve the accuracy of test question recognition results in response to the above technical problems.

[0005] A method for identifying test questions, the method comprising: obtaining a test question description text in a target test question to be identified, and a target test question image corresponding to the target test question; the test question description text comprises a plurality of test question segmentations; performing feature extraction on the test question description text to obtain feature extraction vectors corresponding to the test question segmentations, arranging the feature extraction vectors in the order of the test question segmentations in the test question description text to obtain a test question text feature sequence; performing feature extraction on the target test question image to obtain test question image features; fusing the test question text feature sequence with the test question image features to obtain a target fusion feature; determining the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature; adjusting the feature extraction vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; performing knowledge point identification based on the target text feature sequence to obtain a target knowledge point corresponding to the target test question.

[0006] A test question recognition device, the device comprising: an acquisition module for acquiring a test question description text in a target test question to be recognized, and a target test question picture corresponding to the target test question; the test question description text comprises a plurality of test question segmentations; a text feature extraction module for performing feature extraction on the test question description text to obtain feature extraction vectors corresponding to the test question segmentations, the feature extraction vectors being arranged in the order of the test question segmentations in the test question description text to obtain a test question text feature sequence; a picture feature extraction module for performing feature extraction on the target test question picture to obtain test question picture features; a fusion module for fusing the test question text feature sequence with the test question picture features to obtain a target fusion feature; a feature attention strength determination module for determining, based on the target fusion feature, the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence; an adjustment module for adjusting the feature extraction vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; and a recognition module for performing knowledge point recognition based on the target text feature sequence to obtain a target knowledge point corresponding to the target test question.

[0007] In one embodiment, the fusion module includes: a text feature fusion parameter acquisition unit, used to obtain the text feature fusion parameters corresponding to the test question text feature sequence, and use the text feature fusion parameters to adjust the test question text feature sequence to obtain an adjusted text feature sequence; a picture feature fusion parameter acquisition unit, used to obtain the picture feature fusion parameters corresponding to the test question picture feature, and use the picture feature fusion parameters to adjust the test question picture feature to obtain an adjusted picture feature; and a fusion unit, used to fuse the adjusted text feature sequence and the adjusted picture feature to obtain a target fusion feature.

[0008] In one embodiment, the picture feature fusion parameter acquisition unit is used to: obtain the target feature quantity corresponding to the test question text feature sequence, where the target feature quantity is the number of feature extraction vectors contained in the test question text feature sequence; copy the test question picture features according to the target feature quantity, and arrange the copied test question picture features of the target feature quantity to obtain a picture feature sequence; adjust the picture feature sequence using the picture feature fusion parameter, and use the adjusted picture feature sequence as the adjusted picture feature.

[0009] In one embodiment, the fusion module is used to: use the features in the adjusted text feature sequence as the matrix values ​​in the first matrix object corresponding to the adjusted text feature sequence in the target feature matrix to obtain the first matrix object; use the features in the adjusted image feature sequence as the matrix values ​​in the second matrix object corresponding to the adjusted image feature sequence in the target feature matrix to obtain the second matrix object; use the matrix formed by the first matrix object and the second matrix object as the target feature matrix; and obtain the target fusion feature based on the target feature matrix.

[0010] In one embodiment, the feature attention strength determination module is used to: obtain fusion feature processing parameters, use the fusion feature processing parameters to process the target fusion feature, and obtain an attention strength vector; the attention strength vector includes a vector value of the number of target features, and the number of target features is the number of feature extraction vectors contained in the test text feature sequence; the vector value in the attention strength vector that is consistent with the order of the feature extraction vectors in the test text feature sequence is used as the feature attention strength corresponding to the feature extraction vector.

[0011] In one embodiment, the target test question picture includes a first test question picture, and the acquisition module includes: a content description information acquisition unit, used to obtain content description information corresponding to the target test question, the content description information describes the test question content corresponding to each content type in the target test question and the association relationship between the test question contents; an image generation unit, used to generate a first test question picture corresponding to the target test question according to the content description information; wherein, when generating the first test question picture, the display method of the test question content on the first test question picture is determined according to the content type corresponding to the test question content, and the display position of the test question content on the first test question picture is determined according to the association relationship between the test question contents.

[0012] In one embodiment, the target test question picture includes a first test question picture, and the acquisition module includes: a second test question picture acquisition unit is used to: use the test question picture in picture format in the target test question as the second test question picture; the first test question picture includes the test question description text and the picture content in the second test question picture; the test question picture features include first test question picture features and second test question picture features; the picture feature extraction module is used to: input the first test question picture into a fusion extraction model for feature extraction to obtain a first test question picture feature that fuses the test question description text and the picture content; input the second test question picture into a picture extraction model for feature extraction to obtain a second test question picture feature.

[0013] In one embodiment, the recognition module includes: a test question comprehensive feature acquisition unit, which is used to fuse the target text feature sequence, the first test question image feature, and the second test question image feature to obtain the test question comprehensive feature corresponding to the target test question; and an identification unit, which is used to identify knowledge points based on the test question comprehensive feature to obtain the target knowledge points corresponding to the target test question.

[0014] In one embodiment, a test question comprehensive feature acquisition unit is used to input the target text feature sequence, the first test question image feature, and the second test question image feature into a comprehensive feature extraction layer for feature extraction. The comprehensive feature extraction layer uses the parameters corresponding to each feature to process the target text feature sequence, the first test question image feature, and the second test question image feature respectively to obtain a processed target text feature sequence, a processed first test question image feature, and a processed second test question image feature; the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature are spliced ​​to obtain the test question comprehensive feature corresponding to the target test question.

[0015] In one embodiment, the acquisition module is used to: acquire text information in the target test question; perform character type recognition on the text information to obtain a symbol string and an operation string in the text information; filter out a mixed string including a symbol string and an operation string adjacent to each other from the text information based on the string position of the symbol string and the string position of the operation string; filter out the mixed string from the text information and use the remaining text information as the test question description text.

[0016] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program: obtaining a test question description text in a target test question to be identified, and a target test question image corresponding to the target test question; the test question description text comprises a plurality of test question segmentations; performing feature extraction on the test question description text to obtain feature extraction vectors corresponding to the test question segmentations, and arranging the feature extraction vectors according to the order of the test question segmentations in the test question description text to obtain a test question text feature sequence; performing feature extraction on the target test question image to obtain test question image features; fusing the test question text feature sequence with the test question image features to obtain a target fusion feature; determining the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature; adjusting the feature extraction vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; performing knowledge point recognition based on the target text feature sequence to obtain a target knowledge point corresponding to the target test question.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps: obtaining a test question description text in a target test question to be identified, and a target test question image corresponding to the target test question; the test question description text includes a plurality of test question segmentations; performing feature extraction on the test question description text to obtain feature extraction vectors corresponding to the test question segmentations, and arranging the feature extraction vectors according to the order of the test question segmentations in the test question description text to obtain a test question text feature sequence; performing feature extraction on the target test question image to obtain test question image features; fusing the test question text feature sequence with the test question image features to obtain a target fusion feature; determining the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature; adjusting the feature extraction vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; performing knowledge point recognition based on the target text feature sequence to obtain a target knowledge point corresponding to the target test question.

[0018] The above-mentioned test question recognition method, device, computer equipment and storage medium obtain the test question description text in the target test question to be recognized, and the target test question image corresponding to the target test question; the test question description text includes multiple test question segmentations; feature extraction is performed on the test question description text to obtain feature extraction vectors corresponding to the test question segmentations, and the feature extraction vectors are arranged according to the order of the test question segmentations in the test question description text to obtain a test question text feature sequence; feature extraction is performed on the target test question image to obtain test question image features; the test question text feature sequence is fused with the test question image features to obtain target fusion features; based on the target fusion features, the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence is determined; based on the feature attention strength, the feature extraction vectors in the test question text feature sequence are adjusted to obtain a target text feature sequence; knowledge point recognition is performed based on the target text feature sequence to obtain a target knowledge point corresponding to the target test question. Since the target test question to be identified contains test question description text and test question pictures, after the text features corresponding to the test question description text and the image features corresponding to the test question pictures are fused, the fused features obtained contain the fusion information of text information and image information. Therefore, when performing knowledge point recognition, we can refer to the fusion information containing text information and image information, and pay more attention to the text features that are more relevant to the recognition points corresponding to the target test question. Therefore, based on the target text feature sequence, we can obtain the target knowledge points corresponding to the target test question, thereby improving the accuracy of test point recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A diagram showing an application environment of a test question identification method in one embodiment;

[0020] Figure 2 1 is a flow chart of a test question identification method in one embodiment;

[0021] Figure 3 1. A flowchart of steps for fusing a test question text feature sequence with a test question image feature to obtain a target fusion feature in one embodiment;

[0022] Figure 4 A schematic diagram of converting a target test question in one embodiment;

[0023] Figure 5 1. A flowchart illustrating the steps of fusing a target text feature sequence, a first test question image feature, and a second test question image feature to obtain a comprehensive test question feature corresponding to the target test question in one embodiment.

[0024] Figure 6 Schematic diagram of the principle of a test question recognition model in one embodiment;

[0025] Figure 7 Schematic diagram of the encoding principle of a text encoding model in one embodiment;

[0026] Figure 8 is a schematic diagram of the contents included when the target test question is a mathematics test question in another embodiment;

[0027] Figure 9 is a structural block diagram of a test question identification device in one embodiment;

[0028] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0030] The test question recognition method provided in this application can be applied to Figure 1 In the application environment shown. Figure 1As shown, the application environment includes a terminal 102 and a server 104, and the application environment can be an environment for the interaction between the terminal 102 and the server 104. The terminal 102 communicates with the server 104 via a network. The server 104 can respond to the question recognition request sent by the terminal 102 to obtain the target question to be identified. Furthermore, the server 104 obtains the question description text in the target question to be identified, and the target question image corresponding to the target question. The question description text includes multiple question segmentations. The server 104 performs feature extraction on the question description text to obtain feature extraction vectors corresponding to the question segmentations. The server 104 arranges the feature extraction vectors according to the order of the question segmentations in the question description text to obtain a question text feature sequence. The server 104 performs feature extraction on the target question image to obtain question image features. The server 104 fuses the test question text feature sequence with the test question image feature to obtain a target fusion feature. The server 104 determines the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature. The server 104 adjusts the feature extraction vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence. The server 104 performs knowledge point recognition based on the target text feature sequence to obtain a target knowledge point corresponding to the target test question. After the server 104 obtains the target knowledge point corresponding to the target test question, the server 104 can associate the target test question with the target knowledge point and store it. For example, the recognition result can be used as a knowledge point label for the target test question. The server 104 can also perform statistics on the test question information corresponding to the target knowledge point to obtain statistical results. The server 104 can also send the statistical results to the corresponding terminal 102. Alternatively, after the server 104 performs statistics on the test question information corresponding to the target knowledge point, the server 104 returns all the test question information corresponding to the target knowledge point to the terminal 102, so that the user can obtain all the test question information associated with the target knowledge point. The question identification request may be triggered by the server 104 or sent to the server 104 by another device, such as a terminal. The terminal 102 may be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers. It is understood that the question identification method provided in the embodiments of the present application may also be executed by a terminal.

[0031] The embodiments of the present application relate to natural language processing, which can be based on machine learning to process natural language. Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology generally includes technologies such as text processing, semantic understanding, machine translation, robot question answering, knowledge graphs, etc. The solutions provided in the embodiments of the present application relate to various types of natural language processing technology, which are not specifically limited here.

[0032] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The solutions provided in the embodiments of the present application involve various types of machine learning technology, which are not specifically limited here.

[0033] In one embodiment, Figure 2 As shown, a test question recognition method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the server in the example:

[0034] Step 202: Acquire the test question description text in the target test question to be identified, and the target test question image corresponding to the target test question; the test question description text includes a plurality of test question segmentations.

[0035] Among them, the target test questions refer to the test questions with the knowledge points to be identified, and the target test questions can be of different types. For example, according to the subject classification of the test questions, the test questions can be divided into liberal arts test questions and science test questions, etc., and according to the answer format of the test questions, they can be divided into fill-in-the-blank questions and short-answer questions, etc. The target test questions can contain multiple types of content, and multiple types of content refer to different types of content formats, such as known or multiple types of text, picture, and voice formats. For example, the target test questions can contain data in text format and data in picture format.

[0036] The test question description text is a text used to describe the content of the test question. The test question description text may refer to the content in text format contained in the target test question, that is, the test question description text may be the text content of the stem part of the test question. For example, the test question description text may include text in different languages ​​such as Chinese text and English text. The target test question picture is information that describes the target test question in the form of a picture. The target test question picture may include a picture originally contained in the target test question or a picture obtained by converting the information in the target test question into a picture. For example, if a certain science test question contains a picture of a triangle, then the picture of the triangle is the target test question picture. For another example, the entire target test question can be converted into a picture to obtain the target test question picture. There may also be an association between the test question description text and the target test question picture. For example, the target test question picture can be used to display the content to be expressed by the test question description text, so that the user can understand the content of the test question more intuitively.

[0037] Test question segmentation refers to the segmentation obtained after the test question text is segmented. For example, assuming that the test question description text is "The pencil in the picture below is 6 cm long", the test question description text is segmented into one element according to the preset segmentation method, then the multiple test question segmentations included in the test question description text can be obtained as "below", "picture", "middle", "of", "lead", "pen", "long", "6", "centimeter" and "meter". It can be understood that the granularity of the segmentation can be set as needed. For example, it can be segmented in units of characters, words or sentences to obtain segmented text blocks, and each text block is used as a segmentation word. There can be one or more test question segmentations obtained by segmenting the test question description text, and a plurality of refers to at least two. The test question segmentations obtained by different segmentation methods can be different. For example, the segmentation method can include at least one of a unary segmentation method, a binary segmentation method or other segmentation methods.

[0038] Specifically, the server can obtain a test question uploaded by a user and use the uploaded test question as a target test question to be identified. The server can also obtain the test question description text of the target test question to be identified, as well as the target test question image corresponding to the target test question, where the test question description text includes multiple test question segmentation words. The server can also receive a test question identification instruction and use each test question in the test paper stored on the server as a target test question to be identified.

[0039] For example, assuming the target test question to be identified is science test question 01, the server may obtain the test question description text of science test question 01 as "calculate the area of ​​a triangle" and the target test question image corresponding to science test question 01 as an image containing a triangle. After the server segments the test question text according to the preset unary segmentation method, the test question description text may include multiple test question segmentation words such as "calculate", "calculate", "three", "angle", "shape", "surface", and "area".

[0040] Step 204 , extract features from the test question description text to obtain feature extraction vectors corresponding to the test question segmentation words, and arrange the feature extraction vectors according to the order of the test question segmentation words in the test question description text to obtain a test question text feature sequence.

[0041] The feature extraction vector refers to a feature vector obtained by performing feature extraction on the test question description text, that is, the feature extraction vector can be a corresponding feature extraction vector obtained by the server performing feature extraction on each test question word in the test question description text.

[0042] The test question text feature sequence refers to the feature sequence obtained by arranging the feature extraction vectors according to the order of the test question words in the test question description text. For example, the feature extraction vector corresponding to the test question word "lead" is a1, and the feature extraction vector corresponding to the test question word "pen" is a2. According to the order of the test question words in the test question description text, it is "pencil", and the corresponding test question text feature sequence can be "a1 a2".

[0043] Specifically, the server can use the trained text feature extraction model to extract features from the test question description text, obtain the feature extraction vector corresponding to the test question segmentation, and arrange the feature extraction vector according to the order of the test question segmentation in the test question description text to obtain the test question text feature sequence. Among them, the trained text feature extraction model can be a neural network based on artificial intelligence, for example, it can be a convolutional neural network (CNN), and of course it can also be other types of neural networks. The text feature extraction model in this embodiment can be implemented using a Transformer network or a Transformer-based bidirectional encoder (Bidirectional Encoder Representations from Transformers, BERT) network. The BERT model is constructed based on Transformer and can be a model obtained by training a Masked Language Model and predicting the next sentence task.

[0044] For example, suppose the server obtains the question description text of the science test 01 to be identified as "Calculate the area of ​​a triangle", which has 7 word segments. The server can input the above question description text "Calculate the area of ​​a triangle" into the trained BERT model. After processing by the BERT model, the corresponding question text feature sequence is output as: "a1 a2 a3 a4 a5a6 a7".

[0045] Step 206: extract features from the target test question image to obtain test question image features.

[0046] The test question image features refer to the features obtained by extracting features from the target test question images.

[0047] Specifically, the server can use the trained image feature extraction model to extract features from the target test question image to obtain test question image features. Among them, the trained image feature extraction model can be a neural network based on artificial intelligence, for example, it can be a convolutional neural network, or it can be other types of neural networks. The image feature extraction model in this embodiment can be implemented using a residual convolutional network (ResNet, Residual Network), because the network with a deep number of layers can extract the low-level, middle-level and high-level features of the image. However, when the number of network layers is saturated, it will lead to degradation problems, that is, adding more network layers will lead to optimization difficulties, and the training error and prediction error will be larger. Therefore, in this embodiment, the degradation problem is solved by adding a network with a residual structure. When there are multiple target test question images, each target test question image can be input into the image feature extraction model for feature extraction to obtain the target test question image features corresponding to each target test question image.

[0048] For example, the server can input the target test picture into the pre-trained resnet50 network structure, and after processing by the resnet50 network structure, the corresponding test picture features are output.

[0049] Step 208: Fuse the test question text feature sequence with the test question image feature to obtain the target fusion feature.

[0050] The target fusion feature refers to a feature obtained by fusing text features with image features. For example, multiple text features are fused with multiple image features to obtain the target fusion feature. The fusion method can be splicing.

[0051] Specifically, the server can segment the test question description text according to a preset segmentation method, and there are multiple test question segmentations. The server can obtain the corresponding text feature extraction vectors obtained according to each test question segmentation, form each text feature extraction vector into a matrix, and use the formed matrix as a text extraction feature matrix. At the same time, when there are multiple target test question images, the server can form a matrix of test question image features corresponding to each test question image, and use the formed matrix as a picture feature matrix. Furthermore, the server can further process the text extraction feature matrix and the picture feature matrix according to a preset target activation function to obtain the corresponding target fusion feature. It can be understood that the preset target activation function in this embodiment includes but is not limited to the hyperbolic tangent function, and can also be other functions. The calculation method of the activation function is not limited here.

[0052] Step 210 : determining the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature.

[0053] The feature attention strength reflects the intensity of attention paid to a feature. The greater the feature attention strength, the more attention should be paid to the feature during content recognition. When the test question segmentation corresponding to the text extraction feature is different, the feature attention strength corresponding to the text extraction feature can be different.

[0054] Specifically, the feature attention strength can be obtained by processing the target fusion feature using the model parameters obtained in advance for calculating the feature attention. The server can input the target fusion feature into a pre-trained network model, and after processing the model parameters in the network model, output the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence. For example, the server can input the target fusion feature into the activation layer of the pre-trained network model, and after processing the activation layer, output the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence.

[0055] Step 212: Adjust the feature extraction vector in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence.

[0056] The target text feature sequence refers to a text feature sequence obtained by adjusting the test question text feature sequence based on the feature attention strength. That is, the server can adjust the attention strength of each feature extraction vector in the test question text feature sequence according to different feature attention strengths.

[0057] Specifically, the server can obtain a preset adjustment function and, based on the adjustment function and the feature attention strength, calculate the feature extraction vectors in the test text feature sequence to obtain the target text feature sequence. For example, each feature extraction vector can be multiplied by its corresponding feature attention strength to obtain the text features in the target text feature sequence.

[0058] Step 214 , performing knowledge point recognition based on the target text feature sequence to obtain target knowledge points corresponding to the target test question.

[0059] Target knowledge points refer to the labels of the knowledge points that are identified in the target test questions. The knowledge point labels of the test questions can be the category labels of the knowledge points in the knowledge graph, which can be a knowledge graph of knowledge points pre-constructed using the textbook catalog and relevant domain knowledge. For example, the target knowledge points in a physics test question may include acceleration calculation and velocity calculation.

[0060] Specifically, the server can use a pre-trained recognition model to perform knowledge point recognition on the target text feature sequence to obtain the target knowledge point corresponding to the target test question. For example, the recognition model can output the recognition probability corresponding to each candidate knowledge point, and use the candidate knowledge point with the highest recognition probability as the target knowledge point. It is understood that the knowledge point recognition model in this embodiment includes but is not limited to various pre-trained neural network models, or can also be a combination of various neural network models, or other methods. The implementation method of the knowledge point recognition model is not limited here.

[0061] In the above-mentioned test question recognition method, the test question description text in the target test question to be recognized and the target test question picture corresponding to the target test question are obtained; the test question description text includes multiple test question segmentations; feature extraction is performed on the test question description text to obtain feature extraction vectors corresponding to the test question segmentations, and the feature extraction vectors are arranged according to the order of the test question segmentations in the test question description text to obtain a test question text feature sequence; feature extraction is performed on the target test question picture to obtain test question picture features; the test question text feature sequence is fused with the test question picture features to obtain target fusion features; based on the target fusion features, the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence is determined; based on the feature attention strength, the feature extraction vectors in the test question text feature sequence are adjusted to obtain a target text feature sequence; knowledge point recognition is performed based on the target text feature sequence to obtain target knowledge points corresponding to the target test question. Since the target test question to be identified contains test question description text and test question pictures, the text features corresponding to the test question description text and the picture features corresponding to the test question pictures are fused, and the fused features obtained are fusion information containing text information and picture information. Therefore, based on the target fusion features, the feature attention strength corresponding to each feature extraction vector can be determined, and the feature extraction vector in the test question text feature sequence can be adjusted based on the feature attention strength, so that the target text feature sequence containing the fusion information can be obtained. Therefore, recognition based on the target text feature sequence can obtain the target knowledge point corresponding to the target test question, thereby improving the accuracy of the target knowledge point corresponding to the target test question.

[0062] In one embodiment, Figure 3 As shown in FIG, the steps of fusing the test question text feature sequence with the test question image feature to obtain the target fusion feature include:

[0063] Step 302: Acquire text feature fusion parameters corresponding to the test question text feature sequence, and use the text feature fusion parameters to adjust the test question text feature sequence to obtain an adjusted text feature sequence.

[0064] Step 304: Obtain image feature fusion parameters corresponding to the test question image features, and use the image feature fusion parameters to adjust the test question image features to obtain adjusted image features.

[0065] Step 306: Fusing the adjusted text feature sequence and the adjusted image feature to obtain a target fused feature.

[0066] The text feature fusion parameters refer to the fusion parameters corresponding to the test question text features. The image feature fusion parameters refer to the fusion parameters corresponding to the test question image features. The text feature fusion parameters and image feature fusion parameters are model parameters obtained through pre-machine learning training.

[0067] Specifically, assuming that the server extracts features from the test question description text and the target test question image respectively, and obtains the test question text feature sequence H and the test question image feature g, the server can obtain the text feature fusion parameter W corresponding to the test question text feature sequence H h , using the text feature fusion parameter W h Adjust the test text feature sequence H, that is, the server can adjust the calculation method according to the preset adjustment method. For example, the server adjusts the test text feature sequence H and the text feature fusion parameter W h After performing linear operations, the adjusted text feature sequence H1 can be obtained. Further, the server can obtain the image feature fusion parameter W corresponding to the test question image feature g g , using the image feature fusion parameter W g Adjust the test picture feature g, that is, the server can adjust the calculation method according to the preset adjustment method. For example, the server adjusts the test picture feature g and the picture feature fusion parameter W g After performing the linear operation, the adjusted image feature g1 is obtained. The server then performs a fusion calculation on the adjusted text feature sequence H1 and the adjusted image feature g1 according to a preset target adjustment function to obtain the target fusion feature M. It is understood that the calculation method in this embodiment includes but is not limited to linear operations, and can also be nonlinear operations or other hybrid function calculation methods.

[0068] In this embodiment, by respectively obtaining the feature fusion parameters corresponding to the text features and the image features, the different types of features extracted are fused using a fusion method, thereby improving the expression of the test questions and thus improving the precision and accuracy of the automatic recognition of knowledge points in the test questions containing image information.

[0069] In one embodiment, picture feature fusion parameters corresponding to test question picture features are obtained, and the test question picture features are adjusted using the picture feature fusion parameters to obtain the steps of adjusting the picture features, including: obtaining the target feature quantity corresponding to the test question text feature sequence, where the target feature quantity is the number of feature extraction vectors contained in the test question text feature sequence; copying the test question picture features according to the target feature quantity, arranging the copied test question picture features of the target feature quantity to obtain a picture feature sequence; adjusting the picture feature sequence using the picture feature fusion parameters, and using the adjusted picture feature sequence as the adjusted picture feature.

[0070] The target feature number refers to the number of feature extraction vectors contained in the test text feature sequence. For example, if the test text feature sequence is a1a2a3a4, the target feature number is 4. The image feature sequence refers to the feature sequence obtained by arranging the target number of copied test image features. For example, if the test image feature is g, and 4 test image features g are copied, the arranged image feature sequence is gggg.

[0071] Specifically, assuming that the server extracts features from the test question description text and the target test question image respectively, and obtains the test question text feature sequence H and the test question image feature g, the server can obtain the target feature number corresponding to the test question text feature sequence H, and the target feature number is the number of feature extraction vectors contained in the test question text feature sequence. Assuming that the server obtains the target feature number N corresponding to the test question text feature sequence H, the server can copy the test question image feature g according to the target feature number, that is, the server copies the test question image feature g N times, and the copied image feature sequence G has the same feature number as the test question text feature sequence H, that is, the target feature number corresponding to the image feature sequence G is also N. That is, in this embodiment, the test question text feature and the test question image feature are of the same dimension, for example, they can be vectors or matrices of the same dimension. If the dimensions are different, the test question image feature is converted into the same dimension as the test question text feature. In other words, the number of image feature values ​​included in the test question image feature is the same as the number of text feature values ​​included in the test question text feature.

[0072] For example, assuming that the test question text feature is a vector A = [a1, a2, a3], the server can copy the test question image feature g according to the target feature quantity, that is, the server copies the test question image feature g 3 times, and the copied image feature B has the same number of features as the test question text feature A, that is, the test question image feature is a vector B = [b1, b2, b3], where vector A includes 3 elements, namely a1, a2 and a3, and each element in vector A is a text feature value. Similarly, vector B also includes 3 elements, namely b1, b2 and b3, and each element in vector B is a picture feature value.

[0073] Furthermore, the server can use the image feature fusion parameter W g The image feature B is adjusted, and the adjusted image feature sequence is used as the adjusted image feature B1. It can be understood that the test question image feature in this embodiment can be multiple. For example, the server can obtain the test question image feature g and the test question image feature k, and then can copy and adjust the test question image feature g and the test question image feature k respectively according to the number of target features. As a result, by copying the image features and converting them into a feature sequence with the same dimension as the text features, the image features and text features of the same latitude can be fused subsequently, and the fused features obtained contain both text information and image information, thereby improving the precision and accuracy of automatic recognition of test question knowledge points containing image information.

[0074] In one embodiment, the step of fusing the adjusted text feature sequence and the adjusted image feature to obtain a target fusion feature includes: using the features in the adjusted text feature sequence as matrix values ​​in a first matrix object corresponding to the adjusted text feature sequence in a target feature matrix to obtain a first matrix object; using the features in the adjusted image feature sequence as matrix values ​​in a second matrix object corresponding to the adjusted image feature sequence in a target feature matrix to obtain a second matrix object; using the matrix formed by the first matrix object and the second matrix object as a target feature matrix; and obtaining a target fusion feature based on the target feature matrix.

[0075] The target feature matrix is ​​a matrix formed by the first matrix object and the second matrix object, which can be used to obtain the target fusion features. For example, the target feature matrix can be used as the target fusion matrix, or the target feature matrix can be activated to obtain the target fusion matrix. The first matrix object refers to the text feature matrix, and the second matrix object refers to the image feature matrix. A matrix object is a row or a column. Matrix values ​​refer to the eigenvalues ​​contained in the matrix object, which can include text eigenvalues ​​and image eigenvalues.

[0076] Specifically, assume that the adjusted text feature sequence is A=[a1, a2, a3], and the adjusted image feature sequence is B=[b1, b2, b3], wherein the adjusted text feature sequence A includes 3 elements, namely a1, a2 and a3, and each element in the adjusted text feature sequence A is a text feature value. Similarly, the adjusted image feature sequence B also includes 3 elements, namely b1, b2 and b3, and each element in the adjusted image feature sequence B is an image feature value. The server can use the features in the adjusted text feature sequence A as the matrix values ​​in the first matrix object corresponding to the adjusted text feature sequence in the target feature matrix, and obtain the first matrix object as [a1, a2, a3]. Further, the server uses the features in the adjusted image feature sequence B as the matrix values ​​in the second matrix object corresponding to the adjusted image feature sequence in the target feature matrix, and obtains the second matrix object [b1, b2, b3]. The server uses the matrix formed by the first matrix object and the second matrix object as the target feature matrix, and the target matrix m can be obtained as follows: The server calculates the target fusion feature M based on the target matrix m. For example, the server can calculate the target matrix m based on a preset hyperbolic tangent function to obtain the target fusion feature M. Thus, by fusing image features and text features at the same latitude, the resulting fusion feature contains both text information and image information, providing a fusion feature containing polymorphic information for subsequent knowledge point recognition, thereby improving the precision and accuracy of automatic recognition of knowledge points in test questions containing image information.

[0077] In some embodiments, the formula for calculating the target fusion feature M may be as follows:

[0078]

[0079] Among them, in formula (1) W h ∈R d×d , M represents the target fusion feature, tanh represents the hyperbolic tangent function, W h represents the question feature fusion parameter, H represents the question text feature sequence, W k Represents the first image feature fusion parameter, W g represents the second picture feature fusion parameter, k represents the first test question picture feature, g represents the second test question picture feature, Represents matrix multiplication, which physically means copying k N times and copying g N times. N =[1, 1, ..1], indicating N 1s.

[0080] In one embodiment, the step of determining the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature includes: obtaining the fusion feature processing parameters, processing the target fusion feature using the fusion feature processing parameters, and obtaining the attention strength vector; the attention strength vector includes the vector value of the target feature quantity, and the target feature quantity is the number of feature extraction vectors contained in the test question text feature sequence; the vector value in the attention strength vector that is consistent with the order of the feature extraction vectors in the test question text feature sequence is used as the feature attention strength corresponding to the feature extraction vector.

[0081] The fusion feature processing parameters refer to the parameters corresponding to the target fusion features, which can be model parameters obtained through machine learning training in advance. The attention strength vector is a parameter used to represent the feature attention strength.

[0082] Specifically, the server fuses the test question text feature sequence with the test question image feature to obtain the target fusion feature M. The server can then obtain the fusion feature processing parameter ω T , using fusion feature processing parameter ω T The target fusion feature M is processed to obtain an attention strength vector M1. The attention strength vector M1 includes vector values ​​of the target feature number N, where the target feature number N is the number of feature extraction vectors included in the test text feature sequence H. Furthermore, the server uses the vector value in the attention strength vector M1 that is consistent with the order of the feature extraction vectors in the test text feature sequence H as the feature attention strength α corresponding to the feature extraction vector.

[0083] For example, suppose the server uses the fusion feature processing parameter ω TAfter processing the target fusion feature M, we obtain the attention strength vector M1 = [m1, m2, m3]. The attention strength vector M1 includes three vector values, that is, the attention strength vector M1 includes three elements, namely m1, m2, and m3. Each element in the attention strength vector M1 is a vector value. The server then uses the vector value in the attention strength vector M1 that is consistent with the feature extraction vector in the test text feature sequence H = [h1, h2, h3] as the feature attention strength α corresponding to the feature extraction vector. That is, the server uses m1 as the feature attention strength α1 corresponding to the feature extraction vector h1, m2 as the feature attention strength α2 corresponding to the feature extraction vector h2, and m3 as the feature attention strength α3 corresponding to the feature extraction vector h3. As a result, after processing the target fusion features using the fusion feature processing parameters, the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence can be obtained, so that the feature extraction vectors in the test question text feature sequence can be adjusted based on the feature attention strength to obtain the target text feature sequence, which effectively improves the expression of the target test question, thereby improving the precision and accuracy of automatic recognition of test question knowledge points containing image information.

[0084] In some embodiments, the formula for calculating the feature attention strength α may be as follows:

[0085] α=softmax(ω Τ M) (2)

[0086] Among them, α∈R in formula (2) N , α represents the feature attention strength, M represents the target fusion feature, is the parameter to be learned by the network model, ω T Represents the weight parameter, softmax represents the activation function, and softmax is a normalized exponential function, which is used as the activation function here.

[0087] In one embodiment, the target test question picture includes a first test question picture, and the step of obtaining the target test question picture corresponding to the target test question includes: obtaining content description information corresponding to the target test question, the content description information describing the test question content corresponding to each content type in the target test question and the association relationship between the test question content; generating the first test question picture corresponding to the target test question according to the content description information; wherein, when generating the first test question picture, determining the display method of the test question content on the first test question picture according to the content type corresponding to the test question content, and determining the display position of the test question content on the first test question picture according to the association relationship between the test question content.

[0088] Among them, the first test question image refers to a picture that contains all the information in the target test question. Content description information refers to descriptive text used to describe the test question. For example, the content description information can be HTML (HyperText Markup Language) text. The full name of HTML is Hypertext Markup Language, which is a markup language. The target test question can contain different types of content. For example, the target test question can contain text content and picture content, etc. HTML text can be used to describe various types of content such as text content and picture content in the target test question, that is, it describes all types of information contained in the target test question. The association relationship between text content and picture content is, for example, the positional relationship between text content and picture content displayed on the Internet. Display method refers to the way information is presented, for example, whether it is reflected as text in the picture or as a picture.

[0089] Specifically, the server may obtain content description information corresponding to the target test question, where the content description information describes the test question content corresponding to each content type in the target test question and the associations between the test question content. The server may generate a first test question image corresponding to the target test question based on the content description information. When generating the first test question image, the server may determine how the test question content is displayed on the first test question image based on the content type corresponding to the test question content, and may determine the display position of the test question content on the first test question image based on the associations between the test question content.

[0090] For example, Figure 4 The figure shows a schematic diagram of converting the target test question. The content can be described in HTML. The server can obtain the HTML code corresponding to the target test question. The HTML code describes the text content, image content, and the relationship between the text content and the image content in the target test question.

[0091] In one embodiment, Figure 4 As shown, the server can use imgkit to convert the HTML corresponding to the target test question into an image format. imgkit is a Python toolkit that can convert web pages into images. Using imgkit, you can convert a web page into an image. That is, the input content can be any one of a URL, an HTML file, or a string. Because the target test question needs to be converted into an image format, the imgkit tool can be directly used for processing in this embodiment. That is, the server can use imgkit to convert the content description information corresponding to the target test question, that is, the entire content of the question stem, into an image format, obtaining the first test question image corresponding to the target test question.

[0092] In this embodiment, the target test question is converted into a picture format in order to obtain the original question information, that is, the original test question information containing text information and picture information, which can provide fusion features containing polymorphic information for subsequent knowledge point recognition, thereby making the recognition results obtained when automatically identifying the target test question knowledge points more accurate.

[0093] In one embodiment, the target test question picture further includes a second test question picture, and the step of obtaining the target test question picture corresponding to the target test question includes: using the test question picture in the image format of the target test question as the second test question picture;

[0094] The first test question picture includes the test question description text and the picture content in the second test question picture; the test question picture features include the first test question picture features and the second test question picture features; feature extraction is performed on the target test question picture to obtain the test question picture features, including: inputting the first test question picture into the fusion extraction model for feature extraction to obtain the first test question picture features that are fused with the test question description text and the picture content; inputting the second test question picture into the picture extraction model for feature extraction to obtain the second test question picture features.

[0095] The second question image refers to the question image information in the target question format. For example, in a math question, in addition to text information, there will also be corresponding graphic information in the image format. For example, in a question, in addition to the text information about calculating the area of ​​a triangle, there will also be an image of the triangle whose area is to be calculated.

[0096] Specifically, the server can use the test question image in the image format of the target test question as the second test question image, the first test question image includes the test question description text and the image content in the second test question image, and the test question image features include the first test question image features and the second test question image features. Furthermore, the server can extract the image features corresponding to the two test question images respectively, that is, the server can input the first test question image into the fusion extraction model for feature extraction to obtain the first test question image features that integrate the test question description text and the image content, and input the second test question image into the image extraction model for feature extraction to obtain the second test question image features. Among them, the fusion extraction model and the image extraction model can be pre-trained residual network models, for example, it can be an extraction model of the resnet50 network structure, or other pre-trained feature extraction models can be used to replace resnet50, such as inception v3 or resnet18 network structure models.

[0097] It can be understood that the fusion extraction model and the image extraction model in this embodiment can be the same neural network model, or different neural network models, or a combination of various neural network models, or other methods. The method of pre-training the fusion extraction model and the image extraction model can be different. There is no specific limitation on the model structure and training method of the fusion extraction model and the image extraction model.

[0098] As a result, by extracting the features of image information, text features of text information, and original question features of original question information respectively, it is possible to provide fusion features containing polymorphic information for subsequent knowledge point recognition, thereby making the recognition results obtained when automatically identifying the target test knowledge points more accurate.

[0099] In one embodiment, the steps of performing knowledge point recognition based on the target text feature sequence to obtain target knowledge points corresponding to the target test question include: fusing the target text feature sequence, the first test question image feature, and the second test question image feature to obtain comprehensive test question features corresponding to the target test question; and performing knowledge point recognition based on the comprehensive test question features to obtain target knowledge points corresponding to the target test question.

[0100] Among them, the comprehensive features of the test questions refer to the comprehensive features that integrate text features and image features.

[0101] Specifically, the server adjusts the feature extraction vectors in the test question text feature sequence based on the feature attention strength. After obtaining the target text feature sequence, the server can fuse the target text feature sequence, the first test question image features, and the second test question image features to obtain the test question comprehensive features corresponding to the target test question. Furthermore, the server performs knowledge point recognition based on the test question comprehensive features to obtain the target knowledge point corresponding to the target test question.

[0102] For example, the server can input the target text feature sequence, the first test question image feature, and the second test question image feature into the fully connected layer for fusion processing, and the fusion obtains the test question comprehensive feature corresponding to the target test question. Furthermore, the server can input the test question comprehensive feature into a pre-trained recognition model for knowledge point recognition, and output the recognition result corresponding to the target test question. Among them, the recognition result includes category labels and confidence levels. After the server obtains the recognition result, the server can filter based on a preset confidence threshold. For example, if the confidence threshold is 0.5, the server will output the label with a confidence level greater than 0.5 in the category label as the target knowledge point label, and map it to the real knowledge point label based on the mapping relationship between the category label and the real knowledge point label. In this way, the different types of features extracted are fused using a fusion method, which improves the expression of the test question, thereby improving the precision and accuracy of the automatic recognition of test question knowledge points containing image information.

[0103] In one embodiment, Figure 5 As shown, the step of fusing the target text feature sequence, the first test question image feature, and the second test question image feature to obtain the test question comprehensive feature corresponding to the target test question includes:

[0104] In step 502, the target text feature sequence, the first test question image feature, and the second test question image feature are input into the comprehensive feature extraction layer for feature extraction. The comprehensive feature extraction layer processes the target text feature sequence, the first test question image feature, and the second test question image feature respectively using the parameters corresponding to each feature to obtain the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature.

[0105] Step 504 : Concatenate the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature to obtain a comprehensive test question feature corresponding to the target test question.

[0106] The comprehensive feature extraction layer refers to a network layer used to extract comprehensive features. For example, the comprehensive feature extraction layer can be a linear layer in a comprehensive feature extraction model.

[0107] Specifically, the server adjusts the feature extraction vectors in the test question text feature sequence based on the feature attention strength. After obtaining the target text feature sequence, the server can input the target text feature sequence, the first test question image feature, and the second test question image feature into the comprehensive feature extraction layer for feature extraction. The comprehensive feature extraction layer uses the parameters corresponding to each feature to process the target text feature sequence, the first test question image feature, and the second test question image feature, respectively, to obtain the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature. Further, the server splices the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature to obtain the comprehensive test question feature corresponding to the target test question.

[0108] For example, the server can input the target text feature sequence, the first test question image feature, and the second test question image feature into a pre-trained comprehensive feature extraction model for feature extraction. The comprehensive feature extraction model uses the weight parameters corresponding to each feature to process the target text feature sequence, the first test question image feature, and the second test question image feature, respectively, to obtain the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature. Further, the server inputs the processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature into a splicing layer for splicing to obtain the comprehensive test question feature corresponding to the target test question. The splicing layer can be implemented using a concat layer.

[0109] As a result, by obtaining the corresponding fusion parameters of the text features, the first image features and the second image features respectively, the different types of features extracted are fused using a fusion method, which improves the expression of the test questions and thus improves the precision and accuracy of the automatic recognition of knowledge points in the test questions containing image information.

[0110] In some embodiments, the comprehensive features of the test questions are calculated The formula can be shown as follows:

[0111]

[0112] Among them, in formula (3) represents the comprehensive characteristics of the test questions, tanh represents the hyperbolic tangent function, r is the test question representation that integrates polymorphic image information, and W r It represents the test question feature fusion parameter, which is used to represent the weight parameter and is the parameter that the model needs to learn. n represents the test text feature sequence, W h represents the text feature fusion parameter, k represents the first test question image feature, g represents the second test question image feature, W k Represents the first image feature fusion parameter, W g Represents the second image feature fusion parameter.

[0113] In one embodiment, the step of obtaining the test question description text in the target test question to be identified includes: obtaining the text information in the target test question; performing character type recognition on the text information to identify the symbol string and the numeric string in the text information; filtering the text information based on the string position of the symbol string and the string position of the numeric string to obtain a mixed string including the symbol string and the numeric string adjacent to each other; filtering the mixed string from the text information and using the remaining text information as the test question description text. Among them, the character type refers to the type of the string, for example, the string can include a symbol string and a numeric string, etc. In programming, a string is a continuous sequence of symbols or numerical values, such as a symbol string (a string of characters) or a binary numeric string (a string of binary digits).

[0114] Specifically, the server may obtain text information from the target test question and perform character type recognition on the text information to identify symbol strings and numeric strings in the text information. Furthermore, based on the string positions of the symbol strings and the string positions of the numeric strings, the server may filter the text information to obtain mixed strings containing adjacent symbol strings and numeric strings. The server then removes the mixed strings from the text information and uses the remaining text information as the test question description text. The symbol strings may include arithmetic symbols such as addition, subtraction, multiplication, and division.

[0115] For example, Figure 4 As shown, the server obtains the text information from the target test question, "Test 01: Solve the function value y according to the formula y = -3x + b." The server performs character type recognition on the text information and identifies the symbolic string and numeric string in the text information as "y, =, -3, x, +, b, y." Furthermore, based on the string positions of the symbolic string and the numeric string, the server can filter the text information to obtain a mixed string "y, =, -3, x, +, b" containing adjacent symbolic and numeric strings. The server then removes the mixed string "y, =, -3, x, +, b" from the text information "Solve the function value y according to the formula y = -3x + b" and uses the remaining text information "Solve the function value y according to the formula" as the test question description. Since semantically invalid formulas can be filtered out from the text information, text recognition efficiency can be improved. Semantically invalid formulas can be converted into target test question images for feature extraction, which can also extract the formula information.

[0116] By extracting the test question description text and test question image information from the target test question separately, the subsequent processing steps can use a fusion method to fuse the extracted different types of features, thereby improving the expression of the test question and thus improving the precision and accuracy of automatic recognition of test question knowledge points containing image information.

[0117] This application also provides an application scenario, which applies the above-mentioned test question recognition method. Specifically, the application of the test question recognition method in this application scenario is as follows:

[0118] When a user needs to conduct personalized learning, the above-mentioned test question identification method can be used, that is, the user logs in to the personalized learning system, and the user can customize and upload the target test questions to be identified to the learning system, or directly select the target test questions to be identified from the system, so that the system automatically identifies the knowledge points of the target test questions and obtains the target knowledge points corresponding to the target test questions. The personalized learning system can be a cloud platform education system, which can be implemented with an independent server or a server cluster composed of multiple servers. For example, when a user needs to identify the target knowledge points corresponding to the test questions, the education system can start the automatic identification function of the system, for example, start the application program for identifying the test question knowledge points, so that the education system can automatically identify the target knowledge points corresponding to the test questions, and return all the test questions corresponding to the target knowledge points or related test questions to the user interface, so that the user can directly view the test questions related to the target knowledge points in the display interface.

[0119] For example, Figure 4 As shown, assuming that the server obtains the target test question to be identified as "Solve the function value y according to the formula y=-3x+b" and an attached figure, which is an image in a plane rectangular coordinate system, the server can obtain the test question description text in the above target test question, which is "Solve the function value y according to the formula y=-3x+b", and the target test question image corresponding to the target test question is the image in the plane rectangular coordinate system. The server can use the trained text feature extraction model to extract features from the test question description text, obtain the feature extraction vector corresponding to the test question segmentation, and arrange the feature extraction vector according to the order of the test question segmentation in the test question description text to obtain a test question text feature sequence. The server can also use the trained image feature extraction model to extract features from the target test question image to obtain test question image features. The server fuses the test question text feature sequence with the test question image features to obtain target fusion features. The server determines the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature, and adjusts the feature extraction vector in the test question text feature sequence based on the feature attention strength to obtain the target text feature sequence; then the server identifies the knowledge points based on the target text feature sequence, and obtains the target knowledge point label corresponding to the target test question as a "unary function".

[0120] The method provided in the embodiment of the present application can be applied to the scenario of automatic labeling and identification of test question knowledge points. The following uses the automatic labeling and identification of test question knowledge points as an example to illustrate the test question identification method provided in the embodiment of the present application, including the following steps:

[0121] The prediction of test question knowledge points is widely used in educational products such as test question resource search, test question recommendation, question bank construction, and personalized learning. Through the automatic labeling of test question knowledge points, on the one hand, the problem of inconsistent knowledge point labeling standards can be solved, and on the other hand, the cost of manual labeling can be reduced. However, there are a large number of pictures and formula information in science test questions. If you rely solely on text information for automatic labeling or recognition, the effect is often not good for test questions that lack text information but contain a large amount of picture information. In order to improve the accuracy of automatic labeling and recognition of test question knowledge points with relatively little text content and containing picture information, this embodiment proposes a method for automatic labeling and recognition of test question knowledge points based on multimodal information fusion of a neural network model, which specifically includes:

[0122] First, the server uses the ResNet model (Residual Network) to extract features from the image in the target test question and obtain image features. Then, the server uses the BERT model (Bidirectional Encoder Representations from Transformers, a deep language representation model) to encode and represent the text content and obtain text features. The server then uses the ResNet model to extract features from the original question containing text content and images and obtain the original question image features. The server again uses the attention mechanism to fuse the original question features, image features, and text content encoding to obtain a text representation containing polymorphic image information. Finally, the server concatenates the original question features, image features, text representation, and text representation containing polymorphic image information, and after processing through a fully connected layer, outputs the target text feature sequence corresponding to the target test question. The server then inputs the target text feature sequence into a pre-trained classification and recognition model. After processing through a fully connected layer and a sigmoid classifier (sigmoid as the activation function), the server outputs the final result, which is the knowledge point label category and confidence level corresponding to the target test question. Therefore, by using the attention mechanism combined with the neural network model processing to obtain a text representation containing polymorphic image information, the expression of the test questions is improved by using the fusion feature method, thereby improving the accuracy and precision of the automatic labeling and recognition of test knowledge points.

[0123] The test question recognition method of this embodiment can be used for automatic annotation and recognition of test question knowledge points in the recording and arrangement system for question bank construction, test question recommendation system, and personalized learning system. In order to improve the accuracy of automatic annotation and recognition of test question knowledge points that lack text information but contain image information, this embodiment proposes a method for automatic annotation and recognition of test question knowledge points based on multimodal information fusion of neural network model, and the specific steps are as follows:

[0124] The server obtains the test question description text in the target test question to be identified, as well as the target test question image corresponding to the target test question. The test question description text includes multiple test question segmentation words. Figure 6 As shown in the figure, it is a schematic diagram of the principle of the test question recognition model. Figure 6 The square box represents a feature representation, and the number of circles within the box represents different feature representations. Arrows indicate the flow of data. Each arrow carries data, representing the flow from the output of the previous node to the input of another node. Circles represent point-by-point operations, such as vector addition and point-by-point multiplication. A "+" within a circle indicates vector addition. In this example, a "+" within a circle indicates fusion and concatenation.

[0125] For example, Figure 6 As shown, the server inputs the test image of the target test question into the pre-trained resnet50 network structure in the test question recognition model, and then passes it through a fully connected network layer to output the feature representation g of the test question image. The resnet50 network structure is used in this embodiment because it has a good recognition effect. It is understandable that the training method of the resnet50 network structure is not limited in the embodiments of this application.

[0126] In one embodiment, Figure 6 As shown, the server can use imgkit to convert the HTML of the target test question into image format to obtain the original question image. The server then inputs the complete original question image into a resnet50 network structure, and then passes it through a fully connected network layer to obtain the complete original question image feature representation k.

[0127] In one embodiment, Figure 6 As shown, the server extracts features from the question description text, obtaining feature extraction vectors corresponding to the question segmentation words. The feature extraction vectors are then arranged according to the order of the question segmentation words in the question description text, resulting in a question text feature sequence. Specifically, the server inputs the question description text into a pre-trained BERT model and uses BERT's built-in segmentation tool to segment and concatenate the text content of the question, including the question stem, answer, and explanation. The server then searches the BERT model's dictionary to convert the word IDs corresponding to the words and inputs them into the BERT model. The question description text is then input into the component annotation module for tokenization, outputting a sequence of labeled question segmentation words, Tok1, Tok2, ..., TokN. Furthermore, this labeled segmentation word sequence, Tok1, Tok2, ..., TokN, is used as input parameters for the BERT model and fed into the BERT model for encoding, outputting the segmentation encoding vectors T1, T2, ..., TN corresponding to each segmentation word.

[0128] like Figure 7As shown in the figure, it is a schematic diagram of the encoding principle of the text encoding model. The server can segment the test question description text to obtain N word tokens (tokens), where N is a positive integer, denoted as Tok1, Tok2... TokN. Add the marker "[CLS]" before Tok1. [CLS] represents "classification". E represents an embedding vector. For example, the embedding vector of [CLS] is E[CLS]. T represents the encoded vector, and C is the semantic vector, which is the semantic representation formula corresponding to [CLS], that is, the semantic encoding vector. That is, the server can input the test question description text including N word tokens into the encoding model, that is, the bert model. The encoding model outputs the semantic representation formula C (referred to as the target semantic vector) of [CLS] corresponding to the test question description text, and the word token encoding vectors T corresponding to each word token. Among them, the Bert model comes with a word segmentation tool that can segment the text string according to the dictionary provided by bert. The bert dictionary is a vocab.txt file of the bert dictionary, which contains characters such as "我", "##亳", "##vel". That is, each character corresponds to an id, and each id corresponds to the word vector of a certain word in the bert model. The embedding vectors E[CLS], E1, E2... En corresponding to the character ids are obtained by looking up the table. This is the Embedding layer of the bert model. The Embedding layer is used to convert the id of the word into the word vector of the bert model, which is a process of looking up the table.

[0129] Furthermore, the server can input the word vectors into a 6-layer Transformer model to obtain the expressions h[CLS], h1, h2..., hn of the output test question text, denoted as H = {h[CLS], h1, h2..., hn}. The Transformer model refers to a model based on the encoder-decoder structure. In order to obtain the text representation H, the word vectors need to be input into the 6-layer Transformer. The Bert model in this embodiment can be composed of 12 layers of Transformers, and the expression H of the test question text is the output of the sixth-layer model.

[0130] The server fuses the feature representation g of the test question picture, the feature representation k of the complete original question picture, and the text feature H output by the bert model. That is, the server uses the attention mechanism to fuse the two picture feature information and a test question text expression H. The specific fusion formula is shown in the following formulas (1)(2)(3)(4):

[0131]

[0132] α = softmax(ω Τ M) (2)

[0133]

[0134] r=Hα T (4)

[0135] Among them, r∈R in formula (4) d , r is the question representation that integrates polymorphic image information, H represents the question text feature sequence, α represents the feature attention strength, and T represents the transposed matrix operation.

[0136] As shown in the above formula (3), the server can combine the text representation r that integrates the polymorphic image information, the test question image feature representation g, the complete original question image feature representation k, and the text representation h that contains the polymorphic information. n Perform splicing and fusion, and obtain the comprehensive features of the test questions through a linear layer processing The server will test comprehensive features After being processed by a fully connected layer and a sigmoid classifier, the final recognition result is output.

[0137] When training the test question recognition model, the loss function can be sigmoid entry loss, and the maximum character sequence length can be 500. The Adam algorithm can be used to optimize the parameters of each model layer, with the learning rate set to 0.000125 and the batch_size set to 32. After the model training is complete, save the model. This model can be used to label the knowledge points tested in the pre-test questions, and the output is the label category and confidence rate. The Adam algorithm is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process and iteratively updates the neural network weights based on the training data. It has the advantages of fast speed and simple parameter adjustment.

[0138] For example, Figure 8 The following is a schematic diagram of the content included when the target test question is a mathematics test question. Figure 8 In the example, the content of the target test question can be divided into three parts, namely, the test question description text, the test question image, and the original question image. The server can input the target test question 01 into the test question recognition model, first convert the complete test question HTML into an image, that is, convert it into the original question image, then process the original question image through a RESTNet50 network structure, and output the complete test question expression K. The image in the test question, that is, the test question image, is processed through a RESTNet50 network to obtain the test question image representation G. The text information of the test question "Test question 01: The pencil in the picture below is 6 cm long" is processed through the BERT model, and the output of the sixth layer is taken to obtain the text representation R. Then the text representation R, the image feature representation G, the complete original question image feature representation K, and the text representation H containing polymorphic information are combined. nPerform splicing and fusion, and obtain the comprehensive features of the test questions through a linear layer processing The server will test comprehensive features After processing through a fully connected layer and a sigmoid classifier, the final recognition result is output. After the server receives the recognition result, it can filter it based on a preset confidence threshold. For example, if the confidence threshold is 0.5, the server will use the output category labels with a confidence level greater than 0.5 as the target knowledge point label. Based on the mapping relationship between category labels and real knowledge point labels, the server maps them to the real knowledge point label "Calculate the volume of a cylinder."

[0139] In this embodiment, by using the attention mechanism in combination with the RestNet and Bert models, a text representation containing polymorphic image information is obtained, and a fusion method is used to fuse different types of feature representations, thereby improving the expression of test questions and thus improving the accuracy of automatic labeling and recognition of knowledge points in test questions that lack text information but contain image information.

[0140] It should be understood that although Figure 1-8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1-8 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0141] In one embodiment, Figure 9 As shown, a test question recognition device is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: an acquisition module 902, a text feature extraction module 904, an image feature extraction module 906, a fusion module 908, a feature attention strength determination module 910, an adjustment module 912 and a recognition module 914, wherein:

[0142] The acquisition module 902 is used to acquire the test question description text in the target test question to be identified, and the target test question image corresponding to the target test question; the test question description text includes multiple test question segmentations.

[0143] The text feature extraction module 904 is used to extract features from the test question description text to obtain feature extraction vectors corresponding to the test question segmentation words. The feature extraction vectors are arranged in the order of the test question segmentation words in the test question description text to obtain a test question text feature sequence.

[0144] The image feature extraction module 906 is used to extract features from the target test question image to obtain test question image features.

[0145] The fusion module 908 is used to fuse the test question text feature sequence with the test question image feature to obtain the target fusion feature.

[0146] The feature attention strength determination module 910 is used to determine the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature.

[0147] The adjustment module 912 is used to adjust the feature extraction vectors in the test text feature sequence based on the feature attention strength to obtain the target text feature sequence.

[0148] The recognition module 914 is used to recognize knowledge points based on the target text feature sequence to obtain target knowledge points corresponding to the target test questions.

[0149] In one embodiment, the fusion module includes: a text feature fusion parameter acquisition unit, which is used to obtain text feature fusion parameters corresponding to the test question text feature sequence, and use the text feature fusion parameters to adjust the test question text feature sequence to obtain an adjusted text feature sequence; a picture feature fusion parameter acquisition unit, which is used to obtain picture feature fusion parameters corresponding to the test question picture feature, and use the picture feature fusion parameters to adjust the test question picture feature to obtain an adjusted picture feature; and a fusion unit, which is used to fuse the adjusted text feature sequence and the adjusted picture feature to obtain a target fusion feature.

[0150] In one embodiment, the image feature fusion parameter acquisition unit is used to: obtain the target feature quantity corresponding to the test question text feature sequence, where the target feature quantity is the number of feature extraction vectors contained in the test question text feature sequence; copy the test question image features according to the target feature quantity, and arrange the copied test question image features of the target feature quantity to obtain a picture feature sequence; adjust the picture feature sequence using the picture feature fusion parameter, and use the adjusted picture feature sequence as the adjusted picture feature.

[0151] In one embodiment, the fusion module is used to: use the features in the adjusted text feature sequence as the matrix values ​​in the first matrix object corresponding to the adjusted text feature sequence in the target feature matrix to obtain a first matrix object; use the features in the adjusted image feature sequence as the matrix values ​​in the second matrix object corresponding to the adjusted image feature sequence in the target feature matrix to obtain a second matrix object; use the matrix formed by the first matrix object and the second matrix object as the target feature matrix; and obtain target fusion features based on the target feature matrix.

[0152] In one embodiment, the feature attention strength determination module is used to: obtain fusion feature processing parameters, use the fusion feature processing parameters to process the target fusion feature, and obtain an attention strength vector; the attention strength vector includes a vector value of the number of target features, and the number of target features is the number of feature extraction vectors contained in the test text feature sequence; the vector value in the attention strength vector that is consistent with the order of the feature extraction vectors in the test text feature sequence is used as the feature attention strength corresponding to the feature extraction vector.

[0153] In one embodiment, the target test question picture includes a first test question picture, and the acquisition module includes: a content description information acquisition unit, used to obtain content description information corresponding to the target test question, the content description information describes the test question content corresponding to each content type in the target test question and the association relationship between the test question contents; an image generation unit, used to generate a first test question picture corresponding to the target test question according to the content description information; wherein, when generating the first test question picture, the display method of the test question content on the first test question picture is determined according to the content type corresponding to the test question content, and the display position of the test question content on the first test question picture is determined according to the association relationship between the test question contents.

[0154] In one embodiment, the target test question picture includes a first test question picture, and the acquisition module includes: a second test question picture acquisition unit is used to: use the test question picture in picture format in the target test question as the second test question picture; the first test question picture includes the test question description text and the picture content in the second test question picture; the test question picture features include the first test question picture features and the second test question picture features; the picture feature extraction module is used to: input the first test question picture into the fusion extraction model for feature extraction, and obtain the first test question picture features that integrate the test question description text and the picture content; input the second test question picture into the picture extraction model for feature extraction, and obtain the second test question picture features.

[0155] In one embodiment, the recognition module includes: a test question comprehensive feature acquisition unit, which is used to fuse the target text feature sequence, the first test question image feature, and the second test question image feature to obtain the test question comprehensive feature corresponding to the target test question; and an identification unit, which is used to identify knowledge points based on the test question comprehensive feature to obtain the target knowledge points corresponding to the target test question.

[0156] In one embodiment, a test question comprehensive feature acquisition unit is used to input the target text feature sequence, the first test question picture feature, and the second test question picture feature into the comprehensive feature extraction layer for feature extraction. The comprehensive feature extraction layer uses the parameters corresponding to each feature to process the target text feature sequence, the first test question picture feature, and the second test question picture feature respectively to obtain the processed target text feature sequence, the processed first test question picture feature, and the processed second test question picture feature; the processed target text feature sequence, the processed first test question picture feature, and the processed second test question picture feature are spliced ​​to obtain the test question comprehensive feature corresponding to the target test question.

[0157] In one embodiment, the acquisition module is used to: obtain text information in the target test question; perform character type recognition on the text information to obtain the symbol string and the operation string in the text information; filter the text information based on the string position of the symbol string and the string position of the operation string to obtain a mixed string including the symbol string and the operation string adjacent to each other; filter out the mixed string from the text information and use the remaining text information as the test question description text.

[0158] For the specific definition of the test question identification device, please refer to the definition of the test question identification method above, which will not be repeated here. The various modules in the above-mentioned test question identification device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0159] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store test question recognition data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a test question recognition method is implemented.

[0160] Those skilled in the art will understand that Figure 10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0161] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0162] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0163] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0164] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0165] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A test question recognition method, characterized in that: The method comprises: Obtaining a test question description text in a target test question to be identified, and a target test question image corresponding to the target test question; the test question description text includes a plurality of test question segmentations; the target test question is a science test question; the target test question image includes an image originally included in the target test question, and an image obtained by converting information in the target test question into an image; Performing feature extraction on the test question description text to obtain feature extraction vectors corresponding to the test question segmentation words, and arranging the feature extraction vectors according to the order of the test question segmentation words in the test question description text to obtain a test question text feature sequence; Performing feature extraction on the target test question image to obtain test question image features; Fusing the test question text feature sequence with the test question image feature to obtain a target fusion feature, including: fusing an adjusted text feature sequence and an adjusted image feature to obtain the target fusion feature, wherein the adjusted text feature sequence is obtained by obtaining a text feature fusion parameter corresponding to the test question text feature sequence and adjusting the test question text feature sequence using the text feature fusion parameter; the adjusted image feature is obtained by obtaining a target feature quantity corresponding to the test question text feature sequence, copying the test question image features according to the target feature quantity, arranging the copied test question image features of the target feature quantity to obtain an image feature sequence, and adjusting the image feature sequence using the image feature fusion parameter corresponding to the test question image feature; Determining the feature attention strength corresponding to each of the feature extraction vectors in the test question text feature sequence based on the target fusion feature; the feature attention strength is used to reflect the intensity of attention to the feature; Adjusting the feature extraction vector in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; Knowledge points are identified based on the target text feature sequence to obtain target knowledge points corresponding to the target test question.

2. The method according to claim 1, characterized in that The target feature quantity is the quantity of feature extraction vectors contained in the test question text feature sequence.

3. The method according to claim 1, characterized in that The adjusting text feature sequence and the adjusting picture feature are fused to obtain the target fusion feature, which includes: Using the features in the adjusted text feature sequence as matrix values ​​in a first matrix object corresponding to the adjusted text feature sequence in a target feature matrix to obtain the first matrix object; Using the features in the adjusted picture feature sequence as matrix values ​​in a second matrix object corresponding to the adjusted picture feature sequence in a target feature matrix, to obtain the second matrix object; Using the matrix formed by the first matrix object and the second matrix object as the target feature matrix; The target fusion feature is obtained based on the target feature matrix.

4. The method according to claim 1, wherein Determining the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature includes: Obtaining fusion feature processing parameters, and processing the target fusion feature using the fusion feature processing parameters to obtain an attention strength vector; the attention strength vector includes a vector value of the number of target features, and the number of target features is the number of feature extraction vectors included in the test question text feature sequence; The vector value in the attention strength vector that is consistent with the feature extraction vector in the test question text feature sequence is used as the feature attention strength corresponding to the feature extraction vector.

5. The method according to claim 1, characterized in that The target test question picture includes a first test question picture, and the step of obtaining the target test question picture corresponding to the target test question includes: Obtaining content description information corresponding to the target test question, wherein the content description information describes the test question content corresponding to each content type in the target test question and the association relationship between the test question contents; generating a first test question picture corresponding to the target test question according to the content description information; Among them, when generating the first test question picture, the display method of the test question content on the first test question picture is determined according to the content type corresponding to the test question content, and the display position of the test question content on the first test question picture is determined according to the association relationship between the test question contents.

6. The method according to claim 5, characterized in that The target test question picture also includes a second test question picture, and the step of obtaining the target test question picture corresponding to the target test question includes: Using the test question picture in the picture format of the target test question as the second test question picture; The first test question image includes the test question description text and the image content in the second test question image; the test question image features include the first test question image features and the second test question image features; The feature extraction of the target test question image to obtain the test question image features includes: Inputting the first test question image into a fusion extraction model to perform feature extraction, thereby obtaining a first test question image feature that integrates the test question description text and the image content; The second test question image is input into the image extraction model for feature extraction to obtain the second test question image features.

7. The method according to claim 6, characterized in that The performing of knowledge point recognition based on the target text feature sequence to obtain target knowledge points corresponding to the target test question includes: fusing the target text feature sequence, the first test question image feature, and the second test question image feature to obtain a comprehensive test question feature corresponding to the target test question; Knowledge points are identified based on the comprehensive features of the test questions to obtain target knowledge points corresponding to the target test questions.

8. The method according to claim 7, characterized in that The step of fusing the target text feature sequence, the first test question image feature, and the second test question image feature to obtain a comprehensive test question feature corresponding to the target test question includes: Inputting the target text feature sequence, the first test question image feature, and the second test question image feature into a comprehensive feature extraction layer for feature extraction, wherein the comprehensive feature extraction layer processes the target text feature sequence, the first test question image feature, and the second test question image feature respectively using parameters corresponding to each feature to obtain a processed target text feature sequence, a processed first test question image feature, and a processed second test question image feature; The processed target text feature sequence, the processed first test question image feature, and the processed second test question image feature are spliced ​​together to obtain a comprehensive test question feature corresponding to the target test question.

9. The method according to claim 1, characterized in that The step of obtaining the test question description text in the target test question to be identified includes: Obtaining text information in the target test question; Performing character type recognition on the text information to obtain a symbol string and a numeric string in the text information; Filtering the text information based on the string position of the symbol string and the string position of the numeric string to obtain a mixed string including adjacent symbol strings and numeric strings; The mixed character string is filtered out from the text information, and the remaining text information is used as the test question description text.

10. A test question recognition device, characterized in that: The device comprises: An acquisition module is configured to acquire a test question description text in a target test question to be identified, and a target test question image corresponding to the target test question; the test question description text includes a plurality of test question segmentations; the target test question is a science test question; the target test question image includes an image originally included in the target test question, and an image obtained by converting information in the target test question into an image; A text feature extraction module is used to extract features from the test question description text to obtain feature extraction vectors corresponding to the test question segmentation words, and the feature extraction vectors are arranged according to the order of the test question segmentation words in the test question description text to obtain a test question text feature sequence; An image feature extraction module is used to extract features from the target test question image to obtain test question image features; a fusion module, for fusing the test question text feature sequence with the test question image feature to obtain a target fusion feature, comprising: fusing an adjusted text feature sequence and an adjusted image feature to obtain the target fusion feature, wherein the adjusted text feature sequence is obtained by obtaining a text feature fusion parameter corresponding to the test question text feature sequence and adjusting the test question text feature sequence using the text feature fusion parameter; and the adjusted image feature is obtained by obtaining a target feature quantity corresponding to the test question text feature sequence, copying the test question image features according to the target feature quantity, arranging the copied test question image features of the target feature quantity to obtain an image feature sequence, and adjusting the image feature sequence using the image feature fusion parameter corresponding to the test question image feature; A feature attention strength determination module is used to determine the feature attention strength corresponding to each feature extraction vector in the test question text feature sequence based on the target fusion feature; the feature attention strength is used to reflect the strength of attention to the feature; An adjustment module, configured to adjust the feature extraction vectors in the test question text feature sequence based on the feature attention strength to obtain a target text feature sequence; The recognition module is used to perform knowledge point recognition based on the target text feature sequence to obtain the target knowledge points corresponding to the target test question.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images