Method, device and equipment for identifying question sheet in question sheet image

Abstract

The embodiment of the invention provides a method and device for identifying a question sheet in a question sheet image, equipment, a medium and a product. The scheme comprises the following steps: acquiring a question sheet image shot by a user terminal and submitted by a user; the question sheet image is input to an image quality evaluation model, a quality score of the question sheet image is obtained, the higher the quality score is, the higher the probability that the question sheet in the question sheet image is accurately identified is, and the image quality evaluation model is that the probability that the question sheet in the question sheet image is accurately identified is not less than the probability that the question sheet in the question sheet image is accurately identified. Performing fine tuning training on a neural network model constructed based on meta-learning by using question sheet image samples of various distortion types to obtain a model; and if the quality score of the question sheet image is greater than or equal to a first preset threshold value, identifying a question sheet in the question sheet image. According to the scheme, the accuracy of distortion quality detection for the question sheet image can be improved, and then the accuracy of identification for the question sheet in the question sheet image can be improved.

Description

Technical Field

[0001] The present specification relates to the field of image detection technology, and in particular to a method, device, equipment, medium and product for identifying a question sheet in a question sheet image. Background Art

[0002] With the rapid development of computer technology, computer technology has been gradually applied to assist teachers in teaching tasks, such as online marking of papers or online correction of homework through computers. The premise of online marking or online correction of homework is that the quality of the image of the test paper or homework received by the computer must be good. Once a poor-quality image of the question list is submitted to the computer, it will affect the computer's recognition result of the question list in the question list image, and then affect the evaluation result given by the computer for the question list. Therefore, before uploading the question list image to the computer, it is necessary to first perform an image quality test on the question list image.

[0003] In the prior art, image quality detection is mainly performed based on the image distortion types in the image through artificially designed image features. This detection method only has good detection performance for images in image distortion scenarios related to the image features designed by the user. However, it is unable to accurately detect various types of distortion images that may be encountered in the complex scenario of uploading photo assignments.

[0004] Based on this, how to improve the accuracy of quality detection of question sheet images has become a technical problem that needs to be solved urgently. Summary of the invention In view of this, an embodiment of the present specification provides a method for identifying a question list in a question list image. One or more embodiments of the present specification also relate to an apparatus for identifying a question list in a question list image, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of the present specification, a method for identifying a question list in a question list image is provided, comprising: Obtaining a question list image taken by a user terminal and submitted by a user; Inputting the question sheet image into an image quality assessment model to obtain a quality score of the question sheet image; the image quality assessment model is a model obtained by fine-tuning a neural network model constructed based on meta-learning using question sheet image samples of various distortion types; Determine whether the quality score of the question list image is greater than or equal to a first preset threshold value, and obtain a first determination result; the higher the quality score, the greater the probability that the question list in the question list image is accurately identified; If the first judgment result indicates that the quality score of the question list image is greater than or equal to the first preset threshold, the question list in the question list image is identified.

[0006] According to a second aspect of an embodiment of the present specification, a device for identifying a question list in a question list image is provided, comprising: An acquisition module is configured to acquire a question list image photographed by a user terminal and submitted by a user; An input module is configured to input the question sheet image into an image quality assessment model to obtain a quality score of the question sheet image; the image quality assessment model is a model obtained by fine-tuning a neural network model constructed based on meta-learning using question sheet image samples of various distortion types; A judgment module is configured to judge whether the quality score of the question list image is greater than or equal to a first preset threshold value, and obtain a first judgment result; the higher the quality score, the greater the probability that the question list in the question list image is accurately identified; The identification module is configured to identify the question list in the question list image if the first judgment result indicates that the quality score of the question list image is greater than or equal to the first preset threshold.

[0007] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions. When the computer program or instructions are executed by the processor, the steps of the above-mentioned method for identifying the question list in the question list image are implemented.

[0008] According to a fourth aspect of an embodiment of the present specification, a computer-readable storage medium is provided, which stores a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps of the above-mentioned method for identifying a question list in a question list image are implemented.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-mentioned method for identifying a question list in a question list image.

[0010] An embodiment of the present specification implements a quality assessment model for detecting question list images, which is a model obtained by fine-tuning and training a neural network model constructed based on meta-learning using question list image samples of various distortion types. The neural network model constructed based on meta-learning can learn the distortion features of question list images of various distortion types through question list image samples of various distortion types using its own meta-learning ability, so that the quality assessment model obtained after training can have the performance of identifying the distortion features of question list images of various distortion types, thereby improving the accuracy of distortion quality detection for question list images, and further improving the accuracy of identifying question lists in question list images. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a flow chart of a method for identifying a question list in a question list image provided by an embodiment of the present specification; Figure 2 It is a schematic diagram of a process for preprocessing a question sheet image provided by an embodiment of this specification; Figure 3 It is a flowchart of generating an image quality assessment model based on fine-tuning training of a neural network model provided by an embodiment of this specification; Figure 4 It is a schematic diagram of a process of generating a score for a question list in a question list image provided by an embodiment of this specification; Figure 5 It is a structural schematic diagram of a device for identifying a question list in a question list image provided by an embodiment of the present specification; Figure 6 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0011] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0012] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0014] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0015] First, the terms involved in one or more embodiments of this specification are explained.

[0016] Image Quality Assessment (IQA) algorithm aims to give a quantitative value of image quality through a mathematical model, and to numerically distinguish and compare clear and blurred images instead of subjective judgment and classification.

[0017] Meta-learning: is a machine learning methodology that focuses on how to build a model that can quickly adapt to new tasks, also known as learning to learn. In traditional machine learning, models usually require a large amount of data to train in order to achieve good performance on a specific task. However, in meta-learning, the goal is to create a model that can use past experience to accelerate the learning process of new tasks, even if these new tasks only have a small amount of data.

[0018] Fine-tuning training: Based on the pre-trained model, by further collecting sample data related to a specific task, the model is trained and the model parameters are adjusted so that the model can better understand and perform specific tasks. In this way, the model can be more accurate and efficient when processing specific tasks.

[0019] Currently, the quality of an image is usually judged by the pixel features of the image itself or by performing certain statistical analysis on the pixels in the image and extracting statistical features. These methods can only solve problems under specific scene image distortion types, and their generalization performance and flexibility are poor. For single images involving multiple distortion types, it is impossible to comprehensively and accurately evaluate the image quality.

[0020] In this specification, a method for identifying a question list in a question list image is provided. This specification also relates to an apparatus for identifying a question list in a question list image, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0021] Figure 1 This is a flow chart of a method for identifying a question sheet in a question sheet image provided by an embodiment of this specification. From a hardware perspective, the execution subject of this process can be a device for identifying question sheets, or, from a software perspective, an application installed on the device for identifying question sheets.

[0022] like Figure 1 As shown, the process may include the following steps.

[0023] Step 102: Obtain the question list image photographed by the user terminal and submitted by the user.

[0024] In the embodiments of this specification, the target application in the user terminal obtains the image of the question list submitted by the user, wherein the user terminal can be any terminal with a shooting function, such as a smart phone, a tablet computer, a laptop computer, an intelligent interactive device, a wearable device, a vehicle-mounted intelligent terminal, etc. The target application can be an APP installed on the user terminal for online teaching, such as various homework correction software, various question-solving software, etc. The question list image can be an image taken of an examination paper or a student's homework, such as an image taken of a Chinese test paper.

[0025] In actual applications, the target application on the user terminal may include an image acquisition module, and the image acquisition module is used to acquire the image of the question sheet submitted by the user.

[0026] Step 104: Input the question sheet image into an image quality assessment model to obtain a quality score of the question sheet image; the image quality assessment model is a model obtained by fine-tuning and training a neural network model constructed based on meta-learning using question sheet image samples of various distortion types.

[0027] In the embodiments of this specification, the image quality assessment model can be a model independent of the target application, such as an APP for image quality assessment installed on the user terminal; or the image quality assessment model can also be a part of the target application, such as an image detection module for image detection in the target application, without limitation.

[0028] In the embodiments of this specification, the quality score may be a score generated by the image quality assessment model for the image quality of the question image, and the image quality may include several levels, such as five levels from low to high, namely very poor, poor, ordinary, good, and very good, each level corresponds to a quality score, and different levels correspond to different quality scores. The quality score may be a fixed number, English letter, etc., such as the quality score corresponding to the very good level may be 0.9 or E; or the quality score may also be a number interval, English letter interval, etc., such as the quality score corresponding to the very poor level may be 0 to 0.3 or A to C, etc.

[0029] In the embodiments of this specification, the question image samples with multiple distortion types can be understood as each question image sample containing multiple distortion types; or each question image sample contains one distortion type, and at least some different question image samples contain different types of distortion types.

[0030] In the embodiments of this specification, multiple distortion types of question sheet images may be distorted images. A distorted image refers to a situation where the image quality is degraded or changed due to certain factors during the process of shooting, transmission, compression, processing or storage, and the original image cannot be completely restored.

[0031] In practical applications, distortion types may include: image geometric distortion, image skew distortion, image magnification distortion, image compression distortion, image overlap distortion, image color gamut mapping distortion, image contrast distortion, image saturation distortion, image blur distortion, etc. Among them, image geometric distortion refers to the distortion phenomenon in which the image changes in shape or size, such as when the image is stretched, compressed or twisted; image skew distortion refers to the distortion phenomenon in which the entire image or part of the image is tilted; image magnification distortion refers to the distortion phenomenon in which the image details are blurred due to the limitation of the image processing algorithm or the loss of image data during the image magnification process; image compression distortion refers to the distortion phenomenon in which the image details are lost or artifacts appear due to the loss of part of the data during the compression process; image overlap distortion refers to the distortion phenomenon in which part of the image area is overlapped due to image overlap during multi-camera shooting or image stitching. Image saturation distortion refers to the change in the intensity or brightness of the image, which causes the image color to be unrealistic or too bright, or the color to become pale and lifeless; image blur distortion refers to the decrease in image clarity, which makes the details in the image unable to be clearly identified and the image becomes blunt.

[0032] In the embodiments of this specification, meta-learning can also be called learning how to learn. The core idea of ​​meta-learning is to learn how to quickly adapt to new tasks from multiple tasks through algorithms, rather than just learning from a single task. The neural network model constructed based on meta-learning can learn the prior knowledge shared by various distortion types through meta-learning, so that the neural network model can have the ability to cope with other unknown distortions, thereby improving the image quality assessment model generated based on the training of the neural network model. It can accurately evaluate the quality scores of question sheet images of various distortion types, thereby improving the accuracy of identifying question sheets in question sheet images.

[0033] Step 106: Determine whether the quality score of the question list image is greater than or equal to a first preset threshold value, and obtain a first judgment result; the higher the quality score, the greater the probability that the question list in the question list image is accurately identified.

[0034] In the embodiments of the present specification, the higher the quality score of the question list image, the better the quality of the question list image, and the greater the probability that the question list will be accurately identified when the question list image is used for question list recognition.

[0035] Step 108: If the first judgment result indicates that the quality score of the question list image is greater than or equal to the first preset threshold, then identify the question list in the question list image.

[0036] In an embodiment of the present specification, the target application may further include a question list recognition module for identifying the question list in the question list image. If the quality score of the question list image is greater than or equal to the first preset threshold, the question list image is uploaded to the question list recognition module so that the question list recognition module recognizes the question list in the question list image.

[0037] In actual applications, if the quality score of the question sheet image is greater than or equal to the first preset threshold, the question sheet image can also be uploaded to the server, and the server can identify the question sheet in the question sheet image. The server can be a server corresponding to the target application or a third-party server, which is not limited.

[0038] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification can be interchanged according to actual needs, or some steps can be omitted or deleted.

[0039] Figure 1 The method in uses sample images of question sheets with various distortion types to fine-tune the neural network model built based on meta-learning, so that the quality assessment model obtained after training can have the performance of identifying the distortion features of question sheet images with various distortion types, thereby improving the accuracy of distortion quality detection for question sheet images, and further improving the accuracy of identifying question sheets in question sheet images.

[0040] based on Figure 1 The present specification also provides some improved implementation methods of the method, which are described below.

[0041] In the embodiments of this specification, a specific embodiment is also proposed for the scenario where the quality score of the question sheet image is less than the first threshold.

[0042] Optionally, after obtaining the first judgment result, the method may further include: if the first judgment result indicates that the quality score of the question sheet image is less than the first preset threshold, returning an instruction to the user to resubmit the question sheet image.

[0043] In an embodiment of the present specification, if the quality score of a question image is less than a first preset threshold, it can indicate that the question image is a low-quality question image. While the image quality assessment model generates a quality score for the low-quality question image, it can also determine the corresponding image distortion type information for the question image.

[0044] In an embodiment of the present specification, the instruction returned to the user may include image distortion type information, which is used to reflect the quality defects of the question sheet image most recently submitted by the user, so that the user can submit a question sheet image that avoids the quality defects next time.

[0045] In an embodiment of the present specification, when an instruction to resubmit a question list image is sent to a user, the instruction also includes distortion type information of the question list image most recently submitted by the user, so that the user can be aware of quality defects of the question list image most recently submitted, thereby guiding the user to submit a question list image that meets quality requirements, thereby improving the user experience of submitting question list images.

[0046] In order to facilitate the image quality assessment model to perform quality assessment on the question image, the question image may be preprocessed before being input into the image quality assessment model.

[0047] Optionally, before inputting the question list image into the image quality assessment model, the process may further include: performing format conversion on the question list image to obtain a question list image in RGB format; performing flattening processing on the pixel data in the question list image in RGB format to obtain a one-dimensional data set for the question list image; the flattening processing is used to convert the data format of the pixel data from a three-dimensional data format to a one-dimensional data format; inputting the question list image into the image quality assessment model specifically includes: inputting the one-dimensional data set into the image quality assessment model.

[0048] In an embodiment of the present specification, before flattening the pixel data in the question list image in the RGB format to obtain a one-dimensional data set for the question list image, it may also include: intercepting the middle area of ​​the question list image in the RGB format to obtain a intercepted question list image; adjusting the image size of the intercepted question list image to obtain an adjusted question list image. Flattening the pixel data in the question list image in the RGB format to obtain a one-dimensional data set for the question list image may specifically include: for each pixel data in the adjusted question list image, converting the RGB channel sequence corresponding to the pixel data into the BGR channel sequence to obtain each converted pixel data; flattening each converted pixel data to obtain a one-dimensional data set for the question list image.

[0049] Figure 2 is a schematic diagram of a process for preprocessing a question list image provided by an embodiment of this specification; Figure 2 As shown, steps 202 to 212 may be included.

[0050] Step 202: Determine whether the image format of the question sheet image is in RGB format, and obtain a determination result.

[0051] Step 204: If the judgment result indicates that the image format of the question list image is not in RGB format, the image format of the question list image is converted into RGB format to obtain a question list image in RGB format.

[0052] In practical applications, the image format of the question sheet image can be adjusted according to the input requirements of the image quality assessment model, and there is no limitation on the image format of the question sheet image.

[0053] Step 206: intercept the middle area of ​​the question list image in the RGB format to obtain a intercepted question list image.

[0054] In practical applications, since the text information such as the question list in the question list image is mainly concentrated in the middle area of ​​the question list image, and the edge area may be blank or irrelevant information, the middle area is captured for quality assessment. The range of the middle area can be 90%-99%, or other numerical ranges, and there is no limitation on this.

[0055] Step 208: Adjust the image size of the intercepted question list image to obtain an adjusted question list image.

[0056] In practical applications, in order to meet the input requirements of the image quality assessment model, the image size of the question list image needs to be adjusted. The image size of the adjusted question list image can be between 256×256 pixels and 1024×1024 pixels, or can be in other intervals, without limitation. The larger the image size of the adjusted question list image, the higher the accuracy of the image quality assessment model in quality assessment of the question list image.

[0057] Step 210: for each pixel point data in the adjusted question sheet image, convert the RGB channel sequence corresponding to the pixel point data into the BGR channel sequence to obtain each pixel point data after conversion; Step 212: Flatten the pixel data after each conversion process to obtain a one-dimensional data set for the question sheet image.

[0058] In practical applications, RGB can have three channels, each of which can be represented by a two-dimensional matrix (surface). The three channels of RGB can be understood as three overlapping two-dimensional matrices. Each two-dimensional matrix can represent a color in the question image. The color data corresponding to each pixel in the question image can be three-dimensional data composed of the three channel values ​​in RGB, and each dimension corresponds to one of the channel values ​​in RGB.

[0059] Optionally, the pixel data (three-dimensional data) in the question image is flattened according to a preset channel order, and the flattening process is used to convert the data format of the pixel points in the question image from a three-dimensional data format to a one-dimensional data format, wherein the preset channel order may be a BGR channel order, and this specification does not limit the specific channel order of the preset channel order.

[0060] In order to facilitate technical personnel in this field to understand the present solution, this specification flattens the pixel point data in the subject image according to a preset channel order and also proposes a specific example.

[0061] Assume that each pixel point data in the question image is flattened in the order of the first channel, the second channel, and the third channel. The specific process of the flattening is: first, for the first plane corresponding to the first channel, arrange the pixel point data in sequence from the first row to the last row of the first plane. For example, the first plane contains the 1st row, the 2nd row, and the 3rd row. The data contained in the 1st row is abc, the data contained in the 2nd row is bcd, and the data contained in the 2nd row is ade. Then, the first data obtained after arranging the pixel point data in the first plane is abcbcdade; then, for the second plane corresponding to the second channel, arrange the pixel point data in sequence on the basis of the first data in the order from the first row to the last row of the second plane. For example, the second plane contains the 1st row, the 2nd row, and the 3rd row. , the data contained in the first row is dab, the data contained in the second row is ace, and the data contained in the third row is dfe. Then, the second data obtained after arranging the pixel point data in the second plane based on the first data is abcbcdadedabacedfe; finally, for the third plane corresponding to the third channel, the pixel point data are arranged in sequence from the first row to the last row of the third plane based on the second data. For example, the third plane contains the first row, the second row and the third row, the data contained in the first row is fdc, the data contained in the second row is ade, and the data contained in the third row is bac. Then, the third data obtained after arranging the pixel point data in the third plane based on the second data is abcbcdadedabacedfefdcadebac.

[0062] It should be noted that whether to perform flattening processing on the pixel point data in the question list image can be reasonably set according to the data input requirements of the image quality assessment model. If the data input requirements of the image quality assessment model reflect the need to perform flattening processing on the pixel point data in the question list image, then the pixel point data in the question list image will be flattened; if the data input requirements of the image quality assessment model reflect that there is no need to perform flattening processing on the pixel point data in the question list image, then the pixel point data in the question list image will not be flattened. This manual does not limit this.

[0063] In an embodiment of the present specification, for a question sheet image, pixel data in a three-dimensional format is converted into pixel data in a one-dimensional format to obtain a one-dimensional data set for the question sheet image, and the one-dimensional data set is used as input data for an image quality assessment model to achieve a high success rate for data input to the image quality assessment model.

[0064] In the embodiments of this specification, a specific embodiment of generating an image quality assessment model is also proposed.

[0065] Optionally, before obtaining the question list image taken by the user terminal and submitted by the user, the method may also include: obtaining a training sample set of sample question list images containing multiple distortion types; the distortion type is used to reflect the type of quality loss occurring in the sample question list image; obtaining a neural network model constructed based on meta-learning; and fine-tuning the neural network model based on the training sample set to obtain the image quality assessment model.

[0066] In the embodiments of this specification, a sample question list image containing multiple distortion types can be understood as each sample question list image containing multiple distortion types; or it can also be understood as each sample question list image containing one distortion type and at least some different sample question list images contain different distortion types.

[0067] In the embodiment of the present specification, the training sample set may further include a second sample question list image, and the second sample question list image may be a question list image that does not include any distortion type.

[0068] In the embodiments of this specification, each sample question list image and each second sample question list image may carry a label score, which may be generated by the labeler. The label scores correspond to the classification rules of different image qualities, which can refer to the above-mentioned classification rules of different image qualities corresponding to the quality scores, and will not be repeated here.

[0069] In the embodiments of this specification, the neural network model constructed based on meta-learning can be a pre-trained model, and the pre-trained model can be a model learned on a large-scale data set based on the meta-learning function, and the pre-trained model can provide an effective starting point for a specific task. Fine-tuning training refers to continuing to train the pre-trained model based on the pre-trained model using sample question images containing multiple types of distortion, so that the model can better adapt to the task of image quality assessment for question images.

[0070] In the embodiments of this specification, a specific embodiment is also proposed for a method of obtaining a training sample set of sample single images containing multiple distortion types.

[0071] Optionally, obtaining a training sample set of sample single images containing multiple distortion types may specifically include: Acquire each of the sample question list images containing multiple distortion types; for any sample question list image among each of the sample question list images, obtain the labeling scores of each of the multiple labelers for the any sample question list image; calculate the label score of the any sample question list image based on the labeling scores of at least some of the multiple labelers for the any sample question list image; determine the label score of the any sample question list image and the label score of the any sample question list image as a training sample data.

[0072] In practical applications, methods for obtaining each of the sample question list images containing multiple distortion types may include: selecting the sample question list image from historical question list images uploaded by each user; or, selecting the sample question list image from an existing training sample database; or, performing conversion processing on a high-quality question list image to convert the high-quality question list image into the sample question list image, wherein the high-quality question list image may be an image that does not contain any distortion type.

[0073] In practical applications, the method of converting a high-quality question list image into the sample question list image may include: performing at least one of image geometric distortion processing, image skew distortion processing, image magnification distortion processing, image compression distortion processing, image overlap distortion processing, image color gamut mapping distortion processing, image contrast distortion processing, image saturation distortion processing, and image blur distortion processing on the high-quality question list image to obtain a converted sample question list image. For example: performing stretching processing on the high-quality question list image to obtain a sample question list image of the geometric distortion type; performing image contrast reduction processing and image blur processing on the high-quality question list image to obtain a sample question list image containing two types of distortion, namely, contrast distortion and blur distortion.

[0074] In practical applications, the sample question list image with the first distortion type may be processed with the second distortion type to obtain a sample question list image containing the first distortion type and the second distortion type. For example, the sample question list image containing the compression distortion type may be processed with image contrast reduction to obtain a sample question list image containing both compression distortion and contrast distortion.

[0075] In the embodiments of this specification, for any sample question list image among the sample question list images, the sample question list image is sent to a number of labelers, and the labeling score of each labeler for the sample question list image is obtained. The label score of the sample question list image is determined based on the labeling scores of at least some labelers for the sample question list image, and the sample question list image and the label score of the sample question list image are determined as a training sample data. The number of labelers can be 30 labelers, or other number of labelers, which is not limited to this.

[0076] In the embodiment of this specification, the label score may be an average score or a median score determined based on each annotation score.

[0077] In the embodiments of the present specification, for each of the sample question list images, the label score of each sample question list image may be determined in the same manner as that of determining the label score of any of the sample question list images.

[0078] In the embodiments of this specification, since the quality of a question sheet image displayed on different display devices and in different light and dark environments may be different, and different people have different understandings of the standard of image quality issues, the sample question sheet image is sent to multiple labelers for score labeling, and finally the label score for the question sheet image is determined based on the labeling scores of the multiple labelers, thereby improving the objectivity and accuracy of labeling the question sheet image.

[0079] Optionally, the label score of any sample question list image is calculated based on the label scores of at least some of the multiple labelers for any sample question list image; specifically, it may include: judging whether the label score of any of the multiple labelers for any sample question list image is within a preset confidence interval, and obtaining a second judgment result; if the second judgment result indicates that the label score of any of the sample question list image by any of the multiple labelers is not within the preset confidence interval, then calculating the label score of any sample question list image based on the label scores of other labelers for any sample question list image except the label score of any of the labelers for any sample question list image.

[0080] In an embodiment of the present specification, the preset confidence interval may be a confidence interval determined based on the annotation scores of several annotation personnel, and the preset confidence interval is used to reflect that the ratio between the first number of annotation scores located in the preset confidence interval and the total number of all annotation scores is less than or equal to threshold a and greater than or equal to threshold b, for example, threshold a may be 80%, threshold b may be 50%, or other values, without limitation.

[0081] In actual applications, if the ratio between the first number of annotation scores located in the preset confidence interval and the total number of all annotation scores is greater than the threshold a, the preset confidence interval is reduced; if the ratio between the first number of annotation scores located in the preset confidence interval and the total number of all annotation scores is less than the threshold b, the preset confidence interval is expanded.

[0082] In the embodiment of the present specification, the ratio between the first number of annotation scores located in the preset confidence interval and the total number of all annotation scores is between threshold b and threshold a, so that when determining the corresponding label score based on the annotation score located in the preset confidence interval, the objectivity and accuracy of determining the label score can be improved.

[0083] In order to facilitate those skilled in the art to understand the present method, this specification also proposes a specific embodiment of determining a preset confidence interval.

[0084] Optionally, the preset confidence interval includes an interval from a first interval end value to a second interval end value; the first interval end value is based on the formula: The second interval end value is based on the formula: The determined value; represents the average annotation score calculated for each annotation score, s represents the standard deviation calculated for each annotation score, n represents the number of each annotation score, df represents the degrees of freedom of the t distribution, df=n-1, t α / 2，df represents the α / 2 quantile in the t distribution with df degrees of freedom, CI1 represents the end value of the first interval, and CI2 represents the end value of the second interval.

[0085] In this embodiment, the preset confidence interval formed by the first interval end value to the second interval end value may be an interval starting from the first interval end value and ending at the second interval end value. For example, if CI1 calculated based on formula (1) is 0.6 and CI2 calculated based on formula (2) is 0.8, then the preset confidence interval is [0.6-0.8], including 0.6 and 0.8.

[0086] In practical applications, at a 95% confidence level, α=0.05, so α / 2=0.025, t α / 2，df The specific value of can be obtained by looking up the table through the values ​​of α / 2 and df.

[0087] In practical applications, in order to improve the credibility of the confidence interval, a new preset confidence interval can be generated according to the annotation score within the determined preset confidence interval. The method for generating the new preset confidence interval can refer to the method for generating the determined preset confidence interval, which will not be repeated here.

[0088] In order to improve the accuracy of quality assessment for the subject image, the image quality assessment model used for image quality assessment needs to be a qualified quality assessment model.

[0089] Optionally, the fine-tuning training of the neural network model based on the training sample set to obtain the image quality assessment model may specifically include: splitting the training sample set into a first training sample set and a second training sample set, the first training sample set being used to train the neural network model, and the second training sample set being used to test the trained neural network model; training the neural network model based on the first training sample set to obtain the trained neural network model; inputting the test question image in the second training sample set into the trained neural network model to obtain a predicted score for the test question image; determining the difference between the predicted score and the label score of the test question image; judging whether the difference is less than or equal to a second preset threshold to obtain a third judgment result; if the third judgment result indicates that the difference is less than or equal to the second preset threshold, determining the trained neural network model as the image quality assessment model.

[0090] In the embodiment of this specification, the ratio between the number of samples corresponding to the first training sample set and the number of samples corresponding to the second training sample set may be 9:1, or may be other ratios, which are not limited thereto.

[0091] In the embodiment of the specification, the training of the neural network model based on the first training sample set may include: performing multiple rounds of training on the neural network model based on at least part of the training samples in the first training sample set. In practical applications, the neural network model is trained for a first round based on m1 training samples in the first training sample set to obtain a first trained neural network model; the neural network model is trained for a second round based on m2 training samples in the first training sample set to obtain a second trained neural network model; and so on. n The neural network model is trained for the nth round using training samples to obtain the nth trained neural network model. Therefore, multiple trained neural network models can be obtained. 1、 m2......m n Any two values ​​in can be the same or different.

[0092] In an embodiment of the present specification, for any neural network model among multiple trained neural network models, a test question image in the second training sample set is input into any of the neural network models to obtain a test score for the test question image, and the difference between the test score and the label score of the test question image is determined; if the difference is less than or equal to a second preset threshold, any of the neural network models is determined as an image quality assessment model.

[0093] In the embodiments of this specification, the difference between the test score and the label score of the test question image determined by any neural network model can be the average difference value or the median difference value determined based on each test question image tested by any neural network model. For example, the difference values ​​determined for each test question image include 0.1, 0.3, 0.1, 0.7, and 0.8, then the difference between the test score and the label score of the test question image determined by any neural network model can be the average difference value of 0.4 or the median difference value of 0.3.

[0094] In the embodiments of the present specification, whether each neural network model can be determined as an image quality assessment model can be determined in the same manner as whether any of the neural network models can be determined as an image quality assessment model.

[0095] In practical applications, if it is determined that there are multiple neural network models that meet the conditions for being determined as image quality assessment models, then one of the multiple neural network models can be determined as the image quality assessment model; or the neural network model corresponding to the minimum difference value among the difference values ​​corresponding to the multiple neural network models can be determined as the image quality assessment model.

[0096] In the embodiments of the present specification, the neural network model will only be used to perform image quality assessment on the test question image when the difference between the test score of the neural network model for the test question image and the label score of the test question image is less than or equal to a preset threshold, thereby improving the accuracy of image quality assessment on the test question image.

[0097] In order to improve the accuracy of determining the difference value corresponding to the neural network model, the difference value corresponding to the neural network model can be determined for multiple test question images in the second training sample set based on the similarity between the test score of each test question image and the average test score or the median test score, as well as the similarity between the label score of each test question image and the average label score or the median label score.

[0098] Optionally, determining the difference value between the predicted score and the label score of the test question image may specifically include: determining a first feature score corresponding to the label score of each of the test question images; the first feature score is an average score or a median score determined based on the label scores of each of the test question images; determining a first difference value between the label score of each of the test question images and the first feature score; determining a second feature score corresponding to the test score of each of the test question images; the second feature score is an average score or a median score determined based on the test scores of each of the test question images; the first feature score and the second feature score are the same type of scores; determining a second difference value between the test score of each of the test question images and the second feature score; and determining the difference value based on the first difference value and the second difference value.

[0099] In an embodiment of the present specification, a method for determining the median score corresponding to each label score may include: sorting each label score in order from small to large or from large to small to obtain a label score sequence; if the number of each label score is an odd number, then determining the score in the middle of the label score sequence as the median score; if the number of each label score is an even number, then determining one of the two scores in the middle of the label score sequence as the median score.

[0100] In the embodiments of this specification, the method for determining the median score corresponding to each test score can refer to the method for determining the median score corresponding to each label score, which will not be repeated here.

[0101] In the embodiments of this specification, the first characteristic score and the second characteristic score may both be average scores, or may both be median scores.

[0102] In an embodiment of the present specification, determining the first difference value between the label score of each of the test question images and the first feature score may include: determining, for each of the label scores of the test question images, the score difference between the label score of each of the test question images and the first feature score; and determining the average or median value of each score difference as the first difference value between the label score of each of the test question images and the first feature score.

[0103] In the embodiments of this specification, the method for determining the second difference value can refer to the method for determining the first difference value, which will not be described in detail here. The first difference value and the second difference value can both be average values ​​or median values.

[0104] In order to facilitate those skilled in the art to understand the present solution, in the embodiments of the present application, a specific embodiment is proposed for determining the difference value based on the first difference value and the second difference value.

[0105] Optionally, determining the difference value based on the first difference value and the second difference value may specifically include: based on a formula: , determine the difference value; wherein N represents the number of test question images, represents the label score of the i-th test question image, represents the test score of the i-th test question image, Represents the average score determined based on the scores of each label. represents the average score determined based on the scores of each test, and D1 represents the difference value.

[0106] For multiple test question images in the second training sample set, the difference value corresponding to the neural network model can also be determined based on a test question image sequence generated by the test scores of each test question image, and a test question image sequence generated by the label scores of each test question image.

[0107] Optionally, determining the difference value between the predicted score and the label score of the test question image may specifically include: based on the label scores of the respective test question images, sorting the respective test question images in a preset rule order to obtain a first test question image sequence; the preset rule order includes an order from large to small or a order from small to large; based on the test scores of the respective test question images, sorting the respective test question images in a preset rule order to obtain a second test question image sequence; determining the difference value based on the first position of the target test question image in the first test question image sequence and the second position of the target test question image in the second test question image sequence.

[0108] In an embodiment of the present specification, the determination of the difference value based on the first position of the target test question image in the first question image sequence and the second position of the target test question image in the second question image sequence may include: for any test question image, determining the first position of the test question image in the first question image sequence, and determining the second position of the test question image in the second question image sequence, and determining the position difference between the first position and the second position; wherein the first question image sequence is a sequence generated in the order of label scores from small to large, and the second question image sequence is also a sequence generated in the order of test scores from small to large; or, the first question image sequence is a sequence generated in the order of label scores from large to small, and the second question image sequence is also a sequence generated in the order of test scores from large to small. According to the method of determining the position difference corresponding to any test question image, the position difference corresponding to each test question image is determined, and the difference value is determined based on each position difference. In practical applications, the average position difference or median position difference of each position difference can be determined as the difference value.

[0109] In the embodiment of the present specification, the difference between the test score and the label score can be analyzed more intuitively through the first question single image sequence and the second question single image sequence, so as to improve the intuitiveness of the difference between the test score and the label score.

[0110] In order to facilitate those skilled in the art to understand the present solution, in the embodiments of this specification, a specific embodiment is proposed for the method of determining the difference value based on each position difference.

[0111] Optionally, determining the difference value based on a first position of a target test question image in the first question image sequence and a second position of the target test question image in the second question image sequence may specifically include: Based on the formula: , determine the difference value; where N represents the number of test question images, v i represents the first position of the ith test question single image in the first question single image sequence, pi represents the second position of the ith test question single image in the second question single image sequence, and D2 represents the difference value.

[0112] In practical applications, the problem list image may be an image taken by a user terminal of a math homework completed by a student. The problem list in the problem list image may include the problem of the math homework and the student's answer to the problem.

[0113] Optionally, the identifying of the question list in the question list image may specifically include: for the question list in the question list image, identifying question information in the question list and answer information to the question information; after identifying the question list in the question list image, it may also include: acquiring standard answer information corresponding to the question information; generating a score for the answer information based on the standard answer information; and displaying the score to the user.

[0114] In the embodiments of the present specification, the question information in the question list and the answer information to the question information can be identified by existing technical means, for example, by optical character recognition (OCR) technology.

[0115] In practical applications, the question information may be a question number. For example, the question number identified for the first question in the second question may be “II, 1”.

[0116] In the embodiments of the present specification, based on the question information identified for the question list, standard answer information corresponding to the question information can be queried in the question database.

[0117] In the embodiment of the present specification, the generating of the score for the reply information according to the standard answer information may include: matching the reply information with the standard answer information, and generating the score for the reply information according to the matching result. In practical applications, for different types of questions, the matching method between the reply information and the standard answer information may be different, for example: for a multiple-choice question, the text corresponding to the reply information may be matched to see whether it is consistent with the text corresponding to the standard answer information; for an English conjunction question, the word order between the words corresponding to the reply information may be matched to see whether it is consistent with the word order between the words corresponding to the standard answer information, and the word writing form of the words corresponding to the reply information may be matched to see whether it is consistent with the word writing form of the words corresponding to the standard answer information; for a Chinese reading comprehension question, the semantic information corresponding to the reply information may be matched to see whether it is consistent with the semantic information corresponding to the standard answer information; for a math application question, the first math formula corresponding to the reply information may be matched to see whether it is consistent with the second math formula corresponding to the standard answer information, and the final calculation result corresponding to the first math formula may be matched to see whether it is consistent with the final calculation result corresponding to the second math formula.

[0118] In the embodiments of the present specification, after a corresponding score is generated for a question list, the score may be displayed to the user in a display page of a target application in a user terminal.

[0119] In an embodiment of the present specification, the target application may further include a score generation module for generating a score for the reply information. After the question list identification module in the target application identifies the question information and the reply information for the question list, the question information and the reply information are sent to the score generation module, so that the score generation module obtains the standard answer information based on the question information, and generates a score based on the matching result between the reply information and the standard answer information.

[0120] In practical applications, the standard answer information may be information stored in a server. The score generation module acquires the standard answer information based on the question information, which may include: the score generation module generates a request for acquiring the standard answer information based on the question information, and sends the request to the server, so that the server returns the standard answer information to the score generation module in response to the request. Before generating a score for the answer information, the score generation module requests the latest standard answer information from the server, thereby improving the accuracy of the generated score.

[0121] In actual applications, the server may also generate a corresponding score for the reply information, and there is no limitation to this.

[0122] In order to facilitate those skilled in the art to understand the present solution, a specific embodiment is also proposed in the embodiments of this specification for a method of generating an image quality assessment model based on fine-tuning and training a neural network model.

[0123] Figure 3 1 is a flow chart of generating an image quality assessment model based on fine-tuning training of a neural network model provided in an embodiment of this specification. Figure 3 The process shown includes steps 302 to 324 .

[0124] Step 302: Acquire sample question sheet images containing multiple distortion types.

[0125] Step 304: for any sample question list image, based on the annotation scores of the sample question list image by multiple annotators, a label score for the sample question list image is generated.

[0126] Step 306: Generate a training sample data based on any sample question list image and the label score of any sample question list image.

[0127] Step 308: Generate training sample data corresponding to each sample question list image in the manner of generating training sample data for any of the sample question list images to obtain a training sample set.

[0128] Step 310: Split the training sample set into a first training sample set and a second training sample set.

[0129] Step 312: Obtain a neural network model constructed based on meta-learning.

[0130] Step 314: fine-tune the neural network model based on the first training sample set to obtain a trained neural network model.

[0131] Step 316: Input the test question image in the second training sample set into the trained neural network model to obtain a test score for the test question image.

[0132] Step 318: Determine whether the difference between the label score of the test question image and the test score is less than or equal to a preset threshold.

[0133] Step 320: If the difference between the label score and the test score of the test question image is less than or equal to the preset threshold, the trained neural network model is determined as the image quality assessment model.

[0134] Step 322: If the difference between the label score and the test score of the test question image is greater than the preset threshold, then obtain other trained neural network models that are fine-tuned based on the first training sample set for the neural network model.

[0135] Step 324: According to the method of steps 316 to 320, determine whether the other trained neural network model can be determined as an image quality assessment model.

[0136] In order to facilitate those skilled in the art to understand the present solution, a specific embodiment is also proposed in the embodiments of this specification for the method of generating corresponding scores for the question list in the question list image.

[0137] Figure 4 FIG. 1 is a flow chart of generating scores for a question list in a question list image provided by an embodiment of this specification. Figure 4 The process shown includes steps 402 to 418 .

[0138] Step 402: Obtain the question list image photographed by the user terminal and submitted by the user.

[0139] In the embodiment of this specification, the problem list image may be an image taken for a mathematical word problem.

[0140] Step 404: input the question sheet image into an image quality assessment model to obtain a quality score of the question sheet image.

[0141] Step 406: If the quality score is greater than or equal to a preset threshold, the question sheet in the question sheet image is identified to obtain the question information and answer information contained in the question sheet.

[0142] In the embodiment of the present specification, the question information may be the question number of the math word problem in the test paper; the answer information may be the problem-solving process and result of the user for the math word problem.

[0143] Step 408: Based on the question information, obtain corresponding standard answer information.

[0144] In the embodiment of the present specification, the standard answer information may be the correct problem-solving process and result that matches the mathematical word problem and is retrieved from the problem database.

[0145] Step 410: Match the reply information with the standard answer information to obtain a matching result.

[0146] In the embodiment of the present specification, the user's solution process for the math word problem is matched with the correct solution process to obtain a first matching result; the user's solution result for the math word problem is matched with the correct solution result to obtain a second matching result.

[0147] Step 412: Based on the matching result, generate a score for the reply information.

[0148] In an embodiment of the present specification, if the first matching result reflects that the user's solution process for the math word problem is consistent with the correct solution process, a first score is generated; if the second matching result reflects that the user's solution result for the math word problem is consistent with the correct solution result, a second score is generated; and a final score is generated based on the first score and the second score.

[0149] Step 414: Display the score to the user.

[0150] Step 416: If the quality score is greater than or equal to a preset threshold, an instruction to resubmit the question sheet image is returned to the user.

[0151] Step 418: Process the question sheet image resubmitted by the user in the manner of steps 404 to 416.

[0152] Corresponding to the above method embodiment, this specification also provides a device embodiment for identifying a question list in a question list image. Figure 5 FIG. 1 is a schematic diagram showing a structure of a device for identifying a question list in a question list image provided by an embodiment of the present specification. Figure 5 As shown, the device comprises: The acquisition module 502 is configured to acquire the question list image photographed by the user terminal and submitted by the user.

[0153] The input module 504 is configured to input the question list image into an image quality assessment model to obtain a quality score of the question list image; the image quality assessment model is a model obtained by fine-tuning a neural network model constructed based on meta-learning using question list image samples of various distortion types.

[0154] The judgment module 506 is configured to judge whether the quality score of the question list image is greater than or equal to a first preset threshold value, and obtain a first judgment result; the higher the quality score, the greater the probability that the question list in the question list image is accurately identified.

[0155] The identification module 508 is configured to identify the question list in the question list image if the first judgment result indicates that the quality score of the question list image is greater than or equal to the first preset threshold.

[0156] Optionally, the device may further include: The return module is configured to return an instruction to the user to resubmit the question sheet image if the first judgment result indicates that the quality score of the question sheet image is less than the first preset threshold.

[0157] Optionally, the device may further include: The conversion module is configured to perform format conversion on the question list image to obtain a question list image in RGB format.

[0158] The flattening processing module is configured to perform flattening processing on the pixel point data in the RGB format question list image to obtain a one-dimensional data set for the question list image; the flattening processing is used to convert the data format of the pixel point data from a three-dimensional data format to a one-dimensional data format.

[0159] The input module 504 may specifically include: The first input unit is configured to input the one-dimensional data set into the image quality assessment model.

[0160] Optionally, the device may further include: The second acquisition module is configured to acquire a training sample set of sample question sheet images containing multiple distortion types; the distortion type is used to reflect the type of quality loss of the sample question sheet images.

[0161] The third acquisition module is configured to acquire a neural network model constructed based on meta-learning.

[0162] The training module is configured to fine-tune the neural network model based on the training sample set to obtain the image quality assessment model.

[0163] Optionally, the second acquisition module may specifically include: The first acquisition unit is configured to acquire each of the sample question sheet images containing multiple distortion types.

[0164] The second acquisition unit is configured to obtain, for any sample question list image among the sample question list images, the annotation scores of each of the multiple annotators for the sample question list image.

[0165] The calculation unit is configured to calculate the label score of any sample question list image based on the annotation scores of at least some of the multiple labelers for any sample question list image.

[0166] The first determination unit is configured to determine any one of the sample question list images and the label score of any one of the sample question list images as a training sample data.

[0167] Optionally, the computing unit may specifically include: The judgment subunit is configured to judge whether the annotation score of any one of the multiple annotation personnel for any one of the sample question sheet images is within a preset confidence interval, and obtain a second judgment result.

[0168] The calculation subunit is configured to calculate the label score of any sample question list image based on the annotation scores of any sample question list image by other annotation personnel except the annotation score of any sample question list image by any annotation personnel if the second judgment result indicates that the annotation score of any sample question list image by any annotation personnel is not within the preset confidence interval.

[0169] Optionally, the preset confidence interval includes an interval from a first interval end value to a second interval end value; the first interval end value is a value determined based on formula (1); the second interval end value is a value determined based on formula (2); (1); (2); in, represents the average annotation score calculated for each annotation score, s represents the standard deviation calculated for each annotation score, n represents the number of each annotation score, df represents the degrees of freedom of the t distribution, df=n-1, t α / 2，df represents the α / 2 quantile in the t distribution with df degrees of freedom, CI1 represents the end value of the first interval, and CI2 represents the end value of the second interval.

[0170] Optionally, the training module may specifically include: The splitting unit is configured to split the training sample set into a first training sample set and a second training sample set, wherein the first training sample set is used to train the neural network model, and the second training sample set is used to test the trained neural network model.

[0171] A training unit is configured to train the neural network model based on the first training sample set to obtain the trained neural network model.

[0172] The second input unit is configured to input the test question image in the second training sample set into the trained neural network model to obtain a predicted score for the test question image.

[0173] The second determining unit is configured to determine a difference value between the predicted score and the label score of the test question image.

[0174] The judgment unit is configured to judge whether the difference value is less than or equal to a second preset threshold value, and obtain a third judgment result.

[0175] The third determination unit is configured to determine the trained neural network model as the image quality assessment model if the third judgment result indicates that the difference value is less than or equal to the second preset threshold.

[0176] Optionally, the second determining unit may specifically include: The first determination subunit is configured to determine a first feature score corresponding to the label score of each of the test question images; the first feature score is an average score or a median score determined based on the label scores of each of the test question images.

[0177] The second determination subunit is configured to determine a first difference value between the label score of each of the test question single images and the first feature score.

[0178] The third determination subunit is configured to determine a second feature score corresponding to the test score of each of the test question single images; the second feature score is an average score or a median score determined based on the test scores of each of the test question single images; the first feature score and the second feature score are the same type of scores.

[0179] The fourth determining subunit is configured to determine a second difference value between the test score of each of the test question single images and the second feature score.

[0180] The fifth determining subunit is configured to determine the difference value based on the first difference value and the second difference value.

[0181] Optionally, the fifth determining subunit may be specifically configured as follows: Based on formula (3), the difference value is determined.

[0182] (3).

[0183] Where N represents the number of test question images. represents the label score of the i-th test question image, represents the test score of the i-th test question image, Represents the average score determined based on the scores of each label. represents the average score determined based on the scores of each test, and D1 represents the difference value.

[0184] Optionally, the second determining unit may specifically include: The first sorting subunit is configured to sort the test question images based on the label scores of the test question images in a preset rule order to obtain a first question image sequence; the preset rule order includes an order from large to small or an order from small to large.

[0185] The second sorting subunit is configured to sort the test question images based on the test scores of the test question images in accordance with the preset rule sequence to obtain a second question image sequence.

[0186] The sixth determination subunit is configured to determine the difference value based on a first position of a target test question image in the first question image sequence and a second position of the target test question image in the second question image sequence.

[0187] Optionally, the sixth determining subunit may be specifically configured as follows: Based on formula (4), the difference value is determined.

[0188] (4).

[0189] Where N represents the number of test question images, v i represents the first position of the ith test question single image in the first question single image sequence, pi represents the second position of the ith test question single image in the second question single image sequence, and D2 represents the difference value.

[0190] Optionally, the identification module 508 may specifically include: The recognition unit is configured to recognize, with respect to the question sheet in the question sheet image, question information in the question sheet and answer information to the question information.

[0191] The device may further include: The fourth acquisition module is configured to acquire standard answer information corresponding to the question information.

[0192] A generating module is configured to generate a score for the reply information based on the standard answer information.

[0193] A display module is configured to display the score to the user.

[0194] Figure 5 The device in the invention uses sample images of question sheets with various distortion types to fine-tune the neural network model constructed based on meta-learning, so that the quality assessment model obtained after training can have the performance of identifying the distortion features of question sheet images with various distortion types, thereby improving the accuracy of distortion quality detection for question sheet images, and further improving the accuracy of identifying question sheets in question sheet images.

[0195] The above is a schematic scheme of a device for identifying a list of questions in a list of questions image of the present embodiment. It should be noted that the technical scheme of the device for identifying a list of questions in a list of questions image and the technical scheme of the method for identifying a list of questions in a list of questions image described above belong to the same concept, and the details not described in detail in the technical scheme of the device for identifying a list of questions in a list of questions image can be found in the description of the technical scheme of the method for identifying a list of questions in a list of questions image described above.

[0196] Figure 6 The block diagram of a computing device 600 according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0197] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0198] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0199] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 may also be a mobile or stationary server.

[0200] Among them, the processor 620 is used to execute the following computer program or instructions, which, when executed by the processor, implement the steps of the above-mentioned method for identifying the question list in the question list image.

[0201] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the method embodiment for identifying a question list in a question list image, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment for identifying a question list in a question list image.

[0202] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned method for identifying a question list in a question list image.

[0203] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the method embodiment for identifying a question list in a question list image, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment for identifying a question list in a question list image.

[0204] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned method for identifying a question list in a question list image.

[0205] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the method for identifying the question list in the question list image are of the same concept, and the details not described in detail in the technical scheme of the computer program product can be found in the description of the technical scheme of the method for identifying the question list in the question list image.

[0206] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0207] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0208] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0209] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0210] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A method for identifying a question list in a question list image, characterized in that: include: Obtaining a question list image taken by a user terminal and submitted by a user; Inputting the question sheet image into an image quality assessment model to obtain a quality score of the question sheet image; the image quality assessment model is a model obtained by fine-tuning a neural network model constructed based on meta-learning using question sheet image samples of various distortion types; Determine whether the quality score of the question list image is greater than or equal to a first preset threshold value, and obtain a first determination result; the higher the quality score, the greater the probability that the question list in the question list image is accurately identified; If the first judgment result indicates that the quality score of the question list image is greater than or equal to the first preset threshold, the question list in the question list image is identified.

2. The method according to claim 1, characterized in that After obtaining the first judgment result, the method further includes: If the first judgment result indicates that the quality score of the question sheet image is less than the first preset threshold, an instruction to resubmit the question sheet image is returned to the user.

3. The method according to claim 1, characterized in that Before inputting the single image into the image quality assessment model, the method further includes: Performing format conversion on the question list image to obtain a question list image in RGB format; Flattening is performed on the pixel point data in the question list image in the RGB format to obtain a one-dimensional data set for the question list image; the flattening is used to convert the data format of the pixel point data from a three-dimensional data format to a one-dimensional data format; The step of inputting the single image into the image quality assessment model specifically includes: The one-dimensional data set is input to the image quality assessment model.

4. The method according to claim 1, characterized in that Before obtaining the image of the question list photographed by the user terminal and submitted by the user, the method further includes: Acquire a training sample set of sample single-question images containing multiple types of distortion; the distortion type is used to reflect the type of quality loss of the sample single-question image; Get a neural network model built based on meta-learning; The neural network model is fine-tuned and trained based on the training sample set to obtain the image quality assessment model.

5. The method according to claim 4, characterized in that The step of obtaining a training sample set of sample single-question images containing multiple types of distortion specifically includes: Acquire each of the sample question sheet images containing multiple distortion types; For any sample question list image among the sample question list images, obtaining the annotation scores of the sample question list images by multiple annotators respectively; Calculating a label score of any sample question list image based on the labeling scores of at least some of the multiple labeling personnel for any sample question list image; The label score of any sample question list image and any sample question list image is determined as a training sample data.

6. The method according to claim 5, characterized in that The step of calculating the label score of any sample question list image based on the label scores of at least some of the multiple labelers for any sample question list image specifically includes: Determine whether the annotation score of any one of the multiple annotators for any one of the sample question sheet images is within a preset confidence interval, and obtain a second judgment result; If the second judgment result indicates that the annotation score of any of the sample question list images given by any of the annotation personnel is not within the preset confidence interval, then the label score of any of the sample question list images is calculated based on the annotation scores of other annotation personnel for any of the sample question list images except the annotation score of any of the annotation personnel for any of the sample question list images.

7. The method according to claim 6, characterized in that The preset confidence interval includes an interval from a first interval end value to a second interval end value; the first interval end value is a value determined based on formula (1); the second interval end value is a value determined based on formula (2); （1）； （2）； in, represents the average annotation score calculated for each annotation score, s represents the standard deviation calculated for each annotation score, n represents the number of each annotation score, df represents the degrees of freedom of the t distribution, df=n-1, t α / 2，df represents the α / 2 quantile in the t distribution with df degrees of freedom, CI1 represents the end value of the first interval, and CI2 represents the end value of the second interval.

8. The method according to claim 4, characterized in that The fine-tuning training of the neural network model based on the training sample set to obtain the image quality assessment model specifically includes: Splitting the training sample set into a first training sample set and a second training sample set, wherein the first training sample set is used to train the neural network model, and the second training sample set is used to test the trained neural network model; Training the neural network model based on the first training sample set to obtain the trained neural network model; Inputting the test question single image in the second training sample set into the trained neural network model to obtain a prediction score for the test question single image; Determine a difference between the predicted score and the label score of the test question image; Determine whether the difference value is less than or equal to a second preset threshold value, and obtain a third determination result; If the third judgment result indicates that the difference value is less than or equal to the second preset threshold, the trained neural network model is determined as the image quality assessment model.

9. The method according to claim 8, characterized in that The determining of the difference between the predicted score and the label score of the test question image specifically includes: Determine a first feature score corresponding to the label score of each of the test question images; the first feature score is an average score or a median score determined based on the label scores of each of the test question images; Determine a first difference value between the label score of each of the test question single images and the first feature score; Determine a second feature score corresponding to the test score of each of the test question single images; the second feature score is an average score or a median score determined based on the test scores of each of the test question single images; the first feature score and the second feature score are scores of the same type; Determine a second difference value between the test score of each of the test question single images and the second feature score; The difference value is determined based on the first difference value and the second difference value.

10. The method according to claim 9, characterized in that The determining the difference value based on the first difference value and the second difference value specifically includes: Based on formula (3), determining the difference value; （3）； Where N represents the number of test question images. represents the label score of the i-th test question image, represents the test score of the i-th test question image, Represents the average score determined based on the scores of each label. represents the average score determined based on the scores of each test, and D1 represents the difference value.

11. The method according to claim 8, characterized in that The determining of the difference between the predicted score and the label score of the test question image specifically includes: Based on the label scores of the test question images, the test question images are sorted according to a preset rule order to obtain a first question image sequence; the preset rule order includes a descending order or a descending order; Based on the test scores of the test question images, the test question images are sorted according to the preset rule sequence to obtain a second question image sequence; The difference value is determined based on a first position of a target test question image in the first question image sequence and a second position of the target test question image in the second question image sequence.

12. The method according to claim 11, characterized in that The determining the difference value based on a first position of the target test question image in the first question image sequence and a second position of the target test question image in the second question image sequence specifically includes: Based on formula (4), determining the difference value; （4）； Where N represents the number of test question images, v i represents the first position of the ith test question image in the first question image sequence, pi represents the second position of the ith test question image in the second question image sequence, and D2 represents the difference value.

13. The method according to any one of claims 1 to 12, characterized in that: The identifying the question list in the question list image specifically includes: For the question sheet in the question sheet image, identifying question information in the question sheet and answer information for the question information; After identifying the question list in the question list image, the method further includes: Obtaining standard answer information corresponding to the question information; Generating a score for the response information according to the standard answer information; The score is displayed to the user.

14. A device for identifying a question list in a question list image, characterized in that: include: An acquisition module is configured to acquire a question list image photographed by a user terminal and submitted by a user; An input module is configured to input the question sheet image into an image quality assessment model to obtain a quality score of the question sheet image; the image quality assessment model is a model obtained by fine-tuning a neural network model constructed based on meta-learning using question sheet image samples of various distortion types; A judgment module is configured to judge whether the quality score of the question list image is greater than or equal to a first preset threshold value, and obtain a first judgment result; the higher the quality score, the greater the probability that the question list in the question list image is accurately identified; The identification module is configured to identify the question list in the question list image if the first judgment result indicates that the quality score of the question list image is greater than or equal to the first preset threshold.

15. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions. When the computer program or instructions are executed by the processor, the steps of the method described in any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images