Data processing method and device

Abstract

The embodiment of the invention provides a data processing method and device, and the method comprises the steps: obtaining a to-be-detected image which comprises a target question and at least one to-be-detected object corresponding to the target question; according to the question text corresponding to the target question and the detection box information of the at least one to-be-detected object, constructing a prompt text; the to-be-detected image and the prompt text are input into a multi-modal data processing model, a judgment result corresponding to each to-be-detected object output by the multi-modal data processing model is obtained, the judgment result comprises a detection object type and a detection content type, the detection object type is obtained by judging the type of the to-be-detected object, and the detection content type is obtained by judging the type of the to-be-detected object. The detection content type is obtained by performing correctness judgment on object content in the to-be-detected object; the visual information and the text information are subjected to conjoint analysis by utilizing the multi-modal data processing model, so that each to-be-detected object is accurately judged, and the correction accuracy and the processing efficiency are improved through multi-modal collaborative analysis.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a data processing method. Background Technology

[0002] In the education field, automated grading technology is a key function for improving learning efficiency. Students can upload photos of their handwritten answers, which can then be automatically graded using OCR (Optical Character Recognition) text recognition technology. However, due to the significant differences and diversity in students' handwritten answers, the results of automated grading using OCR text recognition often lack a large amount of relevant information, resulting in low accuracy. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising: Obtain an image to be detected, wherein the image to be detected includes a target question and at least one object to be detected corresponding to the target question; A prompt text is constructed based on the question text corresponding to the target question and the detection box information of at least one object to be detected; The image to be detected and the prompt text are input into the multimodal data processing model to obtain the judgment results corresponding to each object to be detected output by the multimodal data processing model. The judgment results include the detection object type and the detection content type. The detection object type is obtained by judging the type of the object to be detected, and the detection content type is obtained by judging the object content in the object to be detected.

[0005] According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, comprising: The image acquisition module is configured to acquire an image to be detected, wherein the image to be detected includes a target question and at least one object to be detected corresponding to the target question; The prompt construction module is configured to construct prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected. The result acquisition module is configured to input the image to be detected and the prompt text into the multimodal data processing model to obtain the judgment result corresponding to each object to be detected output by the multimodal data processing model. The judgment result includes the detection object type and the detection content type. The detection object type is obtained by judging the type of the object to be detected, and the detection content type is obtained by judging the object content in the object to be detected.

[0006] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0009] This specification provides a data processing method in one embodiment, which acquires a target image containing a target question and its corresponding detection object. It constructs a prompt text using the question text and the detection box information of the detection object, effectively fusing text semantics and spatial positioning information. The target image and the prompt text are then input into a multimodal data processing model, which performs joint analysis of visual and textual information to accurately determine each detection object. Through multimodal collaborative analysis, this method overcomes the limitations of traditional OCR technology in handwritten content recognition due to information loss, significantly improving grading accuracy and processing efficiency. Attached Figure Description

[0010] Figure 1 This is a schematic diagram illustrating a data processing method provided in one embodiment of this specification. Figure 2 This is a flowchart illustrating a data processing method provided in one embodiment of this specification; Figure 3 This is a schematic diagram illustrating the training process of a multimodal data processing model provided in one embodiment of this specification; Figure 4 This is a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification; Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0011] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0012] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0014] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0015] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0016] Multimodal Large Model: An advanced artificial intelligence model based on the Transformer architecture. Compared to large language models that are only pre-trained on massive text modal data, multimodal large models are pre-trained on massive cross-modal data (such as image-text pairs), gaining the ability to deeply understand and generate multiple types of information (i.e., modalities, including but not limited to images and text). Its core breakthrough lies in constructing a shared semantic space, which enables information from different modalities to be mapped and aligned to a unified representation, thereby achieving cross-modal semantic association and reasoning. For example, it can not only identify objects in images and understand text questions separately, but also combine the two to answer complex questions based on image content, generate descriptive text, or create images based on text instructions, demonstrating powerful context awareness, common sense reasoning, and content creation capabilities.

[0017] Optical Character Recognition (OCR) is a technology that converts text information in an image into machine-encodeable text. In this embodiment, as an optional preprocessing step, it is used to initially extract handwritten or printed text content from the captured image, providing text input for subsequent multimodal large-scale model analysis. It should be noted that this invention mainly relies on the end-to-end understanding capability of the multimodal large-scale model, which can reduce the dependence on the accuracy of traditional OCR technology.

[0018] User-generated photo grading: Students upload photos of questions from textbooks, workbooks, and exam papers using their device's camera for answering and grading. Unlike questions in a device's built-in question bank (with clearly defined answer boxes and standard answers), user-generated photo questions typically lack structured standard answers, have no fixed answer area, feature diverse handwriting styles, and have complex layouts. This results in OCR text recognition often missing a significant amount of relevant information, leading to a marked decrease in accuracy for grading methods that rely on OCR+text matching or OCR+text large-scale models. This invention provides intelligent and high-precision grading capabilities for these types of "open-ended photo questions."

[0019] This invention aims to address the common problems of low accuracy and poor adaptability in current photo-based grading systems. Existing technologies primarily rely on a framework of "OCR recognition + question bank matching + rule-based grading," which faces the following key challenges in practical applications: First, errors in answer box recognition can lead to the system misclassifying drafts, question content, or other non-answer content as valid answers, resulting in inaccurate grading results. Second, some questions that can be matched in the question bank cannot be effectively graded due to rule restrictions or excessive question complexity. Third, for simple questions not matched in the question bank, the system lacks autonomous grading capabilities, impacting user experience. Finally, for complex questions requiring step-by-step reasoning, traditional methods cannot understand the solution process and can only match the final answer, making it difficult to achieve precise step-by-step grading. These problems collectively limit the reliability and applicability of existing systems in real-world educational scenarios.

[0020] The existing technical solutions have the following significant shortcomings in design and implementation: First, they heavily rely on OCR recognition accuracy, and the diversity of handwriting and image noise can easily lead to text extraction errors, which cannot be corrected in subsequent processes; second, the grading logic is based on fixed rules, making it difficult to adapt to innovative solutions, partially correct answers, and diverse expressions, resulting in poor flexibility; third, the system cannot utilize visual information in images (such as formula layout, graphic annotations, handwriting layout, etc.), relying solely on text content, leading to incomplete semantic understanding; fourth, it has extremely high requirements for question bank coverage, lacking the ability to autonomously grade unmatched questions based on common sense or reasoning; and fifth, it cannot support structured analysis of complex multi-step questions, only performing binary judgments at the answer level, resulting in limited educational feedback value.

[0021] To address this, this invention proposes an end-to-end grading method based on a multimodal large model. By deeply fusing image visual information and text semantic information, it achieves direct understanding and fine-grained grading of student answers. This method aims to improve the accuracy of answer box recognition, enhance the flexibility of grading logic, expand the ability to autonomously grade unmatched questions, and support step-by-step analysis of complex problem-solving processes, thereby systematically solving the problems of low accuracy, poor adaptability, and high maintenance costs in existing technologies.

[0022] This specification provides a data processing method, and also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0023] See Figure 1 , Figure 1 A schematic diagram of a data processing method according to an embodiment of this specification is shown. Specifically, the data processing method is applied to a data processing system, which includes an end-side device 102 and a server 104. The end-side device 102 is used to send the image to be detected to the server 104.

[0024] A multimodal data processing model is trained in server 104. Server 104 acquires an image to be detected, which includes a target question and at least one object to be detected corresponding to the target question. A prompt text is constructed based on the question text corresponding to the target question and the detection bounding box information of the at least one object to be detected. The image to be detected and the prompt text are input into the multimodal data processing model to obtain the judgment result corresponding to each object to be detected, output by the multimodal data processing model. The judgment result includes the detection object type and the detection content type. The detection object type is obtained by determining the type of the object to be detected, and the detection content type is obtained by determining the object content within the object to be detected. The judgment result corresponding to each object to be detected is returned to the edge device 102.

[0025] The edge device 102 may include a browser, an app (application), or a web application such as an H5 (Hypertext Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. The edge device can be developed based on a software development kit (SDK) provided by the server, such as a real-time communication (RTC) SDK. The edge device can be deployed in an electronic device and depends on the device's operation or certain apps within the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured in the electronic device, such as human-computer interaction applications, model training applications, data processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0026] Server 104 can be understood as a server providing various services, including physical servers and cloud servers. Examples include servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It's important to note that Server 104 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. Server 104 can also be a server in a distributed system, or a server integrated with blockchain. Server 104 can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0027] It is worth noting that the data processing method provided in the embodiments of this specification can be executed by the server 104. In other embodiments of this specification, the multimodal data processing model can be deployed in the end-side device 102, so that the end-side device 102 can also have similar functions to the server 104, thereby executing the data processing method provided in the embodiments of this specification. In other embodiments, the data processing method provided in the embodiments of this specification can also be jointly executed by the end-side device 102 and the server 104.

[0028] See Figure 2 , Figure 2 A flowchart of a data processing method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0029] Step 202: Obtain the image to be detected, wherein the image to be detected includes the target question and at least one object to be detected corresponding to the target question.

[0030] The image to be detected can be understood as an image of homework or test questions provided by the user through a shooting method, which usually contains a complete description of the question and the student's handwriting; the target question can be understood as the specific test question content in the image to be detected that needs to be graded, which can be in the form of text, formulas or charts; the object to be detected can be understood as the answer area selected by the detection box, and each area may contain answer information such as text, handwritten formulas, and drawings.

[0031] Specifically, the system acquires the user-submitted image to be detected, identifies the question region in the image to determine the target question, and identifies each answer region as the object to be detected by detecting and locating one or more answer regions associated with the target question.

[0032] Taking a problem-solving question as an example, a user takes a picture of a math problem and uploads an image (the image to be detected). The image contains the problem description "Given triangle ABC...", and below it are two handwritten areas: the first area contains the derivation steps, and the second area shows a diagram of the triangle. The detection identifies the problem description as the target question and selects the two handwritten areas as the objects to be detected.

[0033] Taking fill-in-the-blank questions as an example, the target question is the text containing one or more blanks and related context. For example, if the target question is "The capital of country A is __", then the object to be detected is the answer area corresponding to each blank. For example, the object to be detected is the text area containing the words "a" that the user fills in at that blank.

[0034] Taking multiple-choice questions as an example, the target question includes the question description and all the options. For example, if the target question is "Which of the following is a mammal? A. Fish B. Bird C. Dog D. Snake", the object to be detected is the area marked with the user's selected option. Alternatively, the object to be detected could be the text area filled in by the user within parentheses at the end of the question description.

[0035] Of course, in complex question types such as drawing questions and experimental questions, the target question may also include non-text information such as attached figures and tables; the object to be tested can be extended to areas such as user-drawn graphics, labeled experimental data, and filled-in table cells, without limitation here.

[0036] Step 204: Construct prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected.

[0037] The title text can be understood as the textual description of the title obtained from the image to be detected through optical character recognition or text extraction technology; the detection box information can be understood as the information for locating and structuring the object to be detected, including the position information, size, and sequence number of the detection box, as well as the recognition content within the detection box; the prompt text can be understood as the text used to guide the model to perform the grading task, which integrates the title text, detection box information, and other necessary instructions.

[0038] Specifically, the text content corresponding to the target question in the image to be detected is extracted to obtain the question text corresponding to the target question, and the detection box information of each object to be detected is determined. The question text and the detection box information of each object to be detected (including but not limited to the position information, sequence number, and recognition content of the detection box) are integrated to form a prompt text for guiding the multimodal data processing model to grade the homework.

[0039] In one or more embodiments of this specification, before constructing the prompt text, text recognition is first performed on the object to be detected to determine the question text in the image and the detection box information of each object to be detected. Once the question text is obtained, it can be compared in a question database to determine relevant reference questions for the target question. Specific implementation methods are as follows: Before constructing the prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected, the method further includes: Text recognition is performed on the image to be detected to obtain the title text corresponding to the target title in the image to be detected, and the detection box information corresponding to the at least one object to be detected, wherein the detection box information includes the detection box position information and the content of the detected object; Based on the comparison of the question text corresponding to the target question in the question database, a reference question corresponding to the target question is obtained, and the question similarity between the target question and the reference question is determined.

[0040] Text recognition can be understood as the process of extracting readable text content from an image using optical character recognition technology; detection box position information is used to describe the spatial position of the object to be detected in the image to be detected, usually represented in coordinate form; the detected object content can be understood as the text, symbol or graphic information obtained within the detection box through recognition technology.

[0041] A question database can be understood as a pre-built structured question bank containing a large number of questions and their reference answers; reference questions can be understood as questions related to the target question content retrieved from the question database through semantic matching; question similarity is used to quantify the degree of content relevance between the target question and the reference questions, and is usually expressed in numerical form.

[0042] Specifically, OCR technology is used to perform text recognition on the image to be detected, obtain the text of the target question, and extract the recognition content (i.e. the object content of the object to be detected) within each detection box based on the location information of the detected boxes. The recognized question text is then input into the question database for semantic similarity matching. The retrieved related questions are used as reference questions, and the question similarity between the reference questions and the target question is calculated.

[0043] For example, for the target question "Solve the equation: 2x+3=7" in the image to be detected, the question text and the corresponding object content (e.g., the object content is "x=2") are obtained through text recognition; the question text is searched in the question database and a reference question "Solve 2x+3=7" is matched, and the question similarity between the two is calculated to be 0.95. Subsequently, different prompt texts are constructed based on the question similarity.

[0044] The data processing method provided in the embodiments of this specification converts image information into structured text data through text recognition, providing a data foundation for subsequent semantic matching and content understanding; based on the question database, semantic similarity retrieval of target questions is performed, and by quantifying the question similarity between the retrieved reference questions and the target questions, differentiated processing strategies can be implemented for questions with different matching degrees, thereby improving the overall processing efficiency.

[0045] In one or more embodiments of this specification, prompt text can be dynamically constructed based on question similarity. For example, for target questions with low question similarity, the reasoning ability of the multimodal data processing model itself can be used for grading, while for target questions with high question similarity, the reference answer corresponding to the reference question can be used for auxiliary judgment. Specific implementation methods are described below: The step of constructing prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected includes: If the similarity between the questions is greater than or equal to a preset similarity threshold, the reference answer corresponding to the reference question is determined, and the prompt text is constructed based on the question text corresponding to the target question, the detection box information of the at least one object to be detected, the reference question, and the reference answer corresponding to the reference question. If the similarity between the questions is less than the preset similarity threshold, the prompt text is constructed based on the question text corresponding to the target question and the detection box information of the at least one object to be detected.

[0046] Among them, the preset similarity threshold can be understood as a pre-set critical value used to distinguish the degree of matching of questions, which can be set according to the actual situation; the reference answer can be understood as the standard answer or official solution corresponding to the reference question in the question database, which can include multimodal data such as text, formulas, and charts.

[0047] Specifically, if the similarity of the questions reaches or exceeds the preset similarity threshold, it is determined that there is a highly matching reference question in the question database. At this time, not only is a prompt text constructed based on the question text of the target question and the detection box information of the object to be detected, but the matched reference question and its reference answer are also included in the prompt text, providing a clear basis for grading for the multimodal data processing model.

[0048] If the question similarity is lower than the preset similarity threshold, it means that no highly matching reference questions were found in the question database. In this case, such questions are regarded as simple questions, and prompt text can be constructed based on the original recognized question text and detection box information. The multimodal data processing model itself relies on its knowledge reserves and reasoning ability to analyze and correct the answers.

[0049] The data processing method provided in the embodiments of this specification introduces reference answer information for target questions with high matching degree, thereby effectively improving the accuracy of grading. For target questions with low matching degree, prompt text is constructed based on the information in the image to be detected to avoid introducing irrelevant or misleading reference information. In this way, the availability of grading service can still be guaranteed even when the question database has limited coverage of questions.

[0050] Step 206: Input the image to be detected and the prompt text into the multimodal data processing model to obtain the judgment result corresponding to each object to be detected output by the multimodal data processing model. The judgment result includes the detection object type and the detection content type. The detection object type is obtained by judging the type of the object to be detected, and the detection content type is obtained by judging the correctness of the object content in the object to be detected.

[0051] Among them, the multimodal data processing model can be understood as a machine learning model that can process multiple modalities of data. It is usually implemented based on a vision-language pre-training architecture. In this embodiment, the multimodal data processing model is a large multimodal model. The detection object type can be understood as the category obtained by classifying the properties of the object to be detected. For example, the detection object type includes answer box, non-answer box, question area, draft area, etc. The detection content type can be understood as the grading judgment of the information contained in the object to be detected. For example, the detection content type includes correct, incorrect, uncertain, etc.

[0052] Specifically, multimodal data processing models can fuse visual images and text information to accurately correct each object to be detected.

[0053] In practice, the multimodal data processing model receives the image to be detected and its corresponding prompt text. Based on the image to be detected and the prompt text, it identifies the type of each object to be detected and determines whether it is a valid answer area. Subsequently, it further analyzes the content of the object within the area confirmed as an answer box and judges the correctness of the object content based on the text recognition results and / or visual features.

[0054] For example, when dealing with object A in a physics problem, the multimodal data processing model might identify the area corresponding to object A as an "answer box" and correct the content within it, such as determining that the content is "correct." However, for another object B that only contains messy lines, the multimodal data processing model might identify it as a "non-answer area."

[0055] Multimodal data processing models can combine image and text information for comprehensive judgment, effectively overcoming the limitations of single modality. For example, OCR recognition of text may not be able to distinguish between answer text and question text, while combining visual layout information can improve the accuracy of object type determination. It can process different forms of answer content, whether it is plain text, formulas or charts, and multimodal data processing models can effectively identify and classify them through multimodal features.

[0056] In one or more embodiments of this specification, when judging each object to be detected, the detection object type of the object to be detected can be determined first. Then, if the detection object type is the expected object type, fine-grained correctness analysis can be performed on the object content in the object to be detected. Specific implementation methods are as follows: The step of inputting the image to be detected and the prompt text into a multimodal data processing model to obtain the judgment results corresponding to each object to be detected output by the multimodal data processing model includes: The image to be detected and the prompt text are input into a multimodal data processing model. Based on the multimodal data processing model, the type of each object to be detected is determined, and the detection object type corresponding to each object to be detected is determined. The detection object type is either a first object type or a second object type. When the detected object type is a first object type, the first object type is determined as the judgment result corresponding to the object to be detected; When the detection object type is the second object type, the object content of the object to be detected is judged to determine the detection content type corresponding to the object to be detected, and the detection content type is determined as the judgment result corresponding to the object to be detected. The detection content type is determined from at least one preset content type, and the preset content type includes unknown types.

[0057] The first object type can be understood as the type that is directly determined to be a non-answer area without further analysis of the object content. For example, when the object to be tested is a non-process question answer area, draft, irrelevant information, or question area, the object to be tested is determined to be the first object type (i.e., a non-answer area). The second object type can be understood as the type that is determined to be an answer area, and its object content needs to be further analyzed to determine the correctness of the object content.

[0058] Preset content types can be understood as a predefined set of categories used to describe the correctness of object content. For example, preset content types include correct, incorrect, uncertain (i.e. unknown type), etc. Of course, preset content types can be refined according to needs. For example, preset content types can also include types such as uncertain due to unclear content or uncertain due to complex questions.

[0059] Specifically, the multimodal model determines whether each object belongs to a valid answer area based on the visual features of the image to be detected and the context provided by the prompt text. If the object is determined to be of the first object type (i.e., "non-answer box"), then the first object type is used as the judgment result corresponding to the object. For example, areas that are obviously question text or draft notes are marked as "non-answer boxes". If the object is determined to be of the second object type, then the object content in the object is further analyzed to determine the detection content type corresponding to the object content from the preset content types.

[0060] For example, considering three bounding boxes (i.e., objects to be detected) in a test question image, the first bounding box contains printed question text. The multimodal data processing model determines this bounding box as the question area, belonging to the first object type, and classifies it as a "non-answer bounding box". The second bounding box contains handwritten content such as "Solution: Let x = ...". The multimodal data processing model determines this bounding box as the answer area, and further analysis of the handwritten content determines the content type to be "correct".

[0061] The data processing method provided in the embodiments of this specification can quickly filter out obvious non-answer areas by first distinguishing the detection object type of the object to be detected, reducing unnecessary computational overhead and improving processing efficiency; subsequently, fine-grained correctness judgment is only performed on the content in the answer area, and by introducing "unknown type", the robustness of the system is enhanced when facing content that is difficult to classify, avoiding erroneous judgments caused by forced classification.

[0062] In one or more embodiments of this specification, the multimodal data processing model is trained through the following steps: Obtain an image sample to be detected, wherein the image sample to be detected includes a target question sample and at least one object sample to be detected corresponding to the target question sample; A prompt text sample is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected, wherein the annotation type includes the detection object type and the detection content type; The image sample to be detected and the prompt text sample are input into the initial data processing model to obtain the prediction and judgment results corresponding to each object sample to be detected output by the initial data processing model. Based on the prediction and judgment results corresponding to each sample to be detected and the labeling type corresponding to each sample to be detected, the initial data processing model is adjusted to obtain the multimodal data processing model.

[0063] The image sample to be detected can be understood as an image with labeled question and answer regions used for model training; the target question sample and the object sample to be detected correspond to the question and its associated answer region in the image sample to be detected, respectively.

[0064] The annotation type can be understood as the annotation result of the pre-annotated sample to be detected. The annotation type covers the detection object type (such as answer box, non-answer box) and the detection content type (such as correct, incorrect, uncertain, etc.). The initial data processing model can be understood as a general multimodal model that has not been specially trained. The prediction result can be understood as the detection object type and detection content type output by the initial data processing model for each sample to be detected.

[0065] The specific implementation is similar to the above embodiments. However, during the model training phase, the prompt text also includes the annotation type for each sample to be detected. This annotation type is used as a label and compared with the prediction result output by the initial data processing model. The loss is calculated by calculating the difference between the prediction result (i.e., the prediction result) and the real label (i.e., the annotation type), and the model parameters are updated accordingly to obtain the trained multimodal data processing model.

[0066] The data processing method provided in the embodiments of this specification supervises and adjusts the initial data processing model based on the annotation type, thereby effectively transferring the general capabilities of the basic model to the specific domain of homework grading, improving training efficiency and practicality while ensuring model performance.

[0067] In one or more embodiments of this specification, during the model training phase, different prompt text samples are constructed based on the similarity of different question samples. This maintains the prompt text input consistent with the real reasoning environment, enabling the training process to simulate two scenarios during actual grading: whether or not reference answer information is available. Specific implementation methods are described below: The step of constructing a prompt text sample based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected includes: Based on the comparison of the question text sample corresponding to the target question sample in the question database, a reference question sample corresponding to the target question sample is obtained, and the question sample similarity between the target question sample and the reference question sample is determined; If the similarity of the question samples is greater than or equal to a preset similarity threshold, a reference answer sample corresponding to the reference question sample is determined, and the prompt text is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object to be detected, the reference question sample, the reference answer sample corresponding to the reference question sample, and the annotation type. If the similarity between the questions is less than the preset similarity threshold, a prompt text sample is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected.

[0068] The reference question sample can be understood as a question retrieved from the question database through semantic matching that is related to the content of the target question sample; the reference answer sample can be understood as the standard answer or official explanation corresponding to the reference question sample.

[0069] For detailed implementation methods, please refer to the above embodiments, which will not be repeated here.

[0070] The data processing method provided in the embodiments of this specification enables the multimodal data processing model to learn stable reasoning capabilities in both modes with reference answer assistance and independent judgment without reference answer by simulating the question database matching and the differentiated components of prompt text in real reasoning during the training phase. This improves adaptability. By exposing the multimodal data processing model to diverse prompt text inputs during the training phase, the robustness of the multimodal data processing model is enhanced, thereby maintaining more stable performance in real complex scenarios.

[0071] The data processing method provided in the embodiments of this specification, leveraging the powerful visual understanding capabilities of a multimodal large model, enables the system to automatically identify and filter invalid answer areas in images, such as drafts, question text, and other irrelevant content, thereby significantly improving the accuracy of extracting students' actual answers. Through the semantic understanding and logical reasoning capabilities of the multimodal large model, the system can not only determine the correctness of the final answer but also understand diverse problem-solving approaches and expressions, thus enabling a more detailed analysis of the problem-solving process. For example, it can provide instructive feedback. By fully integrating visual information from images with text information extracted by OCR, and through cross-modal alignment and semantic fusion using a multimodal data processing model, the accuracy and robustness of grading are improved.

[0072] See Figure 3 , Figure 3 This is a schematic diagram illustrating the training process of a multimodal data processing model provided in one embodiment of this specification.

[0073] Step 302: Obtaining the original question data.

[0074] Obtain the original question data related to user photo correction. The original question data includes, but is not limited to, the question text corresponding to the target question in the captured image (i.e. the image to be detected), the image of the user's answer to the question, and the answer text content extracted after OCR processing.

[0075] Step 304: Question bank matching and filtering.

[0076] Based on the question text obtained in step 302, matching and filtering are performed in the question database (i.e., the question bank).

[0077] Specifically, the system uses OCR to recognize the question text and performs semantic similarity retrieval in the question bank. It then calculates and obtains the most similar reference question and its corresponding similarity score. Based on the similarity score, the questions are divided into two categories: questions with a similarity score at or above a preset similarity threshold (e.g., 0.7) are classified as high-matching questions, and their corresponding reference answers can be obtained; questions with a similarity score below the preset similarity threshold are classified as low-matching questions, and no reference answer information is provided for these questions. The system considers these questions as either relatively simple or relatively complex questions to be processed.

[0078] Step 306: Dynamic construction of multimodal input.

[0079] Based on the question matching results, multimodal input data is dynamically constructed.

[0080] Specifically, for highly matched questions, the reference answer for the corresponding reference question is obtained. If the reference answer includes image information, that image is also used as part of the model input. The image to be detected, the user's answer box information (i.e., the detection box information of each object to be detected), and the reference answer information are integrated together to construct multimodal input data.

[0081] For questions with low matching degree, the input can be constructed based on the image to be detected and the user's answer box information. The multimodal data processing model (i.e., multimodal large model) can then determine whether the question is easy or not and grade it in subsequent stages.

[0082] Step 308: Data labeling.

[0083] User response boxes are manually or automatically annotated, with annotation results categorized into four types: correct, incorrect, uncertain, and non-response box. Before annotation, it's necessary to determine if the area to be annotated is a valid user response box. If determined to be a non-response box, it is directly annotated as "non-response box." If it is a valid user response box, further content assessment is required based on the annotation criteria. It should be noted that the annotation process prioritizes visual content of the image. Specific annotation criteria include: Non-answer box: Solution process, draft content, question, and other information unrelated to answering questions.

[0084] Correct: The user provided a clear answer, and the answer was correct.

[0085] Error: The user provided a clear but incorrect answer, or the user was supposed to answer but did not.

[0086] Uncertain: Users have provided answers, but the content is unclear; the question information is incomplete, key information is missing or unclear; the answer is obviously incomplete and cannot be judged based on the context; the question is too difficult, making accurate grading difficult; it is impossible to clearly determine whether the area is a reasonable answer box.

[0087] Step 310: Model fine-tuning.

[0088] Specifically, to improve the model's performance in homework grading tasks, the aforementioned labeled data was used to fine-tune the model. The model was positioned as a professional homework grader, with its task being to directly provide grading results based on the provided questions and student responses.

[0089] For example, the system prompts: "You are a professional homework grader. You can directly provide grading results based on the question and student's answer. I will provide supplementary text information: the question text content, the number of grading boxes, the text content corresponding to each grading box arranged in numerical order, and the corresponding border color. Please strictly follow the numerical order of the grading boxes to grade the answers. Since grading boxes may be misrecognized, you need to first determine whether the box is a valid user answer box. If so, further grading is required; otherwise, directly grade it as a non-answer box. In addition, there may be discrepancies between the size and position of the grading box and the recognized text content. Sometimes, it is necessary to combine the surrounding information of the box for grading, and prioritize grading based on image content rather than text content. The specific grading criteria are as follows:" Non-answer box: Solution process, draft content, question, and other information unrelated to answering questions.

[0090] Correct: The user provided a clear answer, and the answer was correct.

[0091] Error: The user provided a clear but incorrect answer, or the user was supposed to answer but did not.

[0092] Uncertain: Users have provided answers, but the content is unclear; the question information is incomplete, key information is missing or unclear; the answer is obviously incomplete and cannot be judged based on the context; the question is too difficult, making accurate grading difficult; it is impossible to clearly determine whether the area is a reasonable answer box.

[0093] Additionally, if two boxes overlap in height, treat either box as an answer box and grade it normally, while the other box is considered a non-answer box. You should ultimately return a list of grading results, with each answer box corresponding to one of four grading types: [Correct, Incorrect, Uncertain, Non-answer]. You do not need to return any other content.

[0094] The model input includes: original question image, question text, number of grading boxes, user's answer text in the grading boxes, searched reference questions, reference question images, etc. Among them, the original question image, question text, number of grading boxes, user's answer text in the grading boxes, searched reference questions, and reference question images will be filled with the corresponding content after cleaning and processing.

[0095] The output includes: <Correct>, <Incorrect>, <Uncertain>, and <No Answer>. For any answer box, select one of the four results. The output format is a list, corresponding to the judgment results of each answer box.

[0096] Through the above structured data construction and training methods, the model can gradually learn and optimize its judgment ability and accuracy in multimodal job grading tasks, thus obtaining a multimodal data processing model.

[0097] The data processing methods described in the above embodiments are implemented using this multimodal data processing model, and will not be elaborated here.

[0098] In practice, for questions not matched in the question bank, if the content is relatively simple, the system can utilize the common sense reasoning capabilities of the multimodal data processing model for direct analysis and grading. This reduces reliance on the completeness of the question database and enhances the system's ability to handle open-ended or novel question types. For complex questions, combining question bank matching results with multimodal deep understanding capabilities, the system supports step-by-step grading of answers, identifying partially correct and partially incorrect responses, thereby providing more detailed and comprehensive feedback.

[0099] The data processing method provided in the embodiments of this specification achieves significant improvements in several aspects by introducing an end-to-end grading mechanism based on a multimodal large model. In answer box recognition, the visual understanding capability of the multimodal large model automatically filters invalid answer areas, significantly improving recognition accuracy. Regarding grading effectiveness, the integration of visual information and textual semantics enables autonomous grading of simple questions and supports step-by-step analysis and partial correctness judgment for complex questions, comprehensively improving overall grading accuracy and refinement, and making the judgment results more reliable. The deep fusion of multimodal information allows the system to more fully utilize visual information such as layout, handwriting, and illustrations in images, mimicking the comprehensive judgment logic of human teachers, greatly enhancing the rationality and interpretability of grading. This multimodal large model also has strong transfer adaptability, requiring only minor adjustments to a small amount of labeled data to adapt to new question types, significantly reducing system maintenance and iteration costs. It fundamentally overcomes the technical bottlenecks of traditional rule-based systems—rigidity, reliance on question bank coverage, and difficulty in handling complex questions—providing an effective technical path for promoting intelligent homework grading towards greater accuracy, flexibility, and practicality.

[0100] By fusing user-captured images, OCR-recognized answer boxes, text content, and reference answers matched from the question bank in a multimodal manner, the advantages of different modal information are fully utilized, significantly improving the grading accuracy. The dynamic grading strategy based on question bank matching similarity adaptively adjusts the grading strategy according to the question matching situation, and introduces a model rejection mechanism to output "uncertain" when faced with insufficient information or highly complex questions, rather than forcibly giving potentially incorrect grading results, effectively balancing grading coverage and result reliability.

[0101] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 4 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 4 As shown, the device includes: The image acquisition module 402 is configured to acquire an image to be detected, wherein the image to be detected includes a target question and at least one object to be detected corresponding to the target question; The prompt construction module 404 is configured to construct prompt text based on the question text corresponding to the target question and the detection box information of the at least one object to be detected. The result acquisition module 406 is configured to input the image to be detected and the prompt text into a multimodal data processing model to obtain the judgment result corresponding to each object to be detected output by the multimodal data processing model. The judgment result includes the detection object type and the detection content type. The detection object type is obtained by judging the type of the object to be detected, and the detection content type is obtained by judging the correctness of the object content in the object to be detected.

[0102] The device further includes: The matching module is configured to perform text recognition on the image to be detected, obtain the question text corresponding to the target question in the image to be detected, and the detection box information corresponding to the at least one object to be detected, wherein the detection box information includes the detection box position information and the content of the detected object; compare the question text corresponding to the target question in the question database to obtain the reference question corresponding to the target question, and determine the question similarity between the target question and the reference question.

[0103] Optionally, the prompt construction module 404 is further configured to: If the similarity between the questions is greater than or equal to a preset similarity threshold, the reference answer corresponding to the reference question is determined, and the prompt text is constructed based on the question text corresponding to the target question, the detection box information of the at least one object to be detected, the reference question, and the reference answer corresponding to the reference question. If the similarity between the questions is less than the preset similarity threshold, the prompt text is constructed based on the question text corresponding to the target question and the detection box information of the at least one object to be detected.

[0104] Optionally, the result obtaining module 406 is further configured to: The image to be detected and the prompt text are input into a multimodal data processing model. Based on the multimodal data processing model, the type of each object to be detected is determined, and the detection object type corresponding to each object to be detected is determined. The detection object type is either a first object type or a second object type. When the detected object type is a first object type, the first object type is determined as the judgment result corresponding to the object to be detected; When the detection object type is the second object type, the object content of the object to be detected is judged to determine the detection content type corresponding to the object to be detected, and the detection content type is determined as the judgment result corresponding to the object to be detected. The detection content type is determined from at least one preset content type, and the preset content type includes unknown types.

[0105] The device further includes: The training module is configured to acquire image samples to be detected, wherein the image samples to be detected include target question samples and at least one object sample to be detected corresponding to the target question samples; construct prompt text samples based on the question text samples corresponding to the target question samples, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected, wherein the annotation type includes the object type to be detected and the content type to be detected; input the image samples to be detected and the prompt text samples into an initial data processing model to obtain the prediction and judgment results corresponding to each object sample to be detected output by the initial data processing model; adjust the initial data processing model based on the prediction and judgment results corresponding to each object sample to be detected and the annotation type corresponding to each object sample to be detected to obtain the multimodal data processing model.

[0106] Optionally, the training module is further configured to: Based on the comparison of the question text sample corresponding to the target question sample in the question database, a reference question sample corresponding to the target question sample is obtained, and the question sample similarity between the target question sample and the reference question sample is determined; If the similarity of the question samples is greater than or equal to a preset similarity threshold, a reference answer sample corresponding to the reference question sample is determined, and the prompt text is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object to be detected, the reference question sample, the reference answer sample corresponding to the reference question sample, and the annotation type. If the similarity between the questions is less than the preset similarity threshold, a prompt text sample is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected.

[0107] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0108] Figure 5 A structural block diagram of a computing device 500 according to one embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0109] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0110] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0111] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0112] The processor 520 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0113] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the data processing method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data processing method embodiments.

[0114] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0115] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the data processing method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data processing method embodiments.

[0116] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0117] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0118] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0119] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0120] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0121] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0122] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized in that, include: Obtain an image to be detected, wherein the image to be detected includes a target question and at least one object to be detected corresponding to the target question; A prompt text is constructed based on the question text corresponding to the target question and the detection box information of at least one object to be detected; The image to be detected and the prompt text are input into a multimodal data processing model to obtain the judgment results corresponding to each object to be detected output by the multimodal data processing model. The judgment results include the detection object type and the detection content type. The detection object type is obtained by judging the type of the object to be detected, and the detection content type is obtained by judging the correctness of the object content in the object to be detected.

2. The data processing method according to claim 1, characterized in that, Before constructing the prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected, the method further includes: Text recognition is performed on the image to be detected to obtain the title text corresponding to the target title in the image to be detected, and the detection box information corresponding to the at least one object to be detected, wherein the detection box information includes the detection box position information and the content of the detected object; Based on the comparison of the question text corresponding to the target question in the question database, a reference question corresponding to the target question is obtained, and the question similarity between the target question and the reference question is determined.

3. The data processing method according to claim 2, characterized in that, The step of constructing prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected includes: If the similarity between the questions is greater than or equal to a preset similarity threshold, the reference answer corresponding to the reference question is determined, and the prompt text is constructed based on the question text corresponding to the target question, the detection box information of the at least one object to be detected, the reference question, and the reference answer corresponding to the reference question. If the similarity between the questions is less than the preset similarity threshold, the prompt text is constructed based on the question text corresponding to the target question and the detection box information of the at least one object to be detected.

4. The data processing method according to claim 1, characterized in that, The step of inputting the image to be detected and the prompt text into a multimodal data processing model to obtain the judgment results corresponding to each object to be detected output by the multimodal data processing model includes: The image to be detected and the prompt text are input into a multimodal data processing model. Based on the multimodal data processing model, the type of each object to be detected is determined, and the detection object type corresponding to each object to be detected is determined. The detection object type is either a first object type or a second object type. When the detected object type is a first object type, the first object type is determined as the judgment result corresponding to the object to be detected; When the detection object type is the second object type, the object content of the object to be detected is judged to determine the detection content type corresponding to the object to be detected, and the detection content type is determined as the judgment result corresponding to the object to be detected. The detection content type is determined from at least one preset content type, and the preset content type includes unknown types.

5. The data processing method according to any one of claims 1-4, characterized in that, The multimodal data processing model is trained through the following steps: Obtain an image sample to be detected, wherein the image sample to be detected includes a target question sample and at least one object sample to be detected corresponding to the target question sample; A prompt text sample is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected, wherein the annotation type includes the detection object type and the detection content type; The image sample to be detected and the prompt text sample are input into the initial data processing model to obtain the prediction and judgment results corresponding to each object sample to be detected output by the initial data processing model. Based on the prediction and judgment results corresponding to each sample to be detected and the labeling type corresponding to each sample to be detected, the initial data processing model is adjusted to obtain the multimodal data processing model.

6. The data processing method according to claim 5, characterized in that, The step of constructing a prompt text sample based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected includes: Based on the comparison of the question text sample corresponding to the target question sample in the question database, a reference question sample corresponding to the target question sample is obtained, and the question sample similarity between the target question sample and the reference question sample is determined; If the similarity of the question samples is greater than or equal to a preset similarity threshold, a reference answer sample corresponding to the reference question sample is determined, and the prompt text is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object to be detected, the reference question sample, the reference answer sample corresponding to the reference question sample, and the annotation type. If the similarity between the questions is less than the preset similarity threshold, a prompt text sample is constructed based on the question text sample corresponding to the target question sample, the detection box sample information of the at least one object sample to be detected, and the annotation type corresponding to the at least one object sample to be detected.

7. A data processing apparatus, characterized in that, include: The image acquisition module is configured to acquire an image to be detected, wherein the image to be detected includes a target question and at least one object to be detected corresponding to the target question; The prompt construction module is configured to construct prompt text based on the question text corresponding to the target question and the detection box information of at least one object to be detected. The result acquisition module is configured to input the image to be detected and the prompt text into the multimodal data processing model to obtain the judgment result corresponding to each object to be detected output by the multimodal data processing model. The judgment result includes the detection object type and the detection content type. The detection object type is obtained by judging the type of the object to be detected, and the detection content type is obtained by judging the object content in the object to be detected.

8. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program / instructions that, when executed by a processor, implement the steps of the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Images