Teaching auxiliary method, device, storage medium and electronic device based on image recognition

Through the teaching assistance method based on image recognition and the use of the morphological distribution matrix to identify teaching levels, the problem that the existing intelligent correction system cannot provide in-depth feedback and adjustment of teaching plans is solved, and the accurate identification of teaching results and adaptive adjustment of teaching plans are achieved, which improves teaching efficiency.

CN119229452BActive Publication Date: 2025-03-21SHANGHAI WANRUILONG EDUCATION TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411733802.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-29
Publication Date
2025-03-21
Estimated Expiration
2044-11-29

AI Technical Summary

Technical Problem

The existing intelligent correction systems cannot effectively provide in-depth teaching feedback based on the review results, nor can they adapt to the teaching plan based on these results, mainly because these systems lack the ability to understand and analyze the detailed information in the answering process.

Method used

Through the teaching assistance method based on image recognition, the initial image information of the target object is obtained, the pixel distribution information and morphological distribution information are determined using the preset object recognition model, the morphological distribution matrix is ​​constructed, and the teaching level is determined based on the matrix value calculation results, so as to achieve accurate identification of teaching results and adaptive adjustment of teaching plans.

Benefits of technology

By determining the distribution of pixel points corresponding to the grayscale value, and then judging the teaching level based on the correction, so as to accurately identify the teaching results, and then adaptively adjust the teaching plan based on the teaching results, thereby improving teaching efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119229452B_ABST
    Figure CN119229452B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a teaching assistance method, device, storage medium and electronic device based on image recognition, which relates to the technical field of teaching assistance technology. The method includes: obtaining the initial image information of the target object; determining the pixel distribution information of the first object contained in the target object in a preset area through a preset object recognition model, and determining the morphological distribution information of the first object in the preset area according to the pixel distribution information; constructing a morphological distribution matrix according to the morphological distribution information of different areas; calculating the matrix value of the morphological distribution matrix, and determining the teaching level of the target object according to the matrix value calculation result. Through the present invention, the problem of low teaching efficiency is solved, thereby achieving the effect of improving teaching efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of teaching assistance, and in particular, to a teaching assistance method, device, storage medium and electronic device based on image recognition. Background Art

[0002] With the rapid development of artificial intelligence technology, its application in the education industry has become increasingly widespread. In particular, in terms of intelligent grading systems, the use of artificial intelligence technology to automatically grade oral arithmetic questions and objective test questions has made significant progress, and the technology of these systems is already quite mature. However, for more complex math questions and subjective test questions, existing intelligent grading technologies still face many challenges.

[0003] However, current intelligent grading systems are unable to effectively provide in-depth teaching feedback based on grading results, nor can they make adaptive adjustments to teaching plans based on these results. This is mainly because existing systems lack the ability to understand and analyze detailed information in the answering process, such as whether the steps taken by students when solving math problems are correct, or whether the arguments given when answering subjective questions are sufficient.

[0004] There is currently no better solution to the above problems. Summary of the invention

[0005] The embodiments of the present invention provide a teaching auxiliary method, device, storage medium and electronic device based on image recognition, so as to at least solve the problem of low teaching efficiency in the related art.

[0006] According to an embodiment of the present invention, a teaching auxiliary method based on image recognition is provided, comprising:

[0007] Obtaining initial image information of the target object;

[0008] Determine pixel distribution information of a first object included in the target object in a preset area through a preset object recognition model, and determine morphological distribution information of the first object in the preset area according to the pixel distribution information, wherein the first object includes pixel points whose pixel gray value is less than or equal to a first threshold value;

[0009] Constructing a morphological distribution matrix according to the morphological distribution information of different regions;

[0010] Matrix value calculation is performed on the morphological distribution matrix, and the teaching level of the target object is determined according to the matrix value calculation result.

[0011] In an exemplary embodiment, determining the morphological distribution information of the first object in the preset area according to the pixel distribution information includes:

[0012] Determining a distribution distance of the first object in the preset area according to the pixel distribution information;

[0013] Determine a first threshold relationship between a maximum value and a minimum value of the distance between the first objects according to the distribution distance;

[0014] When the first threshold relationship meets the first condition, the first form relationship between the first objects in the preset area is determined to be the first form, otherwise it is the second form.

[0015] In an exemplary embodiment, when the threshold relationship meets the first condition, after determining that the morphological relationship between the first objects in the preset area is the first form, otherwise the morphological relationship is the second form, the method further includes:

[0016] Constructing a morphological distribution matrix of the first object in the preset area according to the pixel distribution information;

[0017] Calculating distribution values ​​of the morphological distribution matrix, and determining a second morphological relationship of the first object in the preset area according to a second threshold relationship;

[0018] The first morphological relationship is matched with the second morphological relationship, and when the first morphological relationship does not match the second morphological relationship, it is determined that an abnormality exists in the first morphological relationship and / or the second morphological relationship.

[0019] In an exemplary embodiment, before determining the pixel distribution information of the first object included in the target object in a preset area by using a preset object recognition model, the method further includes:

[0020] Acquire preprocessed training data, wherein the training data includes a position label and a category label;

[0021] Dividing the training data into a training set and a test set, wherein the ratio of the data volume of the training set to the data volume of the test set is 3:1;

[0022] Performing object feature extraction on the training set to obtain training object features;

[0023] Training a preset initial recognition model through the training object features, and adjusting the training results through the test set;

[0024] The initial recognition model is repeatedly trained and adjusted a preset number of times to obtain the object recognition model.

[0025] According to another embodiment of the present invention, there is provided a teaching auxiliary device based on image recognition, comprising:

[0026] An image acquisition module, used to obtain initial image information of the target object;

[0027] a distribution information determination module, configured to determine pixel distribution information of a first object contained in the target object in a preset area through a preset object recognition model, and determine morphological distribution information of the first object in the preset area according to the pixel distribution information, wherein the first object includes pixel points whose pixel grayscale values ​​are less than or equal to a first threshold value;

[0028] A first matrix construction module, used to construct a morphological distribution matrix according to the morphological distribution information of different regions;

[0029] The teaching level determination module is used to perform matrix value calculation on the morphological distribution matrix and determine the teaching level of the target object according to the matrix value calculation result.

[0030] In an exemplary embodiment, it further includes:

[0031] The distribution information determination module includes:

[0032] a distribution distance determining unit, configured to determine a distribution distance of the first object in the preset area according to the pixel distribution information;

[0033] a threshold relationship determining unit, configured to determine a first threshold relationship between a maximum value and a minimum value of the distance between the first objects according to the distribution distance;

[0034] The first form determining unit is configured to determine that the first form relationship between the first objects in the preset area is a first form if the first threshold relationship meets a first condition, and is a second form otherwise.

[0035] In an exemplary embodiment, it further includes:

[0036] A second distribution matrix construction module is used to construct a morphological distribution matrix of the first objects in the preset area according to the pixel distribution information after determining that the morphological relationship between the first objects in the preset area is a first morphology if the threshold relationship meets the first condition, and is a second morphology otherwise;

[0037] A second morphology determination module, configured to calculate a distribution value of the morphology distribution matrix and determine a second morphology relationship of the first object in the preset area according to a second threshold relationship;

[0038] A matching module is used to match the first morphological relationship with the second morphological relationship, and when the first morphological relationship does not match the second morphological relationship, determine that there is an abnormality in the first morphological relationship and / or the second morphological relationship.

[0039] In an exemplary embodiment, it further includes:

[0040] A data acquisition module, used to acquire pre-processed training data before determining pixel distribution information of a first object included in the target object in a preset area through a preset object recognition model, wherein the training data includes a position label and a category label;

[0041] A data set division module, used to divide the training data into a training set and a test set, wherein the ratio of the data volume of the training set to the data volume of the test set is 3:1;

[0042] A feature extraction module, used to extract object features from the training set to obtain training object features;

[0043] The training module is used to train the preset initial recognition model through the training object characteristics and adjust the training results through the test set; repeatedly perform a preset number of training and adjustments on the initial recognition model to obtain the object recognition model.

[0044] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0045] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0046] Through the present invention, the correction situation is determined by the distribution of pixel points corresponding to the grayscale value, and the teaching level is judged according to the correction situation, so as to accurately identify the teaching results, and then the teaching plan can be adaptively adjusted according to the teaching results, so as to improve the teaching efficiency. Therefore, the problem of low teaching efficiency can be solved and the effect of improving teaching efficiency can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a teaching auxiliary method based on image recognition according to an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the principle of an embodiment of the present invention;

[0049] Figure 3 It is a structural block diagram of a teaching auxiliary device based on image recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0052] In this embodiment, a teaching auxiliary method based on image recognition is provided. Figure 1 is a flow chart of a teaching auxiliary method based on image recognition according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0053] Step S11, obtaining initial image information of the target object;

[0054] In this embodiment, the target objects include teaching materials such as homework books and exercise books. Correspondingly, the initial image information including the collection of initial image information can be collected through a unified image acquisition device, such as uniformly photographing and scanning the target object through a specific projector or camera, etc., or different students or parents can take pictures of homework books or exercise books in different environments and upload them to a specific area, and then the image can be automatically collected by information collection software or program models (such as a web crawler or image acquisition program). It can also be other methods as long as the collection of the initial image information can be completed.

[0055] Step S12, determining pixel distribution information of a first object contained in the target object in a preset area through a preset object recognition model, and determining morphological distribution information of the first object in the preset area based on the pixel distribution information, wherein the first object includes pixel points whose pixel grayscale values ​​are less than or equal to a first threshold.

[0056] In this embodiment, when the teacher is marking homework or exercise books, the completion status of the homework or exercise books is usually marked by handwriting or annotation marks of different colors. Therefore, at this time, the completion status of the relevant homework or exercise books can be automatically identified by identifying the color of these handwriting or annotation marks and the composition of the handwriting or annotation marks.

[0057] Among them, the pixel distribution information corresponds to the distribution of pixels corresponding to the marking handwriting. For example, when the marking color is red, the grayscale value of the corresponding pixel is usually 128. At this time, the pixel distribution information is the distribution of these pixels with a grayscale value of 128; it can also be other colors, such as blue or yellow, as long as it can be distinguished from the color of the original text on the homework book or exercise book (such as the student's handwriting and the handwriting of the exercises) and the background color (such as the white or khaki color of the homework book and exercise book itself); it should be noted that in order to ensure the accuracy of recognition, it is often necessary to recognize the original text (that is, the black question and the student's blue or black font). At this time, the pixel grayscale value corresponding to the blue font is 158, and so on; the morphological distribution information includes the distribution of the marking font or "√", "×", etc. in the marking area of ​​each question or test paper, such as the number and position of scores, "√", "×", etc.

[0058] Step S13, constructing a morphological distribution matrix according to the morphological distribution information of different regions;

[0059] In this embodiment, the number, position, etc. of "√", "×", fonts, scores, etc. of the marking of the entire test paper or assignment are counted, and the corresponding distribution matrix is ​​constructed based on this.

[0060] For example, the entire assignment or paper is divided into nine areas, and then the number of "√" is filled in as an element in the corresponding matrix to obtain the distribution matrix P: , and then the distribution matrix is ​​processed subsequently. Of course, it can also be to count the number or position relationship of "×", fonts, scores, etc., which is not limited here.

[0061] Step S14, performing matrix value calculation on the morphological distribution matrix, and determining the teaching level of the target object according to the matrix value calculation result.

[0062] In this embodiment, after the matrix values ​​are calculated, the corresponding knowledge mastery status is determined according to the matrix values, thereby determining the corresponding teaching status, and then adjusting the teaching plan accordingly.

[0063] Among them, the teaching level includes the level of mastery of the lesson content, which can be pre-set according to the score or proportion content corresponding to the review semantics.

[0064] For example, when the proportion of "√" or "×" in a certain area is high, it means that the teaching situation of this part of the content is better or worse, and the teaching speed needs to be accelerated or slowed down in a targeted manner; or the overall matrix value is within a certain interval, and the corresponding meaning of this interval is that the overall teaching is better or worse, then it should be accelerated or slowed down, and so on.

[0065] Through the above steps, the annotation handwriting and the color of the annotation handwriting can be accurately identified, so as to accurately judge the teaching situation and adjust the teaching plan according to the teaching situation in time, thereby improving the teaching accuracy and teaching efficiency.

[0066] After performing matrix value calculation on the morphological distribution matrix and determining the teaching level of the target object according to the matrix value calculation result, the method further includes:

[0067] S141. Obtain the discrimination coefficient, difficulty coefficient, and guessing coefficient of each question.

[0068] Exemplarily, based on the examination content of the question, each question is provided with a corresponding discrimination coefficient, difficulty coefficient and guessing coefficient. Among them, the discrimination is used to characterize the ability of the question to distinguish students of different levels. A question with high discrimination can better distinguish high-level and low-level students. The discrimination is used to characterize the ability level required for students to answer the question correctly. The lower the difficulty of the question, the higher the probability of the student answering correctly. The guessing coefficient is used to characterize the probability that the student can get the correct answer by guessing even if he does not understand the question at all. It can be understood that the guessing coefficient is usually used for multiple-choice questions. For non-multiple-choice questions, this parameter is usually set to 0.

[0069] This information can be obtained based on the teacher's pre-evaluation of the questions. For example, a mapping relationship table between questions and corresponding coefficients is pre-established. When the image of the test paper is obtained, the corresponding question number / position / order is obtained by pixel recognition, and the corresponding coefficients are obtained according to the questions.

[0070] S142, based on the discrimination coefficient, difficulty coefficient, guessing coefficient and distribution matrix P, calculate the student's mastery mark of the test paper:

[0071]

[0072] In the formula, Are a student Mastery score; Are a student In the performance on the question (1 for correct and 0 for incorrect); It is The discrimination coefficient of the question; It is The degree of difficulty of the question; is the average of the discrimination coefficients of all questions; is the average difficulty coefficient of all questions; n is the total number of questions.

[0073] In this application, the higher the discrimination coefficient of the proposed question, the greater the contribution of correctly answering the question to the student's ability level. The lower the difficulty coefficient of the question, the smaller the contribution of correctly answering the question to the student's ability level. The contribution of each question (determined by its discrimination coefficient and difficulty coefficient) is weighted and summed to obtain the student's overall mastery score. The discrimination coefficient and difficulty coefficient of the question are used here to adjust the contribution weight of each question.

[0074] In an optional embodiment, determining the morphological distribution information of the first object in the preset area according to the pixel distribution information includes:

[0075] Step S121, determining the distribution distance of the first object in the preset area according to the pixel distribution information;

[0076] Step S122, determining a first threshold relationship between a maximum value and a minimum value of the distance between the first objects according to the distribution distance;

[0077] Step S123: if the first threshold relationship meets the first condition, determine that the first form relationship between the first objects in the preset area is the first form, otherwise it is the second form.

[0078] In this embodiment, in the actual recognition process, the distance relationship between different pixel points can constitute a corresponding morphological relationship, so that the corresponding morphology can be determined based on these morphological relationships.

[0079] For example, the AI ​​distribution of pixels with a grayscale value of 128 is as follows Figure 2 As shown, the pixel point distances between AE, EI, and AI and the pairwise distances between other pixel points are then calculated, and the maximum and minimum values ​​are obtained. When the maximum value is within the preset range, the form is judged to be "√", and so on. It is easy to understand that the first condition can be that the threshold relationship is within the preset range, or meets the preset threshold range; the first form can be "√", or "×", or other preset forms.

[0080] In an optional embodiment, when the threshold relationship meets the first condition, after determining that the morphological relationship between the first objects in the preset area is the first form, otherwise it is the second form, the method further includes:

[0081] Step S124, constructing a morphological distribution matrix of the first object in the preset area according to the pixel distribution information;

[0082] Step S125, calculating the distribution value of the morphological distribution matrix, and determining the second morphological relationship of the first object in the preset area according to a second threshold relationship;

[0083] Step S126, matching the first morphological relationship with the second morphological relationship, and determining that an abnormality exists in the first morphological relationship and / or the second morphological relationship when the first morphological relationship does not match the second morphological relationship.

[0084] In this embodiment, in order to further ensure the accuracy of morphological recognition, morphological recognition may be performed according to the overall pixel distribution, and the two recognition results may be matched to determine whether the relevant morphological recognition is accurate.

[0085] Among them, for the filled pixel distribution information, in addition to the grayscale value, other features such as edge information, texture information, etc. can also be encoded to enrich the matrix content; when performing matrix calculations, the constructed morphological distribution matrix can be numerically calculated to calculate the distribution characteristic values ​​of each pixel or pixel cluster. These characteristic values ​​can include average grayscale value, standard deviation, maximum value, minimum value, etc., and of course, correlation values; and morphological recognition can be achieved using pattern recognition, machine learning algorithms, or rule engines; and when performing morphological matching, if the first morphological relationship indicates a clear "√" shape, and the second morphological relationship also supports this conclusion, then the two are considered to match, and so on.

[0086] In an optional embodiment, before determining the pixel distribution information of the first object included in the target object in a preset area by using a preset object recognition model, the method further includes:

[0087] Step S1201, obtaining preprocessed training data, wherein the training data includes a position label and a category label;

[0088] Step S1202, dividing the training data into a training set and a test set, wherein the ratio of the data volume of the training set to the data volume of the test set is 3:1;

[0089] Step S1203, extracting object features from the training set to obtain training object features;

[0090] Step S1204, training a preset initial recognition model using the training object features, and adjusting the training results using the test set;

[0091] Step S1205, repeatedly performing a preset number of training and adjustments on the initial recognition model to obtain the object recognition model.

[0092] In this embodiment, the collected images are preprocessed, including denoising, contrast enhancement, resizing, cropping and other operations to improve the efficiency and accuracy of subsequent processing; label information is used to improve the accuracy of model training; when allocating data sets, an automated script or tool can be used to randomly divide the preprocessed data; object features include key visual features, such as edges, corners, textures, etc., and the most representative features are selected from the extracted features in order to reduce dimensions and improve the generalization ability of the model; initial recognition models include support vector machines, random forests, convolutional neural network models, etc., and during training, the trained model can be evaluated using a test set to calculate the performance indicators of the model, such as accuracy, recall and F1 score, and based on the evaluation results, the model's hyperparameters, such as learning rate, regularization strength, tree depth, etc., are adjusted to improve the performance of the model.

[0093] From the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0094] In the present embodiment, a teaching auxiliary device based on image recognition is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0095] Figure 3 is a structural block diagram of a teaching auxiliary device based on image recognition according to an embodiment of the present invention, such as Figure 3 As shown, the device comprises:

[0096] An image acquisition module 31 is used to obtain initial image information of a target object;

[0097] a distribution information determination module 32, configured to determine pixel distribution information of a first object included in the target object in a preset area through a preset object recognition model, and determine morphological distribution information of the first object in the preset area according to the pixel distribution information, wherein the first object includes pixel points whose pixel grayscale values ​​are less than or equal to a first threshold value;

[0098] A first matrix construction module 33, used to construct a morphological distribution matrix according to the morphological distribution information of different regions;

[0099] The teaching level determination module 34 is used to perform matrix value calculation on the morphological distribution matrix, and determine the teaching level of the target object according to the matrix value calculation result.

[0100] In an optional embodiment, the distribution information determination module 32 includes:

[0101] a distribution distance determining unit, configured to determine a distribution distance of the first object in the preset area according to the pixel distribution information;

[0102] a threshold relationship determining unit, configured to determine a first threshold relationship between a maximum value and a minimum value of the distance between the first objects according to the distribution distance;

[0103] The first form determining unit is configured to determine that the first form relationship between the first objects in the preset area is a first form if the first threshold relationship meets a first condition, and is a second form otherwise.

[0104] In an optional embodiment, it also includes:

[0105] A second distribution matrix construction module is used to construct a morphological distribution matrix of the first objects in the preset area according to the pixel distribution information after determining that the morphological relationship between the first objects in the preset area is a first morphology if the threshold relationship meets the first condition, and is a second morphology otherwise;

[0106] A second morphology determination module, configured to calculate a distribution value of the morphology distribution matrix and determine a second morphology relationship of the first object in the preset area according to a second threshold relationship;

[0107] A matching module is used to match the first morphological relationship with the second morphological relationship, and when the first morphological relationship does not match the second morphological relationship, determine that there is an abnormality in the first morphological relationship and / or the second morphological relationship.

[0108] In an optional embodiment, it also includes:

[0109] Also includes:

[0110] A data acquisition module, used to acquire pre-processed training data before determining pixel distribution information of a first object included in the target object in a preset area through a preset object recognition model, wherein the training data includes a position label and a category label;

[0111] A data set division module, used to divide the training data into a training set and a test set, wherein the ratio of the data volume of the training set to the data volume of the test set is 3:1;

[0112] A feature extraction module, used to extract object features from the training set to obtain training object features;

[0113] The training module is used to train the preset initial recognition model through the training object characteristics and adjust the training results through the test set; repeatedly perform a preset number of training and adjustments on the initial recognition model to obtain the object recognition model.

[0114] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0115] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0116] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0117] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0118] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0119] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0120] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A teaching auxiliary method based on image recognition, characterized in that: include: Obtaining initial image information of the target object; Determine pixel distribution information of a first object included in the target object in a preset area through a preset object recognition model, and determine morphological distribution information of the first object in the preset area according to the pixel distribution information, wherein the first object includes pixel points whose pixel gray value is less than or equal to a first threshold value; Constructing a morphological distribution matrix according to the morphological distribution information of different regions; Performing matrix value calculation on the morphological distribution matrix, and determining the teaching level of the target object according to the matrix value calculation result; Wherein, determining the morphological distribution information of the first object in the preset area according to the pixel distribution information includes: Determining a distribution distance of the first object in the preset area according to the pixel distribution information; Determine a first threshold relationship between a maximum value and a minimum value of the distance between the first objects according to the distribution distance; When the first threshold relationship meets the first condition, the first form relationship between the first objects in the preset area is determined to be the first form, otherwise it is the second form.

2. The method according to claim 1, characterized in that When the first threshold relationship meets the first condition, after determining that the morphological relationship between the first objects in the preset area is a first form, otherwise it is a second form, the method further includes: Constructing a morphological distribution matrix of the first object in the preset area according to the pixel distribution information; Calculating distribution values ​​of the morphological distribution matrix, and determining a second morphological relationship of the first object in the preset area according to a second threshold relationship; The first morphological relationship is matched with the second morphological relationship, and when the first morphological relationship does not match the second morphological relationship, it is determined that an abnormality exists in the first morphological relationship and / or the second morphological relationship.

3. The method according to claim 1, characterized in that Before determining the pixel distribution information of the first object included in the target object in the preset area by using the preset object recognition model, the method further includes: Acquire preprocessed training data, wherein the training data includes a position label and a category label; Dividing the training data into a training set and a test set, wherein the ratio of the data volume of the training set to the data volume of the test set is 3:1; Performing object feature extraction on the training set to obtain training object features; Training a preset initial recognition model through the training object features, and adjusting the training results through the test set; The initial recognition model is repeatedly trained and adjusted a preset number of times to obtain the object recognition model.

4. A teaching auxiliary device based on image recognition, characterized in that: include: An image acquisition module, used to obtain initial image information of the target object; a distribution information determination module, configured to determine pixel distribution information of a first object contained in the target object in a preset area through a preset object recognition model, and determine morphological distribution information of the first object in the preset area according to the pixel distribution information, wherein the first object includes pixel points whose pixel grayscale values ​​are less than or equal to a first threshold value; A first matrix construction module, used to construct a morphological distribution matrix according to the morphological distribution information of different regions; A teaching grade determination module, used for performing matrix value calculation on the morphological distribution matrix, and determining the teaching grade of the target object according to the matrix value calculation result; Wherein, the distribution information determination module includes: a distribution distance determining unit, configured to determine a distribution distance of the first object in the preset area according to the pixel distribution information; a threshold relationship determining unit, configured to determine a first threshold relationship between a maximum value and a minimum value of the distance between the first objects according to the distribution distance; The first form determining unit is configured to determine that the first form relationship between the first objects in the preset area is a first form if the first threshold relationship meets a first condition, and is a second form otherwise.

5. The device according to claim 4, characterized in that Also includes: A second distribution matrix construction module is used to determine that the morphological relationship between the first objects in the preset area is a first form if the first threshold relationship meets a first condition, and otherwise it is a second form, and then construct a morphological distribution matrix of the first objects in the preset area according to the pixel distribution information; A second morphology determination module, configured to calculate a distribution value of the morphology distribution matrix and determine a second morphology relationship of the first object in the preset area according to a second threshold relationship; A matching module is used to match the first morphological relationship with the second morphological relationship, and when the first morphological relationship does not match the second morphological relationship, determine that there is an abnormality in the first morphological relationship and / or the second morphological relationship.

6. The device according to claim 4, characterized in that Also includes: A data acquisition module, used to acquire pre-processed training data before determining pixel distribution information of a first object included in the target object in a preset area through a preset object recognition model, wherein the training data includes a position label and a category label; A data set partitioning module, used to partition the training data into a training set and a test set, wherein the ratio of the data volume of the training set to the data volume of the test set is 3:1; A feature extraction module, used to extract object features from the training set to obtain training object features; The training module is used to train the preset initial recognition model through the training object characteristics and adjust the training results through the test set; repeatedly perform a preset number of training and adjustments on the initial recognition model to obtain the object recognition model.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 3 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 3.

Citation Information

Patent Citations

  • Teaching assistance method and device based on image recognition

    CN116844163A