A standardized testing method and system for rulers used in teaching and examinations

Abstract

This invention relates to the field of computer vision technology, specifically to a method and system for detecting the standardization of ruler usage in teaching and examinations. The method includes: acquiring an image containing a ruler and the name of the object to be tested; inputting the image into a ruler detection module to obtain the bounding box of the ruler and the centimeter / pixel scale factor; obtaining the image mask and contour vertex set of the object to be tested based on the image and the name of the object; obtaining the set of left-hand keypoint pixel coordinates and the set of right-hand keypoint pixel coordinates; calculating the actual measurement result of the object to be tested, the projection distance to the zero mark line, and the rotation angle difference; determining the hand interaction result and generating an automated detection result; combining the operation experiment record and the automated detection result, generating an automated detection result of ruler usage standardization according to evaluation rules. This invention achieves automated and intelligent detection of ruler usage standardization in teaching and examination scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method and system for detecting the standardization of ruler usage in teaching and examinations. Background Technology

[0002] The use of a ruler is one of the fundamental measurement methods that students must master, and its proper use directly affects the accuracy of the measurement results. In experiments in subjects such as physics, chemistry, and biology, the ruler serves as a basic measurement tool, and its correct use is an important reflection of students' scientific literacy. The demand for automated assessment of practical examinations in the field of educational evaluation is growing, but the detection of the proper use of rulers still faces significant technical challenges. First, even advanced visual large model systems, due to the lack of an effective scale calibration mechanism, struggle to directly analyze the real-world dimensions of objects, thus making accurate detection of the proper use of rulers difficult. Second, existing technologies primarily focus on the numerical extraction of ruler measurement results, neglecting the systematic monitoring of key standardized elements in the operational process. There is an urgent need to develop an automated method for detecting the proper use of rulers in practical examination scenarios, promoting the deep integration of deep learning and educational evaluation theory, and providing an accurate, fair, and efficient automated assessment solution for practical examinations. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides a method and system for detecting the standardization of ruler usage in teaching and examinations. The aim is to achieve automated and intelligent detection of the standardization of ruler usage in teaching and examination scenarios by constructing a technical framework that includes ruler detection, object positioning, hand interaction analysis, and multi-dimensional evaluation rules.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] On one hand, embodiments of the present invention provide a method for checking the standardization of ruler usage in teaching examinations, the method comprising the following steps:

[0006] S100: Obtain an image containing a ruler and the name of the object to be measured; input the image into the ruler detection module to obtain the bounding box where the ruler is located and the centimeter / pixel scale factor.

[0007] S200, Input the image and the name of the object to be tested into the object detection module to obtain the image mask and contour vertex set of the object to be tested;

[0008] S300, the image is input into the hand key point detection module to obtain the left hand key point pixel coordinate set and the right hand key point pixel coordinate set;

[0009] S400, based on the set of contour vertices, centimeter / pixel scaling factor and ruler measurement edge reference line, calculate the actual measurement result of the object to be measured, the projection distance with the zero scale line and the difference in rotation angle;

[0010] S500: Based on the set of left-hand key point pixel coordinates, the set of right-hand key point pixel coordinates and the ruler bounding box, determine the hand interaction result and generate an automated detection result;

[0011] S600, combining the operation experiment record with the automated detection results, generates a standardized automated detection result for ruler use according to the evaluation rules.

[0012] Optionally, in S100, the step of acquiring an image containing the ruler and the name of the object to be measured, and inputting the image into the ruler detection module to obtain the bounding box of the ruler and the centimeter / pixel scale factor, includes:

[0013] S110, perform edge detection on the image to obtain a centimeter edge set, and calculate the diagonal pixel length and pixel area of ​​the bounding box where the ruler is located;

[0014] S120, based on the diagonal pixel length and pixel area, filter the centimeter edge set to obtain the centimeter edge set;

[0015] S130, Calculate the centroid coordinates of each contour in the centimeter edge set, and construct the initial center coordinate set of the centimeter scale;

[0016] S140, by eliminating noise points with excessively high local density through the Euclidean distance matrix, the set of centimeter scale center coordinates is obtained;

[0017] S150, calculate the distance between the coordinate points in the set of centimeter scale center coordinates and the reference line of the scale measurement edge, and filter out outliers to obtain the set of effective centimeter scale center coordinates;

[0018] S160, sort the set of effective center coordinates of the centimeter scale, calculate the Euclidean distance between adjacent coordinate points to obtain the set of centimeter unit pixel ratios, and select the mode or median as the centimeter / pixel ratio factor.

[0019] Optionally, in S200, the step of inputting the image and the name of the object to be tested into the object detection module to obtain the image mask and contour vertex set of the object to be tested includes:

[0020] S210, Input the image and the name of the object to be tested into the open vocabulary instance segmentation model to obtain the image mask of the object to be tested;

[0021] S220, use the Canny operator to extract the edge contours of the image mask to obtain a set of contour vertices.

[0022] Optionally, in S400, the step of calculating the actual measurement result of the object to be measured, the projection distance to the zero scale line, and the rotation angle difference based on the contour vertex set, the centimeter / pixel scale factor, and the ruler measurement edge reference line includes:

[0023] S410, calculate the projection scalar value of each vertex in the contour vertex set on the ruler measurement edge reference line, and obtain the range of the projection value range as the pixel measurement result of the object to be measured.

[0024] S420, Divide the pixel measurement result by the centimeter / pixel scaling factor to obtain the actual measurement result;

[0025] S430, calculate the projection value of the zero mark line on the reference line of the ruler measurement edge, divide the difference between the minimum value of the projection value range and the projection value by the centimeter / pixel scaling factor to obtain the projection distance from the zero mark line;

[0026] S440, calculate the covariance matrix of the set of contour vertices, determine the main direction of the object based on the covariance matrix, and calculate the rotation angle difference with the ruler angle.

[0027] Optionally, in S440, the calculation of the covariance matrix of the contour vertex set, the determination of the object's principal direction based on the covariance matrix, and the calculation of the rotation angle difference with the ruler angle specifically involve:

[0028] S441, calculate the coordinates of the center point of the contour vertex set, construct a centered coordinate matrix with the center point as the origin, and calculate the covariance matrix based on the centered coordinate matrix;

[0029] S442, Calculate the angle between the object and the positive x-axis based on the elements of the covariance matrix, and take the angle as the main direction of the object;

[0030] S443, calculate the minimum rotation angle difference between the principal direction of the object and the scale angle, wherein the value of the rotation angle difference is in the range of 0 to π / 2.

[0031] Optionally, in S500, the step of determining the hand interaction result and generating an automated detection result based on the left-hand keypoint pixel coordinate set, the right-hand keypoint pixel coordinate set, and the ruler bounding box includes:

[0032] S510, determine whether the key hand point is within the ruler bounding box according to the predefined indicator function;

[0033] S520, input each key point in the set of left-hand key point pixel coordinates into the indicator function, and accumulate the results to obtain the left-hand interaction determination result;

[0034] S530, input each key point in the set of right-hand key point pixel coordinates into the indicator function, and accumulate the results to obtain the right-hand interaction determination result;

[0035] The S540 constructs automated detection results based on actual measurement results, projection distance, left-hand interaction judgment results, and right-hand interaction judgment results.

[0036] Optionally, in S600, the step of combining the operational experiment record with the automated detection result to generate a standardized automated detection result for the use of the ruler according to the evaluation rules includes:

[0037] S610, compare the differences between the automated detection results and the operation experiment records;

[0038] S620, Based on the aforementioned differences and evaluation rules, generate prompt words and input them into a large language model for analysis;

[0039] S630 generates a ruler based on the output of a large language model, using standardized automated detection results.

[0040] On the other hand, embodiments of the present invention provide a ruler usage compliance testing system for teaching examinations, comprising:

[0041] At least one processor;

[0042] At least one memory for storing at least one program;

[0043] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0044] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the above-described method.

[0045] The beneficial effects of this invention are as follows: This invention discloses a method and system for detecting the standardization of ruler use in teaching and examinations. The invention achieves accurate calculation of the centimeter / pixel scale factor through a ruler detection module, solving the problem of image scale calibration and laying the foundation for subsequent actual size measurement. Utilizing an open-vocabulary instance segmentation model and edge detection technology, the contour features of the object under test can be accurately extracted. Combined with covariance matrix analysis, the difference between the object's principal direction and rotation angle is quantified, effectively monitoring the standardization of the relative position between the ruler and the object. The hand key point detection module determines the interaction state with the ruler through an indicator function, comprehensively capturing the standardization of students' hand movements during operation. By introducing a large language model to analyze the differences between the automated detection results and experimental records, a standardization detection report conforming to educational evaluation standards is generated, realizing the intelligent transformation from objective data to subjective evaluation. This method not only fills the gap in existing technology for monitoring the standardization of operation processes but also improves the accuracy and robustness of detection through multi-module collaborative work, providing an efficient and fair solution for the automated assessment of ruler use in teaching and examinations. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a method for verifying the standardization of ruler usage in teaching and examinations according to an embodiment of the present invention.

[0048] Figure 2 This is an overall architecture diagram of the standardization testing method for the scale in this embodiment of the invention;

[0049] Figure 3 This is a flowchart illustrating the reasoning process of using a standardized testing method for a ruler in an embodiment of the present invention.

[0050] Figure 4 This is a diagram illustrating the effect of using a standardized testing method for a ruler in an embodiment of the present invention. Detailed Implementation

[0051] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of the present invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of the present invention as detailed in the appended claims.

[0053] It is understood that the terms “first,” “second,” etc., used in this invention may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to determination” as used herein may be interpreted as “when…” or “when…” or “in response to determination.”

[0054] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0056] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0057] The operational examination process in related technologies has the following problems:

[0058] The problem of accurate scale positioning on rulers during practical examinations: In practical teaching or examination environments, the extraction of scale features faces severe challenges due to a combination of factors, including interference from multiple light sources, operator limb obstruction, physical wear and tear on measuring equipment, and resolution limitations of monitoring devices. Specifically, this manifests as: broken scale lines in low-contrast areas, localized overexposure caused by high light reflection, and information loss due to wear and tear on the scale characters. Commonly used scale detection methods, such as optical character recognition technology, experience a significant decrease in accuracy under these non-ideal conditions, severely restricting the reliability assessment of subsequent measurement results.

[0059] The problem of reliability detection of measurement results for test objects during practical examinations: In practical examination scenarios, the test objects are diverse, including regular geometric shapes and irregular objects, with significant differences in shape complexity and size range. Existing detection methods generally employ customized models for specific object categories, lacking the ability to generalize and adapt to unknown object categories. There is an urgent need to construct an adaptive detection framework with strong generalization capabilities to achieve consistent detection of measurement results for any category of test objects.

[0060] The issue of comprehensive evaluation of the standardized use of rulers in practical examinations: Current ruler testing technology has significant methodological limitations, primarily focusing on the numerical extraction of static measurement results while neglecting the systematic monitoring and quantitative evaluation of key standardized elements in the operational process. Specifically, there is currently no effective method to detect core standardized indicators such as zero-scale line alignment error, parallelism deviation between the ruler and the measured object, and adequacy of measurement contact, leading to the inability to accurately attribute various operational errors. More importantly, educational assessment scenarios require multi-dimensional and fine-grained standardized analysis of the operational process, but existing technologies lack a mechanism to integrate geometric measurement data with educational evaluation standards, making it difficult to generate diagnostic feedback with teaching guidance value and failing to meet the modern educational examination needs of "process-oriented evaluation" and "competency-oriented assessment" under the new curriculum standards.

[0061] To address the technical problems in related technologies, this invention proposes a method and system for detecting the standardization of ruler usage in teaching and examinations. By using techniques such as key point detection, image segmentation, contour extraction, statistical analysis, and open-vocabulary instance segmentation to detect information related to the object being measured by the ruler in an image, the method automatically detects the standardization of ruler usage, achieving the goal of intelligently evaluating the teaching and examination process related to ruler usage.

[0062] refer to Figure 1 ,like Figure 1 The figure shown is a method for checking the standardization of ruler use in teaching and examinations provided by an embodiment of the present invention. The method includes the following steps:

[0063] S100: Obtain an image containing a ruler and the name of the object to be measured; input the image into the ruler detection module to obtain the bounding box where the ruler is located and the centimeter / pixel scale factor.

[0064] S200, Input the image and the name of the object to be tested into the object detection module to obtain the image mask and contour vertex set of the object to be tested;

[0065] S300, the image is input into the hand key point detection module to obtain the left hand key point pixel coordinate set and the right hand key point pixel coordinate set;

[0066] S400, based on the set of contour vertices, centimeter / pixel scaling factor and ruler measurement edge reference line, calculate the actual measurement result of the object to be measured, the projection distance with the zero scale line and the difference in rotation angle;

[0067] S500: Based on the set of left-hand key point pixel coordinates, the set of right-hand key point pixel coordinates and the ruler bounding box, determine the hand interaction result and generate an automated detection result;

[0068] S600, combining the operation experiment record with the automated detection results, generates a standardized automated detection result for ruler use according to the evaluation rules.

[0069] In steps S100 to S600 of this embodiment, the ruler detection module accurately calculates the centimeter / pixel ratio factor, providing a scale benchmark for subsequent size measurements; the object detection module uses open vocabulary instance segmentation and edge detection technology to accurately extract the contour of the object to be measured; the hand key point detection module captures the interaction state between the operator's hand and the ruler; and through steps such as projection calculation and covariance matrix analysis, the measurement results, zero-scale line alignment error, and parallelism deviation are quantified, combined with hand interaction judgment to form multi-dimensional automated detection data. Finally, these objective data are compared with the operation experiment records, and a large language model is introduced to conduct in-depth analysis based on educational evaluation rules, thereby generating a standardized detection report that includes both specific operational error quantification values ​​and educational guidance suggestions, realizing fully automated processing from image acquisition, feature extraction, data calculation to intelligent evaluation.

[0070] In some embodiments, S100, obtaining an image containing a ruler and the name of the object to be measured, and inputting the image into a ruler detection module to obtain the bounding box of the ruler and the centimeter / pixel scale factor, includes:

[0071] S110, perform edge detection on the image to obtain a centimeter edge set, and calculate the diagonal pixel length and pixel area of ​​the bounding box where the ruler is located;

[0072] S120, based on the diagonal pixel length and pixel area, filter the centimeter edge set to obtain the centimeter edge set;

[0073] S130, Calculate the centroid coordinates of each contour in the centimeter edge set, and construct the initial center coordinate set of the centimeter scale;

[0074] S140, by eliminating noise points with excessively high local density through the Euclidean distance matrix, the set of centimeter scale center coordinates is obtained;

[0075] S150, calculate the distance between the coordinate points in the set of centimeter scale center coordinates and the reference line of the scale measurement edge, and filter out outliers to obtain the set of effective centimeter scale center coordinates;

[0076] S160, sort the set of effective center coordinates of the centimeter scale, calculate the Euclidean distance between adjacent coordinate points to obtain the set of centimeter unit pixel ratios, and select the mode or median as the centimeter / pixel ratio factor.

[0077] In this embodiment of the invention, edge detection is performed on the image to extract potential centimeter scale edge features, constructing a centimeter edge set. Simultaneously, the diagonal pixel length and pixel area of ​​the bounding box containing the ruler are calculated, providing spatial constraints for subsequent edge selection. Next, based on the diagonal length and area parameters of the bounding box, the initially extracted centimeter edge set is filtered, retaining edge contours that conform to the physical size characteristics of the ruler, thus obtaining a more accurate centimeter edge set. Subsequently, by calculating the centroid coordinates of each contour in this set, an initial centimeter scale center coordinate set is constructed, initially locating the scale center position. Considering the potential for excessively high local density interference points due to image noise, the Euclidean distance matrix analysis method is used to clean the initial center coordinate set, eliminating the influence of noise points, resulting in an optimized centimeter scale center coordinate set. To further ensure the validity of the coordinate points, the vertical distance between these coordinate points and the preset ruler measurement edge reference line is calculated, filtering out outliers with excessively large distances, ultimately determining the effective centimeter scale center coordinate set. Finally, the effective center coordinate set is sorted according to the actual scale order, and the Euclidean distance between adjacent coordinate points is calculated. This yields the pixel ratio set corresponding to centimeter units. By selecting the mode or median of this set as the final centimeter / pixel ratio factor, the impact of a single outlier on the ratio calculation is effectively reduced, ensuring high accuracy of scale calibration and providing a reliable pixel-to-physical unit conversion basis for subsequent actual size measurement of the object under test.

[0078] In some embodiments, S200, the step of inputting the image and the name of the object to be tested into the object detection module to obtain the image mask and contour vertex set of the object to be tested includes:

[0079] S210, Input the image and the name of the object to be tested into the open vocabulary instance segmentation model to obtain the image mask of the object to be tested;

[0080] S220, use the Canny operator to extract the edge contours of the image mask to obtain a set of contour vertices.

[0081] In this embodiment of the invention, by inputting the image and the name of the object to be tested as input parameters into an open-vocabulary instance segmentation model, this model can overcome the dependence of traditional instance segmentation models on predefined category labels. Based on natural language descriptions, it directly locates and segments the target region in the image that matches the name of the object to be tested, thereby generating an accurate binary image mask. The mask region accurately covers the pixel range of the object to be tested. After obtaining the image mask, the Canny operator is used to perform edge detection processing on the mask image. By setting reasonable high and low threshold gradients, noise interference is effectively suppressed while the edge information of the object's contour is completely preserved. Polygon approximation and vertex extraction are performed on the detected edge pixels, ultimately obtaining a contour vertex set composed of a series of ordered pixel coordinates. This set accurately represents the outer contour geometry of the object to be tested, providing basic data support for subsequent calculation of the object's principal direction and actual size.

[0082] In some embodiments, S400, the step of calculating the actual measurement result of the object to be measured, the projection distance to the zero scale line, and the rotation angle difference based on the contour vertex set, the centimeter / pixel scale factor, and the ruler measurement edge reference line includes:

[0083] S410, calculate the projection scalar value of each vertex in the contour vertex set on the ruler measurement edge reference line, and obtain the range of the projection value range as the pixel measurement result of the object to be measured.

[0084] S420, Divide the pixel measurement result by the centimeter / pixel scaling factor to obtain the actual measurement result;

[0085] S430, calculate the projection value of the zero mark line on the reference line of the ruler measurement edge, divide the difference between the minimum value of the projection value range and the projection value by the centimeter / pixel scaling factor to obtain the projection distance from the zero mark line;

[0086] S440, calculate the covariance matrix of the set of contour vertices, determine the main direction of the object based on the covariance matrix, and calculate the rotation angle difference with the ruler angle.

[0087] In this embodiment of the invention, the projection scalar value of each vertex on the reference line is obtained by projecting each vertex in the contour vertex set along a direction perpendicular to the measurement edge reference line of the ruler. Statistical analysis is performed on these projection scalar values, and their maximum and minimum values ​​are extracted. The difference between the two values ​​represents the range of projection values ​​characterizing the pixel size of the object under test in the image; this is the pixel measurement result. Subsequently, using the centimeter / pixel scaling factor obtained in step S100, the pixel measurement result is converted into the actual physical size in centimeters, thus obtaining the actual measurement result of the object under test. To evaluate the zero-scale line alignment specification, the projection scalar value of the ruler's zero-scale line on its measurement edge reference line is first determined. Next, the difference between the minimum value of the projection value range of the object under test (i.e., the projection of the object's starting measurement end) and the projection value of the zero-scale line is calculated. This difference is then divided by the centimeter / pixel scaling factor to obtain the actual projection distance of the object's starting end relative to the zero-scale line. This distance quantifies the zero-scale line alignment error. For parallelism evaluation, the geometric distribution characteristics of the object are analyzed by calculating the covariance matrix of the contour vertex set. The eigenvectors of the covariance matrix reflect the principal extension direction of the object's contour, i.e., the object's principal direction. By comparing this principal direction with the direction of the reference line on the ruler's measuring edge, and using the formula for calculating the vector angle, the rotation angle difference between the two is obtained. This angle difference directly reflects the parallelism deviation between the ruler and the measured object.

[0088] In some embodiments, S440, calculating the covariance matrix of the contour vertex set, determining the principal direction of the object based on the covariance matrix, and calculating the rotation angle difference with the ruler angle specifically involves:

[0089] S441, calculate the coordinates of the center point of the contour vertex set, construct a centered coordinate matrix with the center point as the origin, and calculate the covariance matrix based on the centered coordinate matrix;

[0090] S442, Calculate the angle between the object and the positive x-axis based on the elements of the covariance matrix, and take the angle as the main direction of the object;

[0091] S443, calculate the minimum rotation angle difference between the principal direction of the object and the scale angle, wherein the value of the rotation angle difference is in the range of 0 to π / 2.

[0092] In this embodiment of the invention, the center point coordinates of the contour are obtained by first calculating the arithmetic mean of the coordinates of all vertices in the contour vertex set. Using this center point as the origin, the original coordinates of each vertex are subtracted from the center point coordinates to construct a centered coordinate matrix, thus eliminating the influence of positional offset on direction analysis. Based on this centered coordinate matrix, a covariance matrix is ​​constructed by calculating the covariance values ​​of each coordinate component. The eigenvalues ​​and eigenvectors of this matrix accurately reflect the dispersion of the contour vertices on the two-dimensional plane and the trend of the main direction. Next, based on the elements of the covariance matrix, the angle between the main direction of the object and the positive x-axis is calculated using the arctangent function. This angle represents the overall extension direction of the object in the image. Subsequently, the angle between the reference line of the measuring edge of the ruler and the positive x-axis is obtained as the ruler angle. The difference between the angle of the main direction of the object and the ruler angle is calculated, and the smallest angle value of this difference in the range of [0, π / 2] is taken as the rotation angle difference. The smaller the angle difference, the higher the parallelism between the ruler and the object being measured. Conversely, there is a significant parallelism deviation, which provides key geometric parameter basis for evaluating the standardization of operation.

[0093] In some embodiments, S500, determining the hand interaction result and generating an automated detection result based on the left-hand key point pixel coordinate set, the right-hand key point pixel coordinate set, and the ruler bounding box includes:

[0094] S510, determine whether the key hand point is within the ruler bounding box according to the predefined indicator function;

[0095] S520, input each key point in the set of left-hand key point pixel coordinates into the indicator function, and accumulate the results to obtain the left-hand interaction determination result;

[0096] S530, input each key point in the set of right-hand key point pixel coordinates into the indicator function, and accumulate the results to obtain the right-hand interaction determination result;

[0097] The S540 constructs automated detection results based on actual measurement results, projection distance, left-hand interaction judgment results, and right-hand interaction judgment results.

[0098] In this embodiment of the invention, a predefined indicator function is used to quantify the spatial interaction between the hand and the ruler. The function returns 1 when the pixel coordinates of the hand's key points are within the pixel coordinate range of the ruler's bounding box, and 0 otherwise. For the set of left-hand key point pixel coordinates, each key point is sequentially input into the indicator function, and the returned results are summed to obtain the left-hand interaction determination result. This result reflects the number of left-hand key points falling within the ruler's bounding box. The same logic is used to process the set of right-hand key point pixel coordinates to obtain the right-hand interaction determination result. Subsequently, the actual measurement results calculated in step S400, the projection distance to the zero mark line, the rotation angle difference, and the left-hand and right-hand interaction determination results obtained in this step are integrated to construct an automated detection result dataset containing geometric parameters and interaction states. This dataset comprehensively records the core quantitative indicators during the use of the ruler, laying a data foundation for generating standardized detection reports in conjunction with evaluation rules.

[0099] In some embodiments, S600, the step of combining the operational experiment record with the automated detection result and generating a standardized automated detection result for ruler use according to evaluation rules includes:

[0100] S610, compare the differences between the automated detection results and the operation experiment records;

[0101] S620, Based on the aforementioned differences and evaluation rules, generate prompt words and input them into a large language model for analysis;

[0102] S630 generates a ruler based on the output of a large language model, using standardized automated detection results.

[0103] In this embodiment of the invention, quantitative data such as the actual measurement results, zero-scale projection distance, and rotation angle difference in the automated detection results are compared dimension-by-dimensionally with the measurement values ​​and subjective descriptions of operation steps manually recorded by the operator in the operation experiment record. Differences in numerical accuracy and operational standardization are identified. Based on these differences, and combined with preset educational evaluation rules (such as allowable measurement error thresholds, zero-scale alignment standards, parallelism requirements, and hand operation standards), structured prompts are constructed. These prompts contain the difference data, evaluation indicators, and educational context information. This prompt is then input into a large language model (such as the GPT series model), which utilizes its text understanding and reasoning capabilities to perform in-depth analysis of the difference data. This analysis determines whether the operator has specific problems during ruler use, such as misalignment of the zero-scale, non-parallelism between the ruler and the object, or hand obstruction of the scale. Natural language analysis results containing explanations of the error causes and suggestions for operational improvement are generated. Finally, the analysis results output by the large language model are integrated with the quantitative data in the automated detection results to form a complete automated detection result for the standardization of ruler use. This result not only includes the specific quantitative values ​​of various operational errors (such as a rotation angle difference of 2.5 degrees and a zero-gradient line projection distance of 0.3 cm), but also includes targeted educational guidance suggestions (such as "It is recommended to keep the measuring edge of the ruler parallel to the edge of the object during measurement to reduce measurement errors caused by tilting"). This achieves an organic combination of objective data and subjective evaluation, providing an intelligent and standardized solution for the standardization assessment of ruler use in teaching and examinations.

[0104] The following is a specific embodiment provided by the present invention:

[0105] Overall architecture:

[0106] I. Data Acquisition: The acquisition device is deployed above the experimental table to capture the operation and examination area and construct an RGB video stream.

[0107] II. Model Design:

[0108] This method designs a scale usage normative analysis model containing seven main modules, such as... Figure 2 The modules shown are: input module, ruler analysis module, object detection module, hand key point detection module, detection result parsing module, standardization evaluation module, and output module.

[0109] Input module: Input ruler usage standardization evaluation rules Obtain students' operation and experiment records from the operation and examination terminal. In practical examination scenarios, students typically use rulers with centimeter divisions to perform measurements and experiments on a horizontal experimental platform. A camera is mounted directly above the experimental area to record video. The input module receives the video stream from a top-down view and extracts an RGB format top-down image containing the area where the ruler measurement was performed from the practical examination video stream. For images Perform illumination equalization processing;

[0110] Ruler Analysis Module:

[0111] Create a ruler model: The ruler model includes two key positioning points, and defines the centimeter graduations, the ruler's rotation angle, and the centimeter / pixel ratio.

[0112] Zero mark: denoted as , representing the pixel coordinates of the zero mark of the ruler;

[0113] End mark: denoted as , represents the pixel coordinates of the most distant unknown scale line from the zero mark on the visible scale.

[0114] Centimeter graduations: Rulers with 1-millimeter graduations are commonly used in exams. Generally, a ruler includes both centimeter and millimeter graduations. Centimeter graduations are indicated by larger numbers and lines, while millimeter graduations are indicated by smaller, shorter lines.

[0115] Rotation angle Based on the zero and end graduation points, a reference line for the ruler's measurement edge can be constructed. This reference line represents the ruler's main measurement direction. The angle between the vector originating from the zero graduation point and ending at the end graduation point and the positive x-axis in the image coordinate system is also considered. , .

[0116] cm / pixel scaling factor Given that the actual physical distance between each centimeter mark on a ruler is equal, and under approximately a top-down shooting condition, the ruler rotates only within the XOY plane of the camera coordinate system relative to the camera. The perspective distortion of the ruler is small, so the pixel distance between each centimeter mark can be approximated as equal. Therefore, a centimeter / pixel scaling factor is set. , used to convert pixel length to real-world length.

[0117] (2) Use key point detection algorithm to analyze the image Perform detection to obtain the bounding box of the ruler appearing in the image. The coordinate position of the zero mark on the ruler The coordinate position of the end mark of the ruler , These are the x and y coordinates of the top-left and bottom-right pixels of the bounding box Rec, respectively. These are the x and y coordinates of the zero mark, respectively; These are the x and y coordinates of the end tick point, respectively. and Calculate the reference line for measuring the edge of the ruler. Its normalization equation is , are the variables representing the x-coordinate and y-coordinate in the image coordinate system, respectively, where:

[0118] ;

[0119] ;

[0120] ;

[0121] Calculate the angle between the reference line f and the positive x-axis. ,at this time ;

[0122] (3) Based on the bounding box Crop the image of the area where the ruler is located , , These represent the x-coordinate range and y-coordinate range of the bounding box Rec in image I, respectively. This represents an image cropping operation, specifically extracting the region containing only the ruler from the original image I. To reduce background interference and focus on the detailed features of the ruler, the RGB format area image is converted to a grayscale area image. A mean filter with a kernel size of 3 is used to filter grayscale images. Edge blurring is performed, and the Canny operator is used to extract the set of centimeter scale edge contours from the ruler. , where each contour Has length attribute and area attributes ;

[0123] (4) Set an adjustable length threshold range With area threshold Set of edge contours of centimeter scale Filter by edge contours in the image. These are the lower and upper limits of the length threshold, respectively. These are the lower and upper limits of the area threshold, respectively, preserving the outline of the approximate scale line and removing noise interference. Let... Let be the diagonal pixel length of the bounding box containing the ruler, then we have ,set up Let be the pixel area of ​​the bounding box containing the ruler, then we have The filtered centimeter edge set It can be represented as:

[0124] ; The AND operator "AND" is used to filter contours that simultaneously satisfy both length and area conditions.

[0125] (5) For each contour Calculate its Step Moment , All values ​​are non-negative integers; the centroid coordinates of each contour are calculated based on the first moment. :

[0126] ;

[0127] in, For outline The zeroth moment represents the total number of pixels in the region enclosed by the contour; For outline The first x-moment, For outline The first y-moment, Represents outline The enclosed region is used to obtain the initial set of center coordinates for the centimeter scale. Represented as: ;

[0128] (6) Let It is a set The size is determined by the formula. calculate Construct an Euclidean distance matrix between each coordinate point in the matrix. Its elements are , Let be the element in the i-th row and j-th column of the distance matrix D, representing the Euclidean distance between the i-th coordinate point and the j-th coordinate point in set P. , , Let i and j be the coordinates of the i-th and j-th points in set P, respectively. , Let x and y be the x and y coordinates of the i-th coordinate point, respectively. , Let x and y be the x and y coordinates of the j-th coordinate point, respectively. Let be the Euclidean distance between the i-th and j-th coordinate points. A preset distance threshold is given. >0, based on the Euclidean distance matrix, eliminates noise points with excessively high local density, avoids repeated detection of the same scale line, and obtains the set of centimeter scale center coordinates. ;

[0129] (7) Calculate the set of coordinates of the center of the centimeter scale. Each coordinate point is compared to the reference line of the ruler measurement edge. The distance between them, a preset distance threshold >0 filters out points with excessively large distances to eliminate outliers caused by perspective distortion, background interference, or false detections, thus obtaining the set of valid center coordinates for the centimeter scale. ;

[0130] (8) Let For set Calculate the size of each. The mid-coordinate point is at Axial direction and Total variance along the axis and By selecting the coordinate axes with larger variances and sorting the coordinate points according to an ascending rule, an ordered set of coordinates is obtained. ;

[0131] (9) Calculate the set The Euclidean distance between each coordinate point and the previous coordinate point yields a set of pixel ratios in centimeters. ;

[0132] (10) To avoid the impact of errors in detecting individual scale line coordinate points, select a set The mode or median, which offers better stability and noise resistance, is used as the centimeter / pixel scaling factor displayed on the current scale. That is, the number of pixels corresponding to 1 centimeter in the real world in the current image:

[0133] ;

[0134] Object detection module:

[0135] Image Name of the object to be tested Input an open-vocabulary instance segmentation model (such as DINO, X-SAM, etc.) to obtain the image mask of the object to be tested. ;

[0136] The Canny operator is used to extract the edge contour of the object under test and obtain the set of contour vertices. ,in It is a set Size;

[0137] Hand key point detection module: This module detects the key points of the hand in the image. Input a general hand keypoint detector (such as MMPose, OpenPose, etc.) to obtain the set of pixel coordinates of the left hand keypoints. and the set of pixel coordinates of the right-hand key points ,in and The number of key points for the left and right hands respectively;

[0138] Detection result analysis module:

[0139] For each contour point Calculate its position on the reference line respectively. Projected scalar value on and Euclidean distance Obtain the set of projection values ​​from the contour points to the line. Set of Euclidean distances ;

[0140] Edge contour of the object under test In a straight line The projection length is the range of the projection value range, and the pixel measurement results of the object under test are obtained. pixels, converting the pixel measurement results of the object under test into actual measurement results. cm;

[0141] Further analysis of the spatial position between the object to be measured and the ruler is needed to calculate the zero mark on the reference line. Projection values ​​on The position of the object to be measured relative to the zero mark is on the reference line. Projected distance in the direction cm;

[0142] Calculate the center point of the object to be measured ,in , Construct a centralized coordinate matrix And calculate the covariance matrix. :

[0143] ;

[0144] in , , Calculate the angle between the object and the positive x-axis. ,at this time This represents the principal direction of the object. Calculate the difference in rotation angle between this point and the angle on the ruler. ,at this time The angle difference is used to assess the parallelism deviation between the object and the ruler;

[0145] Definition point Is it in the bounding box? The pointer function inside is:

[0146] ;

[0147] The interaction between the left and right hands and the ruler is calculated separately. If at least one hand keypoint is within the ruler's bounding box, interaction is considered to have occurred. Left-hand interaction result. Interact with the right hand to determine the result They are respectively:

[0148] ;

[0149] ;

[0150] Obtain automated detection results ;

[0151] Standardization assessment module: combining operational experiment records The automated detection results obtained from the detection result analysis module According to the evaluation rules By comparing the differences between the automated testing results and the operational experimental records, a standardized automated testing result for the use of the ruler is established. Optionally, evaluation rules can be combined. Automated detection results Operational Experiment Records Use standardized automated analysis templates with rulers Generate prompt words, input them into a large language model (such as GLM, Qwen, etc.) for analysis, and generate a ruler to use for standardized automated detection results. .

[0152] Output module: Outputs the standardized automated detection results for the ruler. .

[0153] Model reasoning: The overall reasoning process of the model is as follows Figure 3 As shown:

[0154] S1. System Initialization: Start the ruler usage standardization detection system, load the pre-trained deep learning model parameters, ruler usage standardization evaluation rules, and system configuration parameters; initialize the image preprocessing pipeline, feature extraction engine, and standardization evaluation and analysis module.

[0155] S2. Video Stream Processing: Receives high-definition video stream input from a top-down view of the operational examination scene, performs standardized preprocessing on each frame of the image, including color space conversion and illumination equalization, to eliminate the interference of environmental factors on subsequent analysis.

[0156] S3. Key Information Extraction: The preprocessed image is input in parallel to three core analysis modules. The ruler analysis module outputs the ruler bounding box coordinates, zero mark position, measurement reference line equation and centimeter / pixel ratio. The object detection module outputs the precise mask and edge contour vertex set of the object to be measured. The hand key point detection module outputs the two-dimensional spatial coordinates and confidence level of the key points of the left and right hands.

[0157] S4. Based on the output of step S3, perform key information analysis: (1) Calculate the projected length of the object to be measured on the reference line of the ruler and convert it into the actual measurement value; (2) Evaluate the zero scale line alignment error and the parallelism deviation between the ruler and the object to be measured; (3) Analyze the spatial interaction between the operator's hand and the ruler and judge the sufficiency of measurement contact.

[0158] S5. Standardized Comprehensive Assessment: The structured analysis results generated in S4 are compared with the operational experiment records from multiple dimensions. Based on a pre-set knowledge base of educational evaluation rules, various standardized indicators are quantitatively evaluated, and a complete test result including standardization scores, error type identification, and improvement suggestions is output. Optionally, for complex scenarios, a large language model is invoked to assist in generating diagnostic feedback: the evaluation rules, automated test results, operational experiment records, and analysis templates are integrated into structured prompts, which are then input into the large language model to generate a detailed assessment report with teaching guidance value.

[0159] S6. Generate standardized test reports, including visual operation process annotations, key standard indicator scores, and detailed error analysis. The test results can be pushed to the examination management system and saved via API interface for guiding candidates to perform standardized operations, subsequent model optimization, and educational data analysis.

[0160] S7. Process End and Resource Release: After completing the analysis of the current operation frame, release temporary computing resources to prepare for processing the next frame of the image. An example of the effect diagram of a specific embodiment of the present invention is shown below. Figure 4 As shown.

[0161] Compared with related technologies, the present invention has the following advantages:

[0162] This application proposes an automated detection method for the standardized use of rulers in teaching examinations, which can efficiently and accurately perform intelligent scoring on teaching examinations involving ruler usage operations, such as "measuring length with a ruler".

[0163] This application uses key point detection, image segmentation and contour extraction techniques to extract the positional distribution of the centimeter scale lines of a ruler in a video image, and combines statistical analysis techniques to analyze the pose information of the ruler.

[0164] This application detects the measured object and hand movements by using an open vocabulary instance segmentation method and a hand key point detection method, respectively, and further calculates and analyzes the results of several key specification indicators such as zero scale line alignment error, parallelism between the scale and the object, and sufficient hand contact.

[0165] This application combines the analysis results of key normative indicators with educational evaluation rules, and optionally introduces a large language model for analysis, which can generate an analysis report with teaching guidance value.

[0166] This invention also provides a ruler usage standardization testing system for teaching and examinations, comprising:

[0167] At least one processor;

[0168] At least one memory for storing at least one program;

[0169] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0170] The content of the above method embodiments is applicable to this embodiment. The specific functions implemented in this embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. Therefore, they will not be repeated here.

[0171] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0172] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0173] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0174] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0175] This invention also provides a computer program product, including a computer program or computer instructions, which are stored in a memory. A processor of a computer device reads the computer program or computer instructions from the memory and executes the computer program or computer instructions, causing the computer device to perform the above-described method.

[0176] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0177] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0178] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A method for detecting the use of a ruler in a teaching examination, characterized in that, The method comprises the following steps: S100, acquiring an image containing a scale and a name of an object to be measured, and inputting the image into a scale detection module to obtain a boundary box of the scale and a centimeter / pixel scale factor; S200, inputting the image and the name of the object to be measured into an object detection module to obtain an image mask and a contour vertex set of the object to be measured; S300, inputting the image into a hand key point detection module to obtain a left hand key point pixel coordinate set and a right hand key point pixel coordinate set; S400, based on the contour vertex set, the centimeter / pixel scale factor and a scale measurement edge reference line, calculating an actual measurement result of the object to be measured, a projection distance from a zero scale line and a rotation angle difference; S500, determining a hand interaction result according to the left hand key point pixel coordinate set, the right hand key point pixel coordinate set and the scale boundary box, and generating an automatic detection result; S600, combining an operation experiment record and the automatic detection result, and generating a scale use specification automatic detection result according to an evaluation rule.

2. The method of claim 1, wherein, In S100, the image containing the scale and the name of the object to be measured is acquired, and the image is input into the scale detection module to obtain the boundary box of the scale and the centimeter / pixel scale factor, comprising: S110, performing edge detection on the image to obtain a centimeter edge set, and calculating a diagonal line pixel length and a pixel area of the boundary box of the scale; S120, screening the centimeter edge set based on the diagonal line pixel length and the pixel area to obtain the centimeter edge set; S130, calculating the barycentric coordinates of each contour in the centimeter edge set, and constructing an initial centimeter scale center coordinate set; S140, excluding noise points with too high local density through a Euclidean distance matrix to obtain a centimeter scale center coordinate set; S150, calculating the distance between coordinate points in the centimeter scale center coordinate set and a scale measurement edge reference line, filtering outliers to obtain a centimeter scale effective center coordinate set; S160, sorting the centimeter scale effective center coordinate set, calculating the Euclidean distance between adjacent coordinate points to obtain a centimeter unit pixel scale set, and selecting a mode or a median as the centimeter / pixel scale factor.

3. The method of claim 1, wherein, In S200, the image and the name of the object to be measured are input into the object detection module to obtain an image mask and a contour vertex set of the object to be measured, comprising: S210, inputting the image and the name of the object to be measured into an open vocabulary instance segmentation model to obtain an image mask of the object to be measured; S220, using a Canny operator to extract an edge contour of the image mask to obtain a contour vertex set.

4. The method of claim 1, wherein, In S400, based on the contour vertex set, the centimeter / pixel scale factor and the scale measurement edge reference line, the actual measurement result of the object to be measured, the projection distance from the zero scale line and the rotation angle difference are calculated, comprising: S410, calculating the projection scalar value of each vertex in the contour vertex set on the scale measurement edge reference line to obtain a range difference of the projection value range as a pixel measurement result of the object to be measured; S420, dividing the pixel measurement result by the centimeter / pixel scale factor to obtain an actual measurement result; S430, calculating a projection value of the zero mark line on the scale measurement edge reference line, dividing a minimum value in a range of the projection value by a centimeter / pixel scale factor to obtain a zero mark line projection distance; S440, calculating a covariance matrix of the contour vertex set, determining an object main direction based on the covariance matrix, and calculating a rotation angle difference with a scale angle.

5. The method of claim 4, wherein, In S440, the calculation of the covariance matrix of the contour vertex set, the determination of the object main direction based on the covariance matrix, and the calculation of the rotation angle difference with the scale angle are specifically: S441, calculating a center point coordinate of the contour vertex set, constructing a centralized coordinate matrix with the center point as the origin, and calculating a covariance matrix based on the centralized coordinate matrix; S442, calculating an angle between the object and the positive direction of the x-axis according to an element of the covariance matrix, and taking the angle as the main direction of the object; S443, calculating a minimum rotation angle difference between the main direction of the object and the scale angle, and the rotation angle difference is in a range of 0 to π / 2.

6. The method of claim 1, wherein, In S500, the determination of the hand interaction result according to the left hand key point pixel coordinate set, the right hand key point pixel coordinate set, and the scale boundary box, and the generation of the automatic detection result include: S510, determining whether the hand key point is in the scale boundary box according to a predefined indication function; S520, inputting each key point in the left hand key point pixel coordinate set into the indication function, and accumulating the results to obtain a left hand interaction determination result; S530, inputting each key point in the right hand key point pixel coordinate set into the indication function, and accumulating the results to obtain a right hand interaction determination result; S540, constructing an automatic detection result according to the actual measurement result, the projection distance, the left hand interaction determination result, and the right hand interaction determination result.

7. The method of claim 1, wherein, In S600, the combination of the operation experiment record and the automatic detection result, and the generation of the scale use specification automatic detection result according to the evaluation rule include: S610, comparing the difference between the automatic detection result and the operation experiment record; S620, generating a prompt word based on the difference and the evaluation rule, and inputting the large language model for analysis; S630, generating the scale use specification automatic detection result according to the large language model output result.

8. A system for detecting the specification of a scale, characterized by It includes: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images