Correction method of three-view drawing with ruler and compass

Abstract

The present invention discloses a method for correcting three-view drawings of ruler and compass construction, comprising an acquisition step, a preprocessing step, a segmentation step, a correction step, a template matching step, a registration step, a subtraction step, and an error display step. The method effectively utilizes machine vision technology to detect errors in three-view drawings of ruler and compass construction, significantly improving the accuracy and efficiency of detection.

Description

Technical Field

[0001] The present invention relates to a ruler Figure 3 View modification method. Background Art

[0002] The drawing and reading of three views are the premise and foundation for learning to draw and read engineering drawings. Three view drawing methods can be divided into two categories: one is computer software drawing; the other is ruler and compass drawing. At present, the research on computer software drawing error detection has been relatively complete. Ruler and compass drawing Figure 3 Teachers still need to manually grade the views, and their workload increases exponentially with the number of students and daily homework. Furthermore, as the workload increases, teachers are more likely to make mistakes during the grading process.

[0003] In reality, there is a method for automatically correcting drawing assignments based on image processing. First, the image of the ruler and compass drawing is processed, and then the standard answer is compared with the student's assignment (ruler and compass drawing). The errors are marked in the student's assignment, and the correction result is finally obtained. Figure 3 Line thickness and position vary across views, and filming can also cause subtle image distortion. Therefore, the standard answers and student work often differ in line position and thickness, making the aforementioned method of directly comparing the standard answers and student work ineffective. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects in the prior art and provide a Figure 3 View modification method.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A ruler Figure 3 The view modification method includes the following steps:

[0007] Acquisition step: collecting the image to be tested, the entire template image and a single template image of a question;

[0008] A preprocessing step is performed on the image to be tested, grayscale processing and binarization processing to obtain a first image; grayscale processing and binarization processing are performed on the entire template image to obtain a second image; grayscale processing and binarization processing are performed on the single template image to obtain a third image;

[0009] a segmentation step of segmenting the first image to obtain a first three-view area image; and segmenting the second image to obtain a second three-view area image;

[0010] a correction step of performing perspective transformation processing on the first three-view area image to remove distortion, and then performing size transformation processing to obtain a fourth image with the same size as the second three-view area image; in the correction step, the perspective transformation four-point coordinates are obtained based on the contour polygon approximation;

[0011] a template matching step, performing template matching on the third image and the fourth image, finding and separating the fifth image on the fourth image; the process of the template matching step is as follows: selecting a sliding window of the same size as the third image in the fourth image; comparing the grayscale value of each pixel in the sliding window with the grayscale value of the corresponding pixel in the third image, and calculating the similarity between the third image and the sliding window; sliding the sliding window from the upper left corner to the right, sliding it to the rightmost corner, and then sliding it downward one line, and then restarting from the leftmost corner, recording the similarity between the third image and the sliding window calculated after each movement; comparing the similarities at all positions, and selecting the sliding window with the greatest similarity as the fifth image;

[0012] A registration step of aligning the fifth image with the third image to obtain a sixth image; the registration step comprises the following steps: obtaining feature points through SIFT feature point detection; achieving efficient matching of feature points through FLANN matching; preliminarily screening feature points through minimum Hamming distance; optimizing feature point matching through RANSAC to further remove incorrectly matched feature points; then calculating a homography matrix between the two images, and finally performing a perspective transformation;

[0013] a subtraction step of performing a difference operation on the sixth image and the third image to obtain a difference image; wherein the subtraction step further includes the following steps: dilating the sixth image to obtain a first dilated image, and dilating the third image to obtain a second dilated image; extracting the single question template image using the HSV color model to obtain an answer portion image, subtracting the first dilated image from the answer portion image to obtain an underdrawing error, and subtracting the second dilated image from the sixth image to obtain an overdrawing error;

[0014] An error display step is to display the processed differential image on the sixth image.

[0015] In some embodiments, in the acquisition step, the single question template image is obtained by cutting out the entire template image.

[0016] In some embodiments, in the segmentation step, a first three-view area image and a second three-view area image are obtained based on a contour detection method; the contour detection method includes the following steps: performing contour detection on the image, extracting the outermost maximum contour of the image, enclosing a maximum rectangle through the maximum contour, intercepting the three-view area, and removing background information.

[0017] In some embodiments, in the error display step, the differential image is binarized and median filtered before being displayed on the sixth image.

[0018] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present invention.

[0019] The positive improvement effect of the present invention is that: the ruler of this embodiment is Figure 3 The view correction method can effectively use machine vision technology to detect the ruler and compass Figure 3 Errors in the view greatly improve the accuracy and efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The ruler of the preferred embodiment of the present invention is Figure 3 Flowchart of the view modification method.

[0021] Figure 2 This is the image to be tested in the preferred embodiment of the present invention.

[0022] Figure 3 This is the entire template image of the preferred embodiment of the present invention.

[0023] Figure 4 This is a single question template image in a preferred embodiment of the present invention.

[0024] Figure 5 This is the first image of the preferred embodiment of the present invention.

[0025] Figure 6 This is the first three-view area image of the preferred embodiment of the present invention.

[0026] Figure 7 This is the second three-view area image of the preferred embodiment of the present invention.

[0027] Figure 8 This is a four-point coordinate position diagram of the perspective transformation of the image to be measured in the preferred embodiment of the present invention.

[0028] Figure 9 This is the fourth image of the preferred embodiment of the present invention.

[0029] Figure 10 This is the fifth image of the preferred embodiment of the present invention.

[0030] Figure 11 Schematic diagram of the template matching step.

[0031] Figure 12 This is the result image when the subtraction step is performed directly without the registration step.

[0032] Figure 13 This is the sixth image of the preferred embodiment of the present invention.

[0033] Figure 14 for Figure 4 A diagram of the part that requires answering.

[0034] Figure 15 for Figure 4 Schematic diagram of the known part of the problem.

[0035] Figure 16 Schematic diagram of the image expansion process.

[0036] Figure 17 A schematic diagram of the review results. DETAILED DESCRIPTION

[0037] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] It should be noted that in the claims and description of this patent, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a" does not exclude the presence of other identical elements in the process, method, article or apparatus comprising the element.

[0039] like Figure 1 As shown, the ruler of this embodiment is Figure 3The view correction method includes an acquisition step, a preprocessing step, a segmentation step, a correction step, a template matching step, a registration step, a subtraction step and an error display step.

[0040] Acquisition steps: Acquire the image to be tested, the entire template image, and a single question template image.

[0041] Preprocessing steps: grayscale and binarize the image to be tested to obtain the first image; grayscale and binarize the entire template image to obtain the second image; grayscale and binarize a single template image to obtain the third image.

[0042] Segmentation step: segmenting the first image to obtain a first three-view area image; segmenting the second image to obtain a second three-view area image.

[0043] Correction step: performing perspective transformation processing on the first three-view area image to remove distortion, and then performing size transformation processing to obtain a fourth image with the same size as the second three-view area image.

[0044] Template matching step: performing template matching on the third image and the fourth image, and finding and separating the fifth image on the fourth image.

[0045] Registration step: aligning the fifth image with the third image to obtain the sixth image.

[0046] Subtraction step: performing a difference operation on the sixth image and the third image to obtain a difference image.

[0047] Error display step: displaying the processed differential image on the sixth image.

[0048] The ruler of this embodiment is Figure 3 The view correction method is a machine vision-based ruler-making method that takes into account deviations. Figure 3 The view correction method adopts the above steps to effectively use machine vision technology to detect the ruler and compass. Figure 3 Errors in the view greatly improve the accuracy and efficiency of detection.

[0049] In the acquisition step, the template images acquired include the entire template image and a single question template image. The single question template image is obtained by intercepting the entire template image. Figure 2 A demonstrative image to be tested is shown. Figure 3 A whole template image is shown for demonstration purposes. Figure 4 A single question template image is shown for demonstration purposes.

[0050] In the preprocessing step, the processing speed is improved by converting the color image into a grayscale image. The calculation formula is:

[0051] GRAY=0.3R+0.59G+0.11B

[0052] Where GRAY is the obtained grayscale image, R is the red component of the original image, G is the green component of the original image, and B is the blue component of the original image.

[0053] The binarization threshold is obtained by the Otsu method (OTSU), and global binarization is performed to further reduce the amount of calculation. Figure 2 The image to be tested is obtained after the preprocessing step as shown below Figure 5 The first image shown, the second image and the third image obtained after the preprocessing steps of the entire template image and the single question template image are not shown in the accompanying drawings.

[0054] The operation process of Otsu's method is as follows:

[0055] (1) Calculate the image histogram and obtain the frequency distribution of the grayscale value of each pixel in the image.

[0056] (2) Traverse all possible thresholds, and for each possible threshold, calculate the intra-class variance and inter-class variance of the image segmented into foreground and background.

[0057] (3) Find the threshold that maximizes the inter-class variance. This threshold is the optimal binarization threshold. The formula for calculating the inter-class variance is:

[0058]

[0059] in is the between-class variance, ω 1 is the proportion of foreground pixels, ω 2 is the background pixel ratio, μ 1 is the average gray value of the foreground, μ 2 is the average gray value of the background, and t is the current threshold.

[0060] In the segmentation step, the three-view region is obtained based on contour detection. Contour detection is performed on the image to extract the largest contour in the outermost layer of the image. The largest rectangle circumscribed by the largest contour is used to intercept the three-view region and remove background information (including information outside the three-view region, such as student name, student ID, and class). Figure 5 The first image shown is obtained after the segmentation step as follows Figure 6 The first three-view area image is shown. The second image is segmented to obtain Figure 7 The second three-view area image is shown.

[0061] In the correction step, the perspective transformation four-point coordinates are obtained based on the contour polygon approximation. The perspective transformation four-point coordinate positions of the image to be measured are as follows: Figure 8 shown. Figure 6The first three-view area image shown is distorted by perspective transformation and then resized to obtain the image as shown in FIG. Figure 7 The size of the second three-view area image shown is consistent with the fourth image to facilitate detection. The fourth image is as shown in FIG. Figure 9 shown.

[0062] Contour polygon approximation simplifies the contour shape by reducing the number of points while preserving the contour's shape features as much as possible. The Douglas-Peucker algorithm is the most commonly used polygon approximation algorithm. Its steps include:

[0063] (1) Select the starting and ending points and connect them.

[0064] (2) Calculate the distances from the remaining points to the line segment and find the point with the farthest distance.

[0065] (3) If the maximum distance is greater than the given threshold, the point is important, keep it, and divide the contour into two segments. Repeat the above steps for each segment until all points are within the given distance threshold.

[0066] (4) If all points of the current segment are within the threshold distance, replace this segment with a straight line segment between the starting point and the end point.

[0067] The perspective transformation can be represented by a 3x3 transformation matrix in the form of:

[0068]

[0069] Where (x,y) is the point in the original image, (X,Y) is the position after transformation, and Z is used for normalization.

[0070] Sizing actually means changing the length and width of an image to achieve image scaling. During the sizing process, the interpolation method is used to calculate the new pixel value. Bilinear interpolation finds four neighboring pixels near the target pixel position and calculates the new pixel value based on their distance weights.

[0071] In the template matching step, the fifth image (the image to be tested for a separate question) is found on the fourth image based on the third image, and the image is separated from the fourth image as follows: Figure 10 The fifth image shown is independently detected to prevent interference between the three views of different questions.

[0072] The template matching step is often used in the task of finding specific content in an image. Its principle is as follows Figure 11 shown.

[0073] exist Figure 11 In the figure, the 4×4 image on the right is the template image, and the 8×8 image on the left is the image to be matched. The template matching process is as follows:

[0074] (1) Select a sliding window with the same size as the template (third image) in the image to be matched (the fourth image), such as Figure 11 Shown in the shaded area.

[0075] (2) Compare the relationship between the grayscale value of each pixel in the sliding window and the corresponding pixel in the template (third image), and calculate the similarity between the template (third image) and the sliding window.

[0076] (3) Slide the sliding window from the upper left corner to the right, slide it down one line after sliding to the rightmost point, and then start sliding again from the leftmost point. Record the similarity between the template (third image) and the sliding window calculated after each movement.

[0077] (4) Compare the similarities of all positions and select the sliding window with the largest similarity as the fifth image.

[0078] The registration process involves feature point detection, feature point matching, transformation model estimation, image resampling, and transformation. SIFT feature point detection is used to obtain feature points, which are then efficiently matched using FLANN matching. Feature points are initially screened using the minimum Hamming distance, and RANSAC is used to optimize feature point matching and further remove incorrectly matched feature points. The homography matrix between the two images is then calculated, and finally, a perspective transformation is performed.

[0079] In the registration step, due to the possible differences in line thickness and position between the fifth image and the third image, direct subtraction may result in errors. The direct subtraction result is as follows: Figure 12 After the registration step, the fifth image is aligned with the third image to obtain Figure 13 The sixth image shown can align the images as much as possible and reduce the difference in image positions.

[0080] SIFT feature points still have good stability under interference such as lighting, noise, perspective, scaling and rotation. The SIFT feature point detection process is as follows:

[0081] (1) By performing multiple Gaussian blurs on the original image, images of different scales are created. Images of each scale are generated by adjusting the standard deviation of the Gaussian filter.

[0082] (2) By calculating the difference images between adjacent scales to form a scale space, the DOG method can be used to effectively find the local extreme points of the image, which are potential feature points.

[0083] (3) For the detected extreme points, accurate positioning is performed, and by comparing local features, points with strong edge response or low contrast are eliminated to ensure that stable and significant feature points are retained.

[0084] (4) Assign one or more directions to each feature point to ensure that the feature descriptor is invariant to rotation, and determine the main direction by calculating the gradient direction and amplitude around the feature point.

[0085] (5) In the area around the feature point, the feature descriptor is calculated based on the gradient information. Usually, the area around the feature point is divided into several small blocks, and the gradient direction histogram of each small block is calculated to finally generate a 128-dimensional feature descriptor.

[0086] FLANN matching can handle large-scale feature sets and significantly reduce matching time. The general steps of FLANN matching are as follows:

[0087] First, feature points and their descriptors are extracted from the image to be matched.

[0088] The extracted descriptors are used to build a FLANN index for fast retrieval, and the index structure is constructed according to the selected algorithm (such as KD tree).

[0089] For each feature descriptor, FLANN is used to perform a nearest neighbor search, and FLANN returns the descriptor that is most similar to the given descriptor and its distance.

[0090] Filter matches by distance threshold or ratio test to retain high-quality matching point pairs.

[0091] The RANSAC algorithm can better improve the accuracy of feature point matching. Using the RANSAC algorithm to optimize feature point matching can be summarized into the following three steps.

[0092] (1) Randomly select 4 pairs of feature points from the matching results and calculate the homography matrix.

[0093] (2) The feature points in the first frame image are reprojected into the second frame image according to the homography matrix, and the distance between the reprojected coordinates and the matched feature point coordinates is compared. If the distance is less than a certain threshold, it is considered to be a correct matching point pair, otherwise it is considered to be an incorrect match, and the number of correct matching point pairs is recorded.

[0094] (3) Repeat the first and second steps, compare the number of correct matching point pairs counted after multiple cycles, take the case with the largest number of correct matching point pairs as the final result, eliminate the wrong matches, and output the correct matching pairs, thereby realizing the screening of feature point matching.

[0095] In the subtraction step, the answer part and the known part of the question in the template are extracted by the HSV color model. The answer part and the known part are as follows Figure 14 、 15 shown.

[0096] In the subtraction step, the sixth image is dilated to obtain the first dilated image, and the third image is dilated to obtain the second dilated image. The HSV color model is used to extract the answer portion of the template image for each question. The first dilated image is subtracted from the answer portion to obtain the underdrawing error, and the second dilated image is subtracted from the sixth image to obtain the overdrawing error. In the subtraction step, dilation can reduce the impact of line thickness and position differences on the marking process.

[0097] The expansion process diagram is as follows Figure 16 As shown, Figure 16 The left side is the original image to be expanded, the middle is the structure element, the center of the structure element is overlapped with 1 pixel in the original image, and the pixel values ​​of all pixels covered by the structure element are modified to 1. The result of the original image expansion is as follows Figure 16 Shown in the rightmost image.

[0098] Image subtraction is calculated by the following formula:

[0099]

[0100] Where C(x,y) is the pixel value of the result image at position (x,y), A(x,y) is the pixel value of image A at position (x,y), and B(x,y) is the pixel value of image B at position (x,y).

[0101] In the error display step, the difference image is binarized and median filtered before being displayed on the sixth image. In the error display step, the part with less error is displayed in red, and the part with more error is displayed in green. Figure 17 shown.

[0102] In binarization, grayscale values ​​greater than the threshold are set to the maximum value, and other values ​​are set to 0 to remove some interference. Median filtering can effectively remove salt and pepper noise without significantly blurring the image edges. The steps of median filtering are as follows:

[0103] (1) Determine a window size to specify the neighborhood of each pixel.

[0104] (2) Slide the window across the image, covering each pixel location, collect the values ​​of all pixels covered by the window, sort these values, and find the middle value.

[0105] (3) Replace the value of the center pixel of the current window with the calculated median value, and repeat the above process for each pixel in the image until the entire image processing is completed.

Claims

1. A method for correcting three-view drawings using a ruler and compass, characterized in that: The following steps are involved: Acquisition step: collecting the image to be tested, the entire template image and a single template image of a question; A preprocessing step of performing grayscale processing and binarization processing on the image to be measured to obtain a first image; Performing grayscale processing and binarization processing on the entire template image to obtain a second image; Performing grayscale processing and binarization processing on the single question template image to obtain a third image; a segmentation step of segmenting the first image to obtain a first three-view area image; segmenting the second image to obtain a second three-view area image; a correction step of performing perspective transformation on the first three-view area image to remove distortion, and then performing size transformation to obtain a fourth image having the same size as the second three-view area image; In the correction step, the perspective transformed four-point coordinates are obtained based on the contour polygon approximation; a template matching step, performing template matching on the third image and the fourth image, finding and separating the fifth image on the fourth image; the process of the template matching step is as follows: selecting a sliding window of the same size as the third image in the fourth image; comparing the grayscale value of each pixel in the sliding window with the grayscale value of the corresponding pixel in the third image, and calculating the similarity between the third image and the sliding window; sliding the sliding window from the upper left corner to the right, sliding it to the rightmost corner, and then sliding it downward one line, and then restarting from the leftmost corner, recording the similarity between the third image and the sliding window calculated after each movement; comparing the similarities at all positions, and selecting the sliding window with the greatest similarity as the fifth image; A registration step of aligning the fifth image with the third image to obtain a sixth image; the registration step comprises the following steps: obtaining feature points through SIFT feature point detection; achieving efficient matching of feature points through FLANN matching; preliminarily screening feature points through minimum Hamming distance; optimizing feature point matching through RANSAC to further remove incorrectly matched feature points; then calculating a homography matrix between the two images, and finally performing a perspective transformation; a subtraction step of performing a difference operation on the sixth image and the third image to obtain a difference image; wherein the subtraction step further includes the following steps: dilating the sixth image to obtain a first dilated image, and dilating the third image to obtain a second dilated image; extracting the single question template image using the HSV color model to obtain an answer portion image, subtracting the first dilated image from the answer portion image to obtain an underdrawing error, and subtracting the second dilated image from the sixth image to obtain an overdrawing error; An error display step is to display the processed differential image on the sixth image.

2. The method for correcting three-view drawings of ruler and compass construction as claimed in claim 1, characterized in that: In the acquisition step, the single question template image is obtained by cutting out the entire template image.

3. The method for correcting three-view drawings using a ruler and compass as claimed in claim 1, wherein: In the segmentation step, a first three-view area image and a second three-view area image are obtained based on a contour detection method; the contour detection method includes the following steps: performing contour detection on the image, extracting the outermost maximum contour of the image, enclosing a maximum rectangle through the maximum contour, intercepting the three-view area, and removing background information.

4. The method for correcting three-view drawings using a ruler and compass as claimed in claim 1, wherein: In the error display step, the differential image is binarized and median filtered before being displayed on the sixth image.

Images