Ruler-compass drawing three-view correcting method

Abstract

The invention discloses a three-view correction method for ruler and compass drawing. The three-view correction method comprises an acquisition step, a preprocessing step, a segmentation step, a correction step, a template matching step, a registration step, a subtraction step and an error display step. According to the ruler and compass drawing three-view correcting method, errors in the ruler and compass drawing three views can be effectively detected through the machine vision technology, and the detection accuracy and efficiency are greatly improved.

Description

Technical Field

[0001] The present invention relates to a ruler Figure 3 View modification method. Background Art

[0002] The drawing and reading of three views are the premise and foundation for learning to draw and read engineering drawings. Three-view drawing methods can be divided into two categories: one is computer software drawing; the other is ruler and compass drawing. At present, the research on computer software drawing error detection has been relatively complete. Ruler and compass drawing Figure 3 Teachers still need to manually grade the views, and their workload will increase exponentially with the number of students and daily homework. At the same time, as the workload increases, teachers will inevitably make mistakes in the grading process.

[0003] In reality, there is a method for automatically correcting drawing homework based on image processing. First, the image of the ruler and compass drawing homework is processed, and then the standard answer is compared with the student's homework (ruler and compass drawing), and the errors are marked in the student's homework, and finally the correction result is obtained. Figure 3 There are differences in the thickness and position of lines in the view, and shooting will also cause slight deformation of the image. Therefore, the standard answer and the student's homework often have differences in line position and thickness. Therefore, the above method of directly comparing the standard answer and the student's homework is not effective. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defects in the prior art and provide a ruler Figure 3 View modification method.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A ruler Figure 3 The view modification method includes the following steps:

[0007] The acquisition step includes acquiring the image to be tested, the entire template image, and a single template image of a question;

[0008] Preprocessing step: grayscale processing and binarization processing are performed on the image to be tested to obtain a first image; grayscale processing and binarization processing are performed on the entire template image to obtain a second image; grayscale processing and binarization processing are performed on the single template image to obtain a third image;

[0009] A segmentation step, segmenting the first image to obtain a first three-view area image; segmenting the second image to obtain a second three-view area image;

[0010] a correction step of performing perspective transformation processing on the first three-view area image to remove distortion, and then performing size transformation processing to obtain a fourth image having the same size as the second three-view area image; in the correction step, the perspective transformation four-point coordinates are obtained based on contour polygon approximation;

[0011] The template matching step is to perform template matching on the third image and the fourth image, and find and separate the fifth image on the fourth image. The process of the template matching step is as follows: select a sliding window of the same size as the third image in the fourth image; compare the relationship between the grayscale value of each pixel in the sliding window and the corresponding pixel of the third image, and calculate the similarity between the third image and the sliding window; slide the sliding window from the upper left corner to the right, slide it to the rightmost corner, slide it downward one line, and then start sliding again from the leftmost corner, and record the similarity between the third image and the sliding window calculated after each movement; compare the similarity of all positions, and select the sliding window with the largest similarity as the fifth image;

[0012] A registration step, aligning the fifth image with the third image to obtain a sixth image; the process of the registration step is as follows: obtaining feature points through SIFT feature point detection; using FLANN matching to achieve efficient matching of feature points; preliminarily screening feature points through minimum Hamming distance; optimizing feature point matching through RANSAC to further remove feature points with incorrect matching; then calculating the homography matrix between the two images, and finally performing perspective transformation;

[0013] The subtraction step includes performing a difference operation on the sixth image and the third image to obtain a difference image; in the subtraction step, the following steps are also included: dilating the sixth image to obtain a first dilated image, and dilating the third image to obtain a second dilated image; extracting the single question template image through the HSV color model to obtain an answer part image, subtracting the first dilated image from the answer part image to obtain a less-drawing error, and subtracting the second dilated image from the sixth image to obtain an more-drawing error;

[0014] The error display step is to display the processed differential image on the sixth image.

[0015] In some embodiments, in the acquisition step, the single question template image is obtained by cutting out the entire template image.

[0016] In some embodiments, in the segmentation step, a first three-view area image and a second three-view area image are obtained based on a contour detection method; the contour detection method includes the following steps: performing contour detection on the image, extracting the outermost maximum contour of the image, enclosing a maximum rectangle through the maximum contour, intercepting the three-view area, and removing background information.

[0017] In some embodiments, in the error display step, the differential image is binarized and median filtered before being displayed on the sixth image.

[0018] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present invention.

[0019] The positive improvement effect of the present invention is that: the ruler of the present embodiment is Figure 3 The view correction method can effectively use machine vision technology to detect the ruler and compass Figure 3 Errors in the view greatly improve the accuracy and efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The ruler of the preferred embodiment of the present invention is Figure 3 Flowchart of the view modification method.

[0021] Figure 2 This is the image to be tested in the preferred embodiment of the present invention.

[0022] Figure 3 This is the entire template image of the preferred embodiment of the present invention.

[0023] Figure 4 This is a single-question template image of a preferred embodiment of the present invention.

[0024] Figure 5 This is the first image of the preferred embodiment of the present invention.

[0025] Figure 6 It is the first three-view area image of the preferred embodiment of the present invention.

[0026] Figure 7 This is the second three-view area image of the preferred embodiment of the present invention.

[0027] Figure 8 This is a four-point coordinate position diagram of the perspective transformation of the image to be measured in the preferred embodiment of the present invention.

[0028] Fig. 9 This is the fourth image of the preferred embodiment of the present invention.

[0029] Fig.10 This is the fifth image of the preferred embodiment of the present invention.

[0030] Fig.11 Schematic diagram of the template matching step.

[0031] Fig.12 This is the result image of the subtraction step without performing the registration step.

[0032] Fig.13 This is the sixth image of the preferred embodiment of the present invention.

[0033] Fig.14 for Figure 4 A diagram of the part that requires answering.

[0034] Fig.15 for Figure 4 Schematic diagram of the known part of the problem.

[0035] Fig.16 Schematic diagram of the image expansion process.

[0036] Fig.17 A schematic diagram of the review results. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be noted that in the claims and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0039] like Figure 1 As shown, the ruler of this embodiment is Figure 3The view correction method includes an acquisition step, a preprocessing step, a segmentation step, a correction step, a template matching step, a registration step, a subtraction step and an error display step.

[0040] Acquisition steps: Acquire the image to be tested, the entire template image and a single question template image.

[0041] Preprocessing steps: grayscale and binarize the image to be tested to obtain the first image; grayscale and binarize the entire template image to obtain the second image; grayscale and binarize a single template image to obtain the third image.

[0042] Segmentation step: segment the first image to obtain a first three-view area image; segment the second image to obtain a second three-view area image.

[0043] Correction step: performing perspective transformation processing on the first three-view area image to remove distortion, and then performing size transformation processing to obtain a fourth image with the same size as the second three-view area image.

[0044] Template matching step: performing template matching on the third image and the fourth image, and finding and separating the fifth image on the fourth image.

[0045] Registration step: aligning the fifth image with the third image to obtain the sixth image.

[0046] Subtraction step: performing a difference operation on the sixth image and the third image to obtain a difference image.

[0047] Error display step: the differential image is processed and then displayed on the sixth image.

[0048] The ruler of this embodiment is Figure 3 The view correction method is a machine vision-based ruler that takes into account deviations. Figure 3 The view correction method adopts the above steps to effectively use machine vision technology to detect the ruler and compass. Figure 3 Errors in the view greatly improve the accuracy and efficiency of detection.

[0049] In the acquisition step, the template images acquired include the entire template image and a single question template image. The single question template image is obtained by intercepting the entire template image. Figure 2 A demonstrative image to be tested is shown. Figure 3 A whole template image is shown for demonstration purposes. Figure 4 A single question template image is shown for demonstration purposes.

[0050] In the preprocessing step, the processing speed is improved by converting the color image into a grayscale image. The calculation formula is:

[0051] GRAY=0.3R+0.59G+0.11B

[0052] Among them, GRAY is the obtained grayscale image, R is the red component of the original image, G is the green component of the original image, and B is the blue component of the original image.

[0053] The binarization threshold is obtained by the Otsu method (OTSU), and global binarization is performed to further reduce the amount of calculation. Figure 2 The image to be tested is obtained after the preprocessing step as shown in Figure 5 The first image shown, the second image and the third image obtained after the preprocessing steps of the entire template image and the single question template image are not shown in the accompanying drawings.

[0054] The operation process of Otsu's method is as follows:

[0055] (1) Calculate the image histogram and obtain the frequency distribution of the grayscale value of each pixel in the image.

[0056] (2) Traverse all possible thresholds, and for each possible threshold, calculate the intra-class variance and inter-class variance of the image segmented into foreground and background.

[0057] (3) Find the threshold that maximizes the inter-class variance. This threshold is the optimal binarization threshold. The calculation formula for the inter-class variance is:

[0058]

[0059] in is the between-class variance, ω 1 is the proportion of foreground pixels, ω 2 is the background pixel ratio, μ 1 is the average gray value of the foreground, μ 2 is the average gray value of the background, and t is the current threshold.

[0060] In the segmentation step, the three-view area is obtained based on contour detection. Contour detection is performed on the image to extract the largest outer contour of the image. The three-view area is intercepted by the largest rectangle circumscribing the largest contour, and background information (including student name, student number, class, etc.) outside the three-view area is removed. Figure 5 The first image shown is obtained after the segmentation step as follows Figure 6 The first three-view area image is shown. The second image is segmented to obtain Figure 7 The second third view area image is shown.

[0061] In the correction step, the perspective transformation four-point coordinates are obtained based on the contour polygon approximation. The perspective transformation four-point coordinate positions of the image to be measured are as follows: Figure 8 shown. Figure 6The first three-view area image shown in FIG. 1 is distorted by perspective transformation, and then is resized to obtain the image as shown in FIG. Figure 7 The size of the second three-view area image shown is consistent with the fourth image to facilitate detection. The fourth image is as shown in FIG. Fig. 9 shown.

[0062] Contour polygon approximation simplifies the contour shape by reducing the number of points of the contour while trying to preserve the shape characteristics of the contour. The Douglas-Peucker algorithm is the most commonly used polygon approximation algorithm. Its steps include:

[0063] (1) Select the starting and ending points and connect them.

[0064] (2) Calculate the distances from the remaining points to the line segment and find the point with the farthest distance.

[0065] (3) If the maximum distance is greater than the given threshold, the point is important, keep it, and divide the contour into two segments. Repeat the above steps for each segment until all points are within the given distance threshold.

[0066] (4) If all points of the current segment are within the threshold distance, replace this segment with a straight line segment between the start and end points.

[0067] The perspective transformation can be represented by a 3x3 transformation matrix in the form of:

[0068]

[0069] Where (x, y) is the point in the original image, (X, Y) is the transformed position, and Z is used for normalization.

[0070] Sizing actually means changing the length and width of an image to achieve image scaling. During the sizing process, the interpolation method is used to calculate new pixel values. Bilinear interpolation finds four neighboring pixels near the target pixel position and calculates new pixel values ​​based on their distance weights.

[0071] In the template matching step, the fifth image (the image to be tested in a separate question) is found on the fourth image according to the third image, and the following image is separated from the fourth image: Fig.10 The fifth image shown is independently detected to prevent interference between the three views of different questions.

[0072] The template matching step is often used in the task of finding specific content in an image. Its principle is as follows Fig.11 shown.

[0073] exist Fig.11 In the figure, the 4×4 image on the right is the template image, and the 8×8 image on the left is the image to be matched. The template matching process is as follows:

[0074] (1) Select a sliding window of the same size as the template (the third image) in the image to be matched (the fourth image), such as Fig.11 Shown in the shaded area.

[0075] (2) Compare the relationship between the grayscale value of each pixel in the sliding window and the corresponding pixel in the template (third image), and calculate the similarity between the template (third image) and the sliding window.

[0076] (3) Slide the sliding window from the upper left corner to the right, slide it to the rightmost corner, slide it down one line, and then start sliding again from the leftmost corner. Record the similarity between the template (third image) and the sliding window calculated after each movement.

[0077] (4) Compare the similarities of all positions and select the sliding window with the largest similarity as the fifth image.

[0078] In the registration step, the registration process includes feature point detection, feature point matching, estimated transformation model, image resampling and transformation. Feature points are obtained through SIFT feature point detection, and efficient matching of feature points is achieved using FLANN matching. Feature points are initially screened through the minimum Hamming distance, and then feature point matching is optimized through RANSAC to further remove incorrectly matched feature points. Then the homography matrix between the two images is calculated, and finally perspective transformation is performed.

[0079] In the registration step, due to the difference in line thickness and position between the fifth image and the third image, direct subtraction may result in errors. The direct subtraction result is as follows: Fig.12 After the registration step, the fifth image is aligned with the third image to obtain Fig.13 The sixth image shown can align the images as much as possible and reduce the difference in image positions.

[0080] SIFT feature points still have good stability under interference such as lighting, noise, perspective, scaling and rotation. The SIFT feature point detection process is as follows:

[0081] (1) By performing multiple Gaussian blurs on the original image, images of different scales are created. Images of each scale are generated by adjusting the standard deviation of the Gaussian filter.

[0082] (2) By calculating the difference images between adjacent scales to form a scale space, the DOG method can be used to effectively find the local extreme points of the image, which are potential feature points.

[0083] (3) For the detected extreme points, accurate positioning is performed, and by comparing local features, points with strong edge response or low contrast are eliminated to ensure that stable and significant feature points are retained.

[0084] (4) Assign one or more directions to each feature point to ensure that the feature descriptor is invariant to rotation. The main direction is determined by calculating the gradient direction and amplitude around the feature point.

[0085] (5) In the area around the feature point, the feature descriptor is calculated based on the gradient information. Usually, the area around the feature point is divided into several small blocks, and the gradient direction histogram of each small block is calculated to finally generate a 128-dimensional feature descriptor.

[0086] FLANN matching can handle large-scale feature sets and significantly reduce matching time. The general steps of FLANN matching are as follows:

[0087] First, feature points and their descriptors are extracted from the image to be matched.

[0088] The extracted descriptors are used to build a FLANN index for fast retrieval, and the index structure is constructed according to the selected algorithm (such as KD tree).

[0089] For each feature descriptor, FLANN is used to perform a nearest neighbor search, which returns the descriptor most similar to the given descriptor and its distance.

[0090] Filter matches by distance threshold or ratio test to retain high-quality matching point pairs.

[0091] The RANSAC algorithm can better improve the accuracy of feature point matching. Using the RANSAC algorithm to optimize feature point matching can be summarized into the following three steps.

[0092] (1) Randomly select 4 pairs of feature points from the matching results and calculate the homography matrix.

[0093] (2) The feature points in the first frame image are reprojected into the second frame image according to the homography matrix, and the distance between the reprojected coordinates and the matched feature point coordinates is compared. If the distance is less than a certain threshold, it is considered to be a correct matching point pair. Otherwise, it is considered to be an incorrect match, and the number of correct matching point pairs is recorded.

[0094] (3) Repeat the first and second steps, compare the number of correct matching point pairs counted after multiple cycles, take the case with the largest number of correct matching point pairs as the final result, eliminate incorrect matches, and output correct matching pairs, thereby realizing the screening of feature point matching.

[0095] In the subtraction step, the answer part and the known part of the question in the template are extracted through the HSV color model. The answer part and the known part are obtained as follows Fig.14 , 15 shown.

[0096] In the subtraction step, the sixth image is expanded to obtain the first expanded image, and the third image is expanded to obtain the second expanded image; the single question template image is extracted through the HSV color model to obtain the answer part image, the answer part image minus the first expanded image to obtain the underdrawing error, and the sixth image minus the second expanded image to obtain the overdrawing error. In the subtraction step, the expansion process can reduce the influence of line thickness and position on the review.

[0097] The expansion process diagram is as follows: Fig.16 As shown, Fig.16 The left side is the original image to be expanded, and the middle is the structural element. The center of the structural element is overlapped with 1 pixel in the original image, and the pixel values ​​of all pixels covered by the structural element are modified to 1. The result of the original image expansion is as follows Fig.16 Shown in the rightmost image.

[0098] Image subtraction is calculated by the following formula:

[0099]

[0100] Where C(x,y) is the pixel value of the result image at position (x,y), A(x,y) is the pixel value of image A at position (x,y), and B(x,y) is the pixel value of image B at position (x,y).

[0101] In the error display step, the difference image is binarized and median filtered before being displayed on the sixth image. In the error display step, the underdrawn error part is displayed in red, and the overdrawn error part is displayed in green. Fig.17 shown.

[0102] In binarization, the grayscale values ​​greater than the threshold are set to the maximum value, and other values ​​are set to 0 to remove some interference. Median filtering can effectively remove salt and pepper noise without significantly blurring the image edges. The steps of median filtering are as follows:

[0103] (1) Determine a window size to specify the neighborhood of each pixel.

[0104] (2) Slide the window across the image, covering each pixel position, collect the values ​​of all pixels covered by the window, sort these values, and find the middle value.

[0105] (3) Replace the value of the center pixel of the current window with the calculated median value, and repeat the above process for each pixel in the image until the entire image is processed.

Claims

1. A method for correcting three-view drawings of ruler and compass, characterized in that: The following steps are involved: The acquisition step includes acquiring the image to be tested, the entire template image, and a single template image of a question; A preprocessing step of performing grayscale processing and binarization processing on the image to be tested to obtain a first image; Performing grayscale processing and binarization processing on the entire template image to obtain a second image; Performing grayscale processing and binarization processing on the single question template image to obtain a third image; a segmentation step of segmenting the first image to obtain a first three-view area image; Segmenting the second image to obtain a second three-view area image; a correction step of performing perspective transformation processing on the first three-view area image to remove distortion, and then performing size transformation processing to obtain a fourth image having the same size as the second three-view area image; In the correction step, four-point coordinates of perspective transformation are obtained based on contour polygon approximation; The template matching step is to perform template matching on the third image and the fourth image, and find and separate the fifth image on the fourth image. The process of the template matching step is as follows: select a sliding window of the same size as the third image in the fourth image; compare the relationship between the grayscale value of each pixel in the sliding window and the corresponding pixel of the third image, and calculate the similarity between the third image and the sliding window; slide the sliding window from the upper left corner to the right, slide it to the rightmost corner, slide it downward one line, and then start sliding again from the leftmost corner, and record the similarity between the third image and the sliding window calculated after each movement; compare the similarity of all positions, and select the sliding window with the largest similarity as the fifth image; A registration step, aligning the fifth image with the third image to obtain a sixth image; the process of the registration step is as follows: obtaining feature points through SIFT feature point detection; using FLANN matching to achieve efficient matching of feature points; preliminarily screening feature points through minimum Hamming distance; optimizing feature point matching through RANSAC to further remove feature points with incorrect matching; then calculating the homography matrix between the two images, and finally performing perspective transformation; The subtraction step includes performing a difference operation on the sixth image and the third image to obtain a difference image; in the subtraction step, the following steps are also included: dilating the sixth image to obtain a first dilated image, and dilating the third image to obtain a second dilated image; extracting the single question template image through the HSV color model to obtain an answer part image, subtracting the first dilated image from the answer part image to obtain a less-drawing error, and subtracting the second dilated image from the sixth image to obtain an more-drawing error; The error display step is to display the processed differential image on the sixth image.

2. The method for correcting three-view drawings of ruler and compass drawing as claimed in claim 1, characterized in that: In the acquisition step, the single question template image is obtained by intercepting the entire template image.

3. The method for correcting three-view drawings of ruler and compass drawing as claimed in claim 1, characterized in that: In the segmentation step, a first three-view area image and a second three-view area image are obtained based on a contour detection method; the contour detection method comprises the following steps: performing contour detection on the image, extracting the outermost maximum contour of the image, enclosing a maximum rectangle through the maximum contour, intercepting the three-view area, and removing background information.

4. The method for correcting three-view drawings of ruler and compass drawing as claimed in claim 1, characterized in that: In the error display step, the differential image is binarized and median filtered before being displayed on the sixth image.

Images