Deep learning based drawing comparison method

Abstract

The application discloses a kind of based on deep learning's drawing comparison method, comprising: obtaining first image based on original drawing and second image formed based on original drawing scanning;According to key point extraction, matching algorithm, reliable key point matching pair is obtained by first image and second image, and perspective transformation matrix is calculated according to reliable key point matching pair, and second image is aligned to first graph according to perspective transformation matrix, and reprojected second image is obtained after alignment;According to difference comparison model, difference prediction is carried out to first image and reprojected second image, and difference image is obtained;Difference block confirmation image is obtained by dividing difference image according to predetermined parameter, and finally difference confirmation is completed by user;It is more accurate to achieve for small difference capture and noise caused misjudgment processing, suitable for various drawing comparison, especially for drawing quality difference, and better results can be predicted.

Description

Technical Field

[0001] This invention relates to the field of civil engineering drawing comparison technology, specifically to a drawing comparison method based on deep learning. Background Technology

[0002] Currently, the industry generally adopts a manual verification approach for the comparison of architectural design drawings. The specific process is as follows: staff first scan and obtain the revised paper drawings submitted by the user, and simultaneously retrieve the corresponding electronic originals and display them through electronic devices such as computers and monitors; then, staff need to visually compare the paper drawings with the electronic originals page by page and in detail, including but not limited to key information such as dimensions, line directions, graphic structures, text annotations, and symbols in the drawings, to determine whether the two are completely consistent; if differences are found, the location and specific content of the differences need to be recorded and fed back to the user for confirmation or further modifications.

[0003] However, this manual proofreading method has significant limitations and is prone to omissions. From a technical perspective, on the one hand, drawings often contain a large amount of complex and detailed information, such as minor dimensional deviations, variations in line thickness, and subtle differences between similar symbols. During manual visual comparison, these details are difficult to capture fully due to limitations in human visual discrimination and attention span. Especially in scenarios involving long periods and large volumes of drawings, staff are prone to visual fatigue, further increasing the probability of overlooking differences. On the other hand, manual proofreading lacks a unified standardized comparison process and quantitative judgment standards. Different staff members have different proofreading habits and professional levels, and their focus on drawing details and judgment thresholds vary, which may lead to the neglect of some potential differences. For example, minor modifications to non-critical areas are easily overlooked, and these overlooked differences can have a significant impact on the final result.

[0004] The encoding-decoding architecture is derived from Ronneberger O, Fischer P, Brox TU-Net: Convolutional Networks for Biomedical Image Segmentation[J]. Springer, Cham, 2015. DOI:10.1007 / 978-3-662-54345-0_3.

[0005] The feature point extraction algorithm is from Ma J, Jiang X, Fan A, et al. Image Matching from Handcrafted to Deep Features: A Survey[J]. International Journal of Computer Vision, 2021(1).DOI:10.1007 / S11263-020-01359-2. Summary of the Invention

[0006] The purpose of this invention is to address the significant limitations of manual proofreading methods in the prior art, which easily leads to proofreading omissions.

[0007] To address the above problems, this invention provides a deep learning-based drawing comparison method, comprising:

[0008] Step 1: Obtain a first image A based on the original drawing and a second image B1 formed by scanning the original drawing;

[0009] Step 2: Based on the key point extraction and matching algorithm, obtain reliable key point matching pairs G through the first image A and the second image B1. Calculate the perspective transformation matrix P based on the reliable key point matching pairs G. Align the second image B1 to A based on the perspective transformation matrix P. After alignment, obtain the reprojected second image B2.

[0010] Step 3: Based on the difference comparison model J, perform difference prediction on the first image A and the reprojected second image B2 to obtain the difference image K;

[0011] Step 4: Divide the difference image K into blocks according to predetermined parameters to obtain the difference block confirmation image D, and finally the user completes the difference confirmation;

[0012] Step 401: Divide the difference image K into horizontal and vertical blocks according to predetermined parameters. The predetermined parameters are determined by the user to obtain the difference block confirmation image D. Each pixel block of the block result is a pixel block to be confirmed.

[0013] Step 402: Confirm image D based on the difference blocks, count the total number of 1s within each block as the number of difference pixels. If the number of difference pixels within a block divided by the total number of pixels within the block exceeds a threshold, the user must interact to confirm whether the pixel block to be confirmed should be retained. The threshold value is set to 0.001.

[0014] In the preferred approach, the key point extraction and matching algorithms include:

[0015] Step 201: Extract the keypoint results E of the first image A and the keypoint results Z of the second image B1 according to the feature point extraction algorithm. The keypoint results E and Z include location information and feature vectors. The formula is as follows:

[0016] E = F(A)

[0017] Z=F(B1)

[0018] In the formula, F(*) represents the mapping function of any feature point extraction algorithm, where:

[0019] F(*) = {(x,s,θ,d) | x∈detected interest point, s∈scale space, θ∈direction space, d∈feature space}

[0020] In the formula, the scale space s includes the scale hierarchy, the direction space θ includes the main direction of the feature points, and the feature space d includes the feature vectors.

[0021] E = {e p | e p = (x A P , s A P , θ A P ,d A P , p=1, 2,…,m}

[0022] In the formula, e p Let x represent the p-th feature point of the first image A. A P Let s represent the coordinates of the p-th feature point. A P θ represents the level of the scale space in which the p-th feature point is located. A P The principal direction of the p-th feature point is represented by d. A P This represents the eigenvector of the p-th feature point;

[0023] Z={z q | z q = (x B1 q , s B1 q , θ B1 q ,d B1 q , q=1, 2,…,n}

[0024] In the formula, z q Let x represent the q-th feature point of the second image B1.B1 q , represents the coordinates of the q-th feature point, s B1 q θ represents the level of the scale space in which the q-th feature point is located. B1 q Let d represent the principal direction of the q-th feature point. B1 q The eigenvector of the q-th feature point;

[0025] Step 202: Based on the keypoint matching algorithm, obtain a reliable keypoint matching pair G using keypoint results E and Z, as shown in the formula:

[0026] G=M(E,Z)

[0027] In the formula, M(,) represents the matching function. The core logic is to retain matching pairs that meet the reliability conditions through feature vector similarity filtering and outlier removal. The formula is as follows:

[0028]

[0029] In the formula, This is the feature similarity threshold, with a value range of [value range missing]. , The value range is within 4 pixels. This represents a matching pair consisting of the p-th keypoint of the first image A and the q-th keypoint of the second image B1;

[0030] S203: Based on the reliable keypoint matching pair G, calculate the perspective transformation matrix P using the least squares method. The formula is:

[0031]

[0032] In the formula, a, b, c, d, e, f, g, h are the parameters of the perspective transformation matrix P, which are calculated by least squares, and e p If only coordinates are used in the calculation process, then ,in This represents the x-axis coordinate of the p-th feature point in the first image A. This represents the y-axis coordinate of the p-th feature point in the first image A, and similarly... The coordinates of point B1 in the second image;

[0033] Based on the perspective transformation matrix P, the reprojected second image B1 is obtained by projecting the second image B1, using the following formula:

[0034] B2=P*B1

[0035] After mapping, the corresponding pixel (x, y) in the reprojected second image B2 is ( , The formula is:

[0036]

[0037] .

[0039] In the preferred approach, difference prediction includes:

[0040] The difference comparison model J is based on an encoder-decoder architecture, which includes multi-source input fusion and skip connection mechanisms. The specific method is as follows:

[0041] Input section:

[0042] Channel-dimensional concatenation is performed on the first image A and the reprojected second image B2 to obtain the first layer feature input map L0 of model J, as shown in the formula:

[0043] L0 = A + B2 = (C A + C B2 ) * H * W

[0044] In the formula, A = C A * H * W , B2 = C B2 * H * W, C A C is the number of channels in the first image A. B2 H is the number of channels for reprojecting the second image B2, and H and W are the height and width of the image;

[0045] Encoding path section:

[0046] It consists of convolutional blocks, pooling, and fusion jumps, and the specific method is as follows:

[0047] The convolutional block N consists of a convolutional layer, batch normalization, and an activation function, as shown in the formula:

[0048]

[0049] In the formula, These represent the output feature map and input feature map of any layer of the model, respectively. is the activation function, BN(*) is the batch normalization layer, and conv(*) is the convolutional layer;

[0050] Max pooling is used to downsample the feature map, compress the spatial dimension, and retain key features. The formula is:

[0051]

[0052] The fusion jump convolution convolution block outputs from different levels of the encoding path are concatenated with the transposed convolution inputs of the corresponding levels of the decoding path along the channel dimension to pass low-level detailed features, including: line edges and symbol outlines.

[0053] The decoding path consists of convolutional blocks and transposed convolutions. The formula for transposed convolution is:

[0054]

[0055] In the formula, k=2 indicates that a 2×2 convolution kernel is used, and s=2 indicates that upsampling is performed with a stride of 2 to expand the feature map size and gradually restore the spatial details of the image;

[0056] The difference image Q is obtained by performing a pixel-by-pixel subtraction operation between the first image A and the reprojected second image B2, using the following formula:

[0057] Q(i,j) = A(i,j)-B2(i,j)

[0058] In the formula, i and j are row and column indices, and the difference image Q is used as an auxiliary feature;

[0059] The final layer output feature map L of the decoding path n+1 The channel-wise concatenation is performed with the difference image Q, using the following formula:

[0060] R= L n+1 + Q = (C Ln+1 + C Q ) * H * W

[0061] In the formula, R represents the dimensional stitched image, and L... n+1 = C Ln+1 * H * W , Q = C Q * H * W, C Ln+1 It is the number of output channels of the decoding path, C Q H is the number of channels in the difference image Q, and H and W are the height and width of the image;

[0062] After obtaining the dimensional stitched image R, it is processed by a 1×1 convolutional layer. The multi-channel features are compressed into a single channel by the convolution kernel to generate a difference image K, which is a 2-classification image composed of 0 and 1, where 1 represents difference and 0 represents no difference.

[0063] The beneficial effects of the present invention are as follows: The present invention is more accurate in capturing subtle differences and handling misjudgments caused by noise, and is applicable to the comparison of various drawings, especially for drawings with poor quality, it can also predict better results. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0065] Figure 2 This is a schematic diagram of the difference comparison model of the present invention;

[0066] Figure 3 This is a schematic diagram of the first image A of the present invention;

[0067] Figure 4 This is a schematic diagram of the second image B1 of the present invention;

[0068] Figure 5 This is a schematic diagram of the difference image K of the present invention. Detailed Implementation

[0069] Example 1:

[0070] A deep learning-based drawing comparison method includes:

[0071] Step 1: Obtain a first image A based on the original drawing and a second image B1 formed by scanning the original drawing;

[0072] Step 2: Based on the key point extraction and matching algorithm, obtain reliable key point matching pairs G through the first image A and the second image B1. Calculate the perspective transformation matrix P based on the reliable key point matching pairs G. Align the second image B1 to A based on the perspective transformation matrix P. After alignment, obtain the reprojected second image B2.

[0073] Step 3: Based on the difference comparison model J, perform difference prediction on the first image A and the reprojected second image B2 to obtain the difference image K;

[0074] Step 4: Divide the difference image K into blocks according to predetermined parameters to obtain the difference block confirmation image D, and finally the user completes the difference confirmation;

[0075] Step 401: Divide the difference image K into horizontal and vertical blocks according to predetermined parameters. The predetermined parameters are determined by the user to obtain the difference block confirmation image D. Each pixel block of the block result is a pixel block to be confirmed.

[0076] Step 402: Confirm image D based on the difference blocks, count the total number of 1s within each block as the number of difference pixels. If the number of difference pixels within a block divided by the total number of pixels within the block exceeds a threshold, the user must interact to confirm whether the pixel block to be confirmed should be retained. The threshold value is set to 0.001.

[0077] Key point extraction and matching algorithms include:

[0078] Step 201: Extract the keypoint results E of the first image A and the keypoint results Z of the second image B1 according to the feature point extraction algorithm. The keypoint results E and Z include location information and feature vectors. The formula is as follows:

[0079] E = F(A)

[0080] Z=F(B1)

[0081] In the formula, F(*) represents the mapping function of any feature point extraction algorithm, where:

[0082] F(*) = {(x,s,θ,d) | x∈detected interest point, s∈scale space, θ∈direction space, d∈feature space}

[0083] In the formula, the scale space s includes the scale hierarchy, the direction space θ includes the main direction of the feature points, and the feature space d includes the feature vectors.

[0084] E = {e p | e p = (x A P , s A P , θ A P ,d A P , p=1, 2,…,m}

[0085] In the formula, e p Let x represent the p-th feature point of the first image A. A P Let s represent the coordinates of the p-th feature point. A P θ represents the level of the scale space in which the p-th feature point is located. A P The principal direction of the p-th feature point is represented by d. A P This represents the eigenvector of the p-th feature point;

[0086] Z={z q | z q = (x B1 q , s B1 q , θ B1 q ,d B1 q , q=1, 2,…,n}

[0087] In the formula, z q Let x represent the q-th feature point of the second image B1. B1 q , represents the coordinates of the q-th feature point, s B1 q θ represents the level of the scale space in which the q-th feature point is located. B1q Let d represent the principal direction of the q-th feature point. B1 q The eigenvector of the q-th feature point;

[0088] Step 202: Based on the keypoint matching algorithm, obtain a reliable keypoint matching pair G using keypoint results E and Z, as shown in the formula:

[0089] G=M(E,Z)

[0090] In the formula, M(,) represents the matching function. The core logic is to retain matching pairs that meet the reliability conditions through feature vector similarity filtering and outlier removal. The formula is as follows:

[0091]

[0092] In the formula, This is the feature similarity threshold, with a value range of [value range missing]. , The value range is within 4 pixels. This represents a matching pair consisting of the p-th keypoint of the first image A and the q-th keypoint of the second image B1;

[0093] S203: Based on the reliable keypoint matching pair G, calculate the perspective transformation matrix P using the least squares method. The formula is:

[0094]

[0095] In the formula, a, b, c, d, e, f, g, h are the parameters of the perspective transformation matrix P, which are calculated by least squares, and e p If only coordinates are used in the calculation process, then ,in This represents the x-axis coordinate of the p-th feature point in the first image A. This represents the y-axis coordinate of the p-th feature point in the first image A, and similarly... The coordinates of point B1 in the second image;

[0096] Based on the perspective transformation matrix P, the reprojected second image B1 is obtained by projecting the second image B1, using the following formula:

[0097] B2=P*B1

[0098] After mapping, the corresponding pixel (x, y) in the reprojected second image B2 is ( , The formula is:

[0099]

[0100] .

[0102] Difference predictions include:

[0103] The difference comparison model J is based on an encoder-decoder architecture, which includes multi-source input fusion and skip connection mechanisms. The specific method is as follows:

[0104] Input section:

[0105] Channel-dimensional concatenation is performed on the first image A and the reprojected second image B2 to obtain the first layer feature input map L0 of model J, as shown in the formula:

[0106] L0 = A + B2 = (C A + C B2 ) * H * W

[0107] In the formula, A = C A * H * W , B2 = C B2 * H * W, C A C is the number of channels in the first image A. B2 H is the number of channels for reprojecting the second image B2, and H and W are the height and width of the image;

[0108] Encoding path section:

[0109] It consists of convolutional blocks, pooling, and fusion jumps, and the specific method is as follows:

[0110] The convolutional block N consists of a convolutional layer, batch normalization, and an activation function, as shown in the formula:

[0111]

[0112] In the formula, These represent the output feature map and input feature map of any layer of the model, respectively. is the activation function, BN(*) is the batch normalization layer, and conv(*) is the convolutional layer;

[0113] Max pooling is used to downsample the feature map, compress the spatial dimension, and retain key features. The formula is:

[0114]

[0115] The fusion jump convolution convolutional block outputs from different levels of the encoding path are concatenated with the transposed convolutional inputs of the corresponding levels of the decoding path along the channel dimension. This process transmits low-level detailed features, including line edges and symbol outlines, to compensate for the information loss caused by upsampling of the transposed convolution. This achieves feature fusion of high-level semantics and low-level details, improving the accuracy of difference prediction.

[0116] The decoding path consists of convolutional blocks and transposed convolutions. The formula for transposed convolution is:

[0117]

[0118] In the formula, k=2 indicates that a 2×2 convolution kernel is used, and s=2 indicates that upsampling is performed with a stride of 2 to expand the feature map size and gradually restore the spatial details of the image;

[0119] The convolutional blocks in the decoding path use the same convolutional block N as those in the encoding path to further extract and optimize the features output by the transposed convolution; the multi-scale features passed by skip connections are fused to enhance the accuracy of difference localization; the decoding path gradually restores the spatial dimension of the feature map through transposed convolution and convolutional blocks, and combines the multi-scale features fused by skip connections to accurately locate differences;

[0120] The difference image Q is obtained by performing a pixel-by-pixel subtraction operation between the first image A and the reprojected second image B2, using the following formula:

[0121] Q(i,j) = A(i,j)-B2(i,j)

[0122] In the formula, i and j are row and column indices, and the difference image Q is used as an auxiliary feature;

[0123] The final layer output feature L of the decoding path n The channel-wise concatenation is performed with the difference image Q, using the following formula:

[0124] R= L n+1 + Q = (C Ln+1 + C Q ) * H * W

[0125] In the formula, R represents the dimensional stitched image, and L... n = C Ln+1 * H * W , Q = C Q * H * W, C Ln+1 It is the number of output channels of the decoding path, C Q H is the number of channels in the difference image Q, and H and W are the height and width of the image;

[0126] After obtaining the dimensionally stitched image R, it is processed by a 1×1 convolutional layer. The multi-channel features are compressed into a single channel by the convolution kernel to generate a difference image K, which is a binary classification image composed of 0 and 1, where 1 indicates difference and 0 indicates no difference. This enables pixel-level identification of the difference areas between the two images (such as line modifications and size deviations), providing accurate basis for subsequent block labeling and user confirmation.

[0127] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A deep learning-based drawing comparison method, characterized in that, include: Step 1: Obtain a first image A based on the original drawing and a second image B1 formed by scanning the original drawing; Step 2: Based on the key point extraction and matching algorithm, obtain reliable key point matching pairs G through the first image A and the second image B1. Calculate the perspective transformation matrix P based on the reliable key point matching pairs G. Align the second image B1 to A based on the perspective transformation matrix P. After alignment, obtain the reprojected second image B2. Step 3: Based on the difference comparison model J, perform difference prediction on the first image A and the reprojected second image B2 to obtain the difference image K; Step 4: Divide the difference image K into blocks according to predetermined parameters to obtain the difference block confirmation image D, and finally the user completes the difference confirmation; Step 401: Divide the difference image K into horizontal and vertical blocks according to predetermined parameters. The predetermined parameters are determined by the user to obtain the difference block confirmation image D. Each pixel block of the block result is a pixel block to be confirmed. Step 402: Confirm image D based on the difference blocks, count the total number of 1s within each block as the number of difference pixels. If the number of difference pixels within a block divided by the total number of pixels within the block exceeds a threshold, the user must interact to confirm whether the pixel block to be confirmed should be retained. The threshold value is set to 0.

001.

2. The deep learning-based drawing comparison method according to claim 1, characterized in that, Key point extraction and matching algorithms include: Step 201: Extract the keypoint results E of the first image A and the keypoint results Z of the second image B1 according to the feature point extraction algorithm. The keypoint results E and Z include location information and feature vectors. The formula is as follows: E = F(A) Z=F(B1) In the formula, F(*) represents the mapping function of any feature point extraction algorithm, where: F(*)={(x,s,θ,d)|x∈detected interest point, s∈scale space, θ∈direction space, d∈feature space}, In the formula, the scale space s includes the scale hierarchy, the direction space θ includes the main direction of the feature points, and the feature space d includes the feature vectors. E ={e p | and p = (x A P , s A P , θ A P ,d A P , p=1, 2,…,m} In the formula, e p Let x represent the p-th feature point of the first image A. A P Let s represent the coordinates of the p-th feature point. A P θ represents the level of the scale space in which the p-th feature point is located. A P The principal direction of the p-th feature point is represented by d. A P This represents the eigenvector of the p-th feature point; Z={z q | z q = (x B1 q , s B1 q , θ B1 q ,d B1 q , q=1, 2,…,n} In the formula, z q Let x represent the q-th feature point of the second image B1. B1 q Let s represent the coordinates of the q-th feature point. B1 q θ represents the level of the scale space in which the q-th feature point is located. B1 q Let d represent the principal direction of the q-th feature point. B1 q The eigenvector of the q-th feature point; Step 202: Based on the keypoint matching algorithm, obtain a reliable keypoint matching pair G using keypoint results E and Z, as shown in the formula: G=M(E,Z) In the formula, M(,) represents the matching function. The core logic is to retain matching pairs that meet the reliability conditions through feature vector similarity screening and outlier removal. The formula is as follows: In the formula, This is the feature similarity threshold, with a value range of... , The value range is within 4 pixels. This represents a matching pair consisting of the p-th keypoint of the first image A and the q-th keypoint of the second image B1; S203: Based on the reliable keypoint matching pair G, calculate the perspective transformation matrix P using the least squares method. The formula is: In the formula, a, b, c, d, e, f, g, h are the parameters of the perspective transformation matrix P, which are calculated by least squares, and e p If only coordinates are used in the calculation process, then ,in This represents the x-axis coordinate of the p-th feature point in the first image A. This represents the y-axis coordinate of the p-th feature point in the first image A, and similarly... The coordinates of point B1 in the second image; Based on the perspective transformation matrix P, the reprojected second image B1 is obtained by projecting the second image B1, using the following formula: B2=P*B1 After mapping, the corresponding pixel (x, y) in the reprojected second image B2 is ( , The formula is: 。 3. The deep learning-based drawing comparison method according to claim 1, characterized in that, Difference predictions include: The difference comparison model J is based on an encoder-decoder architecture, which includes multi-source input fusion and skip connection mechanisms. The specific method is as follows: Input section: Channel-dimensional concatenation is performed on the first image A and the reprojected second image B2 to obtain the first layer feature input map L0 of model J, as shown in the formula: L0 = A + B2 = (C A + C B2 ) * H * W In the formula, A = C A * H * W , B2 = C B2 * H * W, C A C is the number of channels in the first image A. B2 H is the number of channels for reprojecting the second image B2, and H and W are the height and width of the image; Encoding path section: It consists of convolutional blocks, pooling, and fusion jumps, and the specific method is as follows: The convolutional block N consists of a convolutional layer, batch normalization, and an activation function, as shown in the formula: In the formula, These represent the output feature map and input feature map of any layer of the model, respectively. is the activation function, BN(*) is the batch normalization layer, and conv(*) is the convolutional layer; Max pooling is used to downsample the feature map, compress the spatial dimension, and retain key features. The formula is: The fusion jump concatenates the outputs of convolutional blocks at different levels of the encoding path with the transposed convolutional inputs at the corresponding levels of the decoding path along the channel dimension, thus conveying low-level detailed features, including: line edges and symbol outlines. The decoding path consists of convolutional blocks and transposed convolutions. The formula for transposed convolution is: In the formula, k=2 indicates that a 2×2 convolution kernel is used, and s=2 indicates that upsampling is performed with a stride of 2 to expand the feature map size and gradually restore the spatial details of the image; The difference image Q is obtained by performing a pixel-by-pixel subtraction operation between the first image A and the reprojected second image B2, using the following formula: Q(i,j) = A(i,j)-B2(i,j) In the formula, i and j are row and column indices, and the difference image Q is used as an auxiliary feature; The final layer output feature map L of the decoding path n+1 The channel-wise concatenation is performed with the difference image Q, using the following formula: R= L n+1 + Q=(C Ln+1 + C Q ) * H * W In the formula, R represents the dimensional stitched image, and L... n+1 = C Ln+1 * H * W , Q = C Q * H * W, C Ln+1 It is the number of output channels of the decoding path, C Q H is the number of channels in the difference image Q, and H and W are the height and width of the image; After obtaining the dimensional stitched image R, it is processed by a 1×1 convolutional layer. The multi-channel features are compressed into a single channel by the convolution kernel to generate a difference image K, which is a 2-classification image composed of 0 and 1, where 1 represents difference and 0 represents no difference.

Images