Image-text comparison method and system based on similar image stacking analysis and computer readable storage medium

Abstract

The invention relates to a similar picture stacking analysis-based image-text comparison method and system and a computer readable storage medium. The method comprises the following steps of S1, automatically obtaining document information provided by a client; s2, identifying document information types provided by a client, wherein the document information types at least comprise a PDF image document and a three-dimensional model document; each of the PDF image document and the three-dimensional model document is provided with a first version document and a second version document; s3, performing preprocessing according to the document information type identified in the step S2; s4, starting an image-text comparison engine, and performing superposition comparison on a first version document and a second version document in the PDF image document and / or the three-dimensional model document preprocessed in the step S3; and S5, automatically generating a comparison report of the PDF image document and / or the three-dimensional model document according to a comparison result in the step S4.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent image-text retrieval, and in particular to an image-text comparison method and system based on same-type image superposition analysis and a computer readable storage medium. BACKGROUND

[0002] Traditional drawing comparison methods include manual visual comparison, version switching view comparison, PDF drawing comparison, and code comparison based on text format. These methods have many problems: printing different versions of drawings may damage the integrity of the drawings, differences may be missed during the comparison process, and accurate comparison cannot be achieved. In addition, for complex graphic elements in design drawings, the criteria for judging similarities and differences are also relatively cumbersome.

[0003] Chinese Patent CN118035427B discloses a method and device for enhancing multi-modal image-text retrieval through 3D contrast learning. Based on a 3D contrast learning multi-modal network, the interaction between modal features is supervised by attribute information, fully exploiting the correspondence between modalities, so that complementary and aligned information between modalities can be utilized. In the method, features of the three dimensions are obtained by feature extraction on pairs of pictures, text pairs, and corresponding attribute information in the original data set, and then sent to the 3D contrast learning module for sufficient contrast fusion and feature alignment to obtain complementary information and potential correspondence between visual modalities and text modalities.

[0004] The above-mentioned scheme enhances the retrieval of multi-modal images and texts through 3D contrast learning, which can fully utilize the visual modal information, text modal information, and attribute information shared by these features for joint optimization, thereby greatly improving the accuracy of image-text retrieval. However, 3D contrast learning requires feature extraction and contrast fusion on a large amount of image-text pairs. If the size label of "Φ50H7" in a certain drawing is modified to "Φ50H8", the text layer only needs to identify the change in the tolerance band, while the image layer needs to associate the consistency of the aperture symbol, label position, and lead direction. The existing method may miss or misjudge the differences due to misalignment of text word vectors and image region features.

[0005] Therefore, we propose an image-text comparison method and system based on same-type image superposition analysis and a computer readable storage medium. SUMMARY

[0006] The main purpose of the present application is to provide an image-text comparison method and system based on same-type image superposition analysis and a computer readable storage medium, aiming to solve the problem in the prior art that PDF comparison and switching view require manual intervention or rely on general feature matching, and cannot perform differential analysis on the annotation format and graphic element layout of the same type of drawings, resulting in time-consuming and error-prone version comparison.

[0007] To achieve the above object, the application provides a method for comparing graphics and texts based on the analysis of the superposition of similar pictures, comprising the following steps: S1, automatically obtaining document information provided by a client; S2, identifying the type of the document information provided by the client, wherein the type of the document information at least comprises a PDF image document and a 3D model document, and the PDF image document and the 3D model document each have a first version document and a second version document; S3, pre-processing according to the type of the document information identified in step S2; S4, starting a graphics-text comparison engine to compare the first version document and the second version document in the PDF image document and / or the 3D model document pre-processed in step S3; S5, automatically generating a comparison report of the PDF image document and / or the 3D model document according to the comparison result in step S4; In the pre-processing of the PDF image document, the scanned images in the PDF image document are subjected to gray-scale processing, denoising and binarization processing, and the contrast of the lines of the drawing is enhanced by using a self-adaptive threshold algorithm; the text layer and the image layer in the PDF are extracted, the text part is converted into an editable text format, and the image part is kept as a high-resolution vector graph; the coordinate system of all pages is uniformly adjusted to a standard direction by calculating a rotation matrix and a scaling ratio through the detection of the direction marks in the drawing frame or the title bar. In the pre-processing of the 3D model document, the number of polygons is compressed by using a facet simplification algorithm, the data volume is reduced while the geometric features are kept, the model principal axis direction is extracted by principal component analysis (PCA), the model origin is reset to the geometric center and the coordinate axis direction is unified, the model surface is subjected to Laplacian smoothing to eliminate small noise points, the edge feature lines are extracted and a topological structure graph is generated, and standard six views are automatically generated based on the three-dimensional coordinates of the model and saved in a vectorized SVG format.

[0008] Preferably, the pre-processing of the PDF image document in step S3 further comprises: extracting the text layer part of the first version document and the second version document in the PDF image document by using an OCR technology, and adjusting the text layer parts of the two documents to the same font, font size and format; spatially registering the OCR recognition result of the text layer and the vector graph of the image layer to generate a standardized layer containing superimposed information of graphics and texts; The pre-processing of the 3D model document in step S3 further comprises: selecting at least three points on the surface of the 3D model as reference points, and positioning and adjusting the positions of the 3D model graphs of the first version document and the second version document in the coordinate system through the three selected reference points.

[0009] Preferably, the step S4 of superimposing and comparing the pre-processed PDF image documents comprises at least the following steps: text information comparison and dimension annotation comparison; the text information comparison is used to identify and compare the text content in the PDF image documents, to ensure the consistency and accuracy of the text; and the dimension annotation comparison is used to check the size and position of various graphics, charts or design elements in the documents, to ensure that they are consistent with the original documents or standard specifications.

[0010] Preferably, the text information comparison comprises the following steps: obtaining the text layer information in the pre-processed first version document and the second version document in step S3; stacking the text layer information of the first version document and the second version document, judging whether the text information of the two documents is completely matched, and marking the unmatched text content and its position.

[0011] Preferably, the text information comparison further comprises the following steps: For the unmatched text content, perform semantic similarity analysis to determine whether it is a synonym replacement or a minor expression difference, if so, it is considered as matched, otherwise it is recorded as a substantial difference.

[0012] Preferably, the dimension annotation comparison comprises the following steps: obtaining the standardized layers containing graphic-text superimposed information in the pre-processed first version document and the second version document in step S3; stacking the standardized layers in the first version document and the second version document, judging whether there is a difference in the dimension annotation of the two documents, and dividing the standardized layers into a difference area and a no-difference area according to the judgment result; The division process of the difference area and the no-difference area is as follows: the standardized layers containing graphic-text superimposed information in the two documents are vectorized, then an arrow used to identify the dimension annotation in the difference area is marked on the vector position A1 on the first version document drawing, the text annotation A2 corresponding to the arrow A1 is found, the arrow B1 on the same vector position on the second version drawing is found, then the text annotation B2 corresponding to the arrow B1 is found, and A2 and B2 are compared in terms of graphics and text; when it is judged that A2 and B2 have differences, A2 and B2 are divided into the difference area, otherwise, when it is judged that A2 and B2 have no differences, A2 and B2 are divided into the no-difference area.

[0013] Preferably, the division process of the difference area and the no-difference area further comprises: When the arrow B1 in the corresponding same vector position cannot be found on the second drawing paper, the drawing recognition identifies the smallest closed figure on the drawing where the arrow A1 is located, identifies the arrow that is different from A1 in the smallest closed figure and marks it as B1, annotates the corresponding text of the B1 arrow as B2, and performs a graphic and text comparison of A2 and B2; when it is judged that A2 and B2 have differences, A2 and B2 are divided into a secondary difference area, otherwise, when it is judged that A2 and B2 have no differences, A2 and B2 are divided into a no-difference area.

[0014] Preferably, the comparison processing of the pre-processed 3D model document in the step S4 specifically includes the following steps: Through the pre-processing of the 3D model document in the step S3, six views of the 3D model are obtained, and the six views of the two versions of the 3D model are vectorized and a virtual coordinate system is established in the six views; The absolute position of the drawing in the virtual coordinate system is adjusted, and the six views after the vectorization are stacked, and it is judged whether the two stacked six views have differences; when it is judged that the two six views have differences, the parts in the two versions of the 3D model are classified into the same category, otherwise, when it is judged that the two six views have no differences, the two six views are divided into a no-difference area; The six views of the parts in each corresponding position of the 3D model obtained after the above classification into the same category are compared individually, and it is judged whether the two stacked six views of the parts have differences; when it is judged that the two six views of the parts have differences, the different places are marked and the size information of the difference parts is marked through the virtual coordinate system and the vectorization, otherwise, when it is judged that the two six views of the parts have no differences, the two six views are divided into a no-difference area.

[0015] To achieve the above object, the application provides a graphic-text comparison system based on the same picture superposition analysis, which comprises: A document information acquisition module is configured to automatically acquire document information provided by a customer; A document type identification module is configured to identify the type of the document information provided by the customer, wherein the document information type at least includes a PDF image document and a 3D model document; and the PDF image document and the 3D model document each have a first version document and a second version document; A pre-processing module is configured to perform pre-processing according to the type of the document information identified by the document type identification module; A graphic-text comparison engine starting module is configured to start a graphic-text comparison engine, and perform coincidence comparison between the first version document and the second version document in the pre-processed PDF image document and / or 3D model document; A comparison report generation module is configured to automatically generate a comparison report of the PDF image document and / or the 3D model document according to the comparison result of the image-text comparison engine starting module.

[0016] To achieve the above object, the application provides a computer readable storage medium, comprising: a computer program stored therein, which, when executed by a processor, implements the steps of the image-text comparison method based on the same type of picture superimposition analysis according to any one of the above.

[0017] The technical scheme of the application has the following beneficial effects: In the PDF image document, the noise and background interference in the image can be effectively removed through grayscale processing, denoising and binarization processing, and the clear lines and text information are retained. At the same time, the self-adaptive threshold algorithm further enhances the contrast of the drawing lines, so that the subsequent image-text comparison is more accurate. By detecting the direction mark and calculating the rotation matrix and the scaling ratio, the coordinate system of all pages is uniformly adjusted to the standard direction, so that the positional deviation caused by the rotation or scaling of the drawing is avoided. Further, the text layer and the image layer are separated, the text part is converted into an editable text format, and the image part is retained as a high-resolution vector image, so that the comparison of the text and the image is more independent and clear, and the interference caused by the mixing of the text and the image is avoided.

[0018] In the 3D model document, the facet simplification algorithm and the Laplacian smoothing processing can eliminate the tiny noise points while retaining the geometric features of the model, so that misjudgment caused by the detail noise during comparison is avoided. At the same time, the facet simplification algorithm compresses the number of polygons and reduces the data volume while retaining the geometric features, so that the subsequent comparison calculation is more efficient, and the calculation time and resource consumption can be significantly reduced.

[0019] Through the preprocessing step, the PDF image document and the 3D model document are both converted into a standardized format and coordinate system. For example, all pages of the PDF image document are adjusted to the standard direction, and the coordinate system of the 3D model document is unified to the geometric center, so that the comparability between different versions of the document is ensured, and the comparison error caused by the inconsistency of the format or direction is avoided. At the same time, during the preprocessing process, the key features are extracted and retained. For example, in the PDF image document, the text part is converted into an editable text, and the image part is retained as a high-resolution vector image; in the 3D model document, the edge feature lines are extracted and a topology structure diagram is generated. The retention of the above-mentioned features enables the comparison to focus on the core content of the document, and improves the reliability of the comparison. Different versions of the document are converted into a consistent format and coordinate system, so that the comparison engine can efficiently perform the coincidence comparison and generate an accurate comparison result.

[0020] Through the dual mechanism of text information comparison and size annotation comparison, comprehensive and accurate comparison of PDF image documents is realized, which can not only detect the increase, decrease and modification of text, but also effectively avoid misjudgment caused by format differences, providing a solid guarantee for the accuracy of document content. Among them, as the basis, the editable text extracted by OCR technology is combined with the unified adjustment of font, font size and format in the preprocessing stage to ensure accurate identification and consistency of text content. While the size annotation comparison ensures the accurate preservation of the position and size information of the graphical elements through the high-resolution vector graphics generated in the preprocessing stage and the spatial registration technology, effectively avoiding production errors or design misunderstandings caused by size deviation.

[0021] By selecting three reference points on the surface of the three-dimensional model graph, the positions of the three-dimensional model graphs of the first version document and the second version document in the coordinate system are adjusted, which can effectively eliminate the spatial position differences caused by model translation, rotation or scaling, ensure the alignment of the geometric structures of the two versions of the model in the unified coordinate system, and provide a stable spatial reference basis for subsequent difference comparison, so that the comparison process can focus on the actual shape changes of the model (such as feature increase or decrease, size adjustment) rather than irrelevant displacement or attitude differences. At the same time, the strategy of selecting three reference points meets the minimum necessary condition for determining a unique position in three-dimensional space, which not only ensures the accuracy of positioning, but also avoids redundant calculation. By adjusting the position of the model in the coordinate system, the system can automatically compensate for the model placement differences caused by different design software operations or file export settings (such as origin offset or direction rotation), thereby limiting the comparison range to the geometric characteristics of the model itself. BRIEF DESCRIPTION OF DRAWINGS

[0022] Fig. 1 A flowchart of a graphic-text comparison method based on the same picture overlay analysis in an embodiment of the present application; Fig. 2 A flowchart of a PDF document comparison process of a graphic-text comparison method based on the same picture overlay analysis in an embodiment of the present application; Fig. 3 A flowchart of a 3D model document comparison process of a graphic-text comparison method based on the same picture overlay analysis in an embodiment of the present application. DETAILED DESCRIPTION

[0023] Embodiments of the present application are described below in detail with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations are used to denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below are exemplary and are intended to be illustrative of the present application, and are not to be construed as limiting the present application, all other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0024] In addition, if the description involving "first", "second" and the like in the present application is only for the purpose of description, such as for distinguishing the same or similar elements, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implying the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection required by the present application.

[0025] Referring to Figs. 1-3 The present application proposes a kind of based on the method for comparing text and graphics of same picture superposition analysis, comprising the following steps: S1, automatically obtaining the document information provided by customer; S2, the type of document information provided by customer is identified, and the type of document information at least includes: PDF image document and 3-dimensional model document;And PDF image document and 3-dimensional model document all have a first version document and a second version document; S3, pre-processing according to the type of document information identified in step S2;Wherein, the pre-processing of PDF image document is specifically: the scanned image in PDF image document is carried out gray scale, denoising, binarization processing, and the contrast of drawing line is enhanced by self-adaptive threshold algorithm;Extract the text layer and image layer in PDF, convert the text part into editable text format, and keep the image part as high-resolution vector diagram;By detecting the direction mark in drawing frame or title bar, the rotation matrix and scaling ratio are calculated, and the coordinate system of all pages is uniformly adjusted to standard direction; The pre-processing of 3-dimensional model document is specifically: the number of polygons is compressed using patch simplification algorithm, the data volume is reduced while the geometric characteristics are retained;The model principal axis direction is extracted by principal component analysis (PCA), the model origin is reset to the geometric center and the coordinate axis direction is unified;Laplacian smoothing is carried out on the model surface to eliminate small noise points, and the edge feature line is extracted and the topological structure diagram is generated;Standard six views are automatically generated based on model three-dimensional coordinates, and saved as vectorized SVG format; S4, start the graphic-text comparison engine to superimpose and compare the first version document and the second version document in the PDF image document and / or the 3D model document after the preprocessing in step S3; Specifically, the standard six views generated in the preprocessing of the 3D model document include: Front view: observed from the front of the model, used to show the height and width of the model.

[0026] Rear view: observed from the back of the model, used to show the back features, height and width of the model.

[0027] Left view: observed from the left side of the model, used to show the depth and height of the model.

[0028] Right view: observed from the right side of the model, used to show the right features, depth and height of the model.

[0029] Top view: observed from the top of the model, used to show the width and depth of the model.

[0030] Bottom view: observed from the bottom of the model, used to show the bottom features, width and depth of the model.

[0031] S5, according to the comparison result of step S4, automatically generate a comparison report of the PDF image document and / or the 3D model document.

[0032] In this embodiment, in the PDF image document, through grayscale, denoising and binarization processing, the noise and background interference in the image can be effectively removed, and the clear line and text information is retained. At the same time, the adaptive threshold algorithm further enhances the contrast of the drawing lines, making the subsequent graphic-text comparison more accurate. And by detecting the direction mark and calculating the rotation matrix and scaling ratio, the coordinate system of all pages is adjusted to the standard direction, avoiding the position deviation caused by the rotation or scaling of the drawing. Further, the text layer and the image layer are separated, the text part is converted into editable text format, and the image part is kept as high-resolution vector graphics, making the comparison of text and image more independent and clear, avoiding the interference caused by the mixing of text and image.

[0033] In the 3D model document, the facet simplification algorithm and Laplacian smoothing processing can eliminate small noise points while retaining the geometric features of the model, ensuring that there is no false judgment due to detail noise during comparison. At the same time, the facet simplification algorithm compresses the number of polygons, reduces the data volume, and retains the geometric features, making the subsequent comparison calculation more efficient and significantly reducing the calculation time and resource consumption.

[0034] In addition, through the preprocessing step, both the PDF image document and the 3D model document are converted into a standardized format and coordinate system. For example, all pages of the PDF image document are adjusted to a standard orientation, and the coordinate system of the 3D model document is unified to the geometric center, ensuring comparability between different versions of the document and avoiding comparison errors due to inconsistent formats or orientations. At the same time, during the preprocessing process, key features are extracted and preserved. For example, in the PDF image document, the text part is converted into editable text, and the image part is preserved as a high-resolution vector graph; in the 3D model document, the edge feature lines are extracted and a topology structure graph is generated. The preservation of the above-mentioned features enables the focus to be on the core content of the document during comparison, improving the reliability of the comparison. And converting different versions of the document into a consistent format and coordinate system enables the comparison engine to efficiently perform coincidence comparison and generate accurate comparison results.

[0035] In summary, the present application can effectively improve the accuracy and efficiency of document comparison, ensure the consistency and comparability of the document, and generate a reliable and easy-to-understand comparison report. Whether dealing with simple documents or complex PDF image documents and 3D model documents, this method can provide efficient and accurate comparison results to meet the needs of different scenarios.

[0036] In one embodiment, the preprocessing of the PDF image document in step S3 further includes: extracting the text layer parts of the first version document and the second version document in the PDF image document through OCR technology, and adjusting the text layer parts of the two documents to the same font, font size, and format; Further, the OCR recognition result of the text layer is spatially registered with the vector graph of the image layer to generate a standardized layer containing superimposed text and image information; In this embodiment, the text layer parts of the first version document and the second version document in the PDF image document are extracted through OCR technology, and the text layer parts of the two documents are adjusted to the same font, font size, and format, which can further eliminate comparison interference that may be caused by differences in text style. During text comparison, if the font, font size, or format of the text is inconsistent, even if the content is the same, it may cause the comparison result to deviate, as if it is a different element. After adjusting the text layer to a uniform format, the text can be more accurately matched based on the content during comparison, rather than being affected by the external form.

[0037] Furthermore, the OCR recognition results of the text layer are spatially aligned with the vector graphics of the image layer to ensure that the text and the corresponding image positions are precisely aligned. Therefore, when comparing two versions of the document, the relative position relationship between the text and the image is accurately considered to avoid misjudgment due to position deviation. At the same time, a standardized layer containing image and text overlay information is generated, providing a unified and standardized basis for subsequent image and text comparison, ensuring that in the standardized layer, the text and image are processed as a whole, allowing the comparison engine to compare the two versions of the document more comprehensively and systematically. For example, when detecting updated content in a document, the comparison engine can quickly locate the area where the text content has changed and whether the related image elements are affected based on the standardized layer. At the same time, for image modifications, it can also accurately determine whether the changes involve changes to the parts related to the text annotations. This greatly improves the precision and accuracy of image and text comparison, allowing the comparison results to more realistically reflect the actual changes in the document.

[0038] In one embodiment, the overlap comparison of the pre-processed PDF image document in step S4 includes at least: text information comparison and dimension marking comparison. The text information comparison is used to identify and compare the text content in the PDF image document to ensure the consistency and accuracy of the text; and the dimension marking comparison is used to check the size and position of various graphics, charts or design elements in the document to ensure that they are consistent with the original document or standard specifications.

[0039] In this embodiment, a comprehensive and accurate comparison of PDF image documents is achieved through the dual mechanisms of text information comparison and dimension annotation comparison. This not only detects the addition, deletion, and modification of text, but also effectively avoids misjudgments due to format differences, providing a solid guarantee for the accuracy of the document content. Text information comparison serves as the basis, utilizing editable text extracted using OCR technology, combined with the font, size, and format uniformly adjusted in the preprocessing stage, to ensure accurate recognition and consistent verification of text content. Dimension annotation comparison, on the other hand, uses high-resolution vector graphics generated in the preprocessing stage, combined with spatial registration technology, to ensure that the position and size information of graphic elements are accurately preserved, effectively avoiding production errors or design misunderstandings caused by dimensional deviations.

[0040] In one embodiment, the text information comparison includes the following steps: Obtaining text layer information of the first version document and the second version document preprocessed in step S3; The text layer information of the first version document and the second version document are stacked together to determine whether the text information of the two documents completely matches, and the non-matching text content and its position are marked.

[0041] Further, for the unmatched literal content, semantic similarity analysis is performed to determine whether it is a synonym replacement or a minor expression difference. If so, it is considered as a match; otherwise, it is recorded as a substantial difference.

[0042] The literal layer information of the first version document and the second version document is stacked to determine whether the literal information of the two documents is completely matched. Specifically, a preset text comparison algorithm is used to determine whether the literal information of the two documents is completely matched, and the unmatched literal content and its position are marked.

[0043] The preset text comparison algorithm can be implemented in various ways, such as a character encoding-based exact matching algorithm, a word frequency-based similarity calculation method, or a cosine similarity calculation.

[0044] In this embodiment, by stacking and comparing the preprocessed literal layer information of the first version document and the second version document, the differences in literal content can be accurately located, effectively avoiding false positives caused by format or layout changes. Combined with semantic similarity analysis, the system can not only identify direct literal modifications, but also intelligently judge synonym replacements or minor expression differences, thereby more accurately assessing the substantial changes in document content and improving the accuracy and flexibility of literal information comparison, ensuring that the comparison results not only contain explicit modification points, but also take into account the diversity of language expression.

[0045] Further, by using a preset text comparison algorithm, such as a character encoding-based exact matching, a word frequency-based similarity calculation, or a cosine similarity calculation, the comparison process is not only efficient but also reliable, thereby improving the degree of automation of the comparison and enhancing the system's adaptability to complex text changes.

[0046] In addition, by marking the unmatched literal content and its position, users can intuitively see the specific differences in the document, facilitating subsequent review and processing. At the same time, the introduction of semantic similarity reduces false positives caused by different expression methods, making the comparison results more in line with actual needs and improving the practicality and user satisfaction of document comparison.

[0047] In one embodiment, the size standard comparison includes the following steps: Obtaining the standardized graphic layer containing graphic-text superimposed information in the preprocessed first version document and the second version document; Stacking the standardized graphic layers in the first version document and the second version document to determine whether there are differences in size annotations between the two documents, and dividing the standardized graphic layer into a secondary difference area and a no difference area according to the determination result; The division process of the secondary difference area and the no difference area is as follows: The standardized layers containing graphic and superimposed information in the two documents are vectorized, then an arrow used for size marking in the secondary difference area is marked on the vector position A1 on the first edition document drawing, the arrow A1 is found according to the arrow A1, the arrow B1 on the same vector position on the second edition drawing is found according to the arrow A1, then the arrow B1 is found according to the arrow B1, the arrow A2 and the arrow B2 are compared in graphics and texts, when it is judged that the arrow A2 and the arrow B2 have differences, the arrow A2 and the arrow B2 are divided into the secondary difference area, otherwise, when it is judged that the arrow A2 and the arrow B2 have no differences, the arrow A2 and the arrow B2 are divided into the no-difference area.

[0048] In the embodiment, the arrow of size marking and the text annotation are associated and compared, the subtle differences (such as numerical deviation, unit change or marking position deviation) in size marking between the two documents can be accurately recognized, and the misjudgment caused by layer superimposition in the traditional pixel comparison can be avoided. Meanwhile, the differences are divided into the secondary difference area and the no-difference area, which is helpful to distinguish the key size change (such as tolerance adjustment in the engineering drawing) and the non-key format difference (such as font change of the marked text), and the priority processing efficiency of subsequent audit or revision is improved. The vector position mapping (automatic association of A1→A2 and B1→B2) realizes the rapid pairing of the marked content, and reduces the workload of manual item-by-item checking.

[0049] In one of the embodiments, the division process of the secondary difference area and the no-difference area further includes: When the arrow B1 on the corresponding same vector position on the second edition drawing cannot be found, the smallest closed figure on the drawing where the arrow A1 is located is recognized, the arrow different from A1 in the smallest closed figure is recognized and marked as B1, the text annotation corresponding to the arrow B1 is B2, the arrow A2 and the arrow B2 are compared in graphics and texts, when it is judged that the arrow A2 and the arrow B2 have differences, the arrow A2 and the arrow B2 are divided into the secondary difference area, otherwise, when it is judged that the arrow A2 and the arrow B2 have no differences, the arrow A2 and the arrow B2 are divided into the no-difference area.

[0050] In this embodiment, when the arrow position is offset in the second version document due to design adjustment (such as size line rearrangement), by identifying the smallest closed figure (such as rectangular, circular, etc. geometric boundary) where the arrow A1 is located, the corresponding arrow B1 in the region is automatically matched, avoiding the contrast failure caused by local coordinate changes, and enhancing the stability of difference detection in version iteration. At the same time, the introduction of the smallest closed figure implies the use of the topological relationship of the drawing (such as the size mark is usually associated with a specific geometric feature), which ensures that the matched arrow B1 and A1 belong to the same structure in function (such as marking the diameter of a certain hole position), rather than simply relying on the coordinate position, reducing the risk of mismatch. And for the arrow that cannot be matched directly, it is preferred to search for a substitute mark in the adjacent closed area, rather than directly marked as "add / delete", which is more in line with the gradual revision logic of engineering drawings, reducing the noise of difference report.

[0051] In one embodiment, the preprocessing of the 3D model document in step S3 further includes: At least three points on the surface of the 3D model are selected as reference points, and the positions of the 3D model graphs of the first version document and the second version document in the coordinate system are adjusted by the three reference points selected above.

[0052] In this embodiment, by selecting three reference points on the surface of the three-dimensional model, the positions of the three-dimensional model graphs of the first version document and the second version document in the coordinate system are adjusted, which can effectively eliminate the spatial position difference caused by model translation, rotation or scaling, ensure that the geometric structures of the two version models are aligned in the unified coordinate system, and provide a stable spatial reference basis for subsequent difference comparison, so that the comparison process can focus on the actual shape change of the model (such as feature addition or reduction, size adjustment) rather than irrelevant displacement or posture difference.

[0053] In addition, the strategy of selecting three reference points meets the minimum necessary condition for determining a unique position in three-dimensional space, which not only ensures the accuracy of positioning, but also avoids redundant calculation. By adjusting the position of the model in the coordinate system, the system can automatically compensate for the model placement difference (such as origin offset or direction rotation) caused by different design software operations or file export settings, so as to limit the comparison range to the geometric properties of the model itself.

[0054] In one embodiment, the preprocessing of the 3D model document in step S3 further includes: Drawing comparison, drawing comparison is used to compare the pre-processed PDF image document with the original drawing or standard drawing one by one, to ensure the integrity, accuracy and consistency of the drawing content. Specifically, drawing comparison can include: graphic element comparison: check whether the line, symbol, annotation and other graphic elements in the drawing are consistent with the original drawing, to ensure that there is no omission or error; dimension mark comparison: check whether the dimension mark in the drawing is accurate and consistent with the original drawing or design specification; text information comparison: compare the text description, annotation and other content in the drawing, to ensure that the text information is complete and correct; layout structure comparison: check whether the overall layout, proportion and typesetting of the drawing meet the requirements, to avoid misunderstanding or deviation due to format problems.

[0055] In one embodiment, the comparison processing of the pre-processed 3D model document in step S4 specifically includes the following steps: Through the preprocessing of the 3D model document in step S3, six views of the 3D model are obtained, and the six views of the two versions of the 3D model are vectorized and a virtual coordinate system is established in the six views; Adjust the absolute position of the drawing in the virtual coordinate system and stack the vectorized six views, and determine whether the two stacked six views have differences. When it is determined that the two six views have differences, the parts in the two versions of the 3D model are classified into the same category, otherwise, when it is determined that the two six views have no differences, the two six views are divided into the no-difference area; The six views of the parts in each corresponding position of the 3D model obtained after the above classification are compared separately, and it is determined whether the two stacked six views of the parts have differences. When it is determined that the two six views of the parts have differences, the different places are marked and the size information of the difference part is marked through the virtual coordinate system and vectorization, otherwise, when it is determined that the two six views of the parts have no differences, the two six views are divided into the no-difference area.

[0056] In this embodiment, based on the six views (front view, top view, left view, etc.) of the three-dimensional model, the spatial three-dimensional problem is decomposed into independent analysis of multiple plane views. Through vectorization processing, the line, symbol and other graphic elements in the drawing are converted into mathematical vectors that can be accurately calculated, avoiding the error of pixel-level comparison, while preserving the geometric topological relationship. At the same time, the absolute position of the six views is adjusted in the unified coordinate system, to ensure that the views of the two versions of the model are strictly aligned in space. After stacking, the system quickly locates the difference area (such as newly added contour line or missing annotation) through vector overlap analysis, and uses the graphic topological relationship (such as line connectivity, closed area) to exclude non-key format differences (such as line width change).

[0057] Further, when the overall six views are different, the system further disassembles the model into similar parts (such as bolts, bearings, etc.), and classifies them through part library feature matching. The six views of each part are compared separately, and the positioning information of the virtual coordinate system is combined to accurately mark the size changes (such as an increase of 0.5 mm in shaft diameter) or shape modifications (such as adjustment of the chamfer angle) of the difference parts, avoiding redundant calculations of global comparison.

[0058] The application further provides a picture-text comparison system based on the same picture superposition analysis, comprising: A document information acquisition module is configured to automatically acquire document information provided by a customer. A document type identification module is configured to identify the type of the document information provided by the customer, wherein the document information type at least includes a PDF image document and a 3D model document, and the PDF image document and the 3D model document each have a first version document and a second version document. A preprocessing module is configured to perform preprocessing according to the type of the document information identified by the document type identification module, wherein the preprocessing for the PDF image document specifically includes: performing gray-scale, denoising and binarization processing on the scanned images in the PDF image document, and enhancing the contrast of the lines of the drawing through an adaptive threshold algorithm; extracting the text layer and the image layer in the PDF, converting the text part into an editable text format, and keeping the image part as a high-resolution vector diagram; calculating a rotation matrix and a scaling ratio by detecting the direction marks in the drawing frame or the title bar, and uniformly adjusting the coordinate system of all pages to a standard direction; the preprocessing for the 3D model document specifically includes: compressing the number of polygons by using a facet simplification algorithm, retaining geometric features while reducing data volume; resetting the model origin to the geometric center and unifying the coordinate axis direction by extracting the model principal axis direction through principal component analysis (PCA); performing Laplacian smoothing on the model surface to eliminate small noise points, extracting edge feature lines and generating a topological structure diagram at the same time; automatically generating standard six views based on the three-dimensional coordinates of the model, and saving them in a vectorized SVG format; A picture-text comparison engine starting module is configured to start the picture-text comparison engine, and perform coincidence comparison between the first version document and the second version document in the preprocessed PDF image document and / or 3D model document. A comparison report generation module is configured to automatically generate a comparison report of the PDF image document and / or 3D model document according to the comparison result of the picture-text comparison engine starting module.

[0059] The application further provides a computer readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image-text contrast method based on the same picture superposition analysis. The computer program, when executed, can include the processes of the above respective embodiments of the image-text contrast method based on the same picture superposition analysis. Any reference to memory, storage, databases, or other media in this application and in the embodiments refers to both non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), or external cache memory. As an illustration and not a limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0060] It should be noted that in this document, the terms "comprise", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, device, article, or image-text contrast method based on the same picture superposition analysis that includes a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, device, article, or image-text contrast method based on the same picture superposition analysis. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of another identical element in the process, device, article, or image-text contrast method based on the same picture superposition analysis that includes the element.

[0061] The above description is only the preferred embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for comparing images and texts based on the analysis of overlapping images of the same type, characterized in that: The following steps are involved: S1. Automatically obtain document information provided by customers; S2. Identify the type of document information provided by the customer, where the document information type includes at least: a PDF image document and a 3D model document; and each of the PDF image document and the 3D model document has a first version document and a second version document; S3. Preprocessing according to the document information type identified in step S2; S4. Start the image-text comparison engine to compare the first version of the PDF image document and / or the second version of the 3D model document pre-processed in step S3; S5. Automatically generate a comparison report of the PDF image document and / or the 3D model document based on the comparison result of step S4; Among them, the preprocessing of the PDF image document is specifically as follows: grayscale, denoising, and binarization are performed on the scanned image in the PDF image document, and the contrast of the drawing lines is enhanced through an adaptive threshold algorithm; the text layer and image layer in the PDF are extracted, the text part is converted into an editable text format, and the image part is retained as a high-resolution vector diagram; by detecting the direction mark in the drawing border or title bar, the rotation matrix and scaling ratio are calculated, and the coordinate system of all pages is uniformly adjusted to the standard direction; the preprocessing of the 3D model document is specifically as follows: the number of polygons is compressed using a patch simplification algorithm, retaining geometric features while reducing the amount of data; the main axis direction of the model is extracted through principal component analysis (PCA), the model origin is reset to the geometric center and the coordinate axis direction is unified; Laplacian smoothing is performed on the model surface to eliminate tiny noise points, and edge feature lines are extracted and a topological structure diagram is generated; standard six views are automatically generated based on the three-dimensional coordinates of the model, and saved in vectorized SVG format.

2. The image-text comparison method based on similar image stacking analysis according to claim 1, characterized in that: The pre-processing of the PDF image document in step S3 further includes: Extract the text layer of the first and second versions of the PDF image document using OCR technology, and adjust the text layer of the two documents to the same font, size, and format; Perform spatial registration between the OCR recognition results of the text layer and the vector graphics of the image layer to generate a standardized layer containing text and image overlay information; The pre-processing of the 3D model document in step S3 further includes: At least three points on the surface of the 3D model diagram are selected as reference points, and the positions of the 3D model diagrams of the first version document and the second version document in the coordinate system are adjusted based on the three selected reference points.

3. The image-text comparison method based on similar image stacking analysis according to claim 1, characterized in that: The overlap comparison of the pre-processed PDF image document in step S4 at least includes: text information comparison and dimension comparison; the text information comparison is used to identify and compare the text content in the PDF image document to ensure the consistency and accuracy of the text; and the dimension comparison is used to check the size and position of various graphics, charts or design elements in the document to ensure that they comply with the original document or standard specifications.

4. The image-text comparison method based on similar image stacking analysis according to claim 3, characterized in that: The text information comparison includes the following steps: Obtaining text layer information of the first version document and the second version document preprocessed in step S3; The text layer information of the first version document and the second version document are stacked together to determine whether the text information of the two documents completely matches, and the non-matching text content and its position are marked.

5. The image-text comparison method based on similar image stacking analysis according to claim 4, characterized in that: The text information comparison further includes the following steps: For unmatched text content, semantic similarity analysis is performed to determine whether it is a synonym replacement or a minor difference in expression. If so, it is considered a match; otherwise, it is recorded as a substantial difference.

6. The image-text comparison method based on similar image stacking analysis according to claim 3, characterized in that: The size standard comparison includes the following steps: Obtaining standardized layers containing image and text overlay information in the first version document and the second version document preprocessed in step S3; The standardized layers in the first and second versions of the document are stacked together to determine whether there are any differences in the dimension annotations of the two documents, and based on the determination results, a minor difference area and a no-difference area are divided on the standardized layers; The process of dividing the secondary difference area and the indifference area is specifically as follows: vectorizing the standardized layers containing graphic and text overlay information in the two documents, then marking the arrow used to identify the dimension annotation in the secondary difference area at the vector position A1 on the first version of the document drawing, finding the text annotation A2 corresponding to the arrow A1 according to the arrow A1, and correspondingly finding the arrow B1 at the same vector position on the second version of the drawing, then finding the text annotation B2 corresponding to the arrow B1 according to the arrow B1, and comparing A2 and B2 in graphics and text; when it is judged that A2 and B2 have differences, A2 and B2 are divided into the secondary difference area, otherwise when it is judged that A2 and B2 have no differences, A2 and B2 are divided into the indifference area.

7. The image-text comparison method based on similar image stacking analysis according to claim 6, characterized in that: The process of dividing the sub-difference area and the indifference area also includes: When the arrow B1 at the same vector position cannot be found on the second version of the drawing, the minimum closed figure on the drawing where the arrow A1 is located is identified, the arrow in the minimum closed figure that is different from A1 is identified and marked as B1, the text corresponding to the arrow B1 is annotated as B2, and A2 and B2 are compared in graphics and text; when it is determined that A2 and B2 have differences, A2 and B2 are divided into the secondary difference area, otherwise when it is determined that A2 and B2 have no differences, A2 and B2 are divided into the no difference area.

8. The image-text comparison method based on similar image stacking analysis according to claim 1, characterized in that: The comparison processing of the pre-processed 3D model document in step S4 specifically includes the following steps: By pre-processing the 3D model document in step S3, six views of the 3D model are obtained, the six views of the two versions of the 3D model are vectorized, and a virtual coordinate system is established in the six views; Adjust the absolute position of the drawing in the virtual coordinate system and stack the six vectorized views to determine whether there are any differences between the two stacked six views. If the two six views are different, the parts in the two versions of the 3D model are classified as similar items. Otherwise, if the two six views are not different, the two six views are classified into the indifference area. The six views of the part at each corresponding position in the 3D model obtained after the above classification of similar items are compared separately to determine whether there is any difference between the two stacked six views of the part; when it is determined that the two six views of the part have differences, the different places are marked and the size information of the different parts is marked through a virtual coordinate system and vectorization; conversely, when it is determined that the two six views of the part have no differences, the two six views are divided into the indifference area.

9. A picture-text comparison system based on similar picture stacking analysis, characterized in that: include: Document information acquisition module, used to automatically obtain document information provided by customers; A document type identification module is used to identify the type of document information provided by the customer, wherein the document information type includes at least: a PDF image document and a 3D model document; and each of the PDF image document and the 3D model document has a first version document and a second version document; A preprocessing module, used for preprocessing the document information according to the document type identified by the document type identification module; An image-text comparison engine startup module is used to start the image-text comparison engine and compare the first version document and the second version document in the pre-processed PDF image document and / or 3D model document; The comparison report generation module is used to automatically generate a comparison report of a PDF image document and / or a 3D model document according to the comparison result of the image-text comparison engine startup module.

10. A computer-readable storage medium, characterized in that include: A computer program is stored, and when the program is executed by a processor, the steps of the image-text comparison method based on similar image stacking analysis as described in any one of claims 1 to 8 are implemented.

Images