Intelligent Segmentation and Localization Method, Device, Equipment and Medium for Multi-Page Image Comparison

Abstract

The present invention discloses a multi-page image comparison intelligent segmentation and positioning method, device, equipment and medium, which relates to the technical field of electrical digital data processing. After determining the one-to-one correspondence between multiple first images to be subjected to image comparison and multiple second images, the method first determines a first comparison region positioning frame for each page of the first images and used for positioning the corresponding image comparison target region based on PDF file information, then uses this frame to intercept the first comparison region image and determines a first minimum rectangle frame surrounding the text in the figure. At the same time, a second minimum rectangle frame surrounding the text in the figure is determined for each page of the second images. Then, based on the above three frames, a second comparison region positioning frame and a second comparison region image for each page of the second images and also used for positioning the image comparison target region are determined. Finally, two region images suitable for image comparison are scaled, so as to improve the positioning efficiency of the comparison region and ensure the correctness of the subsequent image comparison result.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and particularly relates to a method, device, equipment and medium for intelligent segmentation and positioning of multi-page image comparison. Background Art

[0002] In the processes of editing, imposition, halftone rasterization, and printing of the original file to be printed for a customer, errors such as misidentifying the original file to be printed, changing areas that should not be modified, pixel loss in the file, incorrect loading of small-page files during imposition, pixel loss in the layers of small-page files, and incorrect graphic interpretation during rasterization may occur. In addition, there are also printing problems caused by plate materials or printing presses, etc. Therefore, after any step is completed, it is necessary to compare the two image files before and after the step to timely detect problems or errors in the latest completed step. For example, after finally obtaining the printed product corresponding to the original file to be printed, it is necessary to compare the scanned image file of the printed product with the original file to be printed imported into the printing press to finally confirm whether there are printing problems caused by plate materials or printing presses, etc. during the printing step.

[0003] During the image comparison process, positioning the image comparison area for the two image files to be compared is a crucial step. If the positioning is inaccurate, it will directly affect the correctness of the subsequent image comparison result. Currently, in existing image comparison solutions, mainly manual drawing is used to mark the image comparison area of the two image files, and then based on the marked area, two images are cropped from the two image files for image comparison to obtain the final image comparison result. However, this manual positioning method obviously has the problems of low efficiency and easy human error, and is particularly unsuitable for positioning the image comparison area of multi-page image files. Therefore, how to automatically position the comparison area during the image comparison process of multi-page image files to improve the positioning efficiency of the comparison area and ensure the correctness of the subsequent image comparison result is an urgent research topic for those skilled in the art. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, device, computer equipment and computer-readable storage medium for intelligent segmentation and positioning of multi-page image comparison, aiming to solve the problems of low efficiency and easy error caused by the need to manually determine the comparison area in existing image comparison solutions for multi-page image files, especially for the intelligent precise segmentation and accurate positioning of multi-page imposition images.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] In the first aspect, a method for intelligent segmentation and positioning of multi-page image comparison is provided, including:

[0007] Obtain a first image file and a second image file to be subjected to image comparison. Among them, the first image file is an electronic file in PDF format or non-PDF format and contains a first image with multiple pages or a multi-page layout of collated plates, and the second image file contains a second image with multiple pages or a multi-page layout of collated plates;

[0008] Determine the one-to-one correspondence or logical layout relationship between the multi-page first image and the multi-page second image;

[0009] For each page of the first image in the first image file, read the PDF file information of the first image file, and determine a corresponding first comparison region positioning frame for positioning the corresponding image comparison target region according to the PDF file printing requirement attributes in the PDF file information. Among them, the image comparison target region is an image region surrounded by a media box, a crop box, a bleed box, a trim box, or a work box; Read the region positioning frame of the second image. When the second image is a typeset and collated image with a multi-page layout, the positioning region is implemented according to the image region surrounded by the media box, the crop box, the bleed box, the trim box, or the work box;

[0010] For each page of the first image, use the corresponding first comparison region positioning frame to intercept a corresponding first comparison region image from the corresponding image;

[0011] For each page of the first image, import the corresponding first comparison region image into a trained recognition model, output corresponding first strings corresponding to each line of text in the corresponding first comparison region image, and determine a corresponding first minimum rectangle according to the first strings of each line. The first minimum rectangle refers to a rectangle with the smallest area that is used to enclose each line of text in the corresponding first comparison region image and has a mapping relationship with the first strings of each line;

[0012] For each page of the second image in the second image file, import the corresponding image into the recognition model, output corresponding second strings corresponding to each line of text in the corresponding image, and determine a corresponding second minimum rectangle according to the second strings of each line. The second minimum rectangle refers to a rectangle with the smallest area that is used to enclose each line of text in the corresponding image and has a mapping relationship with the second strings of each line;

[0013] For each page of the second image, determine a corresponding second comparison region positioning frame for positioning the corresponding image comparison target region according to the corresponding second minimum rectangle and the first minimum rectangle and the first comparison region positioning frame of the corresponding first image;

[0014] For each second image of the pages, use the corresponding second comparison area positioning frame to intercept the corresponding second comparison area image from the corresponding image;

[0015] For each pair of first image and second image with a mapping relationship in the first image file and the second image file, scale the corresponding first comparison area image or the second comparison area image so that the corresponding first comparison area image and the second comparison area image have the same size and resolution.

[0016] In a possible design, after determining the one-to-one correspondence between the multi-page first images and the multi-page second images, the method further includes: for each first image in the multi-page first images, use optical character recognition technology to extract the corresponding page content text from the corresponding image, and count the number of words in the corresponding page content text;

[0017] For each second image in the multi-page second images, use the optical character recognition technology to extract the corresponding page content text from the corresponding image, and count the number of words in the corresponding page content text;

[0018] Arrange the first images of the pages in descending order of the number of words to obtain a first image queue, and also arrange the second images of the pages in descending order of the number of words to obtain a second image queue;

[0019] For each pair of first image and second image with the same serial number in the first image queue and the second image queue, if the corresponding two images do not have a mapping relationship in the one-to-one correspondence between the multi-page first images and the multi-page second images, and the corresponding two page content texts have the characteristic of content consistency, then output and display a reminder message for indicating that there is a mapping relationship mismatch between the corresponding two images, so as to remind the staff to conduct manual inspection and correct the mapping relationship.

[0020] In a possible design, when the first image file is a small-page file, directly read the media box, crop box, bleed box, trim box or work area box to locate the comparison area, read the PDF information of the image file, extract the coordinates of the media box, crop box, bleed box, trim box or work area box, and directly locate the image comparison target area according to the extracted box coordinates. When the image file is a large-format imposition file, use the following precise segmentation method: if the large-format image contains its own trim lines, identify the trim lines in the large-format image, and use the area within each small-page trim line as the precise segmentation area. If it does not contain its own trim lines, use the imposition folding reverse parsing technology to determine the logical order and position of each small page. If it does not contain trim lines, the specific steps of manual segmentation are:

[0021] S31. Specify the boundaries of each small page through interface operations and manually specify some imposition parameters so that the system can assist in splitting more accurately. The imposition parameters are as follows: Image size: The system automatically reads the length and width of the image to determine the overall area of the large file; Number of imposition pages: The user specifies the number of pages for imposition according to the actual situation (such as imposition of 8 pages, 16 pages, etc.); Finished product size: The user specifies the finished product size of each small page (such as 185mm * 260mm), which is the size of the cutting frame.

[0022] S32. Calculate the sizes of the bleed frame and the media frame. The system automatically calculates the size of the media frame or the bleed frame according to the finished product size specified by the user and the built-in bleed size (such as 3mm).

[0023] S33. Built-in imposition model. The system has a built-in imposition model that contains necessary dimensional parameters in the printing process, such as gripper edge size, gutter size, column gutter size, etc. These dimensional parameters are used to ensure that the small page files after splitting meet the actual printing requirements.

[0024] S34. Manually adjust the image and the layout. The user manually draws or selects the boundaries of each small page on the interface provided by the system according to the actual situation of the image and the imposition logic. The system automatically adjusts the layout of each page file according to the boundaries specified by the user and the imposition model to center it on the overall size. If necessary, the user can also manually adjust the layout according to the gripper edge data to ensure that the layout dimensions of each page basically conform to the original data of the imposition logic. After completing the manual splitting and layout adjustment, the system accurately splits the large format file into multiple small page files according to the user's specification.

[0025] In a possible design, for each page of the second image in the second image file, import the corresponding image into the variable-length text recognition model, output the corresponding rows of second strings that correspond one-to-one with the rows of text in the corresponding image, and determine the corresponding second minimum rectangle according to the rows of second strings, including:

[0026] For a certain page of the second image in the second image file, import the corresponding image into the variable-length text recognition model, and output the corresponding rows of second strings that correspond one-to-one with the rows of text in the corresponding image;

[0027] According to the rows of first strings of a certain page of the first image that has a mapping relationship with the certain page of the second image, determine at least one row of strings from the rows of second strings that has the characteristic of content consistency with the rows of first strings;

[0028] Determine the second smallest rectangular box of the second image of a certain page according to the at least one line of strings, where the second smallest rectangular box refers to the rectangular box with the smallest area for enclosing at least one line of text having a mapping relationship with the at least one line of strings within the second image of the certain page.

[0029] In a possible design, for each second image of the pages, determine the corresponding second comparison region positioning box for positioning the corresponding image comparison target region according to the corresponding second smallest rectangular box, the first smallest rectangular box of the corresponding first image, and the first comparison region positioning box, including:

[0030] For any second image in the second image file, determine the corresponding second comparison region positioning box for positioning the corresponding image comparison target region according to the corresponding second smallest rectangular box, the first smallest rectangular box of the corresponding first image, and the first comparison region positioning box according to the following formula:

[0031]

[0032] In the formula, x 21,btl represents the abscissa position of the lower left corner point of the second comparison region positioning box within the any second image, y 21,btl represents the ordinate position of the lower left corner point of the second comparison region positioning box within the any second image, x 21,tpr represents the abscissa position of the upper right corner point of the second comparison region positioning box within the any second image, y 21,tpr represents the ordinate position of the upper right corner point of the second comparison region positioning box within the any second image, x 22,btl represents the abscissa position of the lower left corner point of the second smallest rectangular box within the any second image, y 22,btl represents the ordinate position of the lower left corner point of the second smallest rectangular box within the any second image, x 22,tpr represents the abscissa position of the upper right corner point of the second smallest rectangular box within the any second image, y 22,tpr represents the ordinate position of the upper right corner point of the second smallest rectangular box within the any second image, x 12,btl represents the abscissa position of the lower left corner point of the first smallest rectangular box within the first image corresponding to the any second image, y 12,btl represents the ordinate position of the lower left corner point of the first smallest rectangular box within the first image corresponding to the any second image, x 12,tpr represents the abscissa position of the upper right corner point of the first smallest rectangular box within the first image corresponding to the any second image, y 12,tprRepresents the vertical coordinate position of the upper right corner point of the first minimum rectangular box within the first image corresponding to any one of the second images, x 11,btl Represents the horizontal coordinate position of the lower left corner point of the first comparison region positioning box within the first image corresponding to any one of the second images, y 11,btl Represents the vertical coordinate position of the lower left corner point of the first comparison region positioning box within the first image corresponding to any one of the second images, x 11,tpr Represents the horizontal coordinate position of the upper right corner point of the first comparison region positioning box within the first image corresponding to any one of the second images, y 11,tpr Represents the vertical coordinate position of the upper right corner point of the first comparison region positioning box within the first image corresponding to any one of the second images.

[0033] In a possible design, the variable-length text recognition model adopts a recognition model based on long short-term memory network and connectionist temporal classification LSTM+CTC, convolutional recurrent neural network CRNN, or the text recognition project chineseocr.

[0034] In a possible design, for the manual segmentation method, the system obtains the precise segmentation regions of each small page file obtained after manual segmentation. The system matches each precise segmentation region with a preset or user-specified positioning template, and determines whether the matching is successful by comparing the features of the regions. If a certain precise segmentation region matches the positioning template successfully, the system considers that the region has been precisely located. If there are regions that are not matched or matched inaccurately, the system prompts the user to make manual adjustments or re-segment.

[0035] In a second aspect, a multi-page image comparison intelligent positioning device is provided, including an image file acquisition unit, a correspondence determination unit, a first positioning box determination unit, a first region image interception unit, a first rectangular box determination unit, a second rectangular box determination unit, a second positioning box determination unit, a second region image interception unit, and an image scaling processing unit;

[0036] The image file acquisition unit is used to acquire a first image file and a second image file to be subjected to image comparison, where the first image file is an electronic file in PDF format and contains multiple pages of first images, and the second image file contains multiple pages of second images;

[0037] The correspondence determination unit is communicatively connected to the image file acquisition unit and is used to determine the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images;

[0038] The first positioning frame determination unit, communicatively connected to the image file acquisition unit, is configured to, for each page of the first images in the first image file, read the PDF file information of the first image file, and determine, according to the PDF file printing requirement attributes in the PDF file information, corresponding first comparison region positioning frames for positioning corresponding image comparison target regions, where the image comparison target region is an image region surrounded by a media frame, a crop frame, a bleed frame, a trim frame, or a work frame;

[0039] The first region image extraction unit, communicatively connected to the first positioning frame determination unit, is configured to, for each page of the first images, extract corresponding first comparison region images from the corresponding images by applying the corresponding first comparison region positioning frames;

[0040] The first rectangle determination unit, communicatively connected to the first region image extraction unit, is configured to, for each page of the first images, import the corresponding first comparison region images into a trained variable-length text recognition model, output corresponding first strings corresponding one-to-one to each line of text in the corresponding first comparison region images, and determine corresponding first minimum rectangles according to the first strings, where the first minimum rectangle is a rectangle with the smallest area for enclosing each line of text in the corresponding first comparison region image that has a mapping relationship with the first strings;

[0041] The second rectangle determination unit, communicatively connected to the image file acquisition unit, is configured to, for each page of the second images in the second image file, import the corresponding images into the variable-length text recognition model, output corresponding second strings corresponding one-to-one to each line of text in the corresponding images, and determine corresponding second minimum rectangles according to the second strings, where the second minimum rectangle is a rectangle with the smallest area for enclosing each line of text in the corresponding image that has a mapping relationship with the second strings;

[0042] The second positioning frame determination unit, communicatively connected to the correspondence determination unit, the first positioning frame determination unit, the first rectangle determination unit, and the second rectangle determination unit respectively, is configured to, for each page of the second images, determine corresponding second comparison region positioning frames for positioning corresponding image comparison target regions according to the corresponding second minimum rectangles and the first minimum rectangles and the first comparison region positioning frames of the corresponding first images;

[0043] The second region image intercepting unit, communicatively connected to the second positioning frame determining unit, is configured to intercept corresponding second comparison region images from the corresponding images for each page of the second images by applying the corresponding second comparison region positioning frame.

[0044] The image scaling processing unit is communicatively connected to the corresponding relationship determining unit, the first region image intercepting unit, and the second region image intercepting unit respectively, and is configured to scale the corresponding first comparison region image or the second comparison region image for each pair of first images and second images that have a mapping relationship in the first image file and the second image file, so that the corresponding first comparison region image and the second comparison region image have the same size and resolution.

[0045] In a third aspect, the present invention provides a computer device, including a memory, a processor, and a transceiver communicatively connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to send and receive messages, and the processor is configured to read the computer program and execute the multi-page image comparison intelligent precise segmentation and precise positioning method according to any one of claims 1 to 7.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are run on a computer, the multi-page image comparison intelligent precise segmentation and precise positioning method according to any one of claims 1 to 7 is executed.

[0047] Advantages of the above solution:

[0048] The present invention creatively provides a new solution capable of automatically positioning comparison regions during the image comparison process of multi-page image files. That is, after determining the one-to-one correspondence between multiple pages of first images and multiple pages of second images to be compared, first determine the first comparison region positioning frame for each page of the first images that is used to position the corresponding image comparison target region based on the PDF file information, then use this frame to intercept the first comparison region image and determine the first minimum rectangle frame enclosing the text in the figure. At the same time, determine the second minimum rectangle frame enclosing the text in the figure for each page of the second images, and then determine the second comparison region positioning frame and the second comparison region image for each page of the second images that are also used to position the image comparison target region based on the above three frames. Finally, scale to obtain two region images suitable for image comparison. In this way, the comparison region can be automatically positioned during the image comparison process of two multi-page image files, greatly improving the positioning efficiency of the comparison region, ensuring the correctness of the subsequent image comparison results, and facilitating practical application and promotion. Description of the Drawings

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0050] Figure 1 It is a schematic flowchart of the multi-page image comparison intelligent segmentation and positioning method provided by the embodiment of the present application.

[0051] Figure 2 It is an example diagram of a large-format paper after multi-page layout provided by the embodiment of the present application.

[0052] Figure 3 It is an example diagram of the multi-page layout result on both sides of a large-format paper provided by the embodiment of the present application. Among them, Figure 3 (a) shows an example diagram of the multi-page layout result on the front side of the large-format paper, Figure 3 (b) shows an example diagram of the multi-page layout result on the back side of the large-format paper.

[0053] Figure 4 It is an example diagram of the positional relationship among the cutting area, bleeding area, work area, and media area provided by the embodiment of the present application.

[0054] Figure 5 It is an example diagram of the positional relationship between the comparison area positioning frame and the minimum rectangle frame provided by the embodiment of the present application. Among them, Figure 5 (a) shows an example diagram of the positional relationship between the first comparison area positioning frame and the first minimum rectangle frame in the first image, Figure 5 (b) shows an example diagram of the positional relationship between the second comparison area positioning frame and the second minimum rectangle frame in the second image.

[0055] Figure 6 It is an example diagram of the third minimum rectangle frame in any grayscale image provided by the embodiment of the present application.

[0056] Figure 7 It is a schematic structural diagram of the multi-page image comparison intelligent positioning device provided by the embodiment of the present application.

[0057] Figure 8 It is a schematic structural diagram of the computer device provided by the embodiment of the present application. Detailed implementation manners

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these embodiments. It should be noted here that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation to the present invention.

[0059] It should be understood that although terms such as first and second etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another. For example, the first object can be called the second object, and similarly the second object can be called the first object, without departing from the scope of the exemplary embodiments of the present invention.

[0060] It should be understood that for the term "and / or" that may appear in this article, it is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, B exists alone, or A and B exist simultaneously, etc.; another example, A, B and / or C can mean any one of A, B and C or any combination of them; for the term " / and" that may appear in this article, it is a description of another association object relationship, indicating that there can be two relationships. For example, A / and B can mean: A exists alone or A and B exist simultaneously, etc.; in addition, for the character " / " that may appear in this article, generally it means that the front and back associated objects are an "or" relationship.

[0061] Embodiment

[0062] As Figure 1 shown, the multi-page image comparison intelligent segmentation and positioning method provided in the first aspect of this embodiment can, but is not limited to, be executed by a computer device with certain computing resources, such as a platform server, a personal computer (Personal Computer, PC, referring to a multi-purpose computer suitable for personal use in terms of size, price and performance; desktop computers, laptop computers, small laptop computers, tablet computers, and ultrabooks, etc. all belong to personal computers), a smart phone, a personal digital assistant (Personal Digital Assistant, PDA), or a wearable device, etc. As Figure 1 shown, the multi-page image comparison intelligent precise segmentation and precise positioning method can, but is not limited to, include the following steps S1 to S9.

[0063] S1. Obtain a first image file and a second image file to be subjected to image comparison, wherein the first image file is an electronic file in PDF (Portable Document Format) format and contains multiple pages of first images, and the second image file contains multiple pages of second images.

[0064] In the step S1, the first image file and the second image file are the two image files to be subjected to image comparison. Specifically, the first image file can be, but is not limited to, a multi-page electronic image file corresponding to a multi-page printed matter or a large-format electronic image file after multi-page imposition, etc. The second image file can be, but is not limited to, another multi-page electronic image file corresponding to the multi-page printed matter, another large-format electronic image file after multi-page imposition, a multi-page scanned image file, a large-format scanned image file after multi-page imposition, or a scanned image file obtained by double-sided scanning after folding, cutting, and edge-trimming a large-format printed matter after multi-page imposition. The aforementioned multi-page electronic image file or large-format electronic image file after multi-page imposition can directly be the original file to be printed by the customer, or can also be a new electronic image file obtained by conventional processing of the original file to be printed by the customer. The aforementioned scanned image file obtained by double-sided scanning after folding, cutting, and edge-trimming a large-format printed matter after multi-page imposition refers to an image file obtained by double-sided scanning of the multi-page paper obtained after folding, cutting, and edge-trimming a large-format paper printed with multi-page imposition (for example, as shown in Figure 2 Figure 2 , a large-format paper has 8 pages of content on one side) after folding, cutting, and edge-trimming, and can be obtained by double-sided scanning of the multi-page paper using, but not limited to, an existing sheet-fed scanner. The aforementioned large-format scanned image file after multi-page imposition refers to an image file obtained by directly scanning a large-format paper printed with multi-page imposition, and can be obtained by single-sided or double-sided scanning of the large-format paper printed with multi-page imposition using, but not limited to, an existing large scanner. In addition, the specific obtaining method of the first image file can be, but is not limited to, being imported by a staff member; the specific obtaining method of the second image file can be, but is not limited to, being imported by a staff member or being transmitted after scanning by a scanner; the file format of the second image file can be in PDF format or other file formats.

[0065] S2. Determine the one-to-one correspondence between the multiple pages of the first images and the multiple pages of the second images.

[0066] In step S2, specifically, determining the one-to-one correspondence between the multi-page first images and the multi-page second images includes, but is not limited to: when the page numbers of the multi-page first images are known, if the second image file is a multi-page scanned image file, then matching the scanning order of the multi-page second images with the page number order of the multi-page first images to obtain the one-to-one correspondence between the multi-page first images and the multi-page second images (for example, the first scanned image in the multi-page second images corresponds to the first page of the first images in the multi-page first images); and if the second image file is a multi-page large-format scanned image file after imposition, first determining the page number order of the multi-page second images based on the folding logic (i.e., common knowledge in the printing technology field, such as Figure 3 the multi-page layout result on both sides of a large-format paper and determined based on the folding requirements), and then matching the page number order of the multi-page second images with the page number order of the multi-page first images to obtain the one-to-one correspondence between the multi-page first images and the multi-page second images (for example, the last page of the second images in the multi-page second images corresponds to the last page of the first images in the multi-page first images); and if the second image file is a multi-page electronic image file (i.e., the other multi-page electronic image file or the other multi-page large-format electronic image file after imposition), the page number order of the multi-page second images can be directly obtained based on the order of the electronic images, and then the page number order of the multi-page second images is matched with the page number order of the multi-page first images to obtain the one-to-one correspondence between the multi-page first images and the multi-page second images. Since the first image file is a multi-page electronic image file, the page number order of the multi-page first images can be directly obtained based on the order of the electronic images. In addition, the multi-page first images and the multi-page second images need to be the same in terms of the number of pages. If they are not, the image comparison can be directly determined to fail.

[0067] In step S2, for the purpose of verifying the one-to-one correspondence between the multi-page first images and the multi-page second images, preferably, after determining the one-to-one correspondence between the multi-page first images and the multi-page second images, the method further includes, but is not limited to, the following steps S21 to S24.

[0068] S21. For each page of the first images in the multi-page first images, apply optical character recognition technology to extract the corresponding page content text from the corresponding image, and count the number of words in the corresponding page content text.

[0069] In the step S21, the optical character recognition technology is the existing OCR (Optical Character Recognition) technology. Therefore, the page content text can be routinely extracted from the image.

[0070] S22. For each second image in the multi-page second image, apply the optical character recognition technology to extract the corresponding page content text from the corresponding image, and count the number of words in the corresponding page content text.

[0071] S23. Arrange the first images of each page in descending order of the number of words to obtain a first image queue, and also arrange the second images of each page in descending order of the number of words to obtain a second image queue.

[0072] S24. For each pair of the first image and the second image with the same serial number in the first image queue and the second image queue, if the corresponding two images do not have a mapping relationship in the one-to-one correspondence between the multi-page first image and the multi-page second image, and the corresponding two page content texts have the characteristic of content consistency, then output and display a reminder message indicating a mapping relationship mismatch for the corresponding two images, so as to remind the staff to conduct manual verification and correct the mapping relationship.

[0073] In the step S24, for example, if the fifth first image in the first image queue and the fifth second image in the second image queue do not have a mapping relationship in the one-to-one correspondence between the multi-page first image and the multi-page second image, and the page content text of the fifth first image and the page content text of the fifth second image have the characteristic of content consistency, it means that the fifth first image and the fifth second image should have a mapping relationship but do not. Therefore, it is necessary to output and display a reminder message indicating a mapping relationship mismatch for the fifth first image and the fifth second image, so as to remind the staff to conduct manual verification and correct the mapping relationship. In addition, for each pair of the first image and the second image with the same serial number in the first image queue and the second image queue, if the corresponding two images have a mapping relationship in the one-to-one correspondence between the multi-page first image and the multi-page second image, and the corresponding two page content texts have the characteristic of content inconsistency, then also output and display a reminder message indicating a mapping relationship mismatch for the corresponding two images, so as to remind the staff to conduct manual verification and correct the mapping relationship. In addition, the specific method for determining whether the two page content texts have the characteristic of content consistency can be, but is not limited to, routinely implemented based on indicators such as word frequency, edit distance, and / or Jaccard distance.

[0074] S3. For each page of the first images in the first image file, read the PDF file information of the first image file, and determine a corresponding first comparison area positioning frame for positioning the corresponding image comparison target area according to the PDF file printing requirement attributes in the PDF file information, where the image comparison target area may but is not limited to an image area surrounded by a media frame, a crop frame, a bleed frame, a trimming frame, a work frame, etc.

[0075] Specifically, when the image file is a small-page file, directly read the media frame, crop frame, bleed frame, trimming frame, or work frame to locate the comparison area, read the PDF information of the image file, extract the coordinates of the media frame, crop frame, bleed frame, trimming frame, or work frame, and directly locate the image comparison target area according to the extracted frame coordinates. When the image file is a large-format imposition file, the following precise segmentation method is adopted: If the large-format image contains its own trim lines, identify the trim lines in the large-format image, and use the area within each small-page trim line as the precise segmentation area. If it does not contain its own trim lines, use the imposition folding hand reverse parsing technology to determine the logical order and position of each small page. This technology is related to the registered patent "A method for comparing and quality inspecting large-format imposed images". If it does not contain trim lines, the specific steps for manual segmentation are as follows:

[0076] S31. Specify the boundaries of each small page through interface operations and manually specify some imposition parameters to enable the system to more accurately assist in segmentation. The imposition parameters are as follows: Image size: The system automatically reads the length and width of the image to determine the overall area of the large file; Number of imposition pages: The user specifies the number of pages for imposition according to the actual situation (such as 8-page imposition, 16-page imposition, etc.); Finished size: The user specifies the finished size of each small page (such as 185mm * 260mm), which is the size of the crop frame.

[0077] S32. Calculate the sizes of the bleed frame and the media frame. The system automatically calculates the sizes of the media frame or the bleed frame according to the finished size specified by the user and the built-in bleed size (such as 3mm).

[0078] S33. Built-in imposition model. The system has a built-in imposition model that contains necessary size parameters in the printing process, such as gripper margin size, gutter size, column gutter size, etc. These size parameters are used to ensure that the segmented small-page files meet the actual printing requirements.

[0079] S34. Manually adjust the image and layout. The user manually draws or selects the boundaries of each small page on the interface provided by the system according to the actual situation of the image and the imposition logic. The system automatically adjusts the layout of each page file according to the boundaries specified by the user and the imposition model, making it centered on the overall size. If necessary, the user can also manually adjust the layout according to the gripper data to ensure that the layout size of each page basically conforms to the original data of the imposition logic. After completing the manual splitting and layout adjustment, the system accurately splits the large-format file into multiple small-page files according to the user's specification.

[0080] In step S3, since the first image file is an electronic file in PDF format, the PDF file information can be conventionally read (for example, by using an editing-level PDF kernel, all information of the PDF file can be read). And since there is an item in the PDF file information that describes the printing requirements of the PDF file and describes preset regional attributes such as the trimming area, bleed area, crop area, artwork area, and media area, when the image comparison target area is the trimming area, bleed area, crop area, artwork area, or media area, etc., the first comparison area positioning frame for positioning the image comparison target area can be directly determined according to the PDF file information. In addition, the aforementioned trimming area, bleed area, crop area, artwork area, and media area are all common terms in the PDF file information, and their positional relationships are as Figure 4 shown; if the PDF file information does not contain parameters for describing the media box, crop box, bleed box, trim box, or artwork box, etc., the entire area of the first image can also be directly defaulted to the image comparison target area, or the image comparison target area can be delimited in the first image manually.

[0081] S4. For each page of the first image, use the corresponding first comparison area positioning frame to intercept the corresponding first comparison area image from the corresponding image.

[0082] S5. For each page of the first image, import the corresponding first comparison area image into the trained variable-length text recognition model, and output the corresponding first strings for each row that correspond one-to-one with the rows of text in the corresponding first comparison area image. And according to the first strings for each row, determine the corresponding first minimum rectangle frame, where the first minimum rectangle frame refers to the rectangle frame with the smallest area that is used to enclose the rows of text in the corresponding first comparison area image that have a mapping relationship with the first strings for each row.

[0083] In the step S5, the variable-length text recognition model is an existing model dedicated to variable-length text recognition. Variable-length text recognition is widely applied in reality. For example, it is used for text recognition of printed text, billboard text, etc. Although it has characteristics such as an unfixed number of characters, unpredictability, and relatively high recognition difficulty, it is also the main direction of current research on text recognition. Therefore, common current variable-length text recognition methods include: Long Short-Term Memory and Connectionist Temporal Classifier (LSTM+CTC) (the former is a recurrent neural network with a special structure, Recurrent Neural Networks, used to solve the long-term dependence problem of the recurrent neural network RNN, and the latter is used to solve the alignment problem between input features and output labels), Convolutional Recurrent Neural Network (CRNN) (which is a relatively popular text recognition model at present, does not require character segmentation of sample data, can recognize text sequences of any length, and has the characteristics of fast speed and good performance), and the text recognition project chineseocr (which is a natural scene text recognition project based on YOLO3 and CRNN. This project supports text detection of darknet / opencvdnn / keras, supports direction detection of 0, 90, 180, and 270 degrees, supports variable-length English and Chinese-English recognition, and also supports various scenarios such as general optical character recognition (OCR), ID card recognition, and train ticket recognition), and so on. Therefore, the variable-length text recognition model can but is not limited to adopting a recognition model based on LSTM+CTC, CRNN, or the text recognition project chineseocr, etc. In addition, the training method of the aforementioned existing recognition model is also an existing conventional method, so that the trained variable-length text recognition model has the ability to recognize variable-length text in the imported image and can obtain the first strings corresponding to each line of text in the first comparison area image one by one. In addition, the specific determination method of the first minimum rectangular box can but is not limited to being conventionally determined based on the coordinates of each line of text having a mapping relationship with each line of the first string in the first comparison area image.

[0084] S6. For each second image on the second image file, import the corresponding image into the variable-length text recognition model, output each corresponding second string that corresponds one-to-one with each line of text in the corresponding image, and determine the corresponding second minimum bounding box according to each second string, where the second minimum bounding box refers to a rectangle with the minimum area that is used to enclose each line of text in the corresponding image and has a mapping relationship with each second string.

[0085] In step S6, the specific determination method of the second minimum bounding box can also but is not limited to be determined conventionally based on the coordinates of each line of text in the corresponding second image that has a mapping relationship with each second string. Considering that there may be text in the area outside the bounding box of the second comparison area positioning box of the image comparison target area for positioning the second image, this will cause each second string to contain out-of-box characters, and further cause the second minimum bounding box determined directly based on the coordinates of each line of text in the corresponding second image that has a mapping relationship with each second string to be too large in size, affecting the correctness of the subsequent image comparison result. Therefore, in order to further ensure the correctness of the subsequent image comparison result, preferably, for each second image on the second image file, import the corresponding image into the variable-length text recognition model, output each corresponding second string that corresponds one-to-one with each line of text in the corresponding image, and determine the corresponding second minimum bounding box according to each second string, including but not limited to the following steps S61 to S63.

[0086] S61. For a certain second image on the second image file, import the corresponding image into the variable-length text recognition model, output each corresponding second string that corresponds one-to-one with each line of text in the corresponding image.

[0087] S62. According to each first string of a certain first image that has a mapping relationship with the certain second image, determine at least one string from each second string that has the characteristic of content consistency with each first string.

[0088] In step S62, for example, if the first strings of each row of the first image on a certain page are: "AABBBC; DEEEFF" (that is, there are two rows of first strings, where the first string in the first row is "AABBBC" and the first string in the last row is "DEEEFF"), and the second strings of each row of the second image on the same page are: "IAABBBCJ; MDEEEFFN" (that is, there are also two rows of second strings, where the first string in the first row is "IAABBBCJ" and the first string in the last row is "MDEEEFFN"), then at least one row of strings that has content consistency with the first strings of each row can be determined from the second strings of each row as: "AABBBC; DEEEFF" (that is, there are also two rows of strings, and it can be determined that "I", "J", "M", "N", etc. are all out-of-frame characters). In addition, the specific method for determining whether two strings have content consistency can also, but is not limited to, be conventionally implemented based on indicators such as word frequency, edit distance, and / or Jaccard distance.

[0089] S63. Determine the second minimum rectangular frame of the second image on the certain page according to the at least one row of strings, where the second minimum rectangular frame refers to a rectangular frame with the smallest area that is used to enclose at least one row of text in the second image on the certain page and has a mapping relationship with the at least one row of strings.

[0090] In step S63, the specific method for determining the second minimum rectangular frame can also, but is not limited to, be conventionally determined based on the coordinates of at least one row of text that has a mapping relationship with the at least one row of strings in the second image on the certain page. In addition, the second minimum rectangular frame obtained based on step S63 is used to replace the second minimum rectangular frame obtained based on step S6.

[0091] S7. For each second image, determine a corresponding second comparison region positioning frame for positioning the corresponding image comparison target region according to the corresponding second minimum rectangular frame, the first minimum rectangular frame of the corresponding first image, and the first comparison region positioning frame.

[0092] In step S7, considering that the first comparison region positioning frame will definitely enclose the first minimum rectangular frame, so the second comparison region positioning frame will also definitely enclose the second minimum rectangular frame. Furthermore, specifically, as Figure 5As shown, for each second image of the pages, according to the corresponding second minimum rectangle and the first minimum rectangle and the first comparison area positioning frame of the corresponding first image, to determine the corresponding second comparison area positioning frame for positioning the corresponding image comparison target area, including but not limited to the following steps: For any second image in the second image file, according to the corresponding second minimum rectangle and the first minimum rectangle and the first comparison area positioning frame of the corresponding first image, determine the corresponding second comparison area positioning frame for positioning the corresponding image comparison target area according to the following formula:

[0093]

[0094] In the formula, x 21,btl represents the abscissa position of the lower left corner point of the second comparison area positioning frame in the any second image, y 21,btl represents the ordinate position of the lower left corner point of the second comparison area positioning frame in the any second image, x 21,tpr represents the abscissa position of the upper right corner point of the second comparison area positioning frame in the any second image, y 21,tpr represents the ordinate position of the upper right corner point of the second comparison area positioning frame in the any second image, x 22,btl represents the abscissa position of the lower left corner point of the second minimum rectangle in the any second image, y 22,btl represents the ordinate position of the lower left corner point of the second minimum rectangle in the any second image, x 22,tpr represents the abscissa position of the upper right corner point of the second minimum rectangle in the any second image, y 22,tpr represents the ordinate position of the upper right corner point of the second minimum rectangle in the any second image, x 12,btl represents the abscissa position of the lower left corner point of the first minimum rectangle in the first image corresponding to the any second image, y 12,btl represents the ordinate position of the lower left corner point of the first minimum rectangle in the first image corresponding to the any second image, x 12,tpr represents the abscissa position of the upper right corner point of the first minimum rectangle in the first image corresponding to the any second image, y 12,tpr represents the ordinate position of the upper right corner point of the first minimum rectangle in the first image corresponding to the any second image, x 11,btl represents the abscissa position of the lower left corner point of the first comparison area positioning frame in the first image corresponding to the any second image, y 11,btl represents the ordinate position of the lower left corner point of the first comparison area positioning frame in the first image corresponding to the any second image, x11,tpr represents the abscissa position of the upper right corner point of the first comparison region positioning frame within the first image corresponding to any one of the second images, y 11,tpr represents the ordinate position of the upper right corner point of the first comparison region positioning frame within the first image corresponding to any one of the second images.

[0095] S8. For each page of the second images, use the corresponding second comparison region positioning frame to intercept the corresponding second comparison region image from the corresponding image.

[0096] Specifically, for the manual segmentation method, the system obtains the precise segmentation regions of each small page file obtained after manual segmentation. The system prepares the template data for matching according to a preset or user-specified positioning template. The system matches each precise segmentation region with the positioning template, and determines whether the matching is successful by comparing the features of the regions (such as shape, size, position, etc.). If a precise segmentation region matches the positioning template successfully, the system considers that the region has been precisely located. If there are regions that are not matched or matched inaccurately, the system can prompt the user to make manual adjustments or re-segment. The system finally outputs the result of precise positioning, including the list of small page files with successful positioning, the prompt information for failed positioning, and necessary adjustment suggestions.

[0097] S9. For each pair of the first image and the second image that exist in the first image file and the second image file and have a mapping relationship, scale the corresponding first comparison region image or the second comparison region image so that the corresponding first comparison region image and the second comparison region image have the same size and resolution.

[0098] In step S9, since the image comparison is performed for differences at the pixel point level, it is necessary to make the two comparison region images to be compared have the same size and resolution. In addition, the specific scaling process can but is not limited to include: taking the size and resolution of any one of the first comparison region image and the corresponding second comparison region image as a reference, performing conventional image compression, image expansion, and / or image resolution conversion on the other image in the first comparison region image and the corresponding second comparison region image, so that the two images have the same size and resolution. For example, the resolution of both images is 300 dpi or 600 dpi.

[0099] Based on the multi-page image comparison intelligent precise segmentation and precise positioning method described in the foregoing steps S1 to S9, a new solution that can automatically locate the comparison area during the image comparison process of multi-page image files is provided. That is, after determining the one-to-one correspondence between the multi-page first images and the multi-page second images to be subjected to image comparison, first, based on the PDF file information, determine the first comparison area positioning frames of each page of the first images and used to locate the corresponding image comparison target areas, then use these frames to intercept the first comparison area images and determine the first minimum rectangle frames that enclose the text in the images. At the same time, determine the second minimum rectangle frames that enclose the text in the images of each page of the second images. Then, based on the foregoing three frames, determine the second comparison area positioning frames and the second comparison area images of each page of the second images and also used to locate the image comparison target areas. Finally, scale to obtain the two area images suitable for image comparison. In this way, the comparison area can be automatically located during the image comparison process of two multi-page image files, greatly improving the positioning efficiency of the comparison area, ensuring the correctness of the subsequent image comparison results, and facilitating practical application and promotion.

[0100] Based on the technical solution of the foregoing first aspect, this embodiment further provides a possible design 1 on how to automatically locate the difference area and give a difference description during the image comparison process. That is, after making the first comparison area image and the second comparison area image have the same size and resolution, the method further includes but is not limited to the following steps S101 to S105.

[0101] S101. Perform grayscale processing on the first comparison area image and the second comparison area image respectively to obtain a first grayscale image and a second grayscale image.

[0102] In the step S101, specifically, perform the grayscale processing on the first comparison area image to obtain the first grayscale image; perform the grayscale processing on the second comparison area image to obtain the second grayscale image.

[0103] S102. According to the preset grayscale threshold interval, determine a plurality of first pixel points within any grayscale image and whose grayscale values belong to the grayscale threshold interval, and respectively compare the grayscale values of each first pixel point among the plurality of first pixel points in the first grayscale image and the second grayscale image to determine the second pixel points that do not have the characteristic of grayscale value consistency in the first grayscale image and the second grayscale image, where any grayscale image refers to the first grayscale image or the second grayscale image.

[0104] In the step S102, the grayscale threshold interval is used to screen the feature pixel points participating in the pixel-level comparison. It can be set in advance by the staff or randomly generated in advance, for example, [100, 120]. Considering that the total number of feature pixel points participating in the pixel-level comparison should not be too small to avoid affecting the accuracy of the subsequent differential region positioning result. Preferably, determining a plurality of first pixel points within any grayscale image and having grayscale values belonging to the grayscale threshold interval according to the preset grayscale threshold interval includes, but is not limited to, the following steps: determining a plurality of first pixel points within any grayscale image and having grayscale values belonging to the grayscale threshold interval according to the preset grayscale threshold interval, and judging whether the total number of the plurality of first pixel points is lower than the preset first pixel point number threshold. If so, adjust and update the grayscale threshold interval, and then determine a plurality of first pixel points within the any grayscale image and having grayscale values belonging to the new grayscale threshold interval according to the new grayscale threshold interval until it is determined that the total number of the plurality of first pixel points is greater than or equal to the first pixel point number threshold, where the any grayscale image refers to the first grayscale image or the second grayscale image. The specific way of adjusting and updating the grayscale threshold interval can be, but is not limited to, expanding the interval or changing the interval, etc. In addition, it is also considered that the total number of feature pixel points participating in the pixel-level comparison should not be too large to avoid losing the meaning of screening feature pixel points. Preferably, determining a plurality of first pixel points within any grayscale image and having grayscale values belonging to the grayscale threshold interval according to the preset grayscale threshold interval includes, but is not limited to, the following steps: determining a plurality of first pixel points within any grayscale image and having grayscale values belonging to the grayscale threshold interval according to the preset grayscale threshold interval, and judging whether the total number of the plurality of first pixel points is higher than the preset second pixel point number threshold. If so, adjust and update the grayscale threshold interval, and then determine a plurality of first pixel points within the any grayscale image and having grayscale values belonging to the new grayscale threshold interval according to the new grayscale threshold interval until it is determined that the total number of the plurality of first pixel points is less than or equal to the second pixel point number threshold, where the any grayscale image refers to the first grayscale image or the second grayscale image. The specific way of adjusting and updating the grayscale threshold interval can be, but is not limited to, shrinking the interval or changing the interval, etc. In addition, the second pixel point number threshold needs to be greater than the first pixel point number threshold; the first pixel point number threshold and the second pixel point number threshold can be positively correlated and set in advance based on the corresponding image size or image resolution, that is, the larger the image size or the higher the image resolution, the larger the first pixel point number threshold and the second pixel point number threshold are set, and vice versa.

[0105] In the step S102, it is also considered that multiple feature pixels participating in pixel-level comparison need to have the characteristic of uniform distribution, so as to avoid the situation of pixel-level comparison in a local area, thereby affecting the accuracy of the subsequent differential area positioning result. Preferably, multiple first pixels within any grayscale image and with grayscale values belonging to the preset grayscale threshold interval are determined, including but not limited to the following steps: Determine multiple first pixels within any grayscale image and with grayscale values belonging to the preset grayscale threshold interval, and grid the any grayscale image into multiple image sub-regions of the same size. Then, count the number of the first pixels contained in each of the multiple image sub-regions and the standard deviation of the number of the multiple image sub-regions. Finally, determine whether the standard deviation of the number is higher than the preset standard deviation threshold. If so, adjust and update the grayscale threshold interval, and then determine multiple first pixels within the any grayscale image and with grayscale values belonging to the new grayscale threshold interval again according to the new grayscale threshold interval until it is determined that the standard deviation of the number is less than or equal to the standard deviation threshold. Among them, the any grayscale image refers to the first grayscale image or the second grayscale image. For example, assuming that the size of the any grayscale image is 6000×9000, the any grayscale image can be gridded into 100 image sub-regions with a size of 600×900 each, and then count the number of the first pixels contained in each of these 100 image sub-regions, as well as the average value and standard deviation of the number of these 100 image sub-regions. In addition, the standard deviation threshold and / or the total number of the multiple image sub-regions can also be set in positive correlation in advance based on the corresponding image size or image resolution, that is, the larger the image size or the higher the image resolution, the larger the standard deviation threshold and / or the total number of the multiple image sub-regions are set, and vice versa.

[0106] In the step S102, specifically, the grayscale values of each of the multiple first pixels in the first grayscale image and the second grayscale image are respectively compared to determine second pixels that do not have the characteristic of grayscale value consistency in the first grayscale image and the second grayscale image, including but not limited to: If the absolute value of the difference between the grayscale values of a certain first pixel in the multiple first pixels in the first grayscale image and the second grayscale image is greater than the preset grayscale threshold, it is determined that the certain first pixel does not have the characteristic of grayscale value consistency in the first grayscale image and the second grayscale image, and the certain first pixel is taken as a second pixel. Otherwise, it is determined that the certain first pixel has the characteristic of grayscale value consistency in the first grayscale image and the second grayscale image. In addition, the foregoing grayscale threshold is, for example but not limited to, 10.

[0107] S103. If the total number of the second pixel points is greater than zero, determine, according to all the second pixel points, a third smallest rectangular frame within any one of the grayscale images, where the third smallest rectangular frame refers to a rectangular frame that is used to enclose all the second pixel points and has the smallest area.

[0108] In the step S103, for example, if the second pixel points are represented by black dots, the third smallest rectangular frame can be represented by a dashed frame as shown in Figure 6 the figure. In addition, the specific determination method of the third smallest rectangular frame may, but is not limited to, be determined conventionally based on the coordinates of all the second pixel points within any one of the grayscale images.

[0109] S104. If the area ratio of the first smallest rectangular frame within any one of the grayscale images is greater than a preset first ratio threshold, respectively intercept, from the first comparison region image and the second comparison region image, a first comparison region sub-image and a second comparison region sub-image that are enclosed by the third smallest rectangular frame.

[0110] In the step S104, the third smallest rectangular frame is the located difference region; the first ratio threshold is used as a criterion for measuring whether the difference region can be ignored in the current image comparison process, that is, if the area ratio of the third smallest rectangular frame within any one of the grayscale images is too small, the corresponding difference region can be ignored in the current image comparison process, otherwise it cannot. In addition, the first ratio threshold is, for example, 5%.

[0111] S105. Import the first comparison region sub-image and the second comparison region sub-image together into an image difference description generation model that is pre-trained, output a first image difference description text, and load and display the third smallest rectangular frame and the first image difference description text on the first comparison region image and the second comparison region image.

[0112] In the step S105, the goal of the Image Difference Captioning (IDC) model is to compare two similar pictures, capture the visual differences between them, and then describe these differences in natural language. Specifically, it can be but is not limited to using existing models based on pre-extracted object-level difference features, models based on dynamic correlation attention mechanisms, difference description models based on the Transformer (a classic NLP model proposed by a Google team in 2017) structure, and / or models that combine semantic segmentation models and graph convolutional neural networks to describe image differences, etc. to achieve. Specifically, the first image difference description text can be but is not limited to being composed of several keywords to form difference description content that can be intuitively perceived by the staff. The specific pre-training process of the image difference description model can also refer to the prior art to implement. Specifically, loading and displaying the third minimum rectangle and the first image difference description text on the first comparison region image and the second comparison region image can be: loading and displaying the third minimum rectangle and the first image difference description text on the first comparison region image and / or the second comparison region image, so that the staff can intuitively perceive the difference region positioning result and the given difference description content.

[0113] Due to the possible design one described in the foregoing steps S101 - S105, it is also possible to automatically locate the difference region and give a difference description during the image comparison of two multi-page image files, which is beneficial for corresponding difference elimination processing in the subsequent printing process to ensure the quality of the subsequent printed matter.

[0114] Based on the technical solution of the foregoing possible design one, this embodiment also provides a possible design two on how to further precisely locate the difference region and give the corresponding difference description content, that is, after determining the third minimum rectangle in any one of the grayscale images according to all the second pixel points, the method further includes but is not limited to the following steps S1061 - S1066.

[0115] S1061. If the area ratio of the third minimum rectangle in any one of the grayscale images is greater than a preset second ratio threshold, then from the first grayscale image and the second grayscale image, first grayscale sub-images and second grayscale sub-images surrounded by the third minimum rectangle are respectively intercepted, where the second ratio threshold is greater than the first ratio threshold.

[0116] In the step S1061, the second ratio threshold is used as a criterion for whether to further precisely locate the difference region, and can be exemplified as 10%.

[0117] S1062. Perform grid processing on the first grayscale sub-image and the second grayscale sub-image respectively to obtain a plurality of first grayscale grandchild images and a plurality of second grayscale grandchild images that correspond one-to-one to the plurality of first grayscale grandchild images.

[0118] In step S1062, the first grayscale grandchild image refers to a sub-image of the first grayscale sub-image, and the second grayscale grandchild image refers to a sub-image of the second grayscale sub-image; specifically, perform grid processing on the first grayscale sub-image to obtain the plurality of first grayscale grandchild images; perform the grid processing on the second grayscale sub-image to obtain the plurality of second grayscale grandchild images. In addition, the number of images of the plurality of first grayscale grandchild images or the plurality of second grayscale grandchild images can be positively correlated and set based on the ratio of the second ratio threshold to the first ratio threshold, that is, the larger this ratio is, the more the number of images of the plurality of first grayscale grandchild images or the plurality of second grayscale grandchild images is set, so as to complete the further differential region positioning task in one step.

[0119] S1063. For each pair of first grayscale grandchild images and second grayscale grandchild images that have a mapping relationship among the plurality of first grayscale grandchild images and the plurality of second grayscale grandchild images, determine a plurality of third pixel points within the corresponding grayscale grandchild image and whose grayscale values belong to the grayscale threshold interval according to a preset grayscale threshold interval, and respectively compare the grayscale values of each third pixel point among the plurality of third pixel points in the corresponding pair of grayscale grandchild images to determine the fourth pixel points that correspond and do not have the characteristic of grayscale value consistency in the corresponding pair of grayscale grandchild images.

[0120] In step S1063, the corresponding grayscale grandchild image is either the first grayscale grandchild image or the second grayscale grandchild image among each pair of first grayscale grandchild images and second grayscale grandchild images. In addition, the specific determination details of the third pixel points and the fourth pixel points can be obtained by referring to the derivation in the foregoing step S102, and will not be elaborated here.

[0121] S1064. For each pair of first grayscale grandchild images and second grayscale grandchild images, if the number of the corresponding fourth pixel points is non-zero, then determine the corresponding fourth minimum rectangular frame within the corresponding grayscale grandchild image according to all the corresponding fourth pixel points, where the fourth minimum rectangular frame refers to a rectangular frame that is used to enclose all the corresponding fourth pixel points and has the smallest area.

[0122] In the step S1064, for the specific details of determining the fourth smallest rectangular box, reference may be made to the foregoing step S103, which will not be elaborated herein. In addition, for each pair of the first grayscale grandchild images and the second grayscale grandchild images, if the number of the corresponding fourth pixel points is zero, there is no need to generate the corresponding fourth smallest rectangular box (that is, there is no corresponding difference region).

[0123] S1065. For each pair of the first grayscale grandchild images and the second grayscale grandchild images, if the area ratio of the corresponding fourth smallest rectangular box in any of the grayscale images is greater than a preset third ratio threshold, intercept, from the first comparison region image and the second comparison region image, the corresponding third comparison region sub-image and the fourth comparison region sub-image surrounded by the corresponding fourth smallest rectangular box.

[0124] In the step S1065, the fourth smallest rectangular box is the difference region obtained by further fine positioning; the third ratio threshold is used as a criterion for measuring whether the difference region can be ignored in the current image comparison process, that is, if the area ratio of the fourth smallest rectangular box in any of the grayscale images is too small, the corresponding difference region can be ignored in the current image comparison process, otherwise it cannot. In addition, the third ratio threshold can be specifically set in negative correlation based on the ratio of the second ratio threshold to the first ratio threshold, that is, the larger the ratio, since the number of the multiple first grayscale grandchild images or the number of images of the multiple first grayscale grandchild images is larger, the smaller the third ratio threshold needs to be set; and the third ratio threshold can also be equal to the first ratio threshold, for example, 5%.

[0125] S1066. For each pair of the first grayscale grandchild images and the second grayscale grandchild images, import the corresponding third comparison region sub-image and the fourth comparison region sub-image into an image difference description generation model pre-trained, output the corresponding second image difference description text, and load and display the fourth smallest rectangular box and the second image difference description text on the first comparison region image and the second comparison region image.

[0126] Due to the possible design two described in the foregoing steps S1061 to S1066, it is also possible to further finely position the difference region and give the corresponding difference description content when the initially positioned difference region is too large, so as to improve the user experience.

[0127] Such as Figure 7As shown in the figure, the second aspect of this embodiment provides a virtual device for implementing the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect, including an image file acquisition unit, a correspondence determination unit, a first positioning frame determination unit, a first region image extraction unit, a first rectangle frame determination unit, a second rectangle frame determination unit, a second positioning frame determination unit, a second region image extraction unit, and an image scaling and processing unit;

[0128] The image file acquisition unit is configured to acquire a first image file and a second image file to be subjected to image comparison. Among them, the first image file is an electronic file in PDF format and contains multiple pages of first images, and the second image file contains multiple pages of second images;

[0129] The correspondence determination unit is communicatively connected to the image file acquisition unit and is configured to determine the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images;

[0130] The first positioning frame determination unit is communicatively connected to the image file acquisition unit. For each page of the first images in the first image file, it reads the PDF file information of the first image file and determines a corresponding first comparison region positioning frame for positioning the corresponding image comparison target region according to the PDF file printing requirement attributes in the PDF file information. Among them, the image comparison target region is an image region surrounded by a media frame, a crop frame, a bleed frame, a trim frame, or a work frame;

[0131] The first region image extraction unit is communicatively connected to the first positioning frame determination unit. For each page of the first images, it uses the corresponding first comparison region positioning frame to extract the corresponding first comparison region image from the corresponding image;

[0132] The first rectangle frame determination unit is communicatively connected to the first region image extraction unit. For each page of the first images, it imports the corresponding first comparison region image into a trained variable-length text recognition model, outputs corresponding first strings for each row that are in one-to-one correspondence with the texts in each row of the corresponding first comparison region image, and determines a corresponding first minimum rectangle frame according to the first strings for each row. Among them, the first minimum rectangle frame refers to a rectangle frame with the smallest area that is used to enclose the texts in each row in the corresponding first comparison region image and has a mapping relationship with the first strings for each row;

[0133] The second rectangular frame determination unit is communicatively connected to the image file acquisition unit, and is configured to, for each page of the second image in the second image file, import the corresponding image into the variable-length text recognition model, output corresponding second strings for each row of text in the corresponding image, and determine corresponding second minimum rectangular frames according to the second strings for each row, where the second minimum rectangular frame refers to a rectangular frame with the smallest area for enclosing each row of text in the corresponding image that has a mapping relationship with the second strings for each row;

[0134] The second positioning frame determination unit is communicatively connected to the corresponding relationship determination unit, the first positioning frame determination unit, the first rectangular frame determination unit, and the second rectangular frame determination unit respectively, and is configured to, for each page of the second image, determine a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area according to the corresponding second minimum rectangular frame, the first minimum rectangular frame of the corresponding first image, and the first comparison area positioning frame;

[0135] The second area image extraction unit is communicatively connected to the second positioning frame determination unit, and is configured to, for each page of the second image, extract a corresponding second comparison area image from the corresponding image by applying the corresponding second comparison area positioning frame;

[0136] The image scaling processing unit is communicatively connected to the corresponding relationship determination unit, the first area image extraction unit, and the second area image extraction unit respectively, and is configured to, for each pair of the first image and the second image that have a mapping relationship in the first image file and the second image file, scale and process the corresponding first comparison area image or the second comparison area image so that the corresponding first comparison area image and the second comparison area image have the same size and resolution.

[0137] The working process, working details, and technical effects of the foregoing device provided in the second aspect of this embodiment can be referred to the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect, and will not be elaborated here.

[0138] As Figure 8As shown, in the third aspect of this embodiment, a computer device for executing the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect is provided, including a memory, a processor, and a transceiver that are communicatively connected in sequence. Among them, the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect. Specifically, by way of example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first input first output (FIFO), and / or first input last output (FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0139] For the working process, working details, and technical effects of the foregoing computer device provided in the third aspect of this embodiment, reference may be made to the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect, which will not be elaborated herein.

[0140] In the fourth aspect of this embodiment, a computer-readable storage medium storing instructions including the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect is provided, that is, instructions are stored on the computer-readable storage medium, and when the instructions run on a computer, they execute the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect. Among them, the computer-readable storage medium refers to a carrier for storing data, and may include, but is not limited to, computer-readable storage media such as floppy disks, optical discs, hard disks, flash memories, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices.

[0141] For the working process, working details, and technical effects of the foregoing computer-readable storage medium provided in the fourth aspect of this embodiment, reference may be made to the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect, which will not be elaborated herein.

[0142] In the fifth aspect of this embodiment, a computer program product is provided, including a computer program or instructions, and when the computer program or the instructions are executed by a computer, the multi-page image comparison intelligent precise segmentation and precise positioning method as described in the first aspect or any possible design in the first aspect is implemented. Among them, the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0143] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent segmentation and positioning method for multi-page image comparison, characterized in that Including: Obtain a first image file and a second image file to be subjected to image comparison. Among them, the first image file is an electronic file in PDF format or non-PDF format and contains a first image with multiple pages or a multi-page layout and imposition layout. The second image file contains a second image with multiple pages or a multi-page layout and imposition layout; Determine the one-to-one correspondence or logical layout relationship between the multi-page first image and the multi-page second image; For each page of the first image in the first image file, read the PDF file information of the first image file, and determine a corresponding first comparison region positioning frame for positioning the corresponding image comparison target region according to the PDF file printing requirement attributes in the PDF file information. Among them, the image comparison target region is an image region surrounded by a media box, a crop box, a bleed box, a trim box or a work box; Read the region positioning frame of the second image, including when the second image is a typeset and imposed image with a multi-page layout, the positioning region is implemented according to the image region surrounded by the media box, the crop box, the bleed box, the trim box or the work box; For each page of the first image, use the corresponding first comparison region positioning frame to intercept the corresponding first comparison region image from the corresponding image; For each page of the first image, import the corresponding first comparison region image into a trained recognition model, output corresponding first strings corresponding to each line of text in the corresponding first comparison region image, and determine a corresponding first minimum rectangle according to the first strings of each line. The first minimum rectangle refers to a rectangle with the smallest area used to enclose each line of text in the corresponding first comparison region image that has a mapping relationship with the first strings of each line; For each page of the second image in the second image file, import the corresponding image into the recognition model, output corresponding second strings corresponding to each line of text in the corresponding image, and determine a corresponding second minimum rectangle according to the second strings of each line. The second minimum rectangle refers to a rectangle with the smallest area used to enclose each line of text in the corresponding image that has a mapping relationship with the second strings of each line; For each page of the second image, determine a corresponding second comparison region positioning frame for positioning the corresponding image comparison target region according to the corresponding second minimum rectangle and the first minimum rectangle and the first comparison region positioning frame of the corresponding first image; For each page of the second image, use the corresponding second comparison region positioning frame to intercept the corresponding second comparison region image from the corresponding image; For each pair of the first image and the second image with a mapping relationship in the first image file and the second image file, scale the corresponding first comparison region image or the second comparison region image so that the corresponding first comparison region image and the second comparison region image have the same size and resolution.

2. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that After determining the one-to-one correspondence between the multi-page first image and the multi-page second image, the method further includes: For each first image in the multi-page first image, applying optical character recognition technology to extract the corresponding page content text from the corresponding image, and counting the number of words in the corresponding page content text; For each second image in the multi-page second image, applying the optical character recognition technology to extract the corresponding page content text from the corresponding image, and counting the number of words in the corresponding page content text; Arrange the first images of each page in descending order of the number of words to obtain a first image queue, and also arrange the second images of each page in descending order of the number of words to obtain a second image queue; For each pair of the first image and the second image with the same serial number in the first image queue and the second image queue, if the corresponding two images do not have a mapping relationship in the one-to-one correspondence between the multi-page first image and the multi-page second image, and the corresponding two page content texts have the characteristic of content consistency, then output and display a reminder message for indicating that there is a mapping relationship mismatch between the corresponding two images, so as to remind the staff to conduct manual inspection and correct the mapping relationship.

3. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that When the first image file is a small-page file, directly read the media box, crop box, bleed box, trim box or work box to locate the comparison area, read the PDF information of the image file, extract the coordinates of the media box, crop box, bleed box, trim box or work box, and directly locate the image comparison target area according to the extracted box coordinates. When the image file is a large-format imposition file, the following precise segmentation method is adopted: If the large-format image contains its own trim lines, identify the trim lines in the large-format image, and take the area within each small-page trim line as the precise segmentation area. If it does not contain its own trim lines, adopt the imposition folding hand reverse parsing technology to determine the logical order and position of each small page. The specific steps of manual segmentation are as follows: S31. Specify the boundaries of each small page through interface operations and manually specify some imposition parameters so that the system can more accurately assist in segmentation. The imposition parameters are: Image size: The system automatically reads the length and width of the image to determine the overall area of the large file; Number of imposition pages: The user specifies the number of imposition pages according to the actual situation; Finished size: The user specifies the finished size of each small page, which is the size of the trim box; S32. Calculate the sizes of the bleed box and the media box. The system automatically calculates the size of the media box or the bleed box according to the finished size specified by the user and the built-in bleed size; S33. Built-in imposition model. The system has a built-in imposition model, which contains the necessary size parameters in the printing process, such as gripper size, gutter size, and column gutter size. These size parameters are used to ensure that the segmented small-page files meet the actual printing requirements; S34. Manually adjust the image and its layout. On the interface provided by the system, the user manually draws or selects the boundaries of each small page according to the actual situation of the image and the imposition logic. The system automatically adjusts the layout of each page file according to the boundaries specified by the user and the imposition model, making it centered within the overall size. If necessary, the user can also manually adjust the layout according to the gripper data to ensure that the layout dimensions of each page are basically consistent with the original data of the imposition logic. After completing the manual segmentation and layout adjustment, the system accurately segments the large-format file into multiple small-page files according to the user's specification.

4. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that For each page of the second image in the second image file, import the corresponding image into the non-fixed-length text recognition model, and output the corresponding second strings for each row, which correspond one-to-one with the rows of text in the corresponding image. Then, according to the second strings for each row, determine the corresponding second minimum rectangular frame, including: For a certain page of the second image in the second image file, import the corresponding image into the non-fixed-length text recognition model, and output the corresponding second strings for each row, which correspond one-to-one with the rows of text in the corresponding image. According to the first strings for each row of a certain page of the first image that has a mapping relationship with the certain page of the second image, determine at least one string from the second strings for each row that has the characteristic of content consistency with the first strings for each row. According to the at least one string, determine the second minimum rectangular frame of the certain page of the second image, where the second minimum rectangular frame refers to a rectangular frame with the smallest area that is used to enclose at least one row of text in the certain page of the second image and has a mapping relationship with the at least one string.

5. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that For each page of the second image, according to the corresponding second minimum rectangular frame, the first minimum rectangular frame of the corresponding first image, and the first comparison region positioning frame, determine the corresponding second comparison region positioning frame for positioning the corresponding image comparison target region, including: For any second image in the second image file, according to the corresponding second minimum rectangular frame, the first minimum rectangular frame of the corresponding first image, and the first comparison region positioning frame, determine the corresponding second comparison region positioning frame for positioning the corresponding image comparison target region according to the following formula: where x 21,btl represents the abscissa position of the lower left corner point of the second comparison region positioning frame within any one of the second images, and y 21,btl represents the ordinate position of the lower left corner point of the second comparison region positioning frame within any one of the second images, x 21,tpr represents the abscissa position of the upper right corner point of the second comparison region positioning frame within any one of the second images, and y 21,tpr represents the ordinate position of the upper right corner point of the second comparison region positioning frame within any one of the second images, x 22,btl represents the abscissa position of the lower left corner point of the second minimum rectangle within any one of the second images, and y 22,btl represents the ordinate position of the lower left corner point of the second minimum rectangle within any one of the second images, x 22,tpr represents the abscissa position of the upper right corner point of the second minimum rectangle within any one of the second images, and y 22,tpr represents the ordinate position of the upper right corner point of the second minimum rectangle within any one of the second images, x 12,btl represents the abscissa position of the lower left corner point of the first minimum rectangle within the first image corresponding to any one of the second images, and y 12,btl represents the ordinate position of the lower left corner point of the first minimum rectangle within the first image corresponding to any one of the second images, x 12,tpr represents the abscissa position of the upper right corner point of the first minimum rectangle within the first image corresponding to any one of the second images, and y 12,tpr represents the ordinate position of the upper right corner point of the first minimum rectangle within the first image corresponding to any one of the second images, x 11,btl represents the abscissa position of the lower left corner point of the first comparison region positioning frame within the first image corresponding to any one of the second images, and y 11,btl represents the ordinate position of the lower left corner point of the first comparison region positioning frame within the first image corresponding to any one of the second images, x 11,tpr represents the abscissa position of the upper right corner point of the first comparison region positioning frame within the first image corresponding to any one of the second images, and y 11,tpr represents the ordinate position of the upper right corner point of the first comparison region positioning frame within the first image corresponding to any one of the second images.

6. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that The non-fixed-length text recognition model uses a recognition model based on Long Short-Term Memory network and Connectionist Temporal Classification LSTM+CTC, Convolutional Recurrent Neural Network CRNN, or the text recognition project chineseocr.

7. The multi-page image comparison intelligent segmentation and positioning method according to claim 3, characterized in that For the manual segmentation method, the system obtains the accurate segmentation regions of each small-page file obtained after manual segmentation. The system matches each accurate segmentation region with the positioning template according to a preset or user-specified positioning template. By comparing the features of the regions, it determines whether the matching is successful. If an accurate segmentation region matches the positioning template successfully, the system considers that the region has been accurately positioned. If there are regions that are not matched or inaccurately matched, the system prompts the user to make manual adjustments or re-segment.

8. Intelligent positioning device for multi-page image comparison, characterized in that, It includes an image file acquisition unit, a correspondence determination unit, a first positioning frame determination unit, a first region image cropping unit, a first rectangle frame determination unit, a second rectangle frame determination unit, a second positioning frame determination unit, a second region image cropping unit, and an image scaling and processing unit; The image file acquisition unit is configured to acquire a first image file and a second image file for image comparison. Among them, the first image file is an electronic file in PDF format and contains multiple pages of first images, and the second image file contains multiple pages of second images; The correspondence determination unit is communicatively connected to the image file acquisition unit and is configured to determine the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images; The first positioning frame determination unit is communicatively connected to the image file acquisition unit. For each page of the first images in the first image file, it reads the PDF file information of the first image file and determines a corresponding first comparison region positioning frame for positioning the corresponding image comparison target region according to the PDF file printing requirement attributes in the PDF file information. Among them, the image comparison target region is an image region surrounded by a media frame, a crop frame, a bleed frame, a trim frame, or a work frame; The first region image cropping unit is communicatively connected to the first positioning frame determination unit. For each page of the first images, it crops the corresponding first comparison region image from the corresponding image by applying the corresponding first comparison region positioning frame; The first rectangle frame determination unit is communicatively connected to the first region image cropping unit. For each page of the first images, it imports the corresponding first comparison region image into a trained variable-length text recognition model, outputs corresponding first strings for each row that are in one-to-one correspondence with the lines of text in the corresponding first comparison region image, and determines a corresponding first minimum rectangle frame according to the first strings for each row. Among them, the first minimum rectangle frame is a rectangle frame with the smallest area that is used to enclose the lines of text in the corresponding first comparison region image that have a mapping relationship with the first strings for each row; The second rectangle frame determination unit is communicatively connected to the image file acquisition unit. For each page of the second images in the second image file, it imports the corresponding image into the variable-length text recognition model, outputs corresponding second strings for each row that are in one-to-one correspondence with the lines of text in the corresponding image, and determines a corresponding second minimum rectangle frame according to the second strings for each row. Among them, the second minimum rectangle frame is a rectangle frame with the smallest area that is used to enclose the lines of text in the corresponding image that have a mapping relationship with the second strings for each row; The second positioning frame determination unit is communicatively connected to the correspondence determination unit, the first positioning frame determination unit, the first rectangular frame determination unit, and the second rectangular frame determination unit respectively, and is configured to, for each page of the second images, determine a corresponding second comparison region positioning frame for positioning the corresponding image comparison target region according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison region positioning frame of the corresponding first image; The second region image extraction unit is communicatively connected to the second positioning frame determination unit, and is configured to, for each page of the second images, extract a corresponding second comparison region image from the corresponding image by applying the corresponding second comparison region positioning frame; The image scaling processing unit is communicatively connected to the correspondence determination unit, the first region image extraction unit, and the second region image extraction unit respectively, and is configured to, for each pair of the first image and the second image that have a mapping relationship in the first image file and the second image file, scale and process the corresponding first comparison region image or the second comparison region image so that the corresponding first comparison region image and the second comparison region image have the same size and resolution.

9. A computer device, characterized in that, It includes a memory, a processor, and a transceiver that are communicatively connected in sequence. Among them, the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the multi-page image comparison intelligent precise segmentation and precise positioning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that ,Instructions are stored on the computer-readable storage medium, and when the instructions are run on a computer, the multi-page image comparison intelligent precise segmentation and precise positioning method according to any one of claims 1 to 7 is executed.

Images