Multi-page image comparison intelligent segmentation and positioning method, device, equipment and medium

Abstract

The invention discloses a multi-page image comparison intelligent segmentation and positioning method and device, equipment and a medium, and relates to the technical field of electric digital data processing. The method comprises the following steps: after a one-to-one correspondence relationship between multiple pages of first images to be subjected to image comparison and multiple pages of second images is determined, determining a first comparison area positioning frame of each page of first image and used for positioning a corresponding image comparison target area based on PDF file information; then, intercepting by using the frame to obtain a first contrast area image, determining a first minimum rectangular frame surrounding characters in the image, and determining a second minimum rectangular frame of each page of second image and surrounding the characters in the image at the same time; determining a second comparison area positioning frame and a second comparison area image, which are used for positioning the image comparison target area, of the second image of each page based on the three frames, and finally performing zooming to obtain two area images suitable for image comparison, so that the comparison area positioning efficiency can be improved; and the correctness of a subsequent image comparison result is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and in particular relates to a method, device, equipment and medium for intelligent segmentation and positioning of multi-page image comparison. Background Art

[0002] In the process of editing, large-scale imposition, halftone rasterization and printing of the customer's original files to be printed, there may be errors in the original files to be printed, changes to places that should not be moved, loss of file pixels, errors in loading small page files in imposition, loss of pixels in small page file layers, and errors in image and text interpretation in the rasterization process. There are also printing problems caused by plates or printing machines. Therefore, after any step is completed, it is necessary to compare the two image files before and after the step in order to promptly discover problems or errors in the most recently executed step. For example, after the printed product corresponding to the original file to be printed is finally obtained, it is necessary to compare the scanned image file of the printed product with the original file to be printed imported into the printing machine in order to finally confirm whether there are printing problems caused by plates or printing machines in the printing step.

[0003] In the image comparison process, it is a very critical link to locate the image comparison area of ​​the two image files to be compared. If the positioning is inaccurate, it will directly affect the correctness of the subsequent image comparison results. At present, in the existing image comparison scheme, the image comparison area of ​​the two image files is mainly drawn manually, and then the two images are cut out from the two image files based on the drawn area for image comparison to obtain the final image comparison result. However, this manual positioning method obviously has the problems of low efficiency and easy human error, and is particularly unsuitable for locating the image comparison area of ​​multi-page image files. Therefore, how to automatically locate the comparison area during the image comparison process of multi-page image files in order to improve the efficiency of comparison area positioning and ensure the correctness of subsequent image comparison results is a topic that technicians in this field urgently need to study. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, computer equipment and computer-readable storage medium for intelligent segmentation and positioning of multi-page image comparison, aiming to solve the problems of low efficiency and easy errors caused by the need to manually determine the comparison area in the existing image comparison scheme in multi-page image files, especially for the intelligent and precise segmentation and precise positioning of multi-page imposition images.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In the first aspect, a method for intelligent segmentation and positioning of multi-page image comparison is provided, including:

[0007] Acquire a first image file and a second image file to be compared, wherein the first image file is an electronic file in PDF format or non-PDF format and contains a first image with multiple pages or a multi-page layout, and the second image file contains a second image with multiple pages or a multi-page layout;

[0008] Determining a one-to-one correspondence or a logical layout relationship between the plurality of pages of first images and the plurality of pages of second images;

[0009] For each first image page in the first image file, the PDF file information of the first image file is read, and according to the PDF file printing requirement attribute in the PDF file information, a corresponding first comparison area positioning frame for positioning the corresponding image comparison target area is determined, wherein the image comparison target area is an image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box; reading the area positioning frame of the second image includes, when the second image is a typesetting or imposition image of a multi-page layout, implementing a positioning area according to the image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box;

[0010] For each page of the first image, using the corresponding first comparison area positioning frame to intercept the corresponding image to obtain the corresponding first comparison area image;

[0011] For each page of the first image, the corresponding first comparison area image is imported into the trained recognition model, and the first character strings corresponding to each line of text in the corresponding first comparison area image are outputted, and the corresponding first minimum rectangular frame is determined according to each line of the first character string, wherein the first minimum rectangular frame refers to a rectangular frame with the minimum area used to enclose each line of text having a mapping relationship with each line of the first character string in the corresponding first comparison area image;

[0012] For each page of the second image in the second image file, the corresponding image is imported into the recognition model, and the corresponding second character strings corresponding to each line of text in the corresponding image are outputted, and the corresponding second minimum rectangular frame is determined according to each line of the second character string, wherein the second minimum rectangular frame refers to a rectangular frame with the minimum area used to surround each line of text having a mapping relationship with each line of the second character string in the corresponding image;

[0013] For each page of the second image, according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area is determined;

[0014] For each page of the second image, using the corresponding second comparison area positioning frame to cut out a corresponding second comparison area image from the corresponding image;

[0015] For each pair of first images and second images in the first image file and the second image file and having a mapping relationship, the corresponding first contrast area image or the second contrast area image is scaled so that the corresponding first contrast area image and the second contrast area image have the same size and resolution.

[0016] In a possible design, after determining a one-to-one correspondence between the plurality of pages of first images and the plurality of pages of second images, the method further includes: for each page of the plurality of pages of first images, applying an optical character recognition technology to extract a corresponding page content text from the corresponding image, and counting the number of corresponding words in the page content text;

[0017] For each second image of the plurality of pages of second images, extract the corresponding page content text from the corresponding image using the optical character recognition technology, and count the number of corresponding words in the page content text;

[0018] Arranging the first images of each page in descending order according to the number of words to obtain a first image queue, and arranging the second images of each page in descending order according to the number of words to obtain a second image queue;

[0019] For each pair of first images and second images with the same serial number in the first image queue and the second image queue, if the corresponding two images do not have a mapping relationship in the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, and the corresponding two pages of content text have content consistency characteristics, an output is displayed to indicate that there is a mapping relationship mismatch between the corresponding two images, so as to remind the staff to conduct manual inspection and correct the mapping relationship.

[0020] In a possible design, when the first image file is a small page file, the media frame, crop frame, bleed frame, trim frame or artwork frame is directly read to locate the comparison area, the PDF information of the image file is read, the coordinates of the media frame, crop frame, bleed frame, trim frame or artwork frame are extracted, and the image comparison target area is directly located according to the extracted frame coordinates. When the image file is a large-format imposition file, the following precise segmentation method is used: if the large-format image contains a cutting line, the cutting line in the large-format image is identified, and the area within the cutting line of each small page is used as the precise segmentation area. If there is no cutting line, the imposition folding reverse parsing technology is used to determine the logical order and position of each small page. If there is no cutting line, the specific steps of manual segmentation are as follows:

[0021] S31. Specify the boundaries of each small page through the interface operation, and manually specify some imposition parameters so that the system can assist in segmentation more accurately. The imposition parameters are image size: the system automatically reads the length and width of the image to determine the overall area of ​​the large file; the number of imposition pages: the user specifies the number of imposition pages (such as 8 pages, 16 pages, etc.) according to the actual situation; finished product size: the user specifies the finished product size of each small page (such as 185mm*260mm), which is the size of the trimming box;

[0022] S32. Calculate the size of the bleed box and the media box. The system automatically calculates the size of the media box or the bleed box according to the finished product size specified by the user and the built-in bleed size (eg, 3 mm);

[0023] S33. Built-in imposition model. The system has a built-in imposition model, which contains the necessary size parameters in the printing process, such as grip size, row space size, column space size, etc. These size parameters are used to ensure that the split small page files meet the actual printing requirements;

[0024] S34. Manually adjust the image and layout. The user manually draws or selects the boundaries of each small page on the interface provided by the system according to the actual situation of the image and the layout logic. The system automatically adjusts the layout of each page file according to the boundaries and layout model specified by the user to center it on the overall size. If necessary, the user can also manually adjust the layout according to the gripping data to ensure that the layout size of each page is basically consistent with the original data of the layout logic. After completing the manual segmentation and layout adjustment, the system accurately segments the large version file into multiple small page files according to the user's specifications.

[0025] In a possible design, for each page of the second image in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character strings of each line corresponding to each line of text in the corresponding image are output, and the corresponding second minimum rectangular frame is determined according to the second character strings of each line, including:

[0026] For a second image on a certain page in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character strings of each line corresponding to each line of text in the corresponding image are output;

[0027] According to the first character strings of each row of a certain page of the first image having a mapping relationship with the second image of the certain page, determining at least one row of character strings having content consistency characteristics with the first character strings of each row from the second character strings;

[0028] Based on the at least one line of character string, a second minimum rectangular frame of the second image of the certain page is determined, wherein the second minimum rectangular frame refers to a rectangular frame with a minimum area that is used to enclose at least one line of text that has a mapping relationship with the at least one line of character string in the second image of the certain page.

[0029] In a possible design, for each page of the second image, determining a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image includes:

[0030] For any second image in the second image file, according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area is determined according to the following formula:

[0031]

[0032] In the formula, x 21,btl represents the horizontal coordinate position of the lower left corner of the second comparison area positioning frame in any second image, y 21,btl represents the ordinate position of the lower left corner of the second comparison region positioning frame in any second image, x 21,tpr represents the horizontal coordinate position of the upper right corner of the second comparison area positioning frame in any second image, y 21,tpr represents the vertical coordinate position of the upper right corner of the second comparison area positioning frame in any second image, x 22,btl represents the horizontal coordinate position of the lower left corner of the second smallest rectangular frame in any second image, y 22,btl represents the ordinate position of the lower left corner of the second smallest rectangular frame in any second image, x 22,tpr represents the horizontal coordinate position of the upper right corner of the second smallest rectangular frame in any second image, y 22,tpr represents the vertical coordinate position of the upper right corner of the second smallest rectangular frame in any second image, x 12,btl represents the horizontal coordinate position of the lower left corner of the first minimum rectangular frame in the first image corresponding to any second image, y 12,btl represents the ordinate position of the lower left corner of the first minimum rectangular frame in the first image corresponding to any second image, x 12,tpr represents the horizontal coordinate position of the upper right corner of the first minimum rectangular frame in the first image corresponding to any second image, y 12,tprrepresents the ordinate position of the upper right corner of the first minimum rectangular frame in the first image corresponding to any second image, x 11,btl represents the horizontal coordinate position of the lower left corner point of the first comparison area positioning frame in the first image corresponding to any second image, y 11,btl represents the ordinate position of the lower left corner of the first comparison region positioning frame in the first image corresponding to any one of the second images, x 11,tpr represents the horizontal coordinate position of the upper right corner point of the first comparison area positioning frame in the first image corresponding to any second image, y 11,tpr Indicates the vertical coordinate position of the upper right corner point of the first comparison area positioning frame in the first image corresponding to any one of the second images.

[0033] In a possible design, the variable-length text recognition model adopts a recognition model based on a long short-term memory network and a connection time classifier LSTM+CTC, a convolutional recurrent neural network CRNN, or a text recognition project ChineseOCR.

[0034] In a possible design, for the manual segmentation method, the system obtains the precise segmentation areas of each small page file after manual segmentation. The system matches each precise segmentation area with the positioning template based on a preset or user-specified positioning template, and determines whether the match is successful by comparing the features of the area. If a precise segmentation area successfully matches the positioning template, the system considers that the area has been precisely located. If there are unmatched or inaccurately matched areas, the system prompts the user to make manual adjustments or re-segmentation.

[0035] In a second aspect, a multi-page image comparison intelligent positioning device is provided, comprising an image file acquisition unit, a corresponding relationship determination unit, a first positioning frame determination unit, a first area image capture unit, a first rectangular frame determination unit, a second rectangular frame determination unit, a second positioning frame determination unit, a second area image capture unit and an image scaling processing unit;

[0036] The image file acquisition unit is used to acquire a first image file and a second image file to be compared, wherein the first image file is an electronic file in PDF format and contains multiple pages of first images, and the second image file contains multiple pages of second images;

[0037] The correspondence determination unit is communicatively connected to the image file acquisition unit and is used to determine a one-to-one correspondence between the plurality of pages of first images and the plurality of pages of second images;

[0038] The first positioning frame determining unit is communicatively connected to the image file acquiring unit, and is used to read the PDF file information of the first image file for each page of the first image in the first image file, and determine a corresponding first comparison area positioning frame for locating a corresponding image comparison target area according to the PDF file printing requirement attribute in the PDF file information, wherein the image comparison target area is an image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box;

[0039] The first region image capture unit is communicatively connected to the first positioning frame determination unit, and is used to apply the corresponding first comparison region positioning frame to the first image of each page to capture the corresponding first comparison region image from the corresponding image;

[0040] The first rectangular frame determination unit is communicatively connected to the first area image interception unit, and is used for importing the corresponding first comparison area image into the indefinite-length text recognition model that has been trained for each page of the first image, outputting the first character strings of each line that correspond to each line of text in the corresponding first comparison area image, and determining the corresponding first minimum rectangular frame according to each line of the first character string, wherein the first minimum rectangular frame refers to a rectangular frame with the minimum area that is used to surround each line of text that has a mapping relationship with each line of the first character string in the corresponding first comparison area image;

[0041] The second rectangular frame determination unit is communicatively connected to the image file acquisition unit, and is used for importing the corresponding image into the variable-length text recognition model for each page of the second image in the second image file, outputting the corresponding second character strings of each line corresponding to each line of text in the corresponding image, and determining the corresponding second minimum rectangular frame according to each line of the second character string, wherein the second minimum rectangular frame refers to a rectangular frame with the minimum area used to enclose each line of text having a mapping relationship with each line of the second character string in the corresponding image;

[0042] The second positioning frame determining unit is communicatively connected to the corresponding relationship determining unit, the first positioning frame determining unit, the first rectangular frame determining unit and the second rectangular frame determining unit, respectively, and is used to determine, for each page of the second image, a corresponding second comparison area positioning frame for locating the corresponding image comparison target area according to the corresponding second minimum rectangular frame and the first minimum rectangular frame of the corresponding first image and the first comparison area positioning frame;

[0043] The second region image capture unit is communicatively connected to the second positioning frame determination unit, and is used to apply the corresponding second comparison region positioning frame to the second image of each page to capture the corresponding second comparison region image from the corresponding image;

[0044] The image scaling processing unit is communicatively connected to the correspondence determination unit, the first area image capture unit and the second area image capture unit, respectively, and is used to scale the corresponding first contrast area image or the second contrast area image for each pair of first images and second images in the first image file and the second image file and having a mapping relationship, so that the corresponding first contrast area image and the second contrast area image have the same size and resolution.

[0045] In a third aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the multi-page image comparison intelligent precise segmentation and precise positioning method as described in any one of claims 1 to 7.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are run on a computer, the multi-page image comparison intelligent precise segmentation and precise positioning method as described in any one of claims 1 to 7 is executed.

[0047] Beneficial effects of the above scheme:

[0048] The present invention creatively provides a new solution that can automatically locate a comparison area during the image comparison process of a multi-page image file, that is, after determining a one-to-one correspondence between a plurality of pages of first images to be compared and a plurality of pages of second images, firstly determine a first comparison area positioning frame of each page of the first image and used for locating the corresponding image comparison target area based on PDF file information, then apply the frame to obtain the first comparison area image and determine a first minimum rectangular frame surrounding the text in the image, and simultaneously determine a second minimum rectangular frame of each page of the second image and also surrounding the text in the image, and then determine a second comparison area positioning frame and a second comparison area image of each page of the second image and also used for locating the image comparison target area based on the above three frames, and finally zoom to obtain two area images suitable for image comparison, so that the comparison area can be automatically located during the image comparison process of two multi-page image files, greatly improving the comparison area positioning efficiency, ensuring the correctness of subsequent image comparison results, and facilitating practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of a multi-page image comparison intelligent segmentation and positioning method provided in an embodiment of the present application.

[0051] Figure 2 This is an example diagram of a large-format paper after multiple pages of imposition provided in an embodiment of the present application.

[0052] Figure 3 This is an example diagram of the result of arranging multiple pages on both sides of a large-format paper provided in an embodiment of the present application, wherein: Figure 3 (a) in the figure shows an example of a multi-page layout result on the front side of a large-format paper. Figure 3 (b) in the figure shows an example of the result of arranging multiple pages on the back side of a large format paper.

[0053] Figure 4 This is an example diagram of the positional relationship among the cropping area, bleed area, artwork area, and media area provided in an embodiment of the present application.

[0054] Figure 5 This is an example diagram of the positional relationship between the comparison area positioning frame and the minimum rectangular frame provided in an embodiment of the present application, wherein: Figure 5 (a) shows an example diagram of the positional relationship between the first comparison area positioning frame and the first minimum rectangular frame in the first image. Figure 5 (b) shows an example diagram of the positional relationship between the second comparison area positioning frame and the second minimum rectangular frame in the second image.

[0055] Figure 6 This is an example diagram of the third smallest rectangular frame in any grayscale image provided by an embodiment of the present application.

[0056] Figure 7 A schematic diagram of the structure of a multi-page image comparison intelligent positioning device provided in an embodiment of the present application.

[0057] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structures of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0059] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0060] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0061] Example

[0062] like Figure 1 As shown, the multi-page image comparison intelligent segmentation and positioning method provided in the first aspect of this embodiment can be, but is not limited to, executed by a computer device with certain computing resources, such as a platform server, a personal computer (PC, which refers to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PDA) or wearable devices. Figure 1 As shown, the multi-page image comparison intelligent precise segmentation and precise positioning method may include but is not limited to the following steps S1 to S9.

[0063] S1. Obtain a first image file and a second image file to be compared, wherein the first image file is an electronic file in PDF (Portable Document Format) format and contains multiple pages of first images, and the second image file contains multiple pages of second images.

[0064] In step S1, the first image file and the second image file are two image files to be compared. Specifically, the first image file may be, but is not limited to, a multi-page electronic image file corresponding to a multi-page printed product or a multi-page large-format electronic image file after imposition, etc., and the second image file may be, but is not limited to, another multi-page electronic image file corresponding to the multi-page printed product, another multi-page large-format electronic image file after imposition, a multi-page scanned image file, a multi-page large-format scanned image file, or a multi-page large-format printed product obtained by double-sided scanning after folding, cutting and trimming. The aforementioned multi-page electronic image file or multi-page large-format electronic image file may be the customer's original file to be printed directly, or may be a new electronic image file obtained by conventional processing based on the customer's original file to be printed. The aforementioned scanned image file obtained by double-sided scanning after folding, cutting and trimming of a multi-page large-format printed product refers to a scanned image file obtained by folding, cutting and trimming of a multi-page large-format printed product obtained by printing on a multi-page large-format paper (for example, Figure 2 As shown in the figure, after folding, cutting and de-edging the multiple pages of paper (a single sheet of a large-format paper contains 8 pages of content), the image file obtained by double-sided scanning of the multiple pages of paper can be obtained by, but not limited to, using an existing sheet-fed scanner to perform double-sided scanning on the multiple pages of paper. The aforementioned multi-page large-format scanned image file refers to an image file obtained by directly scanning the printed multiple pages of large-format paper after imposition, and can be obtained by, but not limited to, using an existing large-scale scanner to perform single-sided or double-sided scanning on the multiple pages of large-format paper after imposition. In addition, the specific method for obtaining the first image file can be, but not limited to, importing by a staff member; the specific method for obtaining the second image file can be, but not limited to, importing by a staff member or transmitting by a scanner after scanning; the file format of the second image file can be PDF format or other file formats.

[0065] S2. Determine a one-to-one correspondence between the multiple pages of first images and the multiple pages of second images.

[0066] In the step S2, specifically, determining a one-to-one correspondence between the multiple pages of first images and the multiple pages of second images includes, but is not limited to: in the case where the page number sequence of the multiple pages of first images is a known condition, if the second image file is a multi-page scanned image file, then the scanning sequence of the multiple pages of second images is matched with the page number sequence of the multiple pages of first images to obtain a one-to-one correspondence between the multiple pages of first images and the multiple pages of second images (for example, the first scanned image in the multiple pages of second images corresponds to the first page of the first image in the multiple pages of first images); and if the second image file is a large-format scanned image file after multi-page imposition, then based on the folding logic (that is, the common sense in the field of printing technology, for example Figure 3 The first image file is a multi-page electronic image file (i.e., the second image file of the second page corresponds to the first image of the first page after the multi-page imposition), and the page number sequence of the multi-page second image is determined based on the page number sequence of the multi-page second image on both sides of a large-format paper and based on the folding requirements, and then the page number sequence of the multi-page second image is matched with the page number sequence of the multi-page first image to obtain a one-to-one correspondence between the multi-page first image and the multi-page second image (for example, the last page second image in the multi-page second image corresponds to the last page first image in the multi-page first image); and if the second image file is a multi-page electronic image file (i.e., the other multi-page electronic image file or the other multi-page large-format electronic image file after imposition), the page number sequence of the multi-page second image can be obtained directly based on the sequence of the electronic images, and then the page number sequence of the multi-page second image is matched with the page number sequence of the multi-page first image to obtain a one-to-one correspondence between the multi-page first image and the multi-page second image. Since the first image file is a multi-page electronic image file, the page number sequence of the multi-page first image can be obtained directly based on the sequence of the electronic images. In addition, the plurality of pages of the first image and the plurality of pages of the second image need to be consistent in the number of pages. If they are inconsistent, it can be directly determined that the image comparison fails.

[0067] In the step S2, in order to verify the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, preferably, after determining the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, the method also includes but is not limited to the following steps S21 to S24.

[0068] S21. For each page of the first images in the plurality of pages of first images, extract corresponding page content text from the corresponding image using optical character recognition technology, and count the number of corresponding words in the page content text.

[0069] In the step S21, the optical character recognition technology is the existing OCR (Optical Character Recognition) technology, so the page content text can be conventionally extracted from the image.

[0070] S22. For each page of the second images in the plurality of pages of second images, the optical character recognition technology is applied to extract the corresponding page content text from the corresponding image, and the number of corresponding words in the page content text is counted.

[0071] S23. Arrange the first images of each page in descending order according to the number of words to obtain a first image queue, and also arrange the second images of each page in descending order according to the number of words to obtain a second image queue.

[0072] S24. For each pair of first images and second images having the same serial number in the first image queue and the second image queue, if the corresponding two images do not have a mapping relationship in the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, and the corresponding two pages of content texts have content consistency characteristics, an output is displayed to indicate that there is a mapping relationship mismatch between the corresponding two images, so as to remind the staff to conduct manual inspection and correct the mapping relationship.

[0073] In the step S24, for example, if the fifth first image in the first image queue and the fifth second image in the second image queue do not have a mapping relationship in the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, and the page content text of the fifth first image and the page content text of the fifth second image have content consistency characteristics, it means that the fifth first image and the fifth second image should have a mapping relationship but do not have it, and then it is necessary to output a reminder message indicating that there is a mismatch in the mapping relationship between the fifth first image and the fifth second image, so as to remind the staff to manually check and correct the mapping relationship. In addition, for each pair of first images and second images with the same sequence number in the first image queue and the second image queue, if the corresponding two images have a mapping relationship in the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, and the corresponding two pages of content text have content inconsistency characteristics, then a reminder message indicating that there is a mismatch in the mapping relationship between the corresponding two images is also output to remind the staff to manually check and correct the mapping relationship. In addition, the above-mentioned specific method of determining whether the content texts of two pages have the characteristic of content consistency can be conventionally implemented based on, but not limited to, indicators such as word frequency, edit distance and / or Jaccard distance.

[0074] S3. For each first image page in the first image file, read the PDF file information of the first image file, and determine a corresponding first comparison area positioning frame for locating a corresponding image comparison target area according to the PDF file printing requirement attributes in the PDF file information, wherein the image comparison target area may be, but is not limited to, an image area surrounded by a media box, a crop box, a bleed box, a trim box or a work box.

[0075] Specifically, when the image file is a small page file, the media box, crop box, bleed box, trim box or artwork box is directly read to locate the comparison area, the PDF information of the image file is read, and the coordinates of the media box, crop box, bleed box, trim box or artwork box are extracted. According to the extracted frame coordinates, the image comparison target area is directly located. When the image file is a large-format imposition file, the following precise segmentation method is used: If the large-format image contains a cutting line, the cutting line in the large-format image is identified, and the area within the cutting line of each small page is used as a precise segmentation area. If there is no cutting line, the imposition folding reverse parsing technology is used to determine the logical order and position of each small page. This technology is related to the registered patent "A comparison quality inspection method for imposition images". If there is no cutting line, the specific steps of manual segmentation are:

[0076] S31. Specify the boundaries of each small page through interface operation, and manually specify some imposition parameters so that the system can assist in segmentation more accurately. The imposition parameters are image size: the system automatically reads the length and width of the image to determine the overall area of ​​the large file; number of imposition pages: the user specifies the number of imposition pages (such as 8 pages, 16 pages, etc.) according to actual conditions; finished product size: the user specifies the finished product size of each small page (such as 185mm*260mm), which is the size of the trimming box.

[0077] S32. Calculate the size of the bleed box and the media box. The system automatically calculates the size of the media box or the bleed box according to the finished product size specified by the user and the built-in bleed size (such as 3mm).

[0078] S33. Built-in imposition model. The system has a built-in imposition model, which contains the necessary size parameters in the printing process, such as grip size, row space size, column space size, etc. These size parameters are used to ensure that the divided small page files meet the actual printing requirements.

[0079] S34. Manually adjust the image and layout. The user manually draws or selects the boundaries of each small page on the interface provided by the system according to the actual situation of the image and the layout logic. The system automatically adjusts the layout of each page file according to the boundaries and layout model specified by the user to center it on the overall size. If necessary, the user can also manually adjust the layout according to the gripping data to ensure that the layout size of each page is basically consistent with the original data of the layout logic. After completing the manual segmentation and layout adjustment, the system accurately segments the large version file into multiple small page files according to the user's specifications.

[0080] In step S3, since the first image file is an electronic file in PDF format, the PDF file information can be read conventionally (for example, all information of the PDF file can be read using an editing-level PDF kernel), and since there is an item in the PDF file information that describes the printing requirements of the PDF file, and describes preset area attributes such as the cropping area, bleed area, cropping area, artwork area, and media area, when the image comparison target area is the cropping area, bleed area, cropping area, artwork area, or media area, etc., the first comparison area positioning frame used to locate the image comparison target area can be directly determined based on the PDF file information. In addition, the aforementioned cropping area, bleed area, cropping area, artwork area, and media area are all common terms in PDF file information, and their positional relationships are as follows: Figure 4 As shown; if the PDF file information does not include parameters for describing the media box, the crop box, the bleed box, the trim box or the artwork box, the entire area of ​​the first image may be directly defaulted as the image comparison target area, or the image comparison target area may be manually defined in the first image.

[0081] S4. For each page of the first image, use the corresponding first comparison area positioning frame to capture the corresponding first comparison area image from the corresponding image.

[0082] S5. For the first images of each page, the corresponding first comparison area images are imported into the trained variable-length text recognition model, and the corresponding first character strings of each line that correspond one-to-one to each line of text in the corresponding first comparison area images are output, and the corresponding first minimum rectangular frame is determined based on the first character strings of each line, wherein the first minimum rectangular frame refers to a rectangular frame with the minimum area that is used to enclose each line of text that has a mapping relationship with the first character strings of each line in the corresponding first comparison area image.

[0083] In step S5, the indefinite-length text recognition model is an existing model dedicated to indefinite-length text recognition; indefinite-length text recognition is widely used in reality, such as text recognition of printed text and billboard text, etc. Although it has the characteristics of unfixed and unpredictable number of characters and greater recognition difficulty, it is also the main direction of current research on text recognition, so that the commonly used indefinite-length text recognition methods are: Long Short-Term Memory Network and Connectionist Temporal Classifier LSTM+CTC (the former is a special structure of Recurrent Neural Networks Recurrent Neural Networks, used to solve the long-term dependency problem of Recurrent Neural Network RNN, and the latter is used to solve the alignment problem of input features and output labels), Convolutional Recurrent Neural Network CRNN (Convolutional Recurrent Neural Network Network, which is a popular text recognition model at present, does not need to perform character segmentation on sample data, can recognize text sequences of any length, and has the characteristics of fast speed and good performance) and text recognition project chinsesocr (which is a natural scene text recognition project based on YOLO3 and CRNN, which supports text detection of darknet / opencvdnn / keras, supports direction detection of 0, 90, 180 and 270 degrees, supports variable-length English and Chinese-English recognition, and supports general optical character recognition OCR, ID card recognition and train ticket recognition and other scenarios), etc. Therefore, the variable-length text recognition model can be but not limited to the recognition model based on long short-term memory network and connection time classifier LSTM+CTC, convolutional recurrent neural network CRNN or text recognition project chineseocr. In addition, the training method of the aforementioned existing recognition model is also an existing conventional method, so that the trained variable-length text recognition model has the ability to recognize variable-length text for the imported image, and can obtain the first character string of each line corresponding to each line of text in the first comparison area image. In addition, the specific manner of determining the first minimum rectangular frame may be, but is not limited to, conventionally determined based on the coordinates of each line of text having a mapping relationship with each line of the first character string in the first comparison area image.

[0084] S6. For each page of the second image in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character strings are outputted, which correspond one-to-one to each line of text in the corresponding image, and the corresponding second minimum rectangular frame is determined according to each line of the second character string, wherein the second minimum rectangular frame refers to a rectangular frame with the minimum area that is used to enclose each line of text that has a mapping relationship with each line of the second character string in the corresponding image.

[0085] In the step S6, the specific determination method of the second minimum rectangular frame can also be, but is not limited to, conventionally determined based on the coordinates of each line of text having a mapping relationship with each line of the second character string in the corresponding second image. Considering that there may also be text in the area outside the second comparison area positioning box used to locate the image comparison target area of ​​the second image, this will cause each line of the second character string to contain characters outside the box, thereby causing the second minimum rectangular frame directly determined based on the coordinates of each line of text having a mapping relationship with each line of the second character string in the corresponding second image to be too large, affecting the correctness of the subsequent image comparison results. Therefore, in order to further ensure the correctness of the subsequent image comparison results, preferably, for each page of the second image in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character string of each line corresponding to each line of text in the corresponding image is output, and the corresponding second minimum rectangular frame is determined according to each line of the second character string, including but not limited to the following steps S61 to S63.

[0086] S61. For a second image on a certain page in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character strings of each line that correspond one-to-one to each line of text in the corresponding image are output.

[0087] S62. According to the rows of first character strings of a certain page of the first image having a mapping relationship with the certain page of the second image, determine at least one row of character strings having content consistency characteristics with the rows of first character strings from the rows of second character strings.

[0088] In step S62, for example, if the first character string of each line of the first image of the certain page is: "AABBBC; DEEEFF" (that is, there are two lines of first character strings, wherein the first line of the first character string is "AABBBC" and the last line of the first character string is "DEEEFF"), and the second character string of each line of the second image of the certain page is: "IAABBBCJ; MDEEEFFN" (that is, there are also two lines of second character strings, wherein the first line of the second character string is "IAABBBCJ" and the last line of the second character string is "MDEEEFFN"), then at least one line of character strings having content consistency characteristics with the first character strings of each line can be determined from the second character strings of each line as: "AABBBC; DEEEFF" (that is, there are also two lines of character strings, and it can be determined that "I", "J", "M" and "N" are all out-of-box characters). In addition, the aforementioned specific method of determining whether the two character strings have content consistency characteristics can also be conventionally implemented based on, but not limited to, indicators such as word frequency, edit distance and / or Jaccard distance.

[0089] S63. Determine a second minimum rectangular frame of the second image of the certain page based on the at least one line of character string, wherein the second minimum rectangular frame refers to a rectangular frame with a minimum area that is used to enclose at least one line of text that has a mapping relationship with the at least one line of character string in the second image of the certain page.

[0090] In step S63, the specific method of determining the second minimum rectangular frame can also be, but is not limited to, conventionally determined based on the coordinates of at least one line of text that has a mapping relationship with the at least one line of character string in the second image of the certain page. In addition, the second minimum rectangular frame obtained based on step S63 is used to replace the second minimum rectangular frame obtained based on step S6.

[0091] S7. For each page of the second image, determine a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image.

[0092] In step S7, considering that the first comparison region positioning frame must surround the first minimum rectangular frame, the second comparison region positioning frame must also surround the second minimum rectangular frame. Specifically, as Figure 5As shown, for each page of the second image, according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, a corresponding second comparison area positioning frame for locating the corresponding image comparison target area is determined, including but not limited to the following steps: for any second image in the second image file, according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, a corresponding second comparison area positioning frame for locating the corresponding image comparison target area is determined according to the following formula:

[0093]

[0094] In the formula, x 21,btl represents the horizontal coordinate position of the lower left corner of the second comparison area positioning frame in any second image, y 21,btl represents the ordinate position of the lower left corner of the second comparison region positioning frame in any second image, x 21,tpr represents the horizontal coordinate position of the upper right corner of the second comparison area positioning frame in any second image, y 21,tpr represents the vertical coordinate position of the upper right corner of the second comparison area positioning frame in any second image, x 22,btl represents the horizontal coordinate position of the lower left corner of the second smallest rectangular frame in any second image, y 22,btl represents the ordinate position of the lower left corner of the second smallest rectangular frame in any second image, x 22,tpr represents the horizontal coordinate position of the upper right corner of the second smallest rectangular frame in any second image, y 22,tpr represents the vertical coordinate position of the upper right corner of the second smallest rectangular frame in any second image, x 12,btl represents the horizontal coordinate position of the lower left corner of the first minimum rectangular frame in the first image corresponding to any second image, y 12,btl represents the ordinate position of the lower left corner of the first minimum rectangular frame in the first image corresponding to any second image, x 12,tpr represents the horizontal coordinate position of the upper right corner of the first minimum rectangular frame in the first image corresponding to any second image, y 12,tpr represents the ordinate position of the upper right corner of the first minimum rectangular frame in the first image corresponding to any second image, x 11,btl represents the horizontal coordinate position of the lower left corner point of the first comparison area positioning frame in the first image corresponding to any second image, y 11,btl represents the ordinate position of the lower left corner of the first comparison region positioning frame in the first image corresponding to any one of the second images, x11,tpr represents the horizontal coordinate position of the upper right corner point of the first comparison area positioning frame in the first image corresponding to any second image, y 11,tpr Indicates the vertical coordinate position of the upper right corner point of the first comparison area positioning frame in the first image corresponding to any one of the second images.

[0095] S8. For each page of the second image, use the corresponding second comparison area positioning frame to capture the corresponding second comparison area image from the corresponding image.

[0096] Specifically, for the manual segmentation method, the system obtains the precise segmentation areas of each small page file obtained after manual segmentation. The system prepares template data for matching based on the preset or user-specified positioning template. The system matches each precise segmentation area with the positioning template and determines whether the match is successful by comparing the characteristics of the area (such as shape, size, position, etc.). If a precise segmentation area successfully matches the positioning template, the system considers that the area has been precisely located. If there are unmatched or inaccurately matched areas, the system can prompt the user to make manual adjustments or re-segment. The system finally outputs the results of precise positioning, including a list of small page files that have been successfully located, prompt information for failed positioning, and necessary adjustment suggestions.

[0097] S9. For each pair of first images and second images in the first image file and the second image file that have a mapping relationship, scale the corresponding first contrast area image or the second contrast area image so that the corresponding first contrast area image and the second contrast area image have the same size and resolution.

[0098] In step S9, since the image comparison is a difference comparison at the pixel level, it is necessary to make the two comparison area images to be compared have the same size and resolution. In addition, the specific scaling process may include but is not limited to: taking the size and resolution of any one of the first comparison area image and the corresponding second comparison area image as a reference, performing conventional image compression, image expansion and / or image resolution conversion on the other of the first comparison area image and the corresponding second comparison area image, so that the two images have the same size and resolution, for example, the resolution of the two images is 300dpi or 600dpi.

[0099] Therefore, based on the multi-page image comparison intelligent precise segmentation and precise positioning method described in the aforementioned steps S1 to S9, a new solution is provided that can automatically locate the comparison area during the image comparison process of a multi-page image file, that is, after determining the one-to-one correspondence between the multi-page first images to be compared and the multi-page second images, firstly determine the first comparison area positioning frame of each page of the first image and used to locate the corresponding image comparison target area based on the PDF file information, then apply the frame to obtain the first comparison area image and determine the first minimum rectangular frame surrounding the text in the image, and at the same time determine the second minimum rectangular frame of each page of the second image and also surrounding the text in the image, and then determine the second comparison area positioning frame and the second comparison area image of each page of the second image and also used to locate the image comparison target area based on the aforementioned three frames, and finally zoom to obtain the two area images suitable for image comparison, so that the comparison area can be automatically located during the image comparison process of two multi-page image files, greatly improving the efficiency of comparison area positioning, ensuring the correctness of subsequent image comparison results, and facilitating practical application and promotion.

[0100] Based on the technical solution of the first aspect mentioned above, this embodiment also provides a possible design of how to automatically locate the difference area and give a difference description during the image comparison process. That is, after making the first comparison area image and the second comparison area image have the same size and resolution, the method also includes but is not limited to the following steps S101 to S105.

[0101] S101. Perform grayscale processing on the first contrast region image and the second contrast region image respectively to obtain a first grayscale image and a second grayscale image.

[0102] In the step S101, specifically, the first contrast region image is subjected to the grayscale processing to obtain the first grayscale image; and the second contrast region image is subjected to the grayscale processing to obtain the second grayscale image.

[0103] S102. Determine a plurality of first pixel points in any grayscale image according to a preset grayscale threshold interval and whose grayscale values ​​belong to the grayscale threshold interval, and respectively compare the grayscale values ​​of each first pixel point among the plurality of first pixel points in the first grayscale image and the second grayscale image to determine a second pixel point that does not have a grayscale value consistency characteristic in the first grayscale image and the second grayscale image, wherein the any grayscale image refers to the first grayscale image or the second grayscale image.

[0104] In step S102, the grayscale threshold interval is used to screen feature pixels participating in pixel-level comparison, which can be set in advance by a staff member or randomly generated in advance, for example, [100, 120]. Considering that the total number of feature pixels participating in pixel-level comparison should not be too small, so as not to affect the accuracy of subsequent difference area positioning results, preferably, a plurality of first pixels in any grayscale image whose grayscale values ​​belong to the grayscale threshold interval are determined according to a preset grayscale threshold interval, including but not limited to the following steps: determining a plurality of first pixels in any grayscale image whose grayscale values ​​belong to the grayscale threshold interval according to a preset grayscale threshold interval, and determining whether the total number of the plurality of first pixels is lower than a preset first pixel number threshold, if so, adjusting and updating the grayscale threshold interval, and then determining a plurality of first pixels in any grayscale image whose grayscale values ​​belong to the new grayscale threshold interval according to the new grayscale threshold interval, until it is determined that the total number of the plurality of first pixels is greater than or equal to the first pixel number threshold, wherein the any grayscale image refers to the first grayscale image or the second grayscale image. The specific method of adjusting and updating the grayscale threshold interval mentioned above may be, but not limited to, expanding the interval or changing the interval. In addition, it is also considered that the total number of feature pixels participating in the pixel level comparison should not be too large, so as not to lose the meaning of screening feature pixels. Preferably, multiple first pixels whose grayscale values ​​belong to the grayscale threshold interval in any grayscale image are determined according to the preset grayscale threshold interval, including but not limited to the following steps: multiple first pixels whose grayscale values ​​belong to the grayscale threshold interval in any grayscale image are determined according to the preset grayscale threshold interval, and whether the total number of the multiple first pixels is higher than the preset second pixel number threshold, if so, adjusting and updating the grayscale threshold interval, and then determining again the multiple first pixels whose grayscale values ​​belong to the new grayscale threshold interval in any grayscale image according to the new grayscale threshold interval, until it is determined that the total number of the multiple first pixels is less than or equal to the second pixel number threshold, wherein the any grayscale image refers to the first grayscale image or the second grayscale image. The specific method of adjusting and updating the grayscale threshold interval mentioned above may be, but not limited to, shrinking the interval or changing the interval. In addition, the second pixel number threshold needs to be greater than the first pixel number threshold; the first pixel number threshold and the second pixel number threshold can be pre-set in positive correlation based on the corresponding image size or image resolution, that is, the larger the image size or the higher the image resolution, the larger the first pixel number threshold and the second pixel number threshold are set, and vice versa.

[0105] In the step S102, it is also considered that the multiple feature pixel points involved in the pixel level comparison need to have a uniform distribution characteristic to avoid the situation where the pixel level comparison occurs in the local area, thereby affecting the accuracy of the subsequent difference area positioning results. Preferably, a plurality of first pixel points whose grayscale values ​​belong to the grayscale threshold interval in any grayscale image are determined according to a preset grayscale threshold interval, including but not limited to the following steps: a plurality of first pixel points whose grayscale values ​​belong to the grayscale threshold interval in any grayscale image are determined according to a preset grayscale threshold interval, and any grayscale image is gridded into pixels of the same size. Multiple image sub-regions, then statistically obtain the number of the first pixel points contained in each of the multiple image sub-regions and the number standard deviation of the multiple image sub-regions, and finally determine whether the number standard deviation is higher than the preset standard deviation threshold. If so, adjust and update the gray threshold interval, and then determine again according to the new gray threshold interval a plurality of first pixel points in any gray image whose gray value belongs to the new gray threshold interval, until it is determined that the number standard deviation is less than or equal to the standard deviation threshold, wherein the any gray image refers to the first gray image or the second gray image. For example, assuming that the size of any gray image is 6000×9000, the any gray image can be gridded into 100 image sub-regions of a size of 600×900, and then the number of the first pixel points contained in each of the 100 image sub-regions and the number average and number standard deviation of the 100 image sub-regions are statistically obtained. In addition, the standard deviation threshold and / or the total number of areas of the multiple image sub-areas can also be pre-set in a positive correlation based on the corresponding image size or image resolution, that is, the larger the image size or the higher the image resolution, the larger the standard deviation threshold and / or the total number of areas of the multiple image sub-areas is set, and vice versa.

[0106] In the step S102, specifically, the grayscale values ​​of each first pixel among the multiple first pixel points in the first grayscale image and the second grayscale image are compared respectively to determine the second pixel point that does not have the grayscale value consistency feature in the first grayscale image and the second grayscale image, including but not limited to: if the absolute value of the difference in the grayscale value of a certain first pixel among the multiple first pixel points in the first grayscale image and the second grayscale image is greater than a preset grayscale threshold, then it is determined that the certain first pixel point does not have the grayscale value consistency feature in the first grayscale image and the second grayscale image, and the certain first pixel point is used as a second pixel point, otherwise it is determined that the certain first pixel point has the grayscale value consistency feature in the first grayscale image and the second grayscale image. In addition, the aforementioned grayscale threshold is exemplified but not limited to 10.

[0107] S103. If the total number of the second pixel points is greater than zero, determine a third minimum rectangular frame within any grayscale image based on all the second pixel points, wherein the third minimum rectangular frame refers to a rectangular frame with the smallest area that is used to enclose all the second pixel points.

[0108] In step S103, for example, if the second pixel is represented by a black dot, the third minimum rectangular frame can be represented by Figure 6 In addition, the specific determination method of the third minimum rectangular frame can be, but is not limited to, conventionally determined based on the coordinates of all the second pixel points in any grayscale image.

[0109] S104. If the area proportion of the first minimum rectangular frame in any grayscale image is greater than a preset first ratio threshold, a first contrast area sub-image and a second contrast area sub-image surrounded by the third minimum rectangular frame are respectively captured from the first contrast area image and the second contrast area image.

[0110] In step S104, the third minimum rectangular frame is the located difference area; the first ratio threshold is used as a criterion for measuring whether the difference area can be ignored in the current image comparison process, that is, if the area of ​​the third minimum rectangular frame in any grayscale image is too small, the corresponding difference area can be ignored in the current image comparison process, otherwise it is not. In addition, the first ratio threshold is 5% for example.

[0111] S105. Import the first contrast area sub-image and the second contrast area sub-image into a pre-trained image difference description generation model, output a first image difference description text, and load and display the third minimum rectangular frame and the first image difference description text on the first contrast area image and the second contrast area image.

[0112] In step S105, the goal of the image difference captioning (IDC) model is to compare two similar images and capture the visual differences between them, and then describe these differences in natural language. Specifically, it can be implemented by, but not limited to, using an existing model based on pre-extracted object-level difference features, a model based on a dynamic correlation attention mechanism, a difference description model based on a Transformer (a classic NLP model proposed by a Google team in 2017) structure, and / or a model that combines a semantic segmentation model and a graph convolutional neural network to describe image differences. Specifically, the first image difference description text can be, but not limited to, composed of a number of keywords, so as to form a difference description content that can be intuitively perceived by the staff. The specific pre-training process of the image difference description generation model can also be implemented with reference to the prior art. In detail, the third minimum rectangular box and the first image difference description text are loaded and displayed on the first comparison area image and the second comparison area image, which can be specifically: the third minimum rectangular box and the first image difference description text are loaded and displayed on the first comparison area image and / or the second comparison area image, so that the staff can intuitively perceive the difference area positioning result and the given difference description content.

[0113] Due to the possible design one described in the aforementioned steps S101 to S105, the difference area can be automatically located and a difference description can be given during the image comparison process of two multi-page image files, which is beneficial for performing corresponding difference elimination processing in the subsequent printing process to ensure the quality of subsequent printed products.

[0114] Based on the technical solution of the aforementioned possible design one, this embodiment also provides a possible design two for further finely locating the difference area and giving the corresponding difference description content, that is, after determining the third minimum rectangular frame in any grayscale image based on all the second pixel points, the method also includes but is not limited to the following steps S1061 to S1066.

[0115] S1061. If the area proportion of the third minimum rectangular frame in any grayscale image is greater than a preset second ratio threshold, a first grayscale sub-image and a second grayscale sub-image surrounded by the third minimum rectangular frame are respectively obtained from the first grayscale image and the second grayscale image, wherein the second ratio threshold is greater than the first ratio threshold.

[0116] In the step S1061, the second ratio threshold is used as a criterion for determining whether to further fine-tune the location of the difference region, and may be 10% for example.

[0117] S1062. Perform gridding processing on the first grayscale sub-image and the second grayscale sub-image respectively to obtain a plurality of first grayscale sub-images and a plurality of second grayscale sub-images corresponding to the plurality of first grayscale sub-images one by one.

[0118] In the step S1062, the first grayscale sub-image refers to a sub-image of the first grayscale sub-image, and the second grayscale sub-image refers to a sub-image of the second grayscale sub-image; specifically, the first grayscale sub-image is subjected to gridding processing to obtain the plurality of first grayscale sub-images; the second grayscale sub-image is subjected to the gridding processing to obtain the plurality of second grayscale sub-images. In addition, the number of the plurality of first grayscale sub-images or the plurality of second grayscale sub-images can be set in a positive correlation based on the ratio of the second ratio threshold to the first ratio threshold, that is, the larger the ratio is, the more the number of the plurality of first grayscale sub-images or the plurality of second grayscale sub-images is set, so as to complete the further difference area positioning task in one step.

[0119] S1063. For each pair of first grayscale grandchild images and second grayscale grandchild images that have a mapping relationship among the multiple first grayscale grandchild images and the multiple second grayscale grandchild images, determine a plurality of third pixel points in any corresponding grayscale grandchild image and whose grayscale values ​​belong to the grayscale threshold interval according to a preset grayscale threshold interval, and compare the grayscale values ​​of each third pixel point in the multiple third pixel points in the corresponding grayscale grandchild image pair, and determine the corresponding fourth pixel point that does not have the grayscale value consistency characteristic in the corresponding grayscale grandchild image pair.

[0120] In the step S1063, the any grayscale grandchild image is the first grayscale grandchild image or the second grayscale grandchild image in each pair of the first grayscale grandchild image and the second grayscale grandchild image. In addition, the specific determination details of the third pixel point and the fourth pixel point can be obtained by referring to the derivation of the aforementioned step S102, which will not be repeated here.

[0121] S1064. For each pair of the first grayscale grandson image and the second grayscale grandson image, if the number of the corresponding fourth pixel points is non-zero, determine the corresponding fourth minimum rectangular frame within any of the grayscale grandson images based on all the corresponding fourth pixel points, wherein the fourth minimum rectangular frame refers to a rectangular frame with the smallest area that is used to enclose all the corresponding fourth pixel points.

[0122] In step S1064, the specific details of determining the fourth minimum rectangular frame can be referred to the aforementioned step S103, which will not be repeated here. In addition, for each pair of the first grayscale grandchild image and the second grayscale grandchild image, if the number of the corresponding fourth pixel points is zero, there is no need to generate the corresponding fourth minimum rectangular frame (that is, there is no corresponding difference area).

[0123] S1065. For each pair of first grayscale sub-images and second grayscale sub-images, if the area ratio of the corresponding fourth minimum rectangular frame in any grayscale image is greater than a preset third ratio threshold, then the corresponding third contrast area sub-image and fourth contrast area sub-image surrounded by the corresponding fourth minimum rectangular frame are cut out from the first contrast area image and the second contrast area image.

[0124] In step S1065, the fourth minimum rectangular frame is the difference area obtained by further fine positioning; the third ratio threshold is used as a criterion for measuring whether the difference area can be ignored in the current image comparison process, that is, if the area of ​​the fourth minimum rectangular frame in any grayscale image is too small, the corresponding difference area can be ignored in the current image comparison process, and vice versa. In addition, the third ratio threshold can be set in a negative correlation based on the ratio of the second ratio threshold to the first ratio threshold, that is, the larger the ratio is, the smaller the third ratio threshold needs to be set because the number of the multiple first grayscale grandchild images or the multiple first grayscale grandchild images is larger; and the third ratio threshold can also be equal to the first ratio threshold, that is, 5% for example.

[0125] S1066. For each pair of the first grayscale sub-image and the second grayscale sub-image, the corresponding third contrast area sub-image and the fourth contrast area sub-image are imported into the pre-trained image difference description generation model, the corresponding second image difference description text is output, and the fourth minimum rectangular box and the second image difference description text are loaded and displayed on the first contrast area image and the second contrast area image.

[0126] Due to the possible design 2 described in the aforementioned steps S1061 to S1066, when the initially located difference area is too large, the difference area can be further precisely located and corresponding difference description content can be given to improve the user experience.

[0127] like Figure 7As shown, the second aspect of this embodiment provides a virtual device for implementing the multi-page image comparison intelligent accurate segmentation and accurate positioning method described in the first aspect or any possible design in the first aspect, including an image file acquisition unit, a corresponding relationship determination unit, a first positioning frame determination unit, a first area image capture unit, a first rectangular frame determination unit, a second rectangular frame determination unit, a second positioning frame determination unit, a second area image capture unit and an image scaling processing unit;

[0128] The image file acquisition unit is used to acquire a first image file and a second image file to be compared, wherein the first image file is an electronic file in PDF format and contains multiple pages of first images, and the second image file contains multiple pages of second images;

[0129] The correspondence determination unit is communicatively connected to the image file acquisition unit and is used to determine a one-to-one correspondence between the plurality of pages of first images and the plurality of pages of second images;

[0130] The first positioning frame determining unit is communicatively connected to the image file acquiring unit, and is used to read the PDF file information of the first image file for each page of the first image in the first image file, and determine a corresponding first comparison area positioning frame for locating a corresponding image comparison target area according to the PDF file printing requirement attribute in the PDF file information, wherein the image comparison target area is an image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box;

[0131] The first region image capture unit is communicatively connected to the first positioning frame determination unit, and is used to apply the corresponding first comparison region positioning frame to the first image of each page to capture the corresponding first comparison region image from the corresponding image;

[0132] The first rectangular frame determination unit is communicatively connected to the first area image interception unit, and is used for importing the corresponding first comparison area image into the indefinite-length text recognition model that has been trained for each page of the first image, outputting the first character strings of each line that correspond to each line of text in the corresponding first comparison area image, and determining the corresponding first minimum rectangular frame according to each line of the first character string, wherein the first minimum rectangular frame refers to a rectangular frame with the minimum area that is used to surround each line of text that has a mapping relationship with each line of the first character string in the corresponding first comparison area image;

[0133] The second rectangular frame determination unit is communicatively connected to the image file acquisition unit, and is used for importing the corresponding image into the variable-length text recognition model for each page of the second image in the second image file, outputting the corresponding second character strings of each line corresponding to each line of text in the corresponding image, and determining the corresponding second minimum rectangular frame according to each line of the second character string, wherein the second minimum rectangular frame refers to a rectangular frame with the minimum area used to enclose each line of text having a mapping relationship with each line of the second character string in the corresponding image;

[0134] The second positioning frame determining unit is communicatively connected to the corresponding relationship determining unit, the first positioning frame determining unit, the first rectangular frame determining unit and the second rectangular frame determining unit, respectively, and is used to determine, for each page of the second image, a corresponding second comparison area positioning frame for locating the corresponding image comparison target area according to the corresponding second minimum rectangular frame and the first minimum rectangular frame of the corresponding first image and the first comparison area positioning frame;

[0135] The second region image capture unit is communicatively connected to the second positioning frame determination unit, and is used to apply the corresponding second comparison region positioning frame to the second image of each page to capture the corresponding second comparison region image from the corresponding image;

[0136] The image scaling processing unit is communicatively connected to the correspondence determination unit, the first area image capture unit and the second area image capture unit, respectively, and is used to scale the corresponding first contrast area image or the second contrast area image for each pair of first images and second images in the first image file and the second image file and having a mapping relationship, so that the corresponding first contrast area image and the second contrast area image have the same size and resolution.

[0137] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be referred to the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect, and will not be repeated here.

[0138] like Figure 8As shown, the third aspect of this embodiment provides a computer device for executing the multi-page image comparison intelligent accurate segmentation and accurate positioning method as described in the first aspect or any possible design in the first aspect, including a memory, a processor and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program, and execute the multi-page image comparison intelligent accurate segmentation and accurate positioning method as described in the first aspect or any possible design in the first aspect. For example, the memory may include but is not limited to random access memory (Random-Access Memory, RAM), read-only memory (Read-Only Memory, ROM), flash memory (Flash Memory), first-in-first-out memory (First Input First Output, FIFO) and / or first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may be but is not limited to a microprocessor of the STM32F105 series. In addition, the computer device may also include but is not limited to a power module, a display screen and other necessary components.

[0139] The working process, working details and technical effects of the aforementioned computer device provided in the third aspect of this embodiment can be referred to the multi-page image comparison intelligent precise segmentation and precise positioning method described in the first aspect or any possible design in the first aspect, and will not be repeated here.

[0140] The fourth aspect of this embodiment provides a computer-readable storage medium storing instructions for the multi-page image comparison intelligent accurate segmentation and accurate positioning method as described in the first aspect or any possible design in the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are run on a computer, the multi-page image comparison intelligent accurate segmentation and accurate positioning method as described in the first aspect or any possible design in the first aspect is executed. Wherein, the computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0141] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the multi-page image comparison intelligent precise segmentation and precise positioning method as described in the first aspect or any possible design in the first aspect, and will not be repeated here.

[0142] In a fifth aspect, the present embodiment provides a computer program product, including a computer program or an instruction, wherein when the computer program or the instruction is executed by a computer, the method for intelligent accurate segmentation and accurate positioning of multi-page image comparison as described in the first aspect or any possible design of the first aspect is implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device.

[0143] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-page image comparison intelligent segmentation and positioning method, characterized in that: include: Acquire a first image file and a second image file to be compared, wherein the first image file is an electronic file in PDF format or non-PDF format and contains a first image with multiple pages or a multi-page layout, and the second image file contains a second image with multiple pages or a multi-page layout; Determining a one-to-one correspondence or a logical layout relationship between the plurality of pages of first images and the plurality of pages of second images; For each first image page in the first image file, the PDF file information of the first image file is read, and according to the PDF file printing requirement attribute in the PDF file information, a corresponding first comparison area positioning frame for positioning the corresponding image comparison target area is determined, wherein the image comparison target area is an image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box; reading the area positioning frame of the second image includes, when the second image is a typesetting or imposition image of a multi-page layout, implementing a positioning area according to the image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box; For each page of the first image, using the corresponding first comparison area positioning frame to intercept the corresponding image to obtain the corresponding first comparison area image; For each page of the first image, the corresponding first comparison area image is imported into the trained recognition model, and the first character strings corresponding to each line of text in the corresponding first comparison area image are outputted, and the corresponding first minimum rectangular frame is determined according to each line of the first character string, wherein the first minimum rectangular frame refers to a rectangular frame with the minimum area used to enclose each line of text having a mapping relationship with each line of the first character string in the corresponding first comparison area image; For each page of the second image in the second image file, the corresponding image is imported into the recognition model, and the corresponding second character strings corresponding to each line of text in the corresponding image are outputted, and the corresponding second minimum rectangular frame is determined according to each line of the second character string, wherein the second minimum rectangular frame refers to a rectangular frame with the minimum area used to surround each line of text having a mapping relationship with each line of the second character string in the corresponding image; For each page of the second image, according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area is determined; For each page of the second image, using the corresponding second comparison area positioning frame to cut out a corresponding second comparison area image from the corresponding image; For each pair of first images and second images in the first image file and the second image file and having a mapping relationship, the corresponding first contrast area image or the second contrast area image is scaled so that the corresponding first contrast area image and the second contrast area image have the same size and resolution.

2. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that: After determining a one-to-one correspondence between the plurality of pages of first images and the plurality of pages of second images, the method further includes: For each page of the first images in the plurality of pages of first images, extract corresponding page content text from the corresponding image using optical character recognition technology, and count the number of corresponding words in the page content text; For each second image of the plurality of pages of second images, extract the corresponding page content text from the corresponding image using the optical character recognition technology, and count the number of corresponding words in the page content text; Arranging the first images of each page in descending order according to the number of words to obtain a first image queue, and arranging the second images of each page in descending order according to the number of words to obtain a second image queue; For each pair of first images and second images with the same serial number in the first image queue and the second image queue, if the corresponding two images do not have a mapping relationship in the one-to-one correspondence between the multiple pages of first images and the multiple pages of second images, and the corresponding two pages of content text have content consistency characteristics, an output is displayed to indicate that there is a mapping relationship mismatch between the corresponding two images, so as to remind the staff to conduct manual inspection and correct the mapping relationship.

3. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that: When the first image file is a small page file, the media frame, crop frame, bleed frame, trim frame or artwork frame are directly read to locate the comparison area, the PDF information of the image file is read, the coordinates of the media frame, crop frame, bleed frame, trim frame or artwork frame are extracted, and the image comparison target area is directly located according to the extracted frame coordinates. When the image file is a large-format imposition file, the following precise segmentation method is used: if the large-format image contains a cutting line, the cutting line in the large-format image is identified, and the area within the cutting line of each small page is used as the precise segmentation area. If there is no cutting line, the imposition folding reverse parsing technology is used to determine the logical order and position of each small page. The specific steps of manual segmentation are as follows: S31. Specify the boundaries of each small page through the interface operation, and manually specify some imposition parameters so that the system can assist in segmentation more accurately. The imposition parameters are image size: the system automatically reads the length and width of the image to determine the overall area of ​​the large file; the number of imposition pages: the user specifies the number of imposition pages (such as 8 pages, 16 pages, etc.) according to the actual situation; finished product size: the user specifies the finished product size of each small page (such as 185mm*260mm), which is the size of the trimming box; S32. Calculate the size of the bleed box and the media box. The system automatically calculates the size of the media box or the bleed box according to the finished product size specified by the user and the built-in bleed size (eg, 3 mm); S33. Built-in imposition model. The system has a built-in imposition model, which contains the necessary size parameters in the printing process, such as grip size, row space size, column space size, etc. These size parameters are used to ensure that the split small page files meet the actual printing requirements; S34. Perform manual and layout adjustments on the image. The user manually draws or selects the boundaries of each small page on the interface provided by the system according to the actual situation of the image and the imposition logic. The system automatically adjusts the layout of each page file based on the boundaries and imposition model specified by the user to center it on the overall size. If necessary, the user can also manually adjust the layout based on the gripping data to ensure that the layout size of each page is basically consistent with the original data of the imposition logic. After completing the manual segmentation and layout adjustment, the system accurately segments the large version file into multiple small page files according to the user's specifications.

4. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that: For each page of the second image in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character strings of each line corresponding to each line of text in the corresponding image are outputted, and the corresponding second minimum rectangular frame is determined according to the second character strings of each line, including: For a second image on a certain page in the second image file, the corresponding image is imported into the variable-length text recognition model, and the corresponding second character strings of each line corresponding to each line of text in the corresponding image are output; According to the first character strings of each row of a certain page of the first image having a mapping relationship with the second image of the certain page, determining at least one row of character strings having content consistency characteristics with the first character strings of each row from the second character strings; Based on the at least one line of character string, a second minimum rectangular frame of the second image of the certain page is determined, wherein the second minimum rectangular frame refers to a rectangular frame with a minimum area that is used to enclose at least one line of text that has a mapping relationship with the at least one line of character string in the second image of the certain page.

5. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that: For each page of the second image, determining a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, including: For any second image in the second image file, according to the corresponding second minimum rectangular frame and the first minimum rectangular frame and the first comparison area positioning frame of the corresponding first image, a corresponding second comparison area positioning frame for positioning the corresponding image comparison target area is determined according to the following formula: In the formula, x 21,btl represents the horizontal coordinate position of the lower left corner of the second comparison area positioning frame in any second image, y 21,btl represents the ordinate position of the lower left corner of the second comparison region positioning frame in any second image, x 21,tpr represents the horizontal coordinate position of the upper right corner of the second comparison area positioning frame in any second image, y 21,tpr represents the vertical coordinate position of the upper right corner of the second comparison area positioning frame in any second image, x 22,btl represents the horizontal coordinate position of the lower left corner of the second smallest rectangular frame in any second image, y 22,btl represents the ordinate position of the lower left corner of the second smallest rectangular frame in any second image, x 22,tpr represents the horizontal coordinate position of the upper right corner of the second smallest rectangular frame in any second image, y 22,tpr represents the vertical coordinate position of the upper right corner of the second smallest rectangular frame in any second image, x 12,btl represents the horizontal coordinate position of the lower left corner of the first minimum rectangular frame in the first image corresponding to any second image, y 12,btl represents the ordinate position of the lower left corner of the first minimum rectangular frame in the first image corresponding to any second image, x 12,tpr represents the horizontal coordinate position of the upper right corner of the first minimum rectangular frame in the first image corresponding to any second image, y 12,tpr represents the ordinate position of the upper right corner of the first minimum rectangular frame in the first image corresponding to any second image, x 11,btl represents the horizontal coordinate position of the lower left corner point of the first comparison area positioning frame in the first image corresponding to any second image, y 11,btl represents the ordinate position of the lower left corner of the first comparison region positioning frame in the first image corresponding to any one of the second images, x 11,tpr represents the horizontal coordinate position of the upper right corner point of the first comparison area positioning frame in the first image corresponding to any second image, y 11,tpr Indicates the vertical coordinate position of the upper right corner point of the first comparison area positioning frame in the first image corresponding to any one of the second images.

6. The multi-page image comparison intelligent segmentation and positioning method according to claim 1, characterized in that: The indefinite-length text recognition model adopts a recognition model based on a long short-term memory network and a connection time classifier LSTM+CTC, a convolutional recurrent neural network CRNN or a text recognition project ChineseOCR.

7. The multi-page image comparison intelligent segmentation and positioning method according to claim 3, characterized in that: For the manual segmentation method, the system obtains the precise segmentation areas of each small page file after manual segmentation. The system matches each precise segmentation area with the positioning template based on the preset or user-specified positioning template, and determines whether the match is successful by comparing the features of the area. If a precise segmentation area matches the positioning template successfully, the system considers that the area has been precisely located. If there are unmatched or inaccurately matched areas, the system prompts the user to make manual adjustments or re-segmentation.

8. Multi-page image comparison intelligent positioning device, characterized in that: It includes an image file acquisition unit, a corresponding relationship determination unit, a first positioning frame determination unit, a first area image capture unit, a first rectangular frame determination unit, a second rectangular frame determination unit, a second positioning frame determination unit, a second area image capture unit and an image scaling processing unit; The image file acquisition unit is used to acquire a first image file and a second image file to be compared, wherein the first image file is an electronic file in PDF format and contains multiple pages of first images, and the second image file contains multiple pages of second images; The correspondence determination unit is communicatively connected to the image file acquisition unit and is used to determine a one-to-one correspondence between the plurality of pages of first images and the plurality of pages of second images; The first positioning frame determining unit is communicatively connected to the image file acquiring unit, and is used to read the PDF file information of the first image file for each page of the first image in the first image file, and determine a corresponding first comparison area positioning frame for locating a corresponding image comparison target area according to the PDF file printing requirement attribute in the PDF file information, wherein the image comparison target area is an image area surrounded by a media box, a crop box, a bleed box, a trim box or an artwork box; The first region image capture unit is communicatively connected to the first positioning frame determination unit, and is used to apply the corresponding first comparison region positioning frame to the first image of each page to capture the corresponding first comparison region image from the corresponding image; The first rectangular frame determination unit is communicatively connected to the first area image interception unit, and is used for importing the corresponding first comparison area image into the indefinite-length text recognition model that has been trained for each page of the first image, outputting the first character strings of each line that correspond to each line of text in the corresponding first comparison area image, and determining the corresponding first minimum rectangular frame according to each line of the first character string, wherein the first minimum rectangular frame refers to a rectangular frame with the minimum area that is used to surround each line of text that has a mapping relationship with each line of the first character string in the corresponding first comparison area image; The second rectangular frame determination unit is communicatively connected to the image file acquisition unit, and is used for importing the corresponding image into the variable-length text recognition model for each page of the second image in the second image file, outputting the corresponding second character strings of each line corresponding to each line of text in the corresponding image, and determining the corresponding second minimum rectangular frame according to each line of the second character string, wherein the second minimum rectangular frame refers to a rectangular frame with the minimum area used to enclose each line of text having a mapping relationship with each line of the second character string in the corresponding image; The second positioning frame determining unit is communicatively connected to the corresponding relationship determining unit, the first positioning frame determining unit, the first rectangular frame determining unit and the second rectangular frame determining unit, respectively, and is used to determine, for each page of the second image, a corresponding second comparison area positioning frame for locating the corresponding image comparison target area according to the corresponding second minimum rectangular frame and the first minimum rectangular frame of the corresponding first image and the first comparison area positioning frame; The second region image capture unit is communicatively connected to the second positioning frame determination unit, and is used to apply the corresponding second comparison region positioning frame to the second image of each page to capture the corresponding second comparison region image from the corresponding image; The image scaling processing unit is communicatively connected to the correspondence determination unit, the first area image capture unit and the second area image capture unit, respectively, and is used to scale the corresponding first contrast area image or the second contrast area image for each pair of first images and second images in the first image file and the second image file and having a mapping relationship, so that the corresponding first contrast area image and the second contrast area image have the same size and resolution.

9. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the multi-page image comparison intelligent precise segmentation and precise positioning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the multi-page image comparison intelligent precise segmentation and precise positioning method as described in any one of claims 1 to 7 is executed.

Images