Image correction effect evaluation method and device and electronic equipment

Abstract

The invention relates to an image correction effect evaluation method and device and electronic equipment, and is applied to the technical field of computers, and the method comprises the steps: obtaining an original image and a corrected image obtained after the original image is corrected; extracting first layout element information in the original image and second layout element information in the corrected image; the first layout element information comprises element information forming each component of the original image, and the second layout element information comprises element information forming each component of the corrected image; based on the first layout element information and the second layout element information, optimization indexes of the corrected image relative to the original image after correction are determined, and the optimization indexes comprise layout element integrity, element geometric regularity and semantic association degree; and determining a correction evaluation result of the corrected image at least according to the optimization index. The problems that in the prior art, manual subjective dimension evaluation is high in subjectivity and low in evaluation efficiency are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to an image correction effect evaluation method and device and electronic equipment. BACKGROUND

[0002] Under the trend of education informatization and paperless office, teachers, students and education workers often need to digitize and archive or electronically process paper education teaching aids, test papers, exercise books and the like. The most common and convenient way is to use a mobile phone or tablet computer to take a picture. However, in this daily operation, a common and far-reaching problem arises: due to the thickness of the book binding or the natural bending of the paper when holding the camera, the image obtained by shooting is not an ideal plane, but has obvious curvature, distortion and perspective tilt. These deformations not only affect the appearance of the document, but also pose a serious obstacle to subsequent layout analysis and optical character recognition (OCR). Curved text can cause a sharp decline in OCR recognition rate, and distorted graphs and data can lose their accuracy.

[0003] In the digitalization scenario of education teaching aid documents and test papers, curved surface correction technology is an indispensable bridge connecting rough real shooting and accurate digital application. How to scientifically, effectively and meaningfully evaluate the "good" and "bad" of an algorithm is itself a huge and not yet perfectly solved challenge.

[0004] In related technologies, the method for evaluating the correction effect of a curved surface image mainly relies on artificial subjective direct effect measurement, that is, artificial comparison of the original image and the corrected image for direct first perception comparison to judge the good and bad of the curved surface correction. However, artificial subjective evaluation has strong subjectivity, and the evaluation efficiency will gradually decrease with the increase of evaluation time. SUMMARY

[0005] The present application provides an image correction effect evaluation method, device and electronic equipment to solve the problem of low evaluation efficiency of artificial subjective evaluation in the prior art.

[0006] According to a first aspect of an embodiment of the present application, an image correction effect evaluation method is provided, comprising: obtaining an original image and a corrected image obtained by correcting the original image; extracting first layout element information in the original image and second layout element information in the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image; Based on the first page element information and the second page element information, the optimization index of the corrected image relative to the original image is determined. The optimization index includes page element completeness, element geometric regularity, and semantic relevance. The correction evaluation result for the corrected image is determined based at least on the optimization index.

[0007] Optionally, based on the first layout element information and the second layout element information, the optimization index of the corrected image relative to the original image is determined, including: The first page element information and the second page element information are matched to determine the matching degree of each page element information that is successfully matched. The optimization index is determined based on the matching degree, the first page element information, and the second page element information.

[0008] Optionally, the optimization index is determined based on the matching degree, the first layout element information, and the second layout element information, including: A first quantity of target page element information is determined, wherein the target page element information includes page element information in the second page element information whose matching degree is higher than a first preset matching threshold and whose element type is a target element type, wherein the target element type includes any one of the specified element types; Determine a second quantity of layout element information of the target element type in the first layout element information; The feature recall rate of the target feature type is determined based on the first quantity and the second quantity; The completeness of the layout elements is determined based on the element recall rate.

[0009] Optional, also includes: The third page element information that meets the element loss condition is identified. The element loss condition includes that the page element information is not successfully matched and the element recall rate is lower than the preset recall rate. Determine the location of the third layout element information in the corrected image or the original image; Store the information of the third page elements and the area they are located in.

[0010] Optionally, the optimization index is determined based on the matching degree, the first layout element information, and the second layout element information, including: Determine the first target page element information whose matching degree is higher than the second preset matching threshold in the first page element information, and the second target page element information whose matching degree is higher than the third preset matching threshold in the second page element information; Determine the first tilt angle of each page element information in the first target page element information, and determine the second tilt angle of each page element information in the second target page element information; The geometric regularity of the element is determined based on the first tilt angle and the second tilt angle.

[0011] Optionally, based on the first layout element information and the second layout element information, the optimization index of the corrected image relative to the original image is determined, including: Based on the first layout element information, the text in the original image is extracted to obtain the first text information; Based on the second layout element information, the text in the corrected image is extracted to obtain the second text information; Based on the first text information and the second text information, the semantic quality parameters of the corrected image relative to the original image are determined. The semantic quality parameters include character accuracy improvement rate, sentence fluency improvement rate, and entity integrity improvement rate. The semantic relevance is determined based on the semantic quality parameters.

[0012] Optionally, determining the semantic quality parameters of the corrected image relative to the original image based on the first text information and the second text information includes: Character recognition is performed on the first text information and the second text information respectively to obtain the first character recognition result of the first text information and the second character recognition result of the second text information; The real text information is compared with the first character recognition result to obtain a first comparison result, and the real text information is compared with the second character recognition result to obtain a second comparison result. The real text information is obtained from the annotation information of the original image. The character accuracy improvement rate is determined based on the first comparison result and the second comparison result.

[0013] Optionally, determining the semantic quality parameters of the corrected image relative to the original image based on the first text information and the second text information includes: Character recognition is performed on the first text information and the second text information respectively to obtain the first character recognition result of the first text information and the second character recognition result of the second text information; Based on a pre-trained language model, the first language perplexity of the first character recognition result and the second language perplexity of the second character recognition result are determined respectively. Based on the first language confusion level and the second language confusion level, the sentence fluency improvement rate is determined; Based on a pre-trained language model, the first language completeness of the first character recognition result and the second language completeness of the second character recognition result are determined respectively. The entity completeness improvement rate is determined based on the first language completeness and the second language completeness.

[0014] Optionally, the method further includes: Obtain evaluation metrics for the visual perception dimensions of the original image and the corrected image as determined by human evaluation. Determining the correction evaluation result for the corrected image based at least on the optimization index includes: Based on the optimization index and the evaluation index, the correction evaluation result of the corrected image is determined.

[0015] Optionally, based on the optimization index and the evaluation index, the correction evaluation result for the corrected image is determined, including: Determine whether the optimization index and / or the evaluation index meet the reverse correction constraint condition, wherein the reverse correction constraint condition is used to indicate that the image quality of the corrected image is lower than that of the original image; If satisfied, the optimization index and / or the evaluation index that satisfy the reverse correction constraint are determined to be preset values, and the index weights of the optimization index and the evaluation index are obtained respectively. Based on the index weights, the preset values, the optimization index and the evaluation index, the correction evaluation result of the corrected image is determined. If the conditions are not met, the sum of the optimization index and the evaluation index is determined as the correction evaluation result for the corrected image.

[0016] Optionally, obtain evaluation metrics for the visual perception dimensions of the original image and the corrected image by humans, including: The original image and the corrected image are stitched together to obtain a stitched image; The stitched image is displayed along with the evaluation criteria used to assess the visual perception dimension of the stitched image. Obtain the evaluation indicators for human evaluation of the stitched image based on the evaluation criteria.

[0017] Optionally, the original image includes annotation information on the layout elements in the original image; the extraction of the first layout element information from the original image and the second layout element information from the corrected image includes: Extract the annotation information from the original image to obtain the first layout element information; The elements in the corrected image are identified using an optical character recognition method to obtain the element information of the second page layout.

[0018] According to a second aspect of the embodiments of this application, an image correction effect evaluation apparatus is provided, comprising: An acquisition unit is used to acquire the original image and a corrected image obtained after correcting the original image; An extraction unit is used to extract first layout element information from the original image and second layout element information from the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image. The determining unit is configured to determine the optimization index of the corrected image relative to the original image based on the first page element information and the second page element information. The optimization index includes page element completeness, element geometric regularity, and semantic relevance. An evaluation unit is used to determine a correction evaluation result for the corrected image based at least on the optimization index.

[0019] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the image correction effect evaluation method as described in the first aspect by running the program in the memory.

[0020] According to a fourth aspect of the embodiments of this application, a storage medium is provided, on which a computer program is stored, and when the computer program is run by a processor, it implements the image correction effect evaluation method as described in the first aspect.

[0021] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including computer program instructions, which, when executed by a processor, cause the processor to perform an evaluation method for image correction effects as described in the first aspect.

[0022] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application acquires an original image and a corrected image obtained after correcting the original image; extracts first layout element information from the original image and second layout element information from the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image; based on the first layout element information and the second layout element information, determines the optimization index of the corrected image relative to the original image, the optimization index including layout element completeness, element geometric regularity, and semantic relevance; and determines the correction evaluation result of the corrected image based at least on the optimization index. In this way, by extracting layout element information from the original image and the corrected image, and using the layout element information to determine the layout element completeness, element geometric regularity, and semantic relevance to evaluate the corrected image, optimization indicators can be calculated from different evaluation perspectives. This avoids the subjectivity of manual evaluation. Furthermore, after inputting the original image and the corrected image, corresponding evaluation processing can be performed to obtain the correction evaluation results. Even if the evaluation time increases, it will not affect the evaluation efficiency. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 A flowchart of an image correction effect evaluation method provided for an embodiment of this application.

[0025] Figure 2 This is a schematic diagram of polygonal outline points for labeling layout elements, provided as an embodiment of this application.

[0026] Figure 3 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Exemplary Implementation Environment The image correction effect evaluation method according to the embodiments of this application can be executed by electronic devices such as terminal devices or servers. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. This method can be implemented by a processor calling computer-readable program instructions stored in memory. This application uses the execution of the image correction effect evaluation method by a server as an example for explanation, but does not limit it.

[0029] Exemplary methods Please see Figure 1 In one exemplary embodiment, a method for evaluating image correction effects is provided, comprising: Step 101: Obtain the original image and the corrected image obtained after correcting the original image.

[0030] In some embodiments, the original image may be an image obtained by digitizing or electronically processing paper materials, such as educational supplementary materials, test papers, exercise books, classic works, tender documents, etc. The original image may also be obtained by photographing the paper material using a camera device. Due to the thickness of the book binding or the natural curvature of the paper when photographed by hand, the resulting original image may exhibit curved surfaces, distortions, and perspective tilts; that is, the original image may contain elements that are curved.

[0031] The corrected image is the image obtained after surface correction of the original image. The correction method can employ relevant image correction techniques, which will not be elaborated here. Understandably, the quality of the image correction method will affect the correction effect of the corrected image; therefore, it is necessary to evaluate the correction effect.

[0032] Step 102: Extract the first layout element information from the original image and the second layout element information from the corrected image; the first layout element information includes the element information constituting each component of the original image, and the second layout element information includes the element information constituting each component of the corrected image.

[0033] In some embodiments, both the original image and the corrected image include element information that constitutes each component of the image. By extracting the layout element information, the two are compared and the correction effect is evaluated.

[0034] Taking the original image as an example of an image taken from paper material, the layout elements may include page, layout, region, question area, title, text line, table, graph, header, footer, etc.

[0035] The above-mentioned elements have the following parent-child relationships: A page element can exist as a child element within an image element; a page child element can include a layout element. A layout element can exist as a child element within a page element; a layout child element can include a graph element, a region element, a table element, a title element, etc. A region element can exist as a child element within a layout element; a region child element can include a region element, a questionarea element, a table element, a title element, a graph element, and an annotation element. A title element can exist as a child element within a layout element, a region element, a graph element, and a title element; a title child element can include a questionarea element, etc. The `graph` and `title` elements are considered; the `questionarea` element can exist as a child element within the `region`, `graph`, and `table` elements, specifically within cells, table notes, figure captions, formulas, titles, and annotations. A `questionarea` child element can contain a `textline` element; a `textline` element can exist as a child element within a `questionarea` element, but a `textline` element has no child elements. The `table` element can exist as a child element within the `region`, `graph`, and `layout` elements. A `table` child element can contain a `cell` element, and a `cell` child element can contain `questionarea`, `graph`, `formula`, `region`, and `title` elements. Nested tables are not allowed within cells. The `graph` element can exist as a child element within the `layout`, `region`, `cell`, `title`, and `graph` elements. Illustration child elements can contain `questionarea`, `table`, `graph`, `title`, and `region` elements. The `header` element can exist as a child element within the `page` element, and a `header` child element can contain both `textline` and `graph` elements. The `footer` element can exist as a child element within the `page` element, and a `footer` child element can contain both `textline` and `graph` elements.

[0036] In an optional embodiment, the original image includes annotation information on layout elements in the original image; the extraction of the first layout element information from the original image and the second layout element information from the corrected image includes: Extract the annotation information from the original image to obtain the first layout element information; The elements in the corrected image are identified using an optical character recognition method to obtain the element information of the second page layout.

[0037] In some embodiments, the layout element information in the original image can be annotated, and the layout element information in the original image can be configured in the annotation information. Then, when extracting the first layout element information in the original image, it can be obtained by extracting the annotation information.

[0038] The annotation information can be obtained manually based on experience and observation of the elements in the original image. Alternatively, an annotation model can be pre-trained, and the original image can be input into the annotation model to complete the annotation of the layout elements in the original image.

[0039] The elements in the corrected image can be identified using the Optical Character Recognition (OCR) method. OCR is a technology that can convert printed / handwritten text in an image into a machine-editable text format. By recognizing the corrected image, the corrected image can be divided into regions, and layout element information such as text blocks, tables, pictures, headers, and footers can be identified to obtain the second layout element information.

[0040] Furthermore, the dataset used in the above-mentioned annotation model can include multiple images for each scene, element, and distortion level, with a total size of no less than 5,000 images. Scene types can include educational documents (math homework, textbooks, test papers), handwritten materials (class notes, answer sheets), and mixed media (courseware with formulas / tables, lab reports), with multiple images for each category to ensure sufficient training of the model. Subject types can include mainstream subjects currently taught in schools, such as Chinese, mathematics, English, politics, history, geography, physics, chemistry, and biology. Element types can include pages, layouts, areas, title boxes, headings, text lines, tables, illustrations, headers, and footers. Each element type must account for no less than a specified percentage in the sample, for example, no less than 15%. Headings can also be differentiated by level, such as first-level headings and second-level headings. Text lines can be labeled with printed or handwritten tags. The degree of curvature can be divided into slight curvature (curvature < 15°), moderate wrinkles (curvature between 15° and 30°), and severe distortion (curvature > 30° or wrinkles in multiple areas). Each category accounts for about 1 / 3. Priority is given to collecting textbooks with curved binding edges and folded exercise paper samples.

[0041] The process involves analyzing the layout of the original image using a labeling model, outputting layout element information such as page layout, regions, title boxes, text lines, tables, and illustrations. Then, the coordinates of the original image and each element are coupled to a labeling platform for manual correction. Manual correction can be performed according to the following element standards: Label the polygon outline points using a unified coordinate system (see...). Figure 2 You can configure the maximum point on the Y-axis as p0 and mark 4 or more vertices clockwise: accurately label the bounding boxes of 10 types of features, where "region" must contain subordinate features (such as "paragraph region" containing all text lines within it), and the labeling level is "page → layout → region → paragraph / title / table / illustration / header / footer → text line"; For element type and attribute labeling: basic labels can clearly label 10 element types such as "page / layout / area"; extended labels can distinguish "first-level heading / second-level heading" according to the title, label "printed / handwritten / formula" for text lines, label "table header / table body / table footer" for tables, and label "description text associated ID" for illustrations; for areas with curved surfaces, "curved" needs to be labeled in the corresponding element attribute; Text line transcription: Text lines can be manually transcribed and annotated from images, providing a WYSIWYG (What You See Is What You Get) experience.

[0042] Step 103: Based on the first page element information and the second page element information, determine the optimization index of the corrected image relative to the original image. The optimization index includes page element completeness, element geometric regularity, and semantic relevance.

[0043] In some embodiments, by extracting the first and second layout element information, the original image and the corrected image are compared, and the differences in layout element information between the two are quantified to obtain an optimization index of the corrected image relative to the original image. This optimization index can be calculated from multiple aspects, such as layout element completeness, element geometric regularity, and semantic relevance. Layout element completeness is mainly used to assess whether the corrected image has lost element information; element geometric regularity is mainly used to assess whether the curvature of the corrected image has been improved; and semantic relevance is mainly used to assess whether the corrected image has semantic errors during the correction process.

[0044] By calculating the completeness of layout elements, we can quantify whether core key elements are "retained" and "formally complete," avoiding element loss due to correction, which is the basic baseline for evaluation. Focusing on the common "horizontal and vertical alignment" requirements in document scenarios, we calculate the geometric regularity of elements to evaluate the deviation of layout analysis algorithms. By comparing the semantic relevance between the original image and the corrected image, we measure the practical application value of the correction.

[0045] In an optional embodiment, based on the first layout element information and the second layout element information, determining the optimization index of the corrected image relative to the original image includes: The first page element information and the second page element information are matched to determine the matching degree of each page element information that is successfully matched. The optimization index is determined based on the matching degree, the first page element information, and the second page element information.

[0046] In some embodiments, by matching the first page element information and the second page element information, it is possible to first determine whether element loss occurred during the correction process based on the matching results, and to determine the difference between the page element information between the original image and the corrected image based on the matching degree of the successfully matched page element information, thereby more accurately calculating the optimization index.

[0047] The matching degree between the element information of the first page and the element information of the second page can be calculated by calculating the Intersection over Union (IOU). IOU is a core evaluation index used in the field of computer vision to measure the degree of overlap between two bounding boxes.

[0048] When calculating the matching degree, each first page feature information in the original image can be matched 1v1 or NvN with each second page feature information in the corrected image. If the page feature information in the original image and the corrected image are successfully matched, the matched page feature information can be further filtered according to the feature attributes and matching degree of the matched page feature information. If two page feature information are successfully matched, but their feature attributes are different, they are considered to have failed to match successfully.

[0049] In an optional embodiment, determining the optimization index based on the matching degree, the first layout element information, and the second layout element information includes: A first quantity of target page element information is determined, wherein the target page element information includes page element information in the second page element information whose matching degree is higher than a first preset matching threshold and whose element type is a target element type, wherein the target element type includes any one of the specified element types; Determine a second quantity of layout element information of the target element type in the first layout element information; The feature recall rate of the target feature type is determined based on the first quantity and the second quantity; The completeness of the layout elements is determined based on the element recall rate.

[0050] In some embodiments, after completing the first layout element information and the second layout element information, the number of layout element information of each element type in the corrected image with a matching degree higher than a first preset matching threshold is counted to obtain a first number; and the number of layout element information of each element type in the original image is counted to obtain a second number. Then, by calculating the ratio of the first number to the second number, the element recall rate of the layout element information of each element type is obtained. The element recall rates of each element type are summed to obtain the layout element completeness.

[0051] It is understandable that when calculating the feature recall rate, one can directly count the first total number of first layout feature information in the original image and the second total number of layout feature information in the corrected image with a matching degree higher than the first preset matching threshold. The feature recall rate can be calculated by calculating the ratio of the second total number to the first total number.

[0052] The first preset matching threshold can be set according to the actual situation; for example, it can be set to 0.5.

[0053] The target element types mentioned above may include, but are not limited to, title boxes, headings, text lines, tables, illustrations, headers, footers, etc.

[0054] In an optional embodiment, it further includes: The third page element information that meets the element loss condition is identified. The element loss condition includes that the page element information is not successfully matched and the element recall rate is lower than the preset recall rate. Determine the location of the third layout element information in the corrected image or the original image; Store the information of the third page elements and the area they are located in.

[0055] In some embodiments, to avoid element loss, third-page elements that meet the element loss conditions can also be searched. By determining the position of the third-page element information in the corrected image or the original image, according to its marked area in the original image and its detection area in the corrected image, contour points are marked in the corresponding images, and the third-page element information and its area are stored. Thus, the lost page element information can be saved for subsequent data processing.

[0056] If a layout element fails to match, it means that there is no corresponding element in the other image, indicating that the layout element information may have been lost. The preset recall rate can be set to 100%. When the element recall rate is lower than the preset recall rate, it means that the number of layout element information of the same element type in the original image and the corrected image is different, which also indicates that the layout element information may have been lost.

[0057] In an optional embodiment, determining the optimization index based on the matching degree, the first layout element information, and the second layout element information includes: Determine the first target page element information whose matching degree is higher than the second preset matching threshold in the first page element information, and the second target page element information whose matching degree is higher than the third preset matching threshold in the second page element information; Determine the first tilt angle of each page element information in the first target page element information, and determine the second tilt angle of each page element information in the second target page element information; The geometric regularity of the element is determined based on the first tilt angle and the second tilt angle.

[0058] In some embodiments, after matching the first page element information with the second page element information, first target page element information and second target page element information with a matching degree higher than a second preset matching threshold are selected. The first target page element information and second target page element information include the boundary coordinates of the page elements, which can locate the spatial range of the page elements in the image. For the original image, its page elements are... Figure 2 The bounding boxes shown represent the spatial extent in the image. The bounding boxes of layout elements in the corrected image can also be determined using OCR detection methods. When determining the first and second tilt angles, different tilt angle calculation methods can be used depending on the element types of the first and second target layout element information.

[0059] For example, when the first target page element information or the second target page element information is a text line element, the minimum bounding rectangle of the page element outline can be fitted, and the rotation angle of the minimum bounding rectangle can be extracted as the text line tilt angle.

[0060] When the first or second target page element information is a title element, header element, or footer element, the top edge of the image can be used as the horizontal baseline and the side edge of the image as the vertical baseline. The minimum bounding rectangle is used to fit the element area, and the rotation angle of the minimum bounding rectangle is calculated. This angle is the offset angle between the element and the horizontal baseline.

[0061] When the first or second target page element information is a table element, the table area can be binarized, and then the Hough transform method can be used to detect the horizontal and vertical lines, calculate the angle between each line and the horizontal / vertical baseline, and take the average of all horizontal and vertical line angles as the table's right angle deviation.

[0062] After obtaining the angle data of the different element types mentioned above, the average angle of each element information in the first target page information can be determined as the first tilt angle, and the average tilt angle of each element information in the second target page information can be determined as the second tilt angle.

[0063] Furthermore, when determining the geometric regularity of an element, it can be obtained through the following calculation formula: Geometric regularity of an element = (θ1-θ2) / θ1×100%; where θ1 represents the first tilt angle and θ2 represents the second tilt angle.

[0064] If θ2 < θ1, then the optimization rate is positive, indicating that the geometric shape is more regular.

[0065] Understandably, before calculating the first and second tilt angles, the angles of each element on the page can be filtered to identify outliers (such as angles greater than extreme angles due to annotation errors, for example, 45 degrees) to avoid interfering with the results.

[0066] The second preset matching threshold can be set according to the actual situation; for example, it can be set to 0.5.

[0067] In an optional embodiment, based on the first layout element information and the second layout element information, determining the optimization index of the corrected image relative to the original image includes: Based on the first layout element information, the text in the original image is extracted to obtain the first text information; Based on the second layout element information, the text in the corrected image is extracted to obtain the second text information; Based on the first text information and the second text information, the semantic quality parameters of the corrected image relative to the original image are determined. The semantic quality parameters include character accuracy improvement rate, sentence fluency improvement rate, and entity integrity improvement rate. The semantic relevance is determined based on the semantic quality parameters.

[0068] In some embodiments, the text in the original image can be obtained from the annotation information of the original image. The text in the annotation information containing text is extracted and concatenated according to the reading order in the original image to obtain the first text information. The text in the corrected image can be identified using an OCR method and concatenated according to the reading order in the corrected image to obtain the second text information.

[0069] The reading order mentioned above can be based on the structural order of the page elements, such as the order of area → title box → text line → table, or it can be based on the order from top to bottom and from left to right.

[0070] After obtaining the first and second text information, the semantic quality parameters are determined by comparing and calculating the two to measure the effect of the corrected image on the semantic quality relative to the original image, and thus the semantic relevance is determined.

[0071] In an optional embodiment, determining the semantic quality parameters of the corrected image relative to the original image based on the first text information and the second text information includes: Character recognition is performed on the first text information and the second text information respectively to obtain the first character recognition result of the first text information and the second character recognition result of the second text information; The real text information is compared with the first character recognition result to obtain a first comparison result, and the real text information is compared with the second character recognition result to obtain a second comparison result. The real text information is obtained from the annotation information of the original image. The character accuracy improvement rate is determined based on the first comparison result and the second comparison result.

[0072] In some embodiments, characters in the first and second text information can be recognized using an OCR model to obtain first character recognition results and second character recognition results. Since the layout elements in the original image are pre-annotated, the annotation information includes the actual text information in the image. Then, by comparing the obtained first and second character recognition results with the actual text information, first comparison results and second comparison results are obtained for each. The character accuracy improvement rate is calculated as follows: Character accuracy improvement rate = (second comparison result - first comparison result) / (1 - first comparison result) × 100%.

[0073] The first comparison result can be obtained as follows: calculate the number of first correct characters compared to the real text information, and calculate the ratio of this first correct character count to the total number of characters in the real text information to obtain the first comparison result. The second comparison result can be obtained as follows: calculate the number of second correct characters compared to the real text information, and calculate the ratio of this second correct character count to the total number of characters in the real text information to obtain the second comparison result.

[0074] The first comparison result can represent the accuracy of the characters in the original image, and the second comparison result can represent the accuracy of the characters in the corrected image.

[0075] In an optional embodiment, determining the semantic quality parameters of the corrected image relative to the original image based on the first text information and the second text information includes: Character recognition is performed on the first text information and the second text information respectively to obtain the first character recognition result of the first text information and the second character recognition result of the second text information; Based on a pre-trained language model, the first language perplexity of the first character recognition result and the second language perplexity of the second character recognition result are determined respectively. Based on the first language confusion level and the second language confusion level, the sentence fluency improvement rate is determined; Based on a pre-trained language model, the first language completeness of the first character recognition result and the second language completeness of the second character recognition result are determined respectively. The entity completeness improvement rate is determined based on the first language completeness and the second language completeness.

[0076] In some embodiments, after character recognition of the first and second text information is performed by an OCR model, the recognition results are input into a pre-trained language model (such as GPT series, BERT, etc.) to calculate the corresponding language perplexity and language completeness; then, the calculated sentence fluency improvement rate and entity completeness improvement rate are used.

[0077] The fluency improvement rate can be calculated as follows: Fluency improvement rate = (First language confusion - Second language confusion) / First language confusion × 100%.

[0078] The entity completeness improvement rate can be calculated as follows: Entity completeness improvement rate = (First language completeness - Second language completeness) / First language completeness × 100%.

[0079] In an optional embodiment, determining the semantic relevance based on the semantic quality parameter includes: The first semantic indicator is determined based on the target improvement rate, which is any one of the character accuracy improvement rate, sentence fluency improvement rate, and entity completeness improvement rate. The semantic relevance is determined based on each of the first semantic indicators.

[0080] In some embodiments, the first semantic index can be determined as follows: determine whether the target improvement rate is greater than the preset improvement rate. If yes, it means that the image has been semantically optimized at the target improvement rate level. Obtain the improvement rate weight corresponding to the target improvement rate and determine the product of the target improvement rate and the improvement rate weight as the first semantic index. If no, it means that the image has not been semantically optimized at the target improvement rate level, but has instead played a reverse correction effect. Therefore, the first semantic index is configured as 0.

[0081] After the first semantic index corresponding to the character accuracy improvement rate, sentence fluency improvement rate, and entity completeness improvement rate is calculated, the sum of the obtained first semantic indexes can be determined as the semantic relevance.

[0082] Furthermore, the weighting of the improvement rate corresponding to the character accuracy improvement rate can be adjusted in the following way: When the second comparison result of the corrected image is 100%, it means that the corrected image has a good correction effect at the text level and can accurately correct the characters to be consistent with the original text. Therefore, there is no need to pay attention to the character accuracy improvement rate, entity completeness improvement rate and sentence fluency improvement rate. The improvement rate weight corresponding to the character accuracy improvement rate can be adjusted to 0.

[0083] If the difference between the first comparison result and the second comparison result is 0, the text similarity between the first text information and the second text information is either not completely identical or completely identical, and the weight of the improvement rate corresponding to the character accuracy improvement rate can be adjusted to 0.

[0084] Step 104: Determine the correction evaluation result of the corrected image based at least on the optimization index.

[0085] In some embodiments, the calculated completeness of layout elements, geometric regularity of elements, and semantic relevance are used to evaluate the corrected image to obtain the correction evaluation result. A comprehensive evaluation of the image correction result can avoid the limitations of judging the correction effect by a single visual indicator. The corrected image is evaluated from multiple dimensions such as visual regularity, completeness of layout content, and semantic rationality, making the evaluation result of the corrected image more accurate.

[0086] Among them, the sum of the above-mentioned page element completeness, element geometric regularity and semantic relevance can be directly determined, or the three can be weighted and summed, and the result can be used as the correction evaluation result.

[0087] In an optional embodiment, the method further includes: Obtain evaluation metrics for the visual perception dimensions of the original image and the corrected image as determined by human evaluation. Determining the correction evaluation result for the corrected image based at least on the optimization index includes: Based on the optimization index and the evaluation index, the correction evaluation result of the corrected image is determined.

[0088] In some embodiments, to improve the accuracy of the evaluation results of the corrected image and avoid the matching deviation problem caused by purely technical indicators, human evaluation indicators for the correction effect of the corrected image can be further introduced to score from the user's visual perception perspective. Then, the optimization indicators obtained by machine processing and the evaluation indicators given by humans are used together to determine the correction evaluation result.

[0089] In an optional embodiment, determining the correction evaluation result for the corrected image based on the optimization index and the evaluation index includes: Determine whether the optimization index and / or the evaluation index meet the reverse correction constraint condition, wherein the reverse correction constraint condition is used to indicate that the image quality of the corrected image is lower than that of the original image; If satisfied, the optimization index and / or the evaluation index that satisfy the reverse correction constraint are determined to be preset values, and the index weights of the optimization index and the evaluation index are obtained respectively. Based on the index weights, the preset values, the optimization index and the evaluation index, the correction evaluation result of the corrected image is determined. If the conditions are not met, the sum of the optimization index and the evaluation index is determined as the correction evaluation result for the corrected image.

[0090] In some embodiments, if the corrected image shows a decline in performance after correction, a corresponding penalty mechanism can be set. If the optimization index and / or the evaluation index meet the reverse correction constraint, the optimization index and / or the evaluation index that meet the reverse correction constraint are set to a preset value (e.g., a small value, a value less than 1, or even 0). The result of this assignment is then used to determine the correction evaluation result. If the optimization index and / or the evaluation index do not meet the reverse correction constraint, it indicates that the corrected image has a good correction effect. Therefore, the correction evaluation result can be directly calculated using the optimization index and the evaluation index; for example, the sum of the two can be used to determine the correction evaluation result.

[0091] For example, if the optimization index and / or the evaluation index do not meet the reverse correction constraint, the correction evaluation result can be calculated as follows: The correction evaluation result = (completeness of layout elements + geometric regularity of elements + the evaluation index) × first weight + semantic relevance × second weight; where the first weight can be set to be greater than the second weight, for example, the first weight is 0.6 and the second weight is 0.4.

[0092] If the optimization index and / or the evaluation index meet the reverse correction constraint, the correction evaluation result can be calculated as follows: The correction evaluation result = completeness of page elements + geometric regularity of elements + the evaluation indicators mentioned above + semantic relevance.

[0093] The aforementioned reverse correction constraints may include: any of the aforementioned optimization indicators and evaluation indicators being lower than their corresponding specified thresholds, or the layout element information being mismatched.

[0094] If either the optimization index or the evaluation index is lower than its corresponding specified threshold, it indicates that the correction effect of the image is poor. If the layout element information cannot be matched, it means that elements may have been lost during the correction process.

[0095] For example, the above-mentioned reverse correction constraint conditions may include: During the evaluation process, the evaluator will score any one of the evaluation dimensions (page flattening effect, page integrity effect, text line presentation effect, image clarity) as a first specified value (e.g., 0) or at least two of the scores will be lower than a second specified value (e.g., 3). At least two of the above-mentioned character accuracy improvement rate, sentence fluency improvement rate, and entity completeness improvement rate are lower than the third specified value (e.g., -5%). The title box, table, image, header, and footer in the page layout information have mismatches and are not identified as missing elements.

[0096] Furthermore, after obtaining the correction evaluation results, the correction effectiveness level of the evaluation results can be determined according to preset effectiveness level judgment criteria. The effectiveness level judgment criteria can set multiple score ranges, each corresponding to a correction effectiveness level; the higher the score range, the higher the correction effectiveness level. Corresponding application suggestions can also be configured based on the correction effectiveness level.

[0097] For example, the criteria for determining the level of effectiveness can be found in Table 1 below.

[0098] Table 1 In one optional embodiment, obtaining evaluation metrics for the visual perception dimensions of the original image and the corrected image by a human observer includes: The original image and the corrected image are stitched together to obtain a stitched image; The stitched image is displayed along with the evaluation criteria used to assess the visual perception dimension of the stitched image. Obtain the evaluation indicators for human evaluation of the stitched image based on the evaluation criteria.

[0099] In some embodiments, by stitching the original image and the corrected image together and displaying the resulting stitched image, users can easily evaluate the correction effect of the corrected image from a visual perception perspective. By displaying the evaluation criteria, users can be provided with a basis for evaluation. Furthermore, the corrected image can be manually evaluated based on the evaluation criteria to obtain evaluation indicators.

[0100] The evaluation criteria mainly focus on the completeness of the layout elements, the geometric regularity of the elements, and the semantic relevance.

[0101] For example, the corrected image can be evaluated according to the evaluation criteria provided in Table 2 below.

[0102] Table 2 When evaluating the corrected images, an odd number of users can be set up as an evaluation team to evaluate the spliced ​​images. Each evaluator, based on the above evaluation criteria, independently scores each spliced ​​image according to the grading standards for the completeness of the layout elements, the geometric regularity of the elements, and the semantic relevance. They can also fill in a "Subjective Evaluation Form" and briefly indicate the basis for their judgment (such as "slight text line offset").

[0103] For scores from multiple evaluators, if the standard deviation of scores in the same dimension differs significantly, exceeding a set value (e.g., 2 points (out of 10)), a discussion is organized based on the evaluators' stated criteria to reach a consensus; otherwise, the average is taken as the score for that dimension. Further, the average scores for each dimension are weighted and summed to obtain the evaluation index.

[0104] It is understandable that the evaluation indicators, layout element completeness, element geometric regularity, and semantic relevance parameters used in the evaluation and correction results may be determined according to the data ranges in their set rules. To ensure data consistency, corresponding mapping data ranges can be set in advance for the data ranges of the above parameters. For example, if the data range of the correction evaluation results is 0 to 100, the above parameters can be set according to their degree of influence on the correction evaluation results, and the corresponding mapping data range for each parameter can be set.

[0105] For example, the influence of evaluation indicators, layout element completeness, element geometric regularity, and semantic relevance on the correction evaluation results is in the proportions of 4:2:2:2, respectively; the data range of evaluation indicators is 0 to 10, and their corresponding mapping data range can be set to 0 to 40. When the evaluation indicator is 5, its corresponding mapping value is 20.

[0106] This application provides an image correction effect evaluation method that can be applied to a multi-feature semantic association evaluation system for curved surface correction layouts in educational scenarios. This system can be deployed on a local computer or a cloud server. The system includes: a data and task management module, a layout feature quantification module, a semantic value association module, a human perception evaluation module, and a comprehensive scoring and penalty module.

[0107] The data and task management module receives the original and corrected images to be evaluated, and manages and schedules the constructed standardized test set. It performs layout analysis on the original images and annotates the elements obtained from the analysis.

[0108] The layout feature quantification module incorporates computer vision algorithms to perform quantification calculations, calculating the completeness of layout elements (such as element recall rate) and the geometric regularity of elements (such as text line tilt angle, table straight angle deviation, etc.).

[0109] The semantic value association module has a built-in OCR engine and a pre-trained language model for evaluating semantic value, calculating the relative improvement rate of OCR character recognition and the semantic fluency of text.

[0110] The human perception evaluation module is used to execute the human evaluation process. It is responsible for generating a comparison and stitching image of the original image and the corrected image when specific conditions are met (such as when the difference between automated indicators is not large), and providing a standardized human scoring interface (including four dimensions: page flattening, integrity, text lines, and sharpness).

[0111] The comprehensive scoring and penalty module is used to perform the final calculation, which is responsible for summarizing the scores of the aforementioned modules and adjudicating the scores according to the preset "penalty mechanism" (such as a serious drop in OCR accuracy, loss of key elements, etc.), and finally outputting the total evaluation score (i.e. the corrected evaluation result) and the effect level (S / A / B / C level).

[0112] This system achieves a complete method for evaluating image correction effects through the collaborative work of its various modules.

[0113] The image correction effect evaluation method in this application addresses the core shortcomings of existing technologies, such as inconsistent evaluation standards and significant subjective biases caused by manual evaluation. By introducing standardized test sets and multi-dimensional quantitative indicators, combined with targeted manual calibration that intervenes only at critical moments (such as in areas of indicator ambiguity), a scientific, reliable, and reproducible human-machine coupled measurement standard is constructed. This solves the pain point of existing technologies where objective indicators are disconnected from subjective perception. This invention does not "correct for the sake of correction," but innovatively strongly binds the correction effect to the performance of downstream core tasks (OCR recognition). Through "semantic value association" and "penalty mechanisms," it ensures that high-scoring algorithms are necessarily algorithms with high application value in real educational scenarios (such as subsequent image search and grading). Compared to existing technologies that rely on full manual evaluation, this invention uses automated technology to screen and process the vast majority of samples, sending only a small number (such as samples with no difference in automated indicators) for manual arbitration, greatly reducing the human cost required for evaluation and significantly improving the efficiency of algorithm iteration. The key layout elements in the image are classified and divided, and a comprehensive evaluation framework is constructed from multiple dimensions such as "element integrity", "geometric regularity", "OCR accuracy", "semantic fluency" and "human perception". It can accurately locate the advantages and disadvantages of the algorithm in different dimensions (such as "flattened but lost footer" or "flattened but OCR rate decreased"), and provide clear and quantifiable guidance for algorithm optimization.

[0114] Exemplary device Accordingly, embodiments of this application also provide an image correction effect evaluation device, including: An acquisition unit is used to acquire the original image and a corrected image obtained after correcting the original image; An extraction unit is used to extract first layout element information from the original image and second layout element information from the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image. The determining unit is configured to determine the optimization index of the corrected image relative to the original image based on the first page element information and the second page element information. The optimization index includes page element completeness, element geometric regularity, and semantic relevance. An evaluation unit is used to determine a correction evaluation result for the corrected image based at least on the optimization index.

[0115] The image correction effect evaluation device provided in this embodiment belongs to the same concept as the image correction effect evaluation method provided in the above embodiments of this application. It can execute the method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the specific processing content of the image correction effect evaluation method provided in the above embodiments of this application, and will not be repeated here.

[0116] The functions implemented by each unit in the above image correction effect evaluation device can be implemented by the same or different processors, and this application embodiment does not limit this.

[0117] It should be understood that each unit in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0118] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0119] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0120] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0121] Exemplary electronic devices Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 3 As shown, the device includes: Memory 300 and processor 310; The memory 300 is connected to the processor 310 and is used to store programs; The processor 310 is configured to implement the image correction effect evaluation method disclosed in any of the above embodiments by running the program stored in the memory 300.

[0122] Specifically, the image correction effect evaluation device may also include: a bus, a communication interface 320, an input device 330, and an output device 340.

[0123] The processor 310, memory 300, communication interface 320, input device 330, and output device 340 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components of a computer system.

[0124] The processor 310 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0125] Processor 310 may include a main processor, as well as a baseband chip, modem, etc.

[0126] The memory 300 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 300 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0127] Input device 330 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0128] Output device 340 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0129] The communication interface 320 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0130] The processor 310 executes the program stored in the memory 300 and calls other devices, which can be used to implement the various steps of any of the image correction effect evaluation methods provided in the above embodiments of this application.

[0131] Exemplary computer program products and storage media In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the image correction effect evaluation method according to various embodiments of this application as described in any of the above embodiments of this specification.

[0132] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0133] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor through steps in the image correction effect evaluation method according to various embodiments of this application described above. Specifically, the following steps can be implemented: Acquire the original image and the corrected image obtained after correcting the original image; Extract first layout element information from the original image and second layout element information from the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image; Based on the first page element information and the second page element information, the optimization index of the corrected image relative to the original image is determined. The optimization index includes page element completeness, element geometric regularity, and semantic relevance. The correction evaluation result for the corrected image is determined based at least on the optimization index.

[0134] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0135] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0136] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0137] The modules and sub-modules in the apparatus and terminal in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0138] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0139] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0140] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0141] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0143] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0144] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for evaluating the effect of image correction, characterized in that, include: Acquire the original image and the corrected image obtained after correcting the original image; Extract first layout element information from the original image and second layout element information from the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image; Based on the first page element information and the second page element information, the optimization index of the corrected image relative to the original image is determined. The optimization index includes page element completeness, element geometric regularity, and semantic relevance. The correction evaluation result for the corrected image is determined based at least on the optimization index.

2. The method according to claim 1, characterized in that, Based on the first layout element information and the second layout element information, the optimization index of the corrected image relative to the original image is determined, including: The first page element information and the second page element information are matched to determine the matching degree of each page element information that is successfully matched. The optimization index is determined based on the matching degree, the first page element information, and the second page element information.

3. The method according to claim 2, characterized in that, The optimization index is determined based on the matching degree, the first page element information, and the second page element information, including: A first quantity of target page element information is determined, wherein the target page element information includes page element information in the second page element information whose matching degree is higher than a first preset matching threshold and whose element type is a target element type, wherein the target element type includes any one of the specified element types; Determine a second quantity of layout element information of the target element type in the first layout element information; The feature recall rate of the target feature type is determined based on the first quantity and the second quantity; The completeness of the layout elements is determined based on the element recall rate.

4. The method according to claim 2, characterized in that, Also includes: The third page element information that meets the element loss condition is identified. The element loss condition includes that the page element information is not successfully matched and the element recall rate is lower than the preset recall rate. Determine the location of the third layout element information in the corrected image or the original image; Store the information of the third page elements and the area they are located in.

5. The method according to claim 2, characterized in that, The optimization index is determined based on the matching degree, the first page element information, and the second page element information, including: Determine the first target page element information whose matching degree is higher than the second preset matching threshold in the first page element information, and the second target page element information whose matching degree is higher than the third preset matching threshold in the second page element information; Determine the first tilt angle of each page element information in the first target page element information, and determine the second tilt angle of each page element information in the second target page element information; The geometric regularity of the element is determined based on the first tilt angle and the second tilt angle.

6. The method according to claim 1, characterized in that, Based on the first layout element information and the second layout element information, the optimization index of the corrected image relative to the original image is determined, including: Based on the first layout element information, the text in the original image is extracted to obtain the first text information; Based on the second layout element information, the text in the corrected image is extracted to obtain the second text information; Based on the first text information and the second text information, the semantic quality parameters of the corrected image relative to the original image are determined. The semantic quality parameters include character accuracy improvement rate, sentence fluency improvement rate, and entity integrity improvement rate. The semantic relevance is determined based on the semantic quality parameters.

7. The method according to claim 6, characterized in that, The step of determining the semantic quality parameters of the corrected image relative to the original image based on the first text information and the second text information includes: Character recognition is performed on the first text information and the second text information respectively to obtain the first character recognition result of the first text information and the second character recognition result of the second text information; The real text information is compared with the first character recognition result to obtain a first comparison result, and the real text information is compared with the second character recognition result to obtain a second comparison result. The real text information is obtained from the annotation information of the original image. The character accuracy improvement rate is determined based on the first comparison result and the second comparison result.

8. The method according to claim 6, characterized in that, The step of determining the semantic quality parameters of the corrected image relative to the original image based on the first text information and the second text information includes: Character recognition is performed on the first text information and the second text information respectively to obtain the first character recognition result of the first text information and the second character recognition result of the second text information; Based on a pre-trained language model, the first language perplexity of the first character recognition result and the second language perplexity of the second character recognition result are determined respectively. Based on the first language confusion level and the second language confusion level, the sentence fluency improvement rate is determined; Based on a pre-trained language model, the first language completeness of the first character recognition result and the second language completeness of the second character recognition result are determined respectively. The entity completeness improvement rate is determined based on the first language completeness and the second language completeness.

9. The method according to claim 1, characterized in that, The method further includes: Obtain evaluation metrics for the visual perception dimensions of the original image and the corrected image as determined by human evaluation. Determining the correction evaluation result for the corrected image based at least on the optimization index includes: Based on the optimization index and the evaluation index, the correction evaluation result of the corrected image is determined.

10. The method according to claim 9, characterized in that, Based on the optimization index and the evaluation index, the correction evaluation result for the corrected image is determined, including: Determine whether the optimization index and / or the evaluation index meet the reverse correction constraint condition, wherein the reverse correction constraint condition is used to indicate that the image quality of the corrected image is lower than that of the original image; If satisfied, the optimization index and / or the evaluation index that satisfy the reverse correction constraint are determined to be preset values, and the index weights of the optimization index and the evaluation index are obtained respectively. Based on the index weights, the preset values, the optimization index and the evaluation index, the correction evaluation result of the corrected image is determined. If the conditions are not met, the sum of the optimization index and the evaluation index is determined as the correction evaluation result for the corrected image.

11. The method according to claim 9, characterized in that, Obtain evaluation metrics for the visual perception dimensions of the original image and the corrected image by human evaluation, including: The original image and the corrected image are stitched together to obtain a stitched image; The stitched image is displayed along with the evaluation criteria used to assess the visual perception dimension of the stitched image. Obtain the evaluation indicators for human evaluation of the stitched image based on the evaluation criteria.

12. The method according to claim 1, characterized in that, The original image includes annotation information for layout elements in the original image; the extraction of the first layout element information from the original image and the second layout element information from the corrected image includes: Extract the annotation information from the original image to obtain the first layout element information; The elements in the corrected image are identified using an optical character recognition method to obtain the element information of the second page layout.

13. An evaluation device for image correction effect, characterized in that, include: An acquisition unit is used to acquire the original image and a corrected image obtained after correcting the original image; An extraction unit is used to extract first layout element information from the original image and second layout element information from the corrected image; the first layout element information includes element information constituting each component of the original image, and the second layout element information includes element information constituting each component of the corrected image. The determining unit is configured to determine the optimization index of the corrected image relative to the original image based on the first page element information and the second page element information. The optimization index includes page element completeness, element geometric regularity, and semantic relevance. An evaluation unit is used to determine a correction evaluation result for the corrected image based at least on the optimization index.

14. An electronic device, characterized in that, Including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the image correction effect evaluation method as described in any one of claims 1 to 12 by running the program in the memory.

Images