Analysis method of image-text recognition model and program product

Abstract

The invention provides an analysis method of an image-text recognition model and a program product, and relates to the technical field of image-text recognition. The method comprises the following steps: detecting a similar recognition frame combination of a first image-text recognition model and a second image-text recognition model based on first output data for recognizing a test image by the first image-text recognition model and second output data for recognizing the test image by the second image-text recognition model; determining a first recognition text covered by a first recognition box from the first output data and determining a second recognition text covered by a second recognition box from the second output data under the condition that it is detected that the first image-text recognition model and the second image-text recognition model have similar recognition box combinations; and analyzing a text difference between the first recognition text and the second recognition text. The effects of a plurality of image-text recognition models can be contrastively analyzed, and the contrastive analysis accuracy of the image-text recognition models can be improved.

Description

Technical Field

[0001] This invention relates to the field of image and text recognition technology, and in particular to an analysis method and program product for an image and text recognition model. Background Technology

[0002] Image and text recognition models have achieved large-scale applications in vertical industries such as intelligent government office work, financial document processing, and medical report parsing. Simultaneously, they are deeply empowering K-12 smart education scenarios in the education sector, supporting multimodal information recognition and structured output in complex scenarios such as test paper grading and digitized teaching materials. Therefore, the performance of image and text recognition models is crucial, and analyzing their effectiveness is a pressing technical requirement.

[0003] Currently, the process involves rendering a test image and the text to be recognized by the image recognition model. This allows users to compare and analyze the rendered test image and the recognized text to determine the effectiveness of the image recognition model. However, existing technologies can only analyze a single image recognition model and cannot compare the effectiveness of multiple models. Summary of the Invention

[0004] This invention provides an analysis method and program product for image and text recognition models, which solves the shortcomings of existing technologies that cannot compare and analyze the effects of multiple image and text recognition models, and achieves accurate comparative analysis of multiple image and text recognition models.

[0005] This invention provides an analysis method for an image-text recognition model, comprising: Based on the first output data of the first image-text recognition model recognizing the test image, and the second output data of the second image-text recognition model recognizing the test image, similar recognition box combinations of the first image-text recognition model and the second image-text recognition model are detected; both the first output data and the second output data include the position information of several recognition boxes and the recognition text covered by the several recognition boxes, and the similar recognition box combinations include a first recognition box and a second recognition box that cover similar recognition text, wherein the first recognition box is determined by the first image-text recognition model recognizing the test image, and the second recognition box is determined by the second image-text recognition model recognizing the test image; When a similar recognition box combination is detected between the first image and text recognition model and the second image and text recognition model, the first recognition text covered by the first recognition box is determined from the first output data, and the second recognition text covered by the second recognition box is determined from the second output data. Analyze the textual differences between the first identified text and the second identified text.

[0006] According to the analysis method of the image-text recognition model provided by the present invention, the recognition bounding boxes of the first image-text recognition model and the recognition bounding boxes of the second image-text recognition model are similar based on the following method: Based on the first output data and the second output data, determine the degree of overlap between the recognition box of the first image and text recognition model and the recognition box of the second image and text recognition model; If the degree of overlap is greater than or equal to the overlap threshold, it is determined that the recognition box of the first image-text recognition model is similar to the recognition box of the second image-text recognition model. If the degree of overlap is less than the overlap threshold, it is determined that the recognition box of the first image-text recognition model is not similar to the recognition box of the second image-text recognition model.

[0007] According to the image-text recognition model analysis method provided by the present invention, the analysis of the text differences between the first recognized text and the second recognized text includes: Determine the first string of the first identified text and the second string of the second identified text; The text differences are analyzed based on the edit distance between the first string and the second string, and / or the edit operations that convert the first string and the second string to each other; the edit operations include a first edit operation that converts the first string to the second string and / or a second edit operation that converts the second string to the first string.

[0008] According to the analysis method of the image-text recognition model provided by the present invention, after analyzing the text differences based on the edit distance between the first string and the second string, and / or the editing operations of converting between the first string and the second string, the method further includes: Based on the second editing operation, the first identified text is distinguished and identified to distinguish the inserted characters required to convert the second string into the first string, and the replacement characters required to convert the second string into the first string. Based on the first editing operation, the second identified text is distinguished and identified to distinguish the inserted characters required to convert the first string into the second string, and the replacement characters required to convert the first string into the second string.

[0009] The analysis method for a text-image recognition model provided by the present invention further includes: When it is necessary to analyze the first image recognition model, a target recognition box for rendering is determined based on the first output data; the target recognition box is used to render on the already rendered test image; Based on the position information of the target recognition box in the first output data, the font size of the rendered test image is adjusted.

[0010] According to the image recognition model analysis method provided by the present invention, adjusting the rendered font size of the test image based on the position information of the target recognition box in the first output data includes: Based on the middle font size of the font size range, the first rendering height of the target recognition text covered by the target recognition box is determined; the initial font size range of the font size range is preset, and the first rendering height is the rendering height of the target recognition text after rendering according to the middle font size; Based on the comparison between the first rendering height and the second rendering height of the target recognition box, and the intermediate font size, the font size range is adjusted; the second rendering height is determined based on the position information of the target recognition box. If the difference between the first rendering height and the second rendering height is greater than a preset threshold, or if the first rendering height is greater than the second rendering height, the step of returning the middle font size based on the font size range to determine the first rendering height of the target recognition text covered by the target recognition box is performed. The rendered font size is adjusted based on the adjusted font size range.

[0011] The analysis method for a text-image recognition model provided by the present invention further includes: If a first bounding box to be analyzed exists on a rendered test image, the first data to be analyzed corresponding to the first bounding box is located in the rendered first output data based on the first identification information of the first bounding box; the first bounding box to be analyzed is rendered on the rendered test image based on the first output data. The first preset rendering method is used to render the display position of the first data to be analyzed; the first preset rendering method is used to distinguish the first data to be analyzed from other data in the rendered first output data except for the first data to be analyzed.

[0012] According to the image-text recognition model analysis method provided by the present invention, the step of locating the first data to be analyzed corresponding to the first recognition box in the rendered first output data based on the first identifier information of the first recognition box to be analyzed includes: Based on the identifier-path mapping relationship, the location path corresponding to the first identifier information is determined; Based on the positioning path, locate the first data to be analyzed corresponding to the first identification box to be analyzed in the rendered first output data; The identifier-path mapping relationship includes several pre-defined identifier-path correspondences, and the several identifier-path correspondences include the correspondence between the first identifier information and the positioning path.

[0013] The analysis method for a text-image recognition model provided by the present invention further includes: If a first bounding box to be analyzed exists on the rendered test image, a second preset rendering method is used to render the display position of the first bounding box to be analyzed; the second preset rendering method is used to distinguish the first bounding box to be analyzed from other bounding boxes on the rendered test image except for the first bounding box to be analyzed.

[0014] According to the image recognition model analysis method provided by the present invention, the first recognition box to be analyzed is determined based on the following method: Based on the first user-triggered command, the first identification box to be analyzed is determined; The first user-triggered instruction is the instruction triggered by the user clicking the first identification box to be analyzed.

[0015] The analysis method for a text-image recognition model provided by the present invention further includes: If there is second data to be analyzed in the rendered first output data, a second bounding box corresponding to the second data to be analyzed is matched based on the second identification information of the second data to be analyzed; the second bounding box is rendered on the rendered test image based on the first output data. When the second bounding box to be analyzed is obtained, the display position of the second bounding box to be analyzed is rendered using the second preset rendering method; the second preset rendering method is used to distinguish the second bounding box to be analyzed from other bounding boxes on the rendered test image except for the second bounding box to be analyzed.

[0016] The analysis method for a text-image recognition model provided by the present invention further includes: If there is a second data to be analyzed in the first output data that has already been rendered, a first preset rendering method is used to render the display position of the second data to be analyzed; the first preset rendering method is used to distinguish the second data to be analyzed from the other data in the first output data that has already been rendered except for the second data to be analyzed.

[0017] According to the image recognition model analysis method provided by the present invention, the second data to be analyzed is determined based on the following method: Based on the second user's trigger command, the second data to be analyzed is determined; The second user-triggered instruction is an instruction triggered by a user clicking on at least a portion of the data in the second data to be analyzed.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the analysis method of any of the above-described image recognition models.

[0019] The image-text recognition model analysis method and program product provided by this invention, based on first output data of a first image-text recognition model recognizing a test image and second output data of a second image-text recognition model recognizing a test image, detects similar recognition box combinations between the first and second image-text recognition models. Both the first and second output data include position information of several recognition boxes and the recognition text covered by these boxes. The similar recognition box combinations include first and second recognition boxes that cover similar recognition text. The first recognition box is determined by the first image-text recognition model when recognizing the test image, and the second recognition box is determined by the second image-text recognition model when recognizing the test image. This method is used to detect similar recognition box combinations between the first and second image-text recognition models. When similar recognition box combinations exist, the first recognized text covered by the first recognition box is determined from the first output data, and the second recognized text covered by the second recognition box is determined from the second output data. This allows for analysis of the text differences between the first and second recognized texts, eliminating the need for text difference analysis on the entire recognized text. Instead, text difference analysis can be performed on the recognized text within similar recognition boxes, thereby improving the accuracy of comparative analysis of multi-image-text recognition models. Furthermore, through the aforementioned text difference analysis, the effects of the first and second image-text recognition models can be compared and analyzed, enabling comparative analysis of the effects of multiple image-text recognition models. This improves the comprehensiveness of image-text recognition model analysis, i.e., enhances the analytical effect of image-text recognition models. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts illustrating the analysis method of the image and text recognition model provided by this invention.

[0022] Figure 2 This is the second flowchart illustrating the analysis method of the image and text recognition model provided by this invention.

[0023] Figure 3This is the third flowchart illustrating the analysis method of the image and text recognition model provided by this invention.

[0024] Figure 4 This is the fourth flowchart illustrating the analysis method of the image and text recognition model provided by this invention.

[0025] Figure 5 This is the fifth flowchart illustrating the analysis method of the image and text recognition model provided by this invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Current image recognition model analysis schemes mostly focus on the performance analysis of a single model. This involves rendering a test image and then rendering the text identified by the model from that image, allowing users to compare and analyze the results. However, existing technologies cannot compare and analyze the performance of multiple image recognition models.

[0028] Given that current image recognition model analysis schemes cannot compare and analyze the performance of multiple image recognition models, this invention has conducted research. The initial idea was to set indicators, calculate the indicators of each image recognition model, and then compare and analyze the various image recognition models based on the calculated indicators.

[0029] This invention studies the above approach and finds that although the above approach can compare and analyze the effects of multiple image-text recognition models, it cannot directly perform text difference analysis on the recognized text of multiple image-text recognition models. Since the essential purpose of image-text recognition models is to recognize the recognized text, performing text difference analysis on the recognized text is the key to image-text recognition model analysis.

[0030] To address the problems with the aforementioned approach, this invention continued its research. During the research process, the idea of ​​directly comparing and analyzing the overall recognized text from multiple image-text recognition models on a test image was conceived. However, comparing and analyzing the overall recognized text is too granular, resulting in low accuracy for the image-text recognition models.

[0031] To address the shortcomings of the aforementioned image-text recognition model analysis schemes, this invention further investigates and ultimately proposes an image-text recognition model analysis method. This method can compare and analyze a first image-text recognition model and a second image-text recognition model, and can perform text difference analysis on the first recognized text of the first image-text recognition model and the second recognized text of the second image-text recognition model. It can also perform text difference analysis on the recognized text within similar recognition boxes, without needing to perform text difference analysis on the entire recognized text, thereby improving the accuracy of comparative analysis of multiple image-text recognition models.

[0032] The analysis method of the image and text recognition model provided by the present invention will be introduced through the following embodiments. The following will combine... Figures 1-5 The analysis method of the image recognition model of the present invention is described.

[0033] Figure 1 This is one of the flowcharts illustrating the analysis method of the image and text recognition model provided by the present invention, such as... Figure 1 As shown, the analysis method of this image recognition model includes the following steps 110, 120 and 130.

[0034] Step 110: Based on the first output data of the first image recognition model recognizing the test image and the second output data of the second image recognition model recognizing the test image, detect similar recognition box combinations of the first image recognition model and the second image recognition model.

[0035] Here, the first and second image-text recognition models are different image-text recognition models (image-text recognition engines), both used for text recognition from images. With the rapid development of multimodal large model technology, this image-text recognition model can become a large image-text recognition model. For example, the image-text recognition model could be an OCR (Optical Character Recognition) model.

[0036] Here, the test image is used to test and analyze the performance of the image recognition model. When comparing and analyzing the first and second image recognition models, it is necessary to ensure that both models are recognizing the same test image.

[0037] Both the first and second output data include position information of several recognition boxes and the recognition text covered by the recognition boxes. The number of recognition boxes can be one or more. The position information of any recognition box indicates its location in the test image, and the recognition text covered by any recognition box is the text recognized by the selected location in the test image. The first and second output data can be JSON data.

[0038] In one embodiment, if the recognition box is a rectangle, its position information includes coordinate information, width information, and height information. Further, the coordinate information can be the coordinates of the upper left corner of the recognition box, or it can be the coordinates of the upper right, lower left, or lower right corner; no specific limitation is made here. For example, the position information of a recognition box includes coordinate information (e.g., x-coordinate 109, y-coordinate 204), the width of the recognition box is 307, and the height of the recognition box is 130; the recognition text covered by the recognition box is "The sun is shining brightly today, perfect for going out and playing."

[0039] In another embodiment, if the recognition box is a polygonal box (including but not limited to a quadrilateral box), the position information of the recognition box includes the coordinate information of multiple coordinate points, and the coordinate information of the multiple coordinate points is ordered in sequence so as to connect the coordinate points in sequence based on the coordinate information of each coordinate point to obtain the polygonal box.

[0040] The similarity recognition box combination includes a first recognition box and a second recognition box that cover similar text. The first recognition box is determined by the first image-text recognition model when recognizing the test image, and the second recognition box is determined by the second image-text recognition model when recognizing the test image. In other words, the first and second recognition boxes are similar in position on the test image. Because the recognition performance of the first and second image-text recognition models differs, the first and second recognition boxes may not completely overlap.

[0041] It should be understood that the number of similar bounding box combinations can be 0, 1 or more. For ease of explanation, the following description will use a single similar bounding box combination.

[0042] In one specific embodiment, the bounding boxes of the first image-text recognition model need to be paired with all bounding boxes of the second image-text recognition model for detection. If a similar bounding box is detected, the detection stops. If the bounding boxes of the first image-text recognition model are not similar to any of the bounding boxes of the second image-text recognition model, it is determined that there are no bounding boxes similar to the bounding boxes of the first image-text recognition model. Furthermore, all bounding boxes of the first image-text recognition model need to undergo the above processing to determine whether there are similar bounding box combinations between the first and second image-text recognition models.

[0043] In one embodiment, based on the first output data and the second output data, the degree of overlap between the recognition bounding boxes of the first image-text recognition model and the recognition bounding boxes of the second image-text recognition model is determined, so as to determine whether the two constitute a similar recognition bounding box combination based on the degree of overlap. This degree of overlap can be calculated using the IoU (Intersection over Union) algorithm.

[0044] Step 120: If similar recognition box combinations are detected between the first image recognition model and the second image recognition model, determine the first recognition text covered by the first recognition box from the first output data, and determine the second recognition text covered by the second recognition box from the second output data.

[0045] Here, the first and second recognized texts are similar recognized texts, but due to the difference in recognition performance between the first and second image-text recognition models, the two recognized texts also have textual differences.

[0046] Furthermore, if no similar bounding box combination is detected between the first and second image-text recognition models, no processing is required, i.e., no comparison of the recognized text is performed.

[0047] It should be noted that if a similar bounding box combination is detected, then that similar bounding box combination is identified as the specific area for which text comparison needs to be performed.

[0048] Step 130: Analyze the textual differences between the first identified text and the second identified text.

[0049] In one specific embodiment, the text differences between the first identified text and the second identified text are analyzed based on the edit distance between them and / or the editing operations that convert between the first identified text and the second identified text.

[0050] It should be understood that analyzing the text differences between the first and second recognized texts allows for a comparative analysis of the performance of the first and second image-text recognition models. For example, it can help determine which image-text recognition model performs better, and that text differences can aid in understanding where and why errors occur in the recognized text, thereby helping to understand the correct text.

[0051] It should be noted that if it is necessary to compare and analyze two or more image recognition models, it is only necessary to compare and analyze two image recognition models at a time, so as to ultimately achieve the comparison and analysis of multiple image recognition models.

[0052] The image-text recognition model analysis method provided in this embodiment of the invention detects similar recognition box combinations between the first and second image-text recognition models based on first output data of a first image-text recognition model recognizing a test image and second output data of a second image-text recognition model recognizing a test image. Both the first and second output data include position information of several recognition boxes and the recognition text covered by those boxes. The similar recognition box combinations include first and second recognition boxes that cover similar recognition text. The first recognition box is determined by the first image-text recognition model when recognizing the test image, and the second recognition box is determined by the second image-text recognition model when recognizing the test image. In the presence of similar bounding box combinations, the first recognized text covered by the first recognized box is determined from the first output data, and the second recognized text covered by the second recognized box is determined from the second output data. This allows for the analysis of text differences between the first and second recognized texts. Therefore, text difference analysis can be performed on the recognized text within similar bounding boxes, eliminating the need for analysis of the overall recognized text. This improves the accuracy of comparative analysis of multiple image-text recognition models. Furthermore, through this text difference analysis, the performance of the first and second image-text recognition models can be compared, enabling comparative analysis of the performance of multiple image-text recognition models. This enhances the comprehensiveness of image-text recognition model analysis, i.e., improves the overall analytical effect of the image-text recognition model.

[0053] Based on any of the above embodiments, a specific embodiment of the analysis method for the image and text recognition model is given below. Figure 2 This is the second flowchart illustrating the analysis method of the image and text recognition model provided by this invention, as shown below. Figure 2 As shown, the recognition boxes of the first image and text recognition model and the recognition boxes of the second image and text recognition model are similar to each other based on the following method.

[0054] It should be understood that this explanation only addresses one bounding box from the first image-to-text recognition model and one bounding box from the second image-to-text recognition model; that is, it only addresses whether two bounding boxes are similar. However, the bounding box from the first image-to-text recognition model needs to be paired with all bounding boxes from the second image-to-text recognition model. If a similar bounding box is detected, the detection stops. If the bounding box from the first image-to-text recognition model is dissimilar to all bounding boxes from the second image-to-text recognition model, it is determined that no bounding box similar to the bounding box from the first image-to-text recognition model exists. Furthermore, all bounding boxes from the first image-to-text recognition model need to undergo the above processing to determine whether there are similar bounding box combinations between the first and second image-to-text recognition models.

[0055] Step 111: Based on the first output data and the second output data, determine the degree of overlap between the recognition box of the first image and text recognition model and the recognition box of the second image and text recognition model.

[0056] Since both the first output data and the second output data include the position information of several recognition boxes, the degree of overlap between the two recognition boxes is determined based on the first output data and the second output data.

[0057] In one embodiment, the IoU algorithm is used to determine the degree of overlap, i.e., the overlap degree is expressed as the Intersection over Union (IoU). The IoU algorithm is a commonly used metric for evaluating the performance of object detection algorithms; it measures the degree of overlap between two bounding boxes. This embodiment of the invention cleverly uses the IoU algorithm to determine the degree of overlap, thereby improving the efficiency of overlap determination and achieving efficient image and text recognition model analysis. The formula for calculating IoU is as follows: IoU = (Intersection area) / (Union area); In the formula, the intersection area is the intersection area of ​​the recognition box of the first image and text recognition model and the recognition box of the second image and text recognition model, and the union area is the union area of ​​the recognition box of the first image and text recognition model and the recognition box of the second image and text recognition model.

[0058] In another embodiment, the IoA (Intersection over Area) algorithm is used to determine the degree of overlap.

[0059] Step 112: If the degree of overlap is greater than or equal to the overlap threshold, determine that the recognition box of the first image-text recognition model is similar to the recognition box of the second image-text recognition model.

[0060] Here, the overlap threshold can be preset according to the actual situation, and no specific limitation is made here.

[0061] It should be noted that if the bounding box of the first image-text recognition model is similar to the bounding box of the second image-text recognition model, then the two constitute a similar bounding box combination. Based on this, after confirming that the two constitute a similar bounding box combination, the comparison step of the recognized text covered by the two bounding boxes is performed.

[0062] Step 113: If the degree of overlap is less than the overlap threshold, determine that the recognition box of the first image-text recognition model is not similar to the recognition box of the second image-text recognition model.

[0063] It should be noted that if the recognition box of the first image-text recognition model is not similar to the recognition box of the second image-text recognition model, the comparison step of the recognition text covered by the recognition boxes of the two models will not be entered.

[0064] Furthermore, this embodiment can also evaluate the accuracy of the recognition boxes identified by the first image-text recognition model and the second image-text recognition model. For example, if the recognition boxes of the first image-text recognition model are not similar to all the recognition boxes of the second image-text recognition model, it can be determined that the recognition boxes of the first image-text recognition model may be incorrect.

[0065] The image and text recognition model analysis method provided in this embodiment of the invention detects similar recognition box combinations in the above manner. Specifically, it cleverly determines the degree of overlap between two recognition boxes and compares the degree of overlap with an overlap threshold to confirm whether they are similar, thereby improving the detection efficiency and accuracy of similar recognition box combinations, and thus improving the analysis accuracy and efficiency of the image and text recognition model.

[0066] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, step 130 includes: Determine the first string of the first identified text and the second string of the second identified text; The text differences are analyzed based on the edit distance between the first string and the second string, and / or the editing operations that convert between the first string and the second string.

[0067] Here, the first identified text includes the first string, and the second identified text also includes the second string.

[0068] Here, edit distance refers to the minimum number of single-character edit operations required to convert the first string into the second string.

[0069] The editing operations include a first editing operation to convert the first string into the second string and / or a second editing operation to convert the second string into the first string. These editing operations may include, but are not limited to: insertion operations (inserting a character into the string), deletion operations (deleting a character from the string), and substitution operations (replacing one character in the string with another character). The first editing operation may include, but is not limited to: the insertion operation required to convert the first string into the second string, and the substitution operation required to convert the first string into the second string; the second editing operation may include, but is not limited to: the insertion operation required to convert the second string into the first string, and the substitution operation required to convert the second string into the first string.

[0070] In one embodiment, the textual differences between the first and second identified texts are analyzed based on the edit distance between the first and second strings, thus revealing the edit distance difference (character difference) between the two identified texts.

[0071] In another embodiment, based on the editing operations of converting between the first string and the second string, the text differences between the first identified text and the second identified text can be analyzed, that is, the editing operations of converting between the two identified texts can be known, thereby knowing the reason for the difference between the two identified texts.

[0072] In another embodiment, based on edit distance and edit operations, the text differences between the first and second identified texts are analyzed. This allows us to determine not only the character differences between the two identified texts but also the reasons for those differences. In other words, for the identified texts corresponding to similar bounding box combinations, the edit distance and the edit operation of the phase loop transformation are determined to effectively distinguish the changes in the two texts and thus accurately analyze the text differences.

[0073] In one embodiment, the edit distance and edit operation can be determined using the Levenshtein Distance algorithm. The Levenshtein algorithm is used to calculate the edit distance between two strings. This embodiment of the invention cleverly uses the Levenshtein algorithm to determine the edit distance and edit operation, thereby improving the efficiency and accuracy of determining the edit distance and edit operation, and consequently improving the analysis efficiency and accuracy of the image and text recognition model.

[0074] Specifically, the Levenshtein algorithm uses dynamic programming to calculate the minimum edit distance between two strings. It uses a two-dimensional matrix to track the edit distance between each substring and fills this matrix based on insertion, deletion, and replacement operations.

[0075] In another embodiment, the edit distance and edit operation can be determined using the Damerau-Levenshtein Distance algorithm. In yet another embodiment, the edit distance and edit operation can be determined using the Longest Common Subsequence Distance algorithm.

[0076] The image-text recognition model analysis method provided in this embodiment of the invention analyzes text differences in the manner described above. Specifically, it cleverly determines the first string of the first recognized text and the second string of the second recognized text, thereby determining the edit distance and editing operation based on the first and second strings. This improves the efficiency and accuracy of text difference analysis, and consequently enhances the accuracy and efficiency of the image-text recognition model.

[0077] Based on any of the above embodiments, another embodiment of the analysis method for the image-text recognition model is given below. After analyzing the text differences based on the edit distance between the first string and the second string, and / or the editing operations of converting between the first string and the second string, the method further includes: Based on the second editing operation, the first identified text is distinguished and identified to distinguish the inserted characters required to convert the second string into the first string, and the replacement characters required to convert the second string into the first string. Based on the first editing operation, the second identified text is distinguished and identified to distinguish the inserted characters required to convert the first string into the second string, and the replacement characters required to convert the first string into the second string.

[0078] Since the second editing operation is an editing operation to convert the second string into the first string, it is possible to distinguish the insertion character required to convert the second string into the first string, and the replacement character required to convert the second string into the first string. Accordingly, the second editing operation includes: the insertion operation required to convert the second string into the first string, and the replacement operation required to convert the second string into the first string. Since the first editing operation is an editing operation to convert the first string into the second string, it is possible to distinguish the insertion character required to convert the first string into the second string, and the replacement character required to convert the first string into the second string. Accordingly, the first editing operation includes: the insertion operation required to convert the first string into the second string, and the replacement operation required to convert the first string into the second string.

[0079] It should be noted that the first identified text only includes the deletion characters required to convert the first string to the second string, and these deletion characters are equivalent to the insertion characters required to convert the second string to the first string. Similarly, the second identified text only includes the deletion characters required to convert the second string to the first string, and these deletion characters are equivalent to the insertion characters required to convert the first string to the second string.

[0080] In some embodiments, different identification methods are used to distinguish between inserted characters and replacement characters. In one specific embodiment, inserted characters and replacement characters are distinguished by different colors. Specifically, when rendering and displaying the first and second identified texts, inserted characters and replacement characters are distinguished by different colors; for example, inserted characters are rendered in yellow, and replacement characters are rendered in green.

[0081] The image-text recognition model analysis method provided in this embodiment of the invention highlights text differences by distinguishing and identifying different editing operations, so as to analyze text differences more intuitively, thereby improving the analysis efficiency and accuracy of text differences, and thus improving the analysis accuracy and efficiency of the image-text recognition model.

[0082] Based on any of the above embodiments, another embodiment of the analysis method for the image and text recognition model is given below. Figure 3 This is the third flowchart illustrating the analysis method of the image and text recognition model provided by this invention, as shown below. Figure 3 As shown, the method further includes steps 310 and 320.

[0083] Step 310: If it is necessary to analyze the first image recognition model, determine the target recognition box for rendering based on the first output data.

[0084] It should be noted that when analyzing a single image-text recognition model, the test image and the output data of the image-text recognition model recognizing the test image need to be rendered separately and rendered on the same interface to compare the test image (i.e., the actual data) with the output data (i.e., the recognition data), so as to analyze the image-text recognition model based on the comparison results.

[0085] In one specific embodiment, firstly, adaptive image processing is performed on the test image. This involves loading the image information of the test image and performing precise image rendering using Scalable Vector Graphics (SVG) format. By calculating the proportional relationship between the test image and the display container, dynamic scaling of the test image is achieved to ensure optimal visual effects on display devices of different sizes and resolutions. Secondly, the output data is rendered. This involves reading and parsing the output data, rendering the parsed data using SVG based on the position information of the included recognition boxes, and scaling the parsed data proportionally according to the image proportions using the transform attribute of the element. Finally, the recognized text in the output data is rendered according to coordinates.

[0086] Considering that dynamic layout was not performed when rendering the test image and recognition box, i.e., there was a lack of adaptive adjustment mechanism, this resulted in text rendering overflow in the test image or blank areas in the recognition box; and given the uncertainty of text length and content, using a fixed font size for rendering is difficult to meet the needs of all scenarios. Therefore, this embodiment of the invention fills the entire recognition box with the text in the test image to ensure the rendering effect of the test image and recognition box.

[0087] Here, we will take the first image-text recognition model as an example for explanation. The analysis scheme of the second image-text recognition model is similar, and will not be described in detail here.

[0088] Since the first output data includes the position information of several bounding boxes, the target bounding boxes for rendering can be determined based on the first output data. It should be understood that the several bounding boxes can be rendered and displayed simultaneously, separately or sequentially, or the user can choose to render and display them.

[0089] The target recognition bounding box is used to render on the rendered test image. Based on this, the target recognition bounding box can select the actual text it covers in the test image, so as to facilitate comparative analysis of the actual text and the recognized text covered by the target recognition bounding box.

[0090] Step 320: Based on the position information of the target recognition box in the first output data, adjust the rendering font size of the test image.

[0091] Specifically, based on the position information of the target recognition box, the rendered font size of the test image is adjusted to ensure that the text in the rendered test image fills the entire target recognition box, thereby improving the rendering effect and facilitating efficient analysis of the image-text recognition model. In other words, this ensures that the text content appropriately fills the entire recognition box (recognition area) while avoiding negative impacts on the reading experience due to excessively large or small font sizes. In other words, a dynamic computation-based optimization strategy is proposed, which pre-calculates a relatively suitable rendered font size to ensure that the text fully utilizes the available space without overflowing the recognition box.

[0092] In one embodiment, the font size is adjusted first, and then the test image is rendered based on this font size. In another embodiment, the test image is rendered first, and then re-rendered after the font size is adjusted.

[0093] In one embodiment, the font size of all text in the rendered test image is adjusted based on the rendered font size. In another embodiment, the font size of target recognition text in the test image is adjusted based on the rendered font size, where the target recognition text is the actual text covered by the target recognition bounding box.

[0094] Furthermore, when adjusting the rendered font size of the test image, it is necessary to ensure that the actual text covered by the target recognition box does not overflow the target recognition box.

[0095] The image-text recognition model analysis method provided in this embodiment of the invention adjusts the rendering font size of the test image based on the position information of the target recognition box in the first output data, thereby achieving adaptive font size adjustment of the text in the recognition box, thus improving the rendering and display effect of the recognition box and the test image, and facilitating efficient analysis of the image-text recognition model; that is, ensuring that the text content can properly fill the entire recognition box, while avoiding the impact on the reading experience due to the font being too large or too small, thereby improving the analysis accuracy and efficiency of the image-text recognition model.

[0096] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, step 320 includes: The first rendering height of the target recognition text covered by the target recognition box is determined based on the middle font size within the font size range. Based on the comparison between the first rendering height and the second rendering height of the target recognition box, and the intermediate font size, the font size range is adjusted; If the difference between the first rendering height and the second rendering height is greater than a preset threshold, or if the first rendering height is greater than the second rendering height, the step of returning the middle font size based on the font size range to determine the first rendering height of the target recognition text covered by the target recognition box is performed. The rendered font size is adjusted based on the adjusted font size range.

[0097] The initial font size range is preset; that is, an initial font size range is first set, and then this font size range is continuously reduced. For example, if the font size range is 2-18, then the middle font size is 10.

[0098] Wherein, the first rendering height is the rendering height of the target recognition text rendered according to the intermediate font size. That is, the first rendering height is the actual rendering height of the target recognition text in the test image based on the intermediate font size. This first rendering height can be understood as the DOM element height.

[0099] Since the target recognition box is rendered based on its position information, the second rendering height is determined based on the position information of the target recognition box.

[0100] In one specific embodiment, when the first rendering height is greater than the second rendering height, the maximum value of the font size range is adjusted to the intermediate font size; when the first rendering height is less than the second rendering height, the minimum value of the font size range is adjusted to the intermediate font size; when the first rendering height is equal to the second rendering height, either the maximum or minimum value of the font size range is adjusted to the intermediate font size. Of course, if the two are the same, the intermediate font size can be directly determined as the final rendered font size.

[0101] Here, the preset threshold can be set according to actual needs. Of course, the preset threshold can be 0, that is, to ensure that the first rendering height and the second rendering height are the same. No specific limit is made here.

[0102] It should be noted that, to ensure that the text in the test image does not overflow the target recognition box, the first rendering height should be less than or equal to the second rendering height.

[0103] It should be understood that an initial font size range is first set, and then the possible font size range is gradually narrowed by continuously comparing the first rendering height and the second rendering height and continuously adjusting the font size range, so as to finally obtain the adjusted font size range.

[0104] In one embodiment, any font size within the adjusted font size range is determined as the adjusted rendered font size. In another embodiment, the middle font size within the font size range before the last adjustment is determined as the adjusted rendered font size. In yet another embodiment, the middle font size within the adjusted font size range is determined as the adjusted rendered font size.

[0105] It should be noted that, considering the automatic line wrapping mechanism of the text, only the rendering height is considered, and the rendering width can be ignored.

[0106] It should be understood that, based on the middle font size within the font size range, the first rendering height of the target recognition text covered by the target recognition box is determined. Compared with the traditional methods of increasing or decreasing font size, the binary search method in this embodiment of the invention significantly reduces the number of iterations required, thereby reducing computational complexity and potential performance overhead. It is especially suitable for application scenarios with high requirements for page smoothness, because it can effectively reduce page stuttering caused by frequent redraws.

[0107] The image-text recognition model analysis method provided in this embodiment of the invention determines the first rendering height of the target recognition text covered by the target recognition box based on the middle font size of the font size range, thereby continuously adjusting the font size range based on the middle font size. This can efficiently and accurately determine the optimal rendering font size, thereby improving the analysis efficiency and accuracy of the image-text recognition model.

[0108] Based on any of the above embodiments, another embodiment of the analysis method for the image and text recognition model is given below. Figure 4 This is the fourth flowchart illustrating the analysis method of the image and text recognition model provided by this invention, as shown below. Figure 4 As shown, the method further includes steps 410 and 420.

[0109] Step 410: If a first bounding box to be analyzed exists on the rendered test image, locate the first data to be analyzed corresponding to the first bounding box in the rendered first output data based on the first identification information of the first bounding box.

[0110] It should be noted that when analyzing a single image-text recognition model, the test image and the output data of the image-text recognition model recognizing the test image need to be rendered separately and rendered on the same interface to compare the test image (i.e., the actual data) with the output data (i.e., the recognition data), so as to analyze the image-text recognition model based on the comparison results.

[0111] Considering the lack of correlation between the recognition boxes and output data under multiple factors during rendering, it becomes impossible to quickly locate the data to be analyzed corresponding to the recognition box to be analyzed. Therefore, this embodiment of the invention associates the first recognition box to be analyzed rendered on the test image with the first data to be analyzed in the first rendered output data, quickly locating the data to be analyzed corresponding to the recognition box to be analyzed and displaying them separately, thereby improving the analysis efficiency and accuracy of the image and text recognition model.

[0112] Here, we will take the first image-text recognition model as an example for explanation. The analysis scheme of the second image-text recognition model is similar, and will not be described in detail here.

[0113] The first bounding box to be analyzed is rendered on the rendered test image based on the first output data. Since the first output data includes the position information of several bounding boxes, the first bounding box to be analyzed can be rendered based on the first output data. This first bounding box to be analyzed is the bounding box that needs to be analyzed; that is, the actual text within the first bounding box needs to be compared and analyzed with the recognized text covered by the first bounding box.

[0114] Here, the first identifier is used to uniquely identify the first bounding box to be analyzed. That is, each bounding box is uniquely identified in advance so that the corresponding data to be analyzed can be quickly located later. Furthermore, based on the unique identifier, the data to be analyzed can be accurately located, thereby improving the accuracy of location and ultimately improving the analytical accuracy of the image and text recognition model.

[0115] Here, the first data to be analyzed should include the recognized text covered by the first recognition box to be analyzed; further, the first data to be analyzed also includes the position information of the first recognition box to be analyzed.

[0116] In some embodiments, the first output data is parsed based on the first identification information of the first identification box to be analyzed, so as to locate the first data to be analyzed corresponding to the first identification box to be analyzed in the rendered first output data.

[0117] In one embodiment, if the rendered first output data includes JSON data, the first output data is parsed using a recursive traversal algorithm to locate the data to be analyzed using a depth-first search algorithm.

[0118] In another embodiment, if the rendered first output data includes XML data, the first output data is parsed by integrating a DOM tree parser to compile an XPath query statement based on an XPath expression locator for location execution.

[0119] Step 420: The first preset rendering method is used to render the display position of the first data to be analyzed.

[0120] The first preset rendering method is used to distinguish the first data to be analyzed from other data in the rendered first output data, excluding the first data to be analyzed. In a specific embodiment, the first preset rendering method is to highlight the first data to be analyzed, for example, by injecting CSS Highlight API dynamic styles. Of course, the first preset rendering method can also highlight the first data to be analyzed using different colors, which is not specifically limited here.

[0121] It should be noted that the display position of the first data to be analyzed is rendered to distinguish it from other display positions, so as to make the first data to be analyzed different from other data in the rendered first output data.

[0122] Furthermore, the viewport is positioned at the display location of the first piece of data to be analyzed, facilitating quick and intuitive observation of this data and thus improving the efficiency and accuracy of the image recognition model. For example, the scrollTop value of the target node in the document flow is calculated to achieve viewport positioning.

[0123] Furthermore, when locating the first piece of data to be analyzed, a gradual scrolling animation is implemented through smooth scrolling, thereby improving the rendering and display effects and enhancing the user viewing experience. This is achieved by using color highlighting and automatic scrolling positioning to focus on elements.

[0124] The image recognition model analysis method provided in this embodiment of the invention quickly locates the data to be analyzed corresponding to the recognition box to be analyzed in the above manner, and performs differentiated rendering display, so that the first data to be analyzed is distinguished from other data in the rendered first output data except for the first data to be analyzed, thereby improving the analysis efficiency and accuracy of the image recognition model.

[0125] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, step 410 includes: Based on the identifier-path mapping relationship, the location path corresponding to the first identifier information is determined; Based on the positioning path, the first data to be analyzed, corresponding to the first bounding box to be analyzed, is located in the rendered first output data.

[0126] The identifier-path mapping relationship includes several pre-defined identifier-path correspondences, which include the correspondence between the first identifier information and the positioning path. That is, the identification box is associated with the data to be analyzed in advance, and the positioning path is determined in advance, thus eliminating the need to determine the correspondence and positioning path when positioning is required, thereby improving positioning efficiency. For example, the hierarchical position of the first data to be analyzed in the overall data structure can be calculated in advance to determine its positioning path.

[0127] In one embodiment, if the rendered first output data includes JSON data, the first output data is parsed using a recursive traversal algorithm to generate an identifier-path mapping relationship. In another embodiment, if the rendered first output data includes XML data, the first output data is parsed using an integrated DOM tree parser to generate an identifier-path mapping relationship.

[0128] Specifically, based on the positioning path, the first data to be analyzed, corresponding to the first bounding box, is quickly located in the rendered first output data. This positioning path clearly indicates the path to the first data to be analyzed, thus eliminating the need for re-positioning and improving positioning efficiency.

[0129] The image and text recognition model analysis method provided in this embodiment of the invention, through the above-mentioned method, pre-sets the identifier-path mapping relationship, thereby quickly locating the first data to be analyzed corresponding to the first recognition box in the rendered first output data based on the positioning path, thereby improving the positioning efficiency and ultimately improving the analysis efficiency of the image and text recognition model; and based on the unique identifier information, it can accurately match the positioning path, thereby improving the positioning accuracy and thus improving the analysis accuracy of the image and text recognition model.

[0130] Based on any of the above embodiments, another embodiment of the analysis method for the image-text recognition model is given below. This method further includes: If a first bounding box to be analyzed and identified already exists on the rendered test image, the second preset rendering method is used to render the display position of the first bounding box to be analyzed and identified.

[0131] The second preset rendering method is used to distinguish the first bounding box to be analyzed from other bounding boxes on the rendered test image. In one specific embodiment, the second preset rendering method is to highlight the display position of the first bounding box to be analyzed. Of course, the second preset rendering method can also highlight the first bounding box to be analyzed using different colors, which is not specifically limited here.

[0132] The image-text recognition model analysis method provided in this embodiment of the invention, through the above-described method, links the first recognition box to be analyzed and the first data to be analyzed in a two-way manner, that is, simultaneously performs differentiated rendering and display, so that users can more intuitively and quickly associate the first recognition box to be analyzed with the first data to be analyzed, thereby quickly and intuitively comparing and analyzing the actual text and the recognized text, and ultimately improving the analysis efficiency and accuracy of the image-text recognition model.

[0133] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, the first bounding box to be analyzed is determined based on the following method: Based on the first user-triggered command, the first bounding box to be analyzed is determined.

[0134] The first user-triggered instruction is triggered by the user clicking the first recognition box to be analyzed. That is, which recognition box to analyze is determined by the user's click, allowing the user to select the recognition box to be analyzed as needed, thereby improving the flexibility of the analysis.

[0135] For example, a user triggers a click event on the DOM element of the first box to be analyzed and identified, thereby capturing the first identification information of the first box to be analyzed and identified through event delegation.

[0136] The image-text recognition model analysis method provided in this embodiment of the invention allows users to select the recognition box to be analyzed, thereby improving the flexibility of the analysis and the image-text recognition model. Furthermore, users can quickly select the recognition box to be analyzed simply by clicking, thus improving the analysis efficiency of the image-text recognition model.

[0137] Based on any of the above embodiments, another embodiment of the analysis method for the image and text recognition model is given below. Figure 5This is the fifth flowchart illustrating the analysis method of the image and text recognition model provided by this invention, as shown below. Figure 5 As shown, the method further includes steps 510 and 520.

[0138] Step 510: If there is second data to be analyzed in the rendered first output data, match the second identification box corresponding to the second data to be analyzed based on the second identification information of the second data to be analyzed.

[0139] It should be noted that when analyzing a single image-text recognition model, the test image and the output data of the image-text recognition model recognizing the test image need to be rendered separately and rendered on the same interface to compare the test image (i.e., the actual data) with the output data (i.e., the recognition data), so as to analyze the image-text recognition model based on the comparison results.

[0140] Considering the lack of correlation between the recognition boxes and output data under multiple factors during rendering, it becomes impossible to quickly locate the recognition box corresponding to the data to be analyzed. Therefore, this embodiment of the invention associates the rendered second recognition box to be analyzed on the test image with the second data to be analyzed in the rendered first output data, quickly locating the recognition box corresponding to the data to be analyzed and displaying it separately, thereby improving the analysis efficiency and accuracy of the image and text recognition model.

[0141] Here, we will take the first image-text recognition model as an example for explanation. The analysis scheme of the second image-text recognition model is similar, and will not be described in detail here.

[0142] Here, the second data to be analyzed is the data that needs to be analyzed, that is, the actual text in the second recognition box to be analyzed needs to be compared and analyzed with the recognition text covered by the second recognition box. The second data to be analyzed should include the recognition text covered by the second recognition box; furthermore, the second data to be analyzed also includes the position information of the second recognition box.

[0143] Here, the second identification information is used to uniquely identify the second bounding box to be analyzed. Since the second bounding box to be analyzed is associated with the second data to be analyzed, the second identification information is also used to uniquely identify the second data to be analyzed. That is, each bounding box is uniquely identified in advance so that the bounding box corresponding to the data to be analyzed can be quickly located later. Moreover, based on the unique identification information, the bounding box to be analyzed can be accurately located, thereby improving the location accuracy and thus improving the analysis accuracy of the image and text recognition model.

[0144] The second bounding box to be analyzed is rendered on the already rendered test image based on the first output data. Since the first output data includes the position information of several bounding boxes, the second bounding box to be analyzed can be rendered based on the first output data. This second bounding box is the bounding box that needs to be analyzed.

[0145] It should be noted that, since the second data to be analyzed may be selected incorrectly, or the second bounding box to be analyzed may not need to be rendered, the second bounding box to be analyzed may not be matched with the second data to be analyzed.

[0146] Step 520: If the second bounding box to be analyzed is obtained, the display position of the second bounding box to be analyzed is rendered using the second preset rendering method.

[0147] The second preset rendering method is used to distinguish the second bounding box to be analyzed from other bounding boxes on the rendered test image. In one specific embodiment, the second preset rendering method is to highlight the display position of the second bounding box to be analyzed. Of course, the second preset rendering method can also highlight the second bounding box to be analyzed using different colors, which is not specifically limited here.

[0148] It should be noted that the display position of the second bounding box to be analyzed is rendered to distinguish it from other display positions, so as to make the second bounding box to be analyzed distinct from other bounding boxes.

[0149] Furthermore, the window is positioned to display the second frame to be analyzed, which facilitates quick and intuitive observation of the second frame to be analyzed, thereby improving the analysis efficiency and accuracy of the image and text recognition model.

[0150] Furthermore, when locating the second bounding box to be analyzed, a gradual scrolling animation is implemented through smooth scrolling, thereby improving the rendering and display effects and enhancing the user viewing experience. This is achieved by using color highlighting and automatic scrolling positioning to focus on the element.

[0151] Furthermore, if no second bounding box to be analyzed can be matched, no processing can be performed, or an error message can be displayed.

[0152] The image recognition model analysis method provided in this embodiment of the invention quickly locates the recognition box corresponding to the data to be analyzed in the above manner, and performs differentiated rendering display, so that the second recognition box to be analyzed is distinguished from other recognition boxes on the rendered test image except for the second recognition box to be analyzed, thereby improving the analysis efficiency and accuracy of the image recognition model.

[0153] Based on any of the above embodiments, another embodiment of the analysis method for the image-text recognition model is given below. This method further includes: If there is second data to be analyzed in the first output data that has already been rendered, the first preset rendering method is used to render the display position of the second data to be analyzed.

[0154] The first preset rendering method is used to distinguish the second data to be analyzed from the other data in the rendered first output data, excluding the second data to be analyzed. In one specific embodiment, the first preset rendering method is to highlight the second data to be analyzed. Of course, the first preset rendering method can also highlight the second data to be analyzed using different colors, which is not specifically limited here.

[0155] The image-text recognition model analysis method provided in this embodiment of the invention, through the above-described method, links the second recognition box to be analyzed and the second data to be analyzed in a two-way manner, that is, simultaneously performs differentiated rendering and display, so that users can more intuitively and quickly associate the second recognition box to be analyzed with the second data to be analyzed, thereby quickly and intuitively comparing and analyzing the actual text and the recognized text, and ultimately improving the analysis efficiency and accuracy of the image-text recognition model.

[0156] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, the second data to be analyzed is determined in the following manner: Based on the second user's trigger command, the second data to be analyzed is determined.

[0157] The second user-triggered instruction is triggered by the user clicking on at least a portion of the second set of data to be analyzed. In other words, the user determines which data needs to be analyzed, allowing them to select data as needed and thus improving the flexibility of the analysis.

[0158] For example, a user triggers a click event on a DOM element after the first output data has been rendered, thereby capturing the second identification information of the second data to be analyzed through event delegation.

[0159] The image recognition model analysis method provided in this embodiment of the invention allows users to select the data to be analyzed, thereby improving the flexibility of the analysis and the image recognition model. Furthermore, users can quickly select the data to be analyzed simply by clicking, thus improving the analysis efficiency of the image recognition model.

[0160] Based on any of the above embodiments, another embodiment of the analysis method for the image-text recognition model is given below. After step 510 above, the method further includes: If the second bounding box to be analyzed is not matched, a prompt message will be displayed.

[0161] Here, the prompt message is used to inform the user that no second bounding box to be analyzed was found. Furthermore, the prompt message is an error message, used to inform the user that the second data to be analyzed was selected incorrectly.

[0162] The image and text recognition model analysis method provided in this embodiment of the invention displays a prompt message when no second recognition box to be analyzed is found, thereby making it easier to intuitively confirm that no second recognition box to be analyzed has been found, thus improving the analysis efficiency of the image and text recognition model and enhancing the user experience.

[0163] Based on any of the above embodiments, another embodiment of the analysis method for the image-text recognition model is given below. This method further includes: During the process of saving the labeled data, the labeled data is distinguished and identified based on the labeling target of the labeled data.

[0164] The labeled data refers to the data used to annotate the rendered data. The rendered data refers to the rendered output data (such as first output data or second output data). There can be one or more labeled data points, and the annotation targets for different labeled data points can be the same or different.

[0165] It's important to note that when analyzing a single image-to-text recognition model, the test image and the output data from the model's recognition of the test image need to be rendered separately and displayed on the same screen. This allows for comparison between the test image (the actual data) and the output data (the recognized data), enabling analysis of the image-to-text recognition model based on the comparison results. Furthermore, during the analysis, the rendered output data can be annotated, highlighting the user's analysis results for subsequent reference by others. For example, annotations could include the corresponding correctly recognized text, the reasons for recognition errors, or performance metrics of the model.

[0166] Considering the ambiguous and unclear boundaries of labeled data in various scenarios—specifically, the current labeled data processing suffers from blurred boundaries, making it impossible to distinguish between whole, component, and bounding box labels—this ambiguity not only affects data accuracy but also obscures the boundaries between different types of labeled data. Therefore, this paper proposes an effective differentiation and processing approach for existing labeled scenarios. This involves distinguishing and labeling the labeled data based on its target while preserving the labeled data, thereby improving the readability of the labeled data and allowing others to understand what target was labeled.

[0167] It should be noted that different labeled targets have different identification methods. In one specific embodiment, different types are used to identify different labeled targets. Of course, other methods can also be used for differentiation and identification, and no specific limitation is made here.

[0168] It should be understood that the embodiments of the present invention can compare the output data of the image and text recognition model with the labeled data from the same source, avoid data silos in the analysis and verification process, and thus improve the iterative efficiency of the image and text recognition model.

[0169] The image-text recognition model analysis method provided in this embodiment of the invention distinguishes and identifies the labeled data based on the labeling targets of the labeled data, thereby improving the readability and maintainability of the labeled data. It can also provide a clearer basis in subsequent data processing and analysis, that is, it can clarify the labeling targets of each labeled data, thereby effectively improving the accuracy and consistency of data labeling, and ultimately improving the analysis accuracy and efficiency of the image-text recognition model.

[0170] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, the labeled target is determined based on the following method: If the annotation data is selected annotation data, the annotation target is determined to be the selected box; If the labeled data is not selected labeled data and the labeled data is feature labeled data, then the labeled target is determined to be the selected feature; If the labeled data is not selected label data and is not feature label data, the labeled target is determined to be the whole.

[0171] Here, the bounding box annotation data refers to the annotation data where a bounding box is first selected and then annotated. The feature annotation data refers to the annotation data where a feature is first selected and then annotated.

[0172] Specifically, if a box is selected on the current page, the annotation data will be considered an annotation for the box; if no box is selected but a feature is selected, the annotation will be considered an annotation for the feature; if neither the box nor the feature is selected, the annotation will be considered an annotation for the whole. This is a weighted annotation strategy; during the process of saving annotation data, we will judge the overall state of the page and annotate according to the following priority: box > feature > whole.

[0173] For selected boxes, such as rectangular or polygonal boxes, draw them by listening to mouse behavior.

[0174] The image recognition model analysis method provided in this embodiment of the invention determines the annotation targets of the annotation data according to the priority order of box > element > whole, which can improve the efficiency and accuracy of the annotation target determination, thereby improving the analysis accuracy and efficiency of the image recognition model.

[0175] Based on any of the above embodiments, a specific embodiment of the analysis method for the image-text recognition model is given below. In this method, the first output data is rendered in the following manner: During the process of rendering the first output data based on the text rendering engine, the formula data in the first output data is identified; When formula data is identified, the formula rendering engine is used to render the formula data, and the text rendering engine is used to render other text data in the first output data besides the formula data. If the formula data is not recognized, the first output data is rendered using the text rendering engine.

[0176] It should be noted that when analyzing a single image-text recognition model, the test image and the output data of the image-text recognition model recognizing the test image need to be rendered separately and rendered on the same interface to compare the test image (i.e., the actual data) with the output data (i.e., the recognition data), so as to analyze the image-text recognition model based on the comparison results.

[0177] Considering that rendering formula data, such as LaTeX data, is not supported, this embodiment of the invention loads a formula rendering engine into the text rendering engine. Therefore, it can render not only plain text data but also formula data, such as Markdown format data and formula data.

[0178] Here, we will take the first image and text recognition model as an example for explanation. The analysis scheme of the second image and text recognition model is similar. That is, the rendering method of the second output data can refer to the rendering method of the first output data. We will not go into details here.

[0179] The text rendering engine loads the formula rendering engine. Based on this, formula data can be rendered. This text rendering engine is primarily used to render data other than formulas, such as Markdown data. To meet the rendering requirements of Markdown formatted text, the text rendering engine can use markdown-it. Markdown-it has advantages such as fast parsing, strong extensibility, and high compatibility, thus enabling it to load the formula rendering engine. To meet the rendering requirements of LaTeX data, the formula rendering engine can be the KaTeX rendering engine; and LaTeX.js is used as the formula rendering solution, with advantages including high-quality rendering, broad support, and high flexibility.

[0180] Here, the formula data can include mathematical formulas, chemical formulas, etc., to meet the rendering requirements of mathematical formulas and chemical formulas.

[0181] In one specific embodiment, a deep matching algorithm with nested regular expressions is used to accurately identify formula data through feature symbol detection (such as delimiters such as $$…$$) and contextual semantic analysis.

[0182] For example, LaTeX mathematical formula parsing capabilities are deeply integrated into the standard Markdown rendering workflow. By extending Markdown-it's core parsing rules, a dual-mode text processing channel is constructed. While ensuring the standardized parsing of native Markdown elements, it achieves accurate recognition and independent rendering of LaTeX mathematical formula syntax, ultimately forming a standardized unified processing solution for heterogeneous text. Its core lies in constructing a dynamic parsing architecture based on an Abstract Syntax Tree (AST). After receiving the raw mixed text (i.e., the output data to be rendered) through a JSON interface, the system, relying on Markdown-it's plugin extension mechanism, embeds a LaTeX syntax detection module during the lexical analysis stage. This module uses a deep matching algorithm with nested regular expressions to accurately delineate the grammatical boundaries of mathematical formulas through feature symbol detection and contextual semantic analysis. For the identified LaTeX fragments, the KaTeX rendering engine is invoked for rendering, while maintaining parallel parsing of regular Markdown elements.

[0183] More specifically, the specific steps are as follows: 1. Load a custom LaTeX plugin and register a pre-parsing hook function when initializing a Markdown-it instance; 2. Insert a LaTeX syntax scanner into the Markdown-it core ruler processing stage to perform multi-level regular expression matching on the text stream; 3. Construct a formula syntax validator and ensure paired matching of delimiters through stack balance detection; 4. Generate specific type of token nodes for verified LaTeX fragments and attach metadata annotations; 5. In the post-rendering stage, the corresponding LaTeX renderer is dynamically selected based on the Token type for the final output conversion.

[0184] It should be understood that by integrating Katex.js, high-precision rendering of LaTeX formulas is achieved, powerfully supporting the visualization of complex mathematical and chemical equations and other professional content. Simultaneously, combined with Markdown-it, it provides lightweight and flexible Markdown text rendering capabilities, fully meeting the formatting needs of various scenarios such as essays and documents. Based on this, the synergy between Katex.js and Markdown-it offers advantages such as efficient rendering, strong adaptability to multiple scenarios, seamless integration, lightweight ease of use, cross-platform compatibility, and powerful extensibility.

[0185] The image recognition model analysis method provided in this embodiment of the invention can render not only plain text data but also formula data, thus making it applicable to a variety of complex scenarios, improving the flexibility and applicability of the analysis scenarios.

[0186] Based on the above embodiments, this invention can improve the efficiency of complex scene analysis by more than 50%, driving the efficient iteration and scenario-based implementation of image and text recognition models. Practical application has verified the significant effectiveness of this invention: compared to purely manual analysis, in terms of core performance indicators, the time for single-element detection is reduced from 3 minutes to 1 minute, improving analysis efficiency by 66.7%; the full-element analysis cycle is compressed from 20 minutes to 10 minutes, increasing processing speed by 50%; and the single-version parsing capacity is doubled from 100 images to 200 images. In other words, the analysis system constructed using the above methods, through the establishment of heterogeneous data fusion and an intelligent workflow system, significantly improves processing efficiency and system throughput.

[0187] The analysis apparatus for the image-text recognition model provided by the present invention will be described below. The analysis apparatus for the image-text recognition model described below can be referred to in correspondence with the analysis method for the image-text recognition model described above. The analysis apparatus for the image-text recognition model includes a detection module, a determination module, and an analysis module.

[0188] The detection module is used to generate first output data for recognizing a test image based on a first image-text recognition model, and second output data for recognizing the test image based on a second image-text recognition model, and to detect similar recognition box combinations of the first image-text recognition model and the second image-text recognition model; both the first output data and the second output data include the position information of several recognition boxes and the recognition text covered by the several recognition boxes, and the similar recognition box combination includes a first recognition box and a second recognition box that cover similar recognition text, wherein the first recognition box is determined by the first image-text recognition model for recognizing the test image, and the second recognition box is determined by the second image-text recognition model for recognizing the test image.

[0189] The determining module is used to determine, from the first output data, the first recognized text covered by the first recognized text box, and from the second output data, the second recognized text covered by the second recognized text box, when a similar recognized box combination is detected between the first image and text recognition model and the second image and text recognition model.

[0190] The analysis module is used to analyze the textual differences between the first identified text and the second identified text.

[0191] The present invention also provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The processor can invoke logical instructions in memory to execute an analysis method for a text recognition model. This method includes: first output data for recognizing a test image based on a first text recognition model, and second output data for recognizing the test image based on a second text recognition model; detecting similar bounding box combinations between the first and second text recognition models; both the first and second output data include position information of several bounding boxes and the recognized text covered by the bounding boxes; the similar bounding box combinations include first and second bounding boxes with similar recognized text, where the first bounding box is determined by the first text recognition model in recognizing the test image, and the second bounding box is determined by the second text recognition model in recognizing the test image; when similar bounding box combinations are detected between the first and second text recognition models, determining the first recognized text covered by the first bounding box from the first output data, and determining the second recognized text covered by the second bounding box from the second output data; and analyzing the textual differences between the first and second recognized texts.

[0192] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the analysis method of the image-text recognition model provided by the above methods. The method includes: first output data for recognizing a test image based on a first image-text recognition model, and second output data for recognizing the test image based on a second image-text recognition model; detecting similar recognition box combinations between the first image-text recognition model and the second image-text recognition model; both the first output data and the second output data include position information of several recognition boxes and recognition text covered by the several recognition boxes, the similar recognition box combination includes a first recognition box and a second recognition box with similar recognition text, the first recognition box being determined by the first image-text recognition model for recognizing the test image, and the second recognition box being determined by the second image-text recognition model for recognizing the test image; when a similar recognition box combination is detected between the first image-text recognition model and the second image-text recognition model, determining the first recognition text covered by the first recognition box from the first output data, and determining the second recognition text covered by the second recognition box from the second output data; and analyzing the text differences between the first recognition text and the second recognition text.

[0194] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an analysis method for the image-text recognition model provided by the above methods. The method includes: first output data for recognizing a test image based on a first image-text recognition model, and second output data for recognizing the test image based on a second image-text recognition model; detecting similar recognition box combinations between the first image-text recognition model and the second image-text recognition model; both the first output data and the second output data include position information of a plurality of recognition boxes and recognition text covered by the plurality of recognition boxes, the similar recognition box combination including a first recognition box and a second recognition box with similar recognition text, the first recognition box being determined by the first image-text recognition model for recognizing the test image, and the second recognition box being determined by the second image-text recognition model for recognizing the test image; when a similar recognition box combination is detected between the first image-text recognition model and the second image-text recognition model, determining the first recognition text covered by the first recognition box from the first output data, and determining the second recognition text covered by the second recognition box from the second output data; and analyzing the text differences between the first recognition text and the second recognition text.

[0195] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An analysis method for an image-text recognition model, characterized in that, include: Based on the first output data of the first image-text recognition model recognizing the test image, and the second output data of the second image-text recognition model recognizing the test image, similar recognition box combinations of the first image-text recognition model and the second image-text recognition model are detected; both the first output data and the second output data include the position information of several recognition boxes and the recognition text covered by the several recognition boxes, and the similar recognition box combinations include a first recognition box and a second recognition box that cover similar recognition text, wherein the first recognition box is determined by the first image-text recognition model recognizing the test image, and the second recognition box is determined by the second image-text recognition model recognizing the test image; When a similar recognition box combination is detected between the first image and text recognition model and the second image and text recognition model, the first recognition text covered by the first recognition box is determined from the first output data, and the second recognition text covered by the second recognition box is determined from the second output data. Analyze the textual differences between the first identified text and the second identified text.

2. The analysis method for the image-text recognition model according to claim 1, characterized in that, The similarity between the bounding boxes of the first image-text recognition model and the bounding boxes of the second image-text recognition model is detected based on the following method: Based on the first output data and the second output data, determine the degree of overlap between the recognition box of the first image and text recognition model and the recognition box of the second image and text recognition model; If the degree of overlap is greater than or equal to the overlap threshold, it is determined that the recognition box of the first image-text recognition model is similar to the recognition box of the second image-text recognition model. If the degree of overlap is less than the overlap threshold, it is determined that the recognition box of the first image-text recognition model is not similar to the recognition box of the second image-text recognition model.

3. The analysis method for the image-text recognition model according to claim 1, characterized in that, The analysis of the textual differences between the first identified text and the second identified text includes: Determine the first string of the first identified text and the second string of the second identified text; The text differences are analyzed based on the edit distance between the first string and the second string, and / or the edit operations that convert the first string and the second string to each other; the edit operations include a first edit operation that converts the first string to the second string and / or a second edit operation that converts the second string to the first string.

4. The analysis method for the image-text recognition model according to claim 3, characterized in that, After analyzing the text differences based on the edit distance between the first string and the second string, and / or the editing operations that convert between the first string and the second string, the method further includes: Based on the second editing operation, the first identified text is distinguished and identified to distinguish the inserted characters required to convert the second string into the first string, and the replacement characters required to convert the second string into the first string. Based on the first editing operation, the second identified text is distinguished and identified to distinguish the inserted characters required to convert the first string into the second string, and the replacement characters required to convert the first string into the second string.

5. The analysis method for the image-text recognition model according to any one of claims 1 to 4, characterized in that, Also includes: When it is necessary to analyze the first image recognition model, the target recognition box for rendering is determined based on the first output data; The target recognition box is used to render on the already rendered test image; Based on the position information of the target recognition box in the first output data, the font size of the rendered test image is adjusted.

6. The analysis method for the image-text recognition model according to claim 5, characterized in that, Adjusting the rendered font size of the test image based on the position information of the target recognition box in the first output data includes: Based on the middle font size of the font size range, the first rendering height of the target recognition text covered by the target recognition box is determined; the initial font size range of the font size range is preset, and the first rendering height is the rendering height of the target recognition text after rendering according to the middle font size; Based on the comparison between the first rendering height and the second rendering height of the target recognition box, and the intermediate font size, the font size range is adjusted; the second rendering height is determined based on the position information of the target recognition box. If the difference between the first rendering height and the second rendering height is greater than a preset threshold, or if the first rendering height is greater than the second rendering height, the step of returning the middle font size based on the font size range to determine the first rendering height of the target recognition text covered by the target recognition box is performed. The rendered font size is adjusted based on the adjusted font size range.

7. The analysis method for the image-text recognition model according to any one of claims 1 to 4, characterized in that, Also includes: If a first bounding box to be analyzed exists on a rendered test image, the first data to be analyzed corresponding to the first bounding box is located in the rendered first output data based on the first identification information of the first bounding box. The first bounding box to be analyzed is rendered on the rendered test image based on the first output data; The first preset rendering method is used to render the display position of the first data to be analyzed; the first preset rendering method is used to distinguish the first data to be analyzed from other data in the rendered first output data except for the first data to be analyzed.

8. The analysis method for the image-text recognition model according to claim 7, characterized in that, The step of locating the first data to be analyzed corresponding to the first identification box in the rendered first output data based on the first identification information of the first identification box to be analyzed includes: Based on the identifier-path mapping relationship, the location path corresponding to the first identifier information is determined; Based on the positioning path, locate the first data to be analyzed corresponding to the first identification box to be analyzed in the rendered first output data; The identifier-path mapping relationship includes several pre-defined identifier-path correspondences, and the several identifier-path correspondences include the correspondence between the first identifier information and the positioning path.

9. The analysis method for the image-text recognition model according to claim 7, characterized in that, Also includes: If a first bounding box to be analyzed exists on the rendered test image, a second preset rendering method is used to render the display position of the first bounding box to be analyzed; the second preset rendering method is used to distinguish the first bounding box to be analyzed from other bounding boxes on the rendered test image except for the first bounding box to be analyzed.

10. The analysis method for the image-text recognition model according to claim 7, characterized in that, The first bounding box to be analyzed is determined based on the following method: Based on the first user-triggered command, the first identification box to be analyzed is determined; The first user-triggered instruction is the instruction triggered by the user clicking the first identification box to be analyzed.

11. The analysis method for the image-text recognition model according to any one of claims 1 to 4, characterized in that, Also includes: If there is second data to be analyzed in the rendered first output data, the second identification box corresponding to the second data to be analyzed is matched based on the second identification information of the second data to be analyzed. The second bounding box to be analyzed is rendered on the already rendered test image based on the first output data; When the second bounding box to be analyzed is obtained, the display position of the second bounding box to be analyzed is rendered using the second preset rendering method; the second preset rendering method is used to distinguish the second bounding box to be analyzed from other bounding boxes on the rendered test image except for the second bounding box to be analyzed.

12. The analysis method for the image-text recognition model according to claim 11, characterized in that, Also includes: If there is a second data to be analyzed in the first output data that has already been rendered, a first preset rendering method is used to render the display position of the second data to be analyzed; the first preset rendering method is used to distinguish the second data to be analyzed from the other data in the first output data that has already been rendered except for the second data to be analyzed.

13. The analysis method for the image-text recognition model according to claim 11, characterized in that, The second set of data to be analyzed was determined in the following manner: Based on the second user's trigger command, the second data to be analyzed is determined; The second user-triggered instruction is an instruction triggered by a user clicking on at least a portion of the data in the second data to be analyzed.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the analysis method of the image recognition model as described in any one of claims 1 to 13.

Images