Image marking method, device, equipment, medium and product

Abstract

The invention discloses an image marking method and device, equipment, a medium and a product, and relates to the technical field of image processing. Correcting the to-be-processed image based on a standard image to obtain a corrected image; performing table extraction on the corrected image and the standard image to obtain corrected cells and standard cells; comparing the correction cell with the standard cell to obtain a difference cell; and performing mapping marking on a to-be-processed image based on the difference cells to obtain a marked image. By adopting the technical scheme, the table is corrected, extracted and subjected to difference comparison through the multi-fusion algorithm, so that the difference comparison efficiency of the table in the image is improved, the problem that the existing image marking cannot be subjected to accurate reduction and efficient comparison is further solved, and the visualization of the table difference in the image is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image labeling method, apparatus, device, medium, and product. Background Technology

[0002] Currently, when comparing and verifying table images, manual verification of table content is required, which consumes a huge amount of manpower and resources when comparing a large amount of table document data.

[0003] Traditional image difference comparison uses pixel-based difference comparison methods, which can accurately locate the difference. However, pixel-based document comparison and recognition methods usually have high requirements for two table documents, and the layout of the two table documents must remain unchanged. Since image acquisition can cause the table position to shift, the consistency of the layout cannot be guaranteed, resulting in large errors in pixel-based single-value comparison methods.

[0004] Therefore, there is an urgent need for an image labeling method that can perform full-process image difference comparison and achieve accurate analysis and visual labeling of the difference tables in the image. Summary of the Invention

[0005] This invention provides an image labeling method, apparatus, device, medium, and product that solves the problem that existing image labeling methods cannot accurately restore and efficiently compare images. By using a multi-fusion algorithm to correct, extract, and compare differences in tables, the invention improves the efficiency of comparing differences in tables in images and further realizes the visualization of differences in tables in images.

[0006] According to one aspect of the present invention, an image tagging method is provided, comprising: Obtain the image to be processed; The image to be processed is corrected based on a standard image to obtain the corrected image; Tables are extracted from the corrected image and the standard image respectively to obtain corrected cells and standard cells; The correction cells and the standard cells are compared to obtain the difference cells; Based on the difference cells, the image to be processed is mapped and labeled to obtain a labeled image.

[0007] According to another aspect of the present invention, an image marking device is provided, comprising: The acquisition module is used to acquire the image to be processed; The correction module is used to correct the image to be processed based on a standard image to obtain the corrected image; The extraction module is used to extract tables from the corrected image and the standard image respectively, to obtain corrected cells and standard cells; The comparison module is used to compare the correction cell and the standard cell to obtain the difference cell; The labeling module is used to map and label the image to be processed based on the difference cells to obtain a labeled image.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image tagging method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the image tagging method according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the image tagging method according to any embodiment of the present invention.

[0011] The technical solution of this invention improves the efficiency of comparing differences between tables in an image by using a multi-fusion algorithm to correct, extract, and compare the differences between tables. This further solves the problem that existing image markers cannot be accurately restored and efficiently compared, and enables the visualization of differences between tables in an image.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of an image tagging method provided according to an embodiment of the present invention; Figure 2This is a flowchart of a method for obtaining difference cells according to an embodiment of the present invention; Figure 3 This is a flowchart of an image tagging method provided according to an embodiment of the present invention; Figure 4 This is a flowchart of an image tagging method provided according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an image marking device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device that implements the image marking method of this invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Furthermore, it should be noted that the information collected in the technical solution of this invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data all comply with the relevant laws, regulations and standards of relevant countries and regions, necessary confidentiality measures have been taken, and public order and good morals are not violated. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0018] Figure 1This invention provides a flowchart of an image labeling method according to an embodiment of the present invention. This embodiment is applicable to labeling difference tables in images. The method can be executed by an image labeling device, which can be implemented in hardware and / or software and can be configured in a server. Figure 1 As shown, the method includes: S110. Obtain the image to be processed.

[0019] The image to be processed is the image to be marked with differences, which can be an image containing a table.

[0020] Specifically, obtain the table image containing the table that needs to be marked for difference.

[0021] S120. Correct the image to be processed based on the standard image to obtain the corrected image.

[0022] The standard image is the reference image corresponding to the image to be processed, and the table it contains can be a blank table in the unprocessed reference image; the correction image is the image to be processed based on the standard image as a template, and the tilted or distorted tables in the image to be processed are corrected into orthographic projection tables.

[0023] Specifically, using the rectangle formed by the four points of the outer border of the table in the standard image as the standard template, the perspective transformation matrix between the standard image and the image to be processed is determined. Based on this matrix, the tilted and distorted table image in the image to be processed is corrected into an orthographic projection image, and this orthographic projection image is used as the correction image.

[0024] Understandably, by aligning the outer border of the table with the image coordinate axes, errors caused by perspective distortion and camera tilt are eliminated, accurately restoring the original form of the table and improving the visualization of differences in subsequent tables.

[0025] S130. Extract tables from the corrected image and the standard image respectively to obtain the corrected cell and the standard cell.

[0026] Among them, table extraction can be performed using multiple fusion algorithms, such as mask filtering algorithm, contour filtering algorithm, etc., to remove duplicate lines, filter and supplement the table; the corrected cell is at least one cell of the table in the corrected image; the standard cell is at least one cell of the table in the standard image.

[0027] Specifically, a multi-fusion algorithm can be used to process the table lines in the corrected image and the standard image separately, and the processed table lines can be extracted to obtain the corrected cells in the corrected image and the standard cells in the standard image.

[0028] S140. Compare the correction cells and the standard cells to obtain the difference cells.

[0029] The comparison method can be to compare the background pixel ratio of the cells; the difference cells are the cells that are different from the standard cells.

[0030] Specifically, the correction cells and standard cells are compared based on pixel ratio to find cells in the correction cells that are different from the standard cells, and these cells are identified as the difference cells.

[0031] Optional, such as Figure 2 The method shown involves comparing the correction cell and the standard cell to obtain the following: S141. Calculate the coordinates of the four vertices of the correction cell and the standard cell respectively to determine the detection area corresponding to the correction image and the standard area corresponding to the standard image.

[0032] Among them, vertex coordinates are the coordinates of the corner points of the cell, such as top left, top right, bottom right, and bottom left; the detection area and the standard area are the areas where the vertex coordinates of the cell are used to perform rectangular calculations.

[0033] Specifically, the corner coordinates of the cells corresponding to the correction cells and the standard cells are traversed separately. Based on the vertex coordinates of each cell, the smallest orthogonal rectangle containing these four points is calculated using the OpenCV built-in function (cv::boundingRect()). This rectangle is then used as the detection region corresponding to the correction image and the standard region corresponding to the standard image.

[0034] In one optional embodiment of the present invention, the initially obtained detection area and standard area can be reduced in range by pixel reduction. The rectangle width can be reduced to half its original width minus 4 pixels, and the height reduced by 5 pixels. For example, horizontally, the starting point of the rectangle can be moved half its original width to the right to ensure that the detection area only includes the right half of the cell, eliminating interference from the left side. Vertically, the starting point of the rectangle can be slightly moved downwards by 4 pixels to avoid interference from the top border. By globally reducing the range of the rectangle, the focus is more on the location of the different cells based on user habits, effectively eliminating the influence of cell borders and ensuring that the detection area is completely within the cell's filled area. It should be noted that the specific number of pixels for range reduction can be set by those skilled in the art based on practical experience and needs; this embodiment of the present invention does not impose specific limitations on this.

[0035] S142. Segment the correction image based on the detection region to obtain at least one correction patch.

[0036] The correction patch is a cell region cropped from the detection area.

[0037] Specifically, at least one detection region is calculated and the corresponding cell region is cropped from the correction image to obtain the corresponding correction patch.

[0038] S143. Segment the standard image based on the standard region to obtain at least one standard patch.

[0039] The standard tile is a cell region cropped from a standard area.

[0040] Specifically, at least one standard region is calculated and cropped from the standard image to obtain the corresponding cell region, thus obtaining the corresponding standard image patch.

[0041] S144. Calculate the pixel ratio for the calibration block and the standard block respectively to obtain the calibration pixel ratio for the calibration block and the standard pixel ratio for the standard block.

[0042] Among them, the corrected pixel ratio and the standard pixel ratio are the ratios of the total pixel value to the area of ​​the patch.

[0043] Specifically, the built-in OpenCV function `cv::sum()` can be used to calculate the sum of pixel values ​​corresponding to the calibration patch and the sum of pixel values ​​corresponding to the standard patch, respectively. The area of ​​the region corresponding to the calibration patch and the area of ​​the region corresponding to the standard patch are obtained by multiplying the number of rows and columns of the calibration patch and the standard patch, respectively. The ratio of the sum of pixel values ​​corresponding to the calibration patch to the area of ​​the calibration patch is used as the percentage of calibration pixels; the ratio of the sum of pixel values ​​corresponding to the standard patch to the area of ​​the standard patch is used as the percentage of standard pixels.

[0044] In an optional embodiment of the present invention, before calculating the pixel ratio of the calibration block and the standard block respectively, the size of the calibration block and the standard block can be normalized. A bilinear interpolation algorithm can be used to adjust the calibration block to be exactly the same size as the standard block, eliminating perspective distortion and scale differences caused by differences in image acquisition angle and distance, and ensuring that subsequent pixel statistics are performed at the same spatial resolution.

[0045] S145. Determine the percentage difference based on the corrected pixel percentage and the standard pixel percentage.

[0046] The percentage difference is the difference in pixel percentage between the correction cell and the corresponding standard cell.

[0047] Specifically, the percentage difference is calculated based on the absolute difference between the corrected pixel percentage and the standard pixel percentage for cells at the same location; the percentage difference is calculated as follows: diff=∣testRatio-refRatio∣ Where diff is the percentage difference; testRatio is the percentage of corrected pixels; and refRatio is the percentage of standard pixels.

[0048] S146. Normalize the percentage difference to obtain the normalized difference.

[0049] The normalized difference is the value that normalizes the percentage difference to the range of 0-255.

[0050] Specifically, the percentage difference is normalized, and the normalized value is taken as the value within the range of 0-255.

[0051] S147. Determine the difference cells from the corrected cells based on the normalized difference and the label threshold.

[0052] The marking threshold is a pre-set value used to distinguish whether a cell has been marked; the difference cells are cells that have been marked by the user.

[0053] Specifically, the normalized difference is compared with the labeling threshold. If the normalized difference exceeds the labeling threshold, it indicates that the cell has been labeled by the user and is treated as a difference cell.

[0054] Understandably, by comparing the foreground and background of cells, the differences between the tables in the image to be processed and the standard image are presented intuitively, ensuring that the difference recognition accuracy is over 98%, without the need for manual row-by-row comparison; and it supports the extraction of tables from images of different sizes, different border styles, and different shooting conditions without the need to adjust parameters, making it suitable for diverse table scenarios such as office documents, reports, and forms.

[0055] S150. Map and label the image to be processed based on the difference cells to obtain the labeled image.

[0056] The mapping markers can be highlighted or the cell outline color can be redrawn; the marked image is the image to be processed after mapping, and the user's markings can be clearly seen.

[0057] Specifically, for cells with differences, a red polygon can be used to draw the cell outline to differentiate the cells marked by the user and obtain a marked image.

[0058] Understandably, by setting mapping markers to display the differences in cells, and by setting cell-level foreground-background contrast or outline color drawing, the changes in the image to be processed can be presented intuitively, ensuring that the difference recognition accuracy reaches more than 98%, eliminating the need for manual row-by-row comparison, and further improving the efficiency and accuracy of the entire process for recognizing differences in tables.

[0059] This invention involves acquiring an image to be processed; correcting the image to be processed based on a standard image to obtain a corrected image; extracting tables from both the corrected image and the standard image to obtain corrected cells and standard cells; comparing the corrected cells and standard cells to obtain difference cells; and mapping and labeling the image to be processed based on the difference cells to obtain a labeled image. This technical solution improves the efficiency of comparing table differences in images by using a multi-fusion algorithm for table correction, extraction, and difference comparison, further solving the problem that existing image labeling methods cannot accurately restore and efficiently compare tables, thus enabling the visualization of table differences in images.

[0060] Figure 3 This is a flowchart of an image labeling method according to an embodiment of the present invention. The embodiments of the present invention supplement the above embodiments with improvements to the method of acquiring the corrected image. It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the relevant descriptions in other embodiments. For example... Figure 3 As shown, the method includes: S210. Obtain the image to be processed.

[0061] S220. Preprocess the image to be processed and the standard image respectively to obtain a pixel set; the pixel set includes the pixel set to be processed of the image to be processed and the standard pixel set of the standard image.

[0062] The preprocessing includes grayscale conversion, adaptive binarization, and global binarization of the image; the pixel set is the set of pixels in the image.

[0063] Specifically, grayscale, adaptive binarization, and global binarization are performed on the image to be processed and the standard image to obtain at least one pixel. The at least one pixel in the image to be processed is taken as the set of pixels to be processed; and the at least one pixel in the standard image is taken as the set of standard pixels.

[0064] S230. Traverse the pixel set to determine the set of border coordinate points; the set of border coordinate points includes the set of border coordinate points of the image to be processed and the set of border coordinate points of the standard image.

[0065] The set of border coordinate points is the set of border coordinate points located at the top left, top right, bottom right, and bottom left corners of the table in the image.

[0066] Specifically, the set of pixels to be processed in the image to be processed is traversed row by row or column by column, and the pixels with gradients greater than a threshold in the top border, bottom border, left border, and right border are used as the set of border coordinate points of the image to be processed; the set of standard pixels in the standard image is traversed row by row or column by column, and the pixels with gradients greater than a threshold in the top border, bottom border, left border, and right border are used as the set of border coordinate points of the standard image.

[0067] Optionally, the pixel set is traversed separately to determine the set of bounding box coordinate points, including: Traverse the pixel set from top to bottom and calculate the first vertical gradient based on the pixel coordinates; Use at least one pixel whose first vertical gradient is greater than the gradient threshold as the set of coordinate points for the top border. Traverse the pixel set from bottom to top and calculate the second vertical gradient based on the pixel coordinates; Use at least one pixel whose second vertical gradient is greater than the gradient threshold as the set of coordinate points for the bottom border. Traverse the set of pixels from left to right and calculate the first horizontal gradient based on the coordinates of the pixels; Use at least one pixel whose first horizontal gradient is greater than the gradient threshold as the set of coordinate points for the left border. Traverse the set of pixels from right to left and calculate the second horizontal gradient based on the coordinates of the pixels; Use at least one pixel with a second horizontal gradient greater than the gradient threshold as the set of coordinate points of the right frame; The coordinate point set of the top border, the coordinate point set of the bottom border, the coordinate point set of the left border, and the coordinate point set of the right border are used as the border coordinate point set.

[0068] Wherein, the first vertical gradient and the second vertical gradient are the gradient values ​​corresponding to each column of pixels; the first horizontal gradient and the second horizontal gradient are the gradient values ​​corresponding to each row of pixels; the gradient threshold is a preset gradient threshold, which can be set by relevant technicians according to actual needs; the set of upper border coordinate points is the upper border coordinate point of the table in the image to be processed and the standard image; the set of lower border coordinate points is the lower border coordinate point of the table in the image to be processed and the standard image; the set of left border coordinate points is the left border coordinate point of the table in the image to be processed and the standard image; and the set of right border coordinate points is the right border coordinate point of the table in the image to be processed and the standard image.

[0069] Specifically, the process iterates through the pixel set from top to bottom, calculating the first vertical gradient corresponding to each pixel using symmetric difference based on its coordinates. At least one pixel whose first vertical gradient is greater than a gradient threshold is selected as the top border coordinate set. The process then iterates through the pixel set from bottom to top, calculating the second vertical gradient corresponding to each pixel using symmetric difference based on its coordinates. At least one pixel whose second vertical gradient is greater than a gradient threshold is selected as the bottom border coordinate set. The process then iterates through the pixel set from left to right, calculating the first horizontal gradient corresponding to each pixel using symmetric difference based on its coordinates. At least one pixel whose first horizontal gradient is greater than a gradient threshold is selected as the left border coordinate set. The process then iterates through the pixel set from right to left, calculating the second horizontal gradient corresponding to each pixel using symmetric difference based on its coordinates. At least one pixel whose second horizontal gradient is greater than a gradient threshold is selected as the right border coordinate set. Finally, the top border coordinate set, bottom border coordinate set, left border coordinate set, and right border coordinate set are combined to form the border coordinate set.

[0070] For example, for any pixel (x, y), its vertical gradient is calculated as follows: in, This represents the vertical gradient.

[0071] The horizontal gradient is calculated as follows: in, For horizontal gradient.

[0072] The gradient threshold T can be set to 100. Pixels in the image are scanned from top to bottom column by column, and the first vertical gradient corresponding to each pixel is calculated. The first vertical gradient corresponding to the first pixel. When the value is greater than T, record the coordinates of that pixel as the set of coordinates of the top border; scan the pixels in the image column by column from bottom to top, and calculate the second vertical gradient corresponding to each pixel. The second vertical gradient corresponding to the first pixel appears. When the value is greater than T, record the coordinates of that pixel as the set of coordinate points of the lower border; scan the pixels in the image from left to right row by row, and calculate the first horizontal gradient corresponding to each pixel. The first horizontal gradient corresponding to the first pixel that appears. When the time is greater than (T+100), record the coordinates of the pixel as the set of coordinates of the left border; scan the pixels in the image row by row from right to left, and calculate the second horizontal gradient corresponding to each pixel. The second horizontal gradient corresponding to the first pixel is... When the value is greater than (T+100), the coordinates of the pixel are recorded as the set of coordinate points of the right frame. The set of coordinate points of the top frame, bottom frame, left frame, and right frame of the image to be processed is used as the set of coordinate points of the frame of the image to be processed. The set of coordinate points of the top frame, bottom frame, left frame, and right frame of the standard image is used as the set of coordinate points of the frame of the standard image.

[0073] Understandably, symmetrical difference calculations are used to calculate the gradients in the horizontal and vertical directions, and pixel scanning is performed row by row or column by column based on the gradient threshold. The process stops when the first point that meets the conditions is found in each row or column, and this point is used as the coordinate point of the border. This can effectively detect the upward abrupt change from a black table to a white background. Furthermore, the gradient threshold in the horizontal direction corresponding to the left and right borders is increased by 100 margin to enhance detection robustness, ensuring that the coordinates of the four points of the outer border of the table (top left, top right, bottom right, and bottom left) can be accurately identified as the reference basis for perspective transformation.

[0074] S240. Perform fitting operations on the set of border coordinate points respectively to obtain at least four border intersection coordinates; the border intersection coordinates include the border intersection coordinates of the image to be processed and the border intersection coordinates of the standard image.

[0075] The fitting operation can be performed by using the RANSAC algorithm to fit the border lines; the coordinates of the border intersection points are the coordinates of the four corner points of the table in the image.

[0076] Specifically, the RANSAC algorithm is used to fit the set of border coordinate points to obtain four border lines. The four corner points of the table in the image are determined based on the intersection of adjacent border lines, and these corner points are used as the coordinates of the border intersection points of the image.

[0077] Optionally, a fitting operation is performed on the set of border coordinate points to obtain the coordinates of at least four border intersection points, including: By performing a fitting operation on the set of bounding coordinate points, the candidate line parameters corresponding to the set of bounding coordinate points are obtained; The coordinate distance value is determined based on the coordinate points in the set of bounding coordinate points and the parameters of the candidate line. The candidate line parameters are iteratively updated based on the coordinate distance value and the distance threshold to determine the target line parameters; Based on the target line parameters, obtain the border lines corresponding to the set of border coordinate points; The coordinates of the intersection point of the borders are determined based on the straight lines of the adjacent borders.

[0078] Among them, the candidate line parameter is obtained by calculating the line parameter of two random candidate coordinate points in the set of border coordinate points; the coordinate distance value is the distance between the coordinate point in each set of border coordinate points and the candidate line parameter corresponding to the set of border coordinate points; the target line parameter is the line parameter corresponding to the final border line corresponding to each set of border coordinate points; the border line is the line determined based on the target line parameter.

[0079] Specifically, the process involves fitting the coordinates of the bounding box coordinate points to obtain candidate line parameters; determining the distance from each coordinate point in the bounding box coordinate point set to the line based on the candidate line parameters; iteratively updating the candidate line parameters based on the distance values ​​and a distance threshold to determine the target line parameters; obtaining the bounding box lines corresponding to each bounding box coordinate point set based on the target line parameters; and, after obtaining the line equations for the four bounding boxes, determining the four corner points of the table by calculating the intersection points of adjacent lines, which serve as the coordinates of the bounding box intersection points for the image. For example, for any set of bounding box coordinate points in an image, two different coordinate points P1(x1,y1) and P2(x2,y2) are randomly selected from each set of bounding box coordinate points. Candidate line parameters corresponding to these bounding box coordinate points are obtained through line parameter calculation and parameter normalization. The calculation method for the candidate line parameters is as follows: Where a, b, and c are candidate line parameters.

[0080] After obtaining the line parameters, for any coordinate point P(x,y) in the set of coordinate points of the border, calculate the coordinate distance from the point to the line corresponding to the line parameter. The calculation formula is as follows: dist=|a*x+b*y+c| Where dist is the coordinate distance value.

[0081] The distance value of `dist` is compared with a distance threshold. If the distance value is less than the distance threshold, the coordinate point corresponding to the distance value is marked as an interior point. After all coordinate points in the border coordinate points are counted, the number of interior points of the candidate line parameter is counted. If it is greater than the historical maximum number of interior points, the candidate line parameter is used as the target line parameter. Then, the line equation corresponding to the border coordinate point set is found based on the target line parameter. By solving the line equations of adjacent borders to construct a system of two linear equations, the coordinates of the intersection points of adjacent borders are obtained, and then the coordinates of the four border intersection points corresponding to the image border coordinate point set are determined.

[0082] Understandably, by fitting the set of border coordinate points of each image, the coordinates of the intersection points of the borders of each image are obtained, and the outer frame of the table in the image is accurately located, ensuring that the accuracy of table shape restoration reaches more than 99.5%; this further improves the efficiency and accuracy of subsequent table correction.

[0083] S250. Based on the coordinates of the intersection points of the borders of the standard image and the coordinates of the intersection points of the borders of the image to be processed, the image to be processed is corrected to obtain the corrected image.

[0084] The corrected image is the image after perspective correction of the table in the image.

[0085] Specifically, the table outline in the image is located by using the coordinates of the intersection points of the borders of the standard image and the image to be processed. A transformation matrix is ​​then determined based on the coordinates of the intersection points of the borders of the standard image and the image to be processed. The table in the image to be processed is then corrected based on the transformation matrix to obtain a corrected image.

[0086] Optionally, the image to be processed is corrected based on the coordinates of the intersection points of the borders of the standard image and the coordinates of the intersection points of the borders of the image to be processed, to obtain a corrected image, including: The transformation matrix is ​​determined based on the coordinates of the intersection points of the borders in the standard image and the coordinates of the intersection points of the borders in the image to be processed. The pixel coordinates in the image to be processed are corrected based on the transformation matrix to obtain the corrected image.

[0087] The transformation matrix is ​​determined by comparing the homography matrix, the coordinates of the border intersection points of the standard image, and the coordinates of the border intersection points of the image to be processed.

[0088] Specifically, the transformation matrix between the image to be processed and the standard image is determined by comparing the homography matrix, the coordinates of the intersection points of the borders of the standard image and the coordinates of the intersection points of the borders of the image to be processed; based on this transformation matrix, the coordinates of each pixel in the image to be processed are transformed one by one to obtain the corrected image.

[0089] Understandably, perspective transformation is performed on the image to be processed and the standard image based on the homography matrix. According to the perspective transformation matrix, the tilted and distorted table image is corrected into an orthographic projection image, making the outer border of the table parallel to the image coordinate axis. This eliminates the effects of perspective distortion and shooting tilt, effectively solving the problem that existing Hough transform detection of straight line angles for rotation correction can only handle simple tilt problems and cannot solve table deformation caused by perspective distortion. After correction, the table still has problems such as border stretching and irregular cell shape. By combining outer frame positioning and perspective transformation, shooting tilt and perspective distortion are completely eliminated. After correction, the vertical and horizontal errors of the outer border of the table are less than 0.1°, and the accuracy of table shape restoration reaches more than 99.5%.

[0090] S260. Extract tables from the corrected image and the standard image respectively to obtain the corrected cell and the standard cell.

[0091] S270. Compare the correction cells and the standard cells to obtain the difference cells.

[0092] S280. Map and label the image to be processed based on the difference cells to obtain the labeled image.

[0093] This invention uses the set of border points of the image to be processed and the standard image, and accurately locates the coordinates of the outer frame corner points based on fitting operations. By combining the coordinates of the outer frame corner points with perspective transformation, it completely eliminates shooting tilt and perspective distortion, thereby improving the accuracy of subsequent fully automated table extraction.

[0094] Figure 4 This is a flowchart of an image marking method according to an embodiment of the present invention. The embodiments of the present invention supplement the cell extraction method based on the above embodiments. It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the relevant descriptions in other embodiments. For example... Figure 4 As shown, the method includes: S310. Obtain the image to be processed.

[0095] S320. Correct the image to be processed based on the standard image to obtain the corrected image.

[0096] S330. Perform binarization extraction on the corrected image and the standard image respectively to obtain the corrected binarized image and the standard binarized image.

[0097] The binarization extraction employs both adaptive binarization and global binarization algorithms.

[0098] Specifically, adaptive binarization and global binarization algorithms are used to perform binarization extraction on the corrected image to obtain the corrected binarized image; and binarization extraction is performed on the standard image to obtain the standard binarized image.

[0099] S340. Traverse the pixels in the corrected binarized image and the pixels in the standard binarized image row by row or column by column to obtain at least one corrected line corresponding to the corrected binarized image and at least one standard line corresponding to the standard binarized image.

[0100] The traversal can be done row by row or column by column, recording the start and end positions of consecutive white pixels; the correction lines are the corresponding table lines in the corrected binarized image; and the standard lines are the corresponding table lines in the standard binarized image.

[0101] Specifically, the pixels in the corrected binarized image are traversed row by row or column by column to record the start and end positions of consecutive white pixels; the line length is determined based on the start and end positions, and lines with a length less than a length threshold are filtered out to obtain the corresponding table lines in the corrected binarized image as the corrected lines; the pixels in the standard binarized image are traversed row by row or column by column to record the start and end positions of consecutive white pixels; the line length is determined based on the start and end positions, and lines with a length less than a length threshold are filtered out to obtain the corresponding table image in the standard binarized image as the standard lines.

[0102] In an optional embodiment of the present invention, a contour filtering algorithm can be used to filter out excessively short lines in the binarized table image to obtain purified lines; and for an image containing at least two tables, an extraction region can be set, with the table to be marked and detected as the main table, and a mask filtering algorithm can be used to shield redundant secondary table lines in the binarized table image.

[0103] Understandably, by filtering the table lines in the image and integrating morphological processing, mask filtering, and contour filtering techniques, interfering lines are effectively filtered out, and the line extraction purity reaches 99%, with no obvious false lines or breaks.

[0104] S350. Based on the similarity threshold and standard lines, the correction lines in the corrected binarized image are supplemented to obtain the corrected supplemented image.

[0105] The similarity threshold is the threshold corresponding to the line parameter, which is used to merge and remove duplicate lines with similar parameters; the corrected and supplemented image is the image after the lines in the image are supplemented.

[0106] Specifically, a similarity threshold is set, lines with similar line parameters are merged and deduplicated, retaining only the unique and valid table line to avoid duplicate lines affecting cell division, and the correction lines in the correction binarized image are supplemented again based on the standard lines to obtain the correction supplemented image.

[0107] Optionally, the correction lines in the corrected binarized image are supplemented based on a similarity threshold and standard lines to obtain a corrected supplemented image, including: Based on the similarity threshold and the line coordinates corresponding to the correction line, at least one correction line is deduplicated to obtain candidate lines; The correction spacing and standard spacing are calculated separately for the candidate lines and the standard lines. The location of missing lines in the corrected binarized image is determined based on the correction interval and the standard interval; Based on the standard spacing, the missing lines in the corrected binarized image are supplemented to obtain the corrected supplemented image.

[0108] Among them, candidate lines are correction lines after deduplication; correction spacing is the line spacing between adjacent candidate lines; standard spacing is the line spacing between adjacent standard lines; and missing line positions are the positions of lines that should exist but were not detected.

[0109] Specifically, a similarity threshold is set, and correction lines with similar line parameters determined by the line coordinates of each correction line are merged and deduplicated to obtain at least one candidate line. The candidate spacing corresponding to the candidate lines in the correction binarized image and the standard spacing corresponding to the standard lines in the standard binarized image are traversed row by row or column by column according to the line position. The candidate spacing and standard spacing can be compared row by row or column by column according to the line position order. If the candidate spacing is greater than the standard spacing, it indicates that there is enough space between these two lines, and other lines should exist but have not been detected, which is judged as a missing line. Then, according to the standard line direction parameters, each interpolation line is drawn on the correction binarized image: if it is a horizontal line, the line is drawn between the inherited start and end X coordinates and the calculated fixed Y coordinates; if it is a vertical line, the line is drawn between the fixed X coordinates and the inherited start and end Y coordinates. After drawing, the correction supplementary image is obtained.

[0110] In one optional embodiment of the invention, the number of missing lines can be determined based on the candidate spacing and the standard spacing. Further, the number of missing lines can be determined based on the ratio of the candidate spacing to the standard spacing. Starting from the position of the "current line," the positions of each supplementary line are calculated sequentially with the standard spacing as the step size, and line supplementation is performed based on these positions. For example, if the standard spacing is 15 pixels, and there are two adjacent candidate lines A and B in the corrected binarized image, with candidate line A at position 30 and candidate line B at position 75, and their position difference (candidate spacing) is 45 pixels, then the candidate spacing 45 is greater than the standard spacing 15. Further, based on the ratio of the candidate spacing to the standard spacing, the number of missing lines is determined to be 2. Therefore, at position 30 of candidate line A, adding a standard spacing of 15 determines the missing position of the first missing line to be 45. At position 45 of the first missing line, adding another standard spacing of 15 determines the missing position of the second missing line to be 60. Based on these two missing positions, a horizontal line of the same length as candidate line A is created to fill the gap between candidate line A and candidate line B.

[0111] Understandably, by filtering and deduplicating the table image in the image, and using a standard image as a template to supplement the lines in the image to be processed, interference lines are effectively filtered out while avoiding line breaks. This allows for subsequent cell division based on the effective lines in the image, ensuring that the cell coordinate positioning error is less than 1 pixel, and providing a precise positional reference for table content extraction and comparison.

[0112] S360. Perform intersection point extraction on the line set in the standard image and the line set in the corrected supplementary image respectively to obtain the corrected cell corresponding to the corrected binarized image and the standard cell corresponding to the standard binarized image.

[0113] The set of lines in the standard image is the standard line set; the set of lines in the corrected and supplemented image is the supplemented effective line set.

[0114] Specifically, the slope of each horizontal or vertical line in the line set can be calculated, non-horizontal and vertical lines with excessively large slopes can be filtered out, and duplicates can be removed after sorting. The y-coordinate of the midpoint of each valid horizontal line can be taken as the reference value of the line's y-coordinate. The obtained y-coordinate set can be sorted in ascending order, and duplicate coordinates can be removed by judging the distance between adjacent coordinates, resulting in the final sequence of y-coordinates of the horizontal dividing line. The y-coordinate sequence and the x-coordinate sequence are then traversed, and the y-coordinates of adjacent horizontal lines and the x-coordinates of adjacent vertical lines are used as the four vertices of each cell.

[0115] S370. Compare the correction cells and the standard cells to obtain the difference cells; S380. Map and label the image to be processed based on the difference cells to obtain the labeled image.

[0116] This invention traverses the coordinate points of the calibrated and standard images row by row or column by column to obtain at least one calibrated line and at least one standard line. It integrates morphological processing, mask filtering, and contour filtering techniques to effectively filter out interfering lines and avoid line breakage. Based on the line set, it extracts the coordinates of each intersection point in the image to obtain the vertex coordinates of the cell. Based on the deduplicated effective lines, it divides the cells to provide accurate position references. Moreover, the modular design of each step of the above line extraction and traversal algorithm improves efficiency by more than 70% compared with traditional segmented processing methods.

[0117] Figure 5 This invention provides a schematic diagram of an image labeling device according to an embodiment of the present invention. This embodiment is applicable to labeling difference tables in images. The image labeling device can be implemented in hardware and / or software and can be configured in a server. Figure 5 As shown, the image labeling device 400 includes an acquisition module 410, a correction module 420, an extraction module 430, a comparison module 440, and a labeling module 450. The acquisition module 410 is used to acquire the image to be processed; The correction module 420 is used to correct the image to be processed based on a standard image to obtain a corrected image; The extraction module 430 is used to extract tables from the corrected image and the standard image respectively, to obtain the corrected cells and the standard cells; The comparison module 440 is used to compare the correction cell and the standard cell to obtain the difference cell; The labeling module 450 is used to map and label the image to be processed based on the difference cells to obtain a labeled image.

[0118] This invention involves acquiring an image to be processed; correcting the image to be processed based on a standard image to obtain a corrected image; extracting tables from both the corrected image and the standard image to obtain corrected cells and standard cells; comparing the corrected cells and standard cells to obtain difference cells; and mapping and labeling the image to be processed based on the difference cells to obtain a labeled image. This technical solution improves the efficiency of comparing table differences in images by using a multi-fusion algorithm for table correction, extraction, and difference comparison, further solving the problem that existing image labeling methods cannot accurately restore and efficiently compare tables, thus enabling the visualization of table differences in images.

[0119] Optionally, the comparison module 440 is also used to traverse the coordinates of the four vertices of the correction cell and the standard cell respectively to calculate and determine the detection area corresponding to the correction image and the standard area corresponding to the standard image. The corrected image is segmented based on the detection region to obtain at least one corrected patch; The standard image is segmented based on the standard region to obtain at least one standard patch; Pixel percentages are calculated for the calibration patch and the standard patch respectively to obtain the calibration pixel percentage for the calibration patch and the standard pixel percentage for the standard patch; The percentage difference is determined based on the corrected pixel percentage and the standard pixel percentage. The difference in proportions is normalized to obtain the normalized difference. Differential cells are identified from the corrected cells based on normalized differences and labeling thresholds.

[0120] Optionally, the correction module 420 includes a preprocessing unit, a traversal unit, a fitting unit, and a correction unit; The preprocessing unit is used to preprocess the image to be processed and the standard image respectively to obtain a pixel set; the pixel set includes the pixel set to be processed of the image to be processed and the standard pixel set of the standard image. The traversal unit is used to traverse the pixel set separately to determine the bounding box coordinate point set; the bounding box coordinate point set includes the bounding box coordinate point set of the image to be processed and the bounding box coordinate point set of the standard image. The fitting unit is used to perform fitting operations on the set of bounding box coordinate points to obtain at least four bounding box intersection coordinates; the bounding box intersection coordinates include the bounding box intersection coordinates of the image to be processed and the bounding box intersection coordinates of the standard image. The correction unit is used to correct the image to be processed based on the coordinates of the intersection points of the borders of the standard image and the coordinates of the intersection points of the borders of the image to be processed, so as to obtain a corrected image.

[0121] Optionally, the traversal unit is also used to traverse the pixels in the pixel set from top to bottom and calculate the first vertical gradient based on the coordinates of the pixels. Use at least one pixel whose first vertical gradient is greater than the gradient threshold as the set of coordinate points for the top border. Traverse the pixel set from bottom to top and calculate the second vertical gradient based on the pixel coordinates; Use at least one pixel whose second vertical gradient is greater than the gradient threshold as the set of coordinate points for the bottom border. Traverse the set of pixels from left to right and calculate the first horizontal gradient based on the coordinates of the pixels; Use at least one pixel whose first horizontal gradient is greater than the gradient threshold as the set of coordinate points for the left border. Traverse the set of pixels from right to left and calculate the second horizontal gradient based on the coordinates of the pixels; Use at least one pixel with a second horizontal gradient greater than the gradient threshold as the set of coordinate points of the right frame; The coordinate point set of the top border, the coordinate point set of the bottom border, the coordinate point set of the left border, and the coordinate point set of the right border are used as the border coordinate point set.

[0122] Optionally, the fitting unit is also used to perform fitting operations on the set of bounding box coordinate points to obtain the candidate line parameters corresponding to the set of bounding box coordinate points. The coordinate distance value is determined based on the coordinate points in the set of bounding coordinate points and the parameters of the candidate line. The candidate line parameters are iteratively updated based on the coordinate distance value and the distance threshold to determine the target line parameters; Based on the target line parameters, obtain the border lines corresponding to the set of border coordinate points; The coordinates of the intersection point of the borders are determined based on the straight lines of the adjacent borders.

[0123] Optionally, the correction unit is also used to determine the transformation matrix based on the coordinates of the border intersection points of the standard image and the coordinates of the border intersection points of the image to be processed; The pixel coordinates in the image to be processed are corrected based on the transformation matrix to obtain the corrected image.

[0124] Optionally, the extraction module 430 is also used to perform binarization extraction on the corrected image and the standard image respectively to obtain the corrected binarized image and the standard binarized image; The pixels in the corrected binarized image and the pixels in the standard binarized image are traversed row by row or column by column to obtain at least one corrected line corresponding to the corrected binarized image and at least one standard line corresponding to the standard binarized image. The correction lines in the corrected binarized image are supplemented based on the similarity threshold and standard lines to obtain the corrected supplemented image; Intersection points are extracted from the line sets in the standard image and the line sets in the corrected supplementary image to obtain the corrected cells corresponding to the corrected binarized image and the standard cells corresponding to the standard binarized image.

[0125] Optionally, the extraction module 430 is also used to remove duplicates from at least one correction line based on a similarity threshold and the line coordinates corresponding to the correction line to obtain candidate lines; The correction spacing and standard spacing are calculated separately for the candidate lines and the standard lines. The location of missing lines in the corrected binarized image is determined based on the correction interval and the standard interval; Based on the standard spacing, the missing lines in the corrected binarized image are supplemented to obtain the corrected supplemented image.

[0126] The image marking device provided in the embodiments of the present invention can execute the image marking method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0127] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0128] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0129] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image labeling methods.

[0132] In some embodiments, the image tagging method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image tagging method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image tagging method by any other suitable means (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and dedicated virtual services, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image tagging method, characterized in that, include: Obtain the image to be processed; The image to be processed is corrected based on a standard image to obtain a corrected image; Tables are extracted from the corrected image and the standard image respectively to obtain corrected cells and standard cells; The correction cells and the standard cells are compared to obtain the difference cells; Based on the difference cells, the image to be processed is mapped and labeled to obtain a labeled image.

2. The method according to claim 1, characterized in that, The step of comparing the corrected cell and the standard cell to obtain the difference cell includes: The four vertex coordinates of the correction cell and the standard cell are traversed separately to calculate and determine the detection area corresponding to the correction image and the standard area corresponding to the standard image. The corrected image is segmented based on the detection region to obtain at least one corrected image patch; The standard image is segmented based on the standard region to obtain at least one standard patch; The pixel ratios of the correction patch and the standard patch are calculated respectively to obtain the correction pixel ratio corresponding to the correction patch and the standard pixel ratio corresponding to the standard patch; The percentage difference is determined based on the corrected pixel percentage and the standard pixel percentage; The ratio difference is normalized to obtain the normalized difference. The difference cells are determined from the corrected cells based on the normalized difference and the labeling threshold.

3. The method according to claim 1, characterized in that, The process of correcting the image to be processed based on a standard image to obtain the corrected image includes: The image to be processed and the standard image are preprocessed respectively to obtain a pixel set; the pixel set includes the pixel set of the image to be processed and the standard pixel set of the standard image. The pixel set is traversed to determine the border coordinate point set; the border coordinate point set includes the border coordinate point set of the image to be processed and the border coordinate point set of the standard image. The set of border coordinate points is fitted to obtain at least four border intersection coordinates; the border intersection coordinates include the border intersection coordinates of the image to be processed and the border intersection coordinates of the standard image. The image to be processed is corrected based on the coordinates of the intersection points of the borders of the standard image and the intersection points of the borders of the image to be processed, to obtain a corrected image.

4. The method according to claim 3, characterized in that, The step of traversing the pixel set to determine the set of border coordinate points includes: Traverse the pixel set from top to bottom and calculate the first vertical gradient based on the coordinates of the pixel; Use at least one pixel whose first vertical gradient is greater than the gradient threshold as the set of coordinate points of the top border. Traverse the pixel set from bottom to top and calculate the second vertical gradient based on the coordinates of the pixel; At least one pixel with a second vertical gradient greater than the gradient threshold is used as the set of coordinate points of the bottom border. Traverse the pixels in the set of pixels from left to right, and calculate the first horizontal gradient based on the coordinates of the pixels; At least one pixel with the first horizontal gradient greater than the gradient threshold is taken as the set of coordinate points of the left border. Traverse the pixels in the set of pixels from right to left, and calculate the second horizontal gradient based on the coordinates of the pixels; Use at least one pixel whose second horizontal gradient is greater than the gradient threshold as the set of coordinate points of the right frame; The set of coordinate points of the top border, the set of coordinate points of the bottom border, the set of coordinate points of the left border, and the set of coordinate points of the right border are used as the set of border coordinate points.

5. The method according to claim 3, characterized in that, The process of fitting the set of border coordinate points to obtain at least four border intersection coordinates includes: The set of border coordinate points is fitted to obtain the candidate line parameters corresponding to the set of border coordinate points. The coordinate distance value is determined based on the coordinate points in the set of border coordinate points and the candidate line parameters; The candidate line parameters are iteratively updated based on the coordinate distance value and the distance threshold to determine the target line parameters; Based on the target line parameters, obtain the border lines corresponding to the set of border coordinate points; The coordinates of the intersection point of the borders are determined based on the straight lines of the adjacent borders.

6. The method according to claim 3, characterized in that, The process of correcting the image to be processed based on the coordinates of the intersection points of the borders of the standard image and the coordinates of the intersection points of the borders of the image to be processed, to obtain a corrected image, includes: The transformation matrix is ​​determined based on the coordinates of the border intersection points of the standard image and the coordinates of the border intersection points of the image to be processed; The pixel coordinates in the image to be processed are corrected based on the transformation matrix to obtain a corrected image.

7. The method according to claim 1, characterized in that, The step of extracting tables from the corrected image and the standard image respectively to obtain corrected cells and standard cells includes: The corrected image and the standard image are binarized separately to obtain a corrected binarized image and a standard binarized image; The pixels in the corrected binarized image and the pixels in the standard binarized image are traversed row by row or column by column to obtain at least one corrected line corresponding to the corrected binarized image and at least one standard line corresponding to the standard binarized image. The correction lines in the corrected binarized image are supplemented based on the similarity threshold and the standard lines to obtain a corrected supplemented image; The intersection points of the line sets in the standard image and the line sets in the corrected supplementary image are extracted by traversing the intersection points to obtain the corrected cell corresponding to the corrected binarized image and the standard cell corresponding to the standard binarized image.

8. The method according to claim 7, characterized in that, The step of supplementing the correction lines in the corrected binarized image based on a similarity threshold and the standard lines to obtain a corrected supplemented image includes: Based on the similarity threshold and the line coordinates corresponding to the correction line, at least one correction line is deduplicated to obtain candidate lines; The correction spacing and standard spacing are calculated for the candidate lines and the standard lines respectively. The location of missing lines in the corrected binarized image is determined based on the correction interval and the standard interval; Based on the standard spacing, the missing line positions in the corrected binarized image are filled in to obtain a corrected supplemented image.

9. An image marking device, characterized in that, include: The acquisition module is used to acquire the image to be processed; The correction module is used to correct the image to be processed based on a standard image to obtain the corrected image; The extraction module is used to extract tables from the corrected image and the standard image respectively, to obtain corrected cells and standard cells; The comparison module is used to compare the correction cell and the standard cell to obtain the difference cell; The labeling module is used to map and label the image to be processed based on the difference cells to obtain a labeled image.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image tagging method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image tagging method according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the image tagging method according to any one of claims 1-8.

Images