A photographed document image segmentation and annotation system and method combined with unsupervised learning

Abstract

The present invention provides a system and method for the segmentation and annotation of photographed document images combined with unsupervised learning, wherein the method comprises the following steps: extracting image segmentation and annotation data and then performing grayscale conversion of traditional image processing, and then obtaining matrix position coordinate data through unsupervised target segmentation model processing, which is used for unsupervised image annotation to obtain the result contour data of photographed document segmentation. Next, the traditional image processing data and the result contour data are received, and the pixel intersection and union ratio algorithm is used to compare and traverse the pixel points to determine whether they belong to the segmented area. The inaccurately annotated data is transmitted to the data annotation module for re-annotation. Finally, the accurately annotated data is received and stored in the data server. The whole process realizes the functions of image segmentation, annotation, inspection and storage. Through the scheme of the present application, the training data of photographed document image segmentation can be enriched, and the problem of poor effect of deep learning image segmentation method in complex situations can be solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a photographed document image segmentation and annotation system and method combined with unsupervised learning. Background Art

[0002] Photo document recognition is a difficult scenario in document recognition tasks, and there is still no good and complete solution. In the solution of photo document recognition, it is necessary to perform document image segmentation and document image recovery on the photo document. In the technology of photo document image segmentation, it is often used to directly use traditional image processing algorithms or deep learning image segmentation methods to obtain document segmentation images. However, traditional image processing methods can only handle photo document segmentation work under set modes, so in order to enhance the generalization of functions, deep learning image segmentation methods are often chosen. However, deep learning algorithms also have capabilities, and the segmentation ability of the model is related to the training data, but most of the training data needs to be manually labeled or segmented using traditional image processing methods. There is a problem with this method, that is, it cannot label difficult data. At the same time, it also takes a lot of time to perform data labeling and data cleaning. Summary of the invention

[0003] This patent proposes a photographed document image segmentation model training system and method combined with unsupervised learning to solve the problem that the document image segmentation training data is too simple and the manual labeling cost of complex labeling data is too high and difficult. It is conducive to training a more effective photographed document recognition model.

[0004] One embodiment of the present invention provides a method for image segmentation and annotation of photographed documents combined with unsupervised learning, including the following steps: collecting and reading image segmentation annotation data from a data source; after extracting the image segmentation annotation data, performing grayscale conversion through traditional image processing, performing unsupervised image processing on the data after traditional image processing according to an unsupervised target segmentation model, obtaining matrix position coordinate data of unsupervised image segmentation, inputting the matrix position coordinate data of unsupervised image segmentation into the model as preset points, performing unsupervised image annotation on the annotated image, and obtaining the result contour data of photographed document segmentation; receiving the traditional image processing data and the result contour data of photographed document segmentation, comparing the traditional image processing data and the result contour data of photographed document segmentation according to a pixel intersection-over-union algorithm, and traversing each pixel point in the image matrix corresponding to the two data. For each pixel point, determine whether it belongs to the segmentation area in the traditional image processing data and the result contour data of photographed document segmentation, and if it does not belong to the segmentation area, perform annotation inspection processing and re-annotation; receive and store the accurate annotation data after the annotation inspection processing to a data server.

[0005] In one embodiment, the traditional image processing steps include: photographing the image data according to the RGB pixel channel, and converting the image data into a grayscale image; obtaining contour edge data in the grayscale image according to an edge detection algorithm; obtaining a closed contour frame of the contour edge data according to the contour edge data, and selecting the frame with the largest area as the result contour of the photographed document segmentation.

[0006] In one embodiment, the unsupervised image processing step includes: selecting four peripheral points in the contour edge data as matrix position coordinates for unsupervised image segmentation based on the image data after traditional image processing. According to the unsupervised target segmentation model, the segmentation position coordinates are used as prior knowledge to perform unsupervised image annotation on the annotated image to obtain the result contour of the photographed document segmentation.

[0007] In one of the embodiments, the labeling inspection processing step also includes: defining image segmentation pixel points, setting the image segmentation pixel points that have undergone traditional image processing as a first pixel range, and setting the image segmentation pixel points that have undergone unsupervised image processing as a second pixel range; using an algorithm to calculate the pixel intersection and union ratio of the first pixel range and the second pixel range; judging the labeling results, judging whether there are abnormalities in the labeling, and selecting the labeling results between the first pixel range and the second pixel range; obtaining a new image segmentation result, and using an image labeling tool to optimize the labeling of the image segmentation result.

[0008] Furthermore, the unsupervised image processing process is performed as follows: First, obtain the image data after traditional image processing. In the traditional image processing stage, filtering is used to remove noise, grayscale transformation is used to adjust contrast, and other operations are used to improve image clarity and feature recognition, providing a usable data basis for the follow-up. Next, the image contour edge data is mined, and four special points on its periphery are selected with the help of a special algorithm. After complex calculations and multi-dimensional comparisons, these four points are determined as the matrix position coordinates of the unsupervised image segmentation, and the key areas for subsequent processing are delineated.

[0009] After the introduction of the unsupervised object segmentation model, the model can automatically learn the deep features and hidden patterns of the image. The selected segmentation position coordinates are input into the model as prior knowledge, so that it can quickly anchor the key focus areas in the annotated image. Based on the coordinate clues and the knowledge base of the characteristic rules learned by itself, the model performs unsupervised image annotation operations on the annotated image, continuously accurately classifies and distinguishes each pixel and subtle area in the image, and determines the category to which it belongs. Finally, after a series of internal calculations and processing processes, the result outline of the photographed document segmentation is obtained, which provides a basis for subsequent text extraction, content analysis and other operations.

[0010] Furthermore, the labeling inspection processing steps include: first defining the image segmentation pixel points, and defining the image segmentation pixel points after traditional image processing as the first pixel range, which carries the pixel feature set processed by traditional means and reflects the general appearance of the image structure; the image segmentation pixel points after unsupervised image processing are set as the second pixel range, and the deep feature pixel information mined by the unsupervised model is integrated. Then, the pixel intersection and union ratio of the first pixel range and the second pixel range is calculated using the designed algorithm, which is used as the key quantitative basis. Based on this, the deep judgment process of the labeling results is started to identify abnormal situations such as labeling omissions, misclassification, and blurred boundaries. Reasonable and accurate labeling results are carefully selected from the information covered by the two pixel ranges, and the complementary advantages of the two are fully utilized. Finally, the new image segmentation results are used to call professional image annotation tools to perform refined annotation optimization, eliminate residual defects, ensure high-quality output of image processing, and provide adapted image data for terminal applications.

[0011] In one of the embodiments, it also includes a model training step: receiving the training image data in the traditional image processing data and the unsupervised image processing data; performing image enhancement operations on the read training data; training the model according to the image segmentation model, inputting the training image data after image enhancement processing into the selected image segmentation model, and adjusting the parameters of the model.

[0012] In one embodiment, the unsupervised object segmentation model may be selected as the SAM open source unsupervised image segmentation model.

[0013] The present invention also provides an embodiment, including a photographed document image segmentation and annotation system combined with unsupervised learning, which is applied to the above-mentioned photographed document image segmentation and annotation method combined with unsupervised learning, and includes the following modules: a data reading module, used to collect and read image segmentation and annotation data from a data source; a data annotation module, used to extract the image segmentation and annotation data, perform grayscale conversion through traditional image processing, perform unsupervised image processing on the data after traditional image processing according to an unsupervised target segmentation model, obtain matrix position coordinate data of unsupervised image segmentation, input the matrix position coordinate data of unsupervised image segmentation into the model as preset points, and perform unsupervised image segmentation on the annotated image. The method comprises the following steps: labeling, obtaining the result contour data of the photographed document segmentation; a labeling inspection module, which is used to receive the traditional image processing data and the result contour data of the photographed document segmentation, compare the traditional image processing data and the result contour data of the photographed document segmentation according to the pixel intersection-over-union algorithm, and traverse each pixel point in the image matrix corresponding to the two types of data; for each pixel point, determining whether it belongs to the segmented area in the traditional image processing data and the result contour data of the photographed document segmentation, and if it does not belong to the segmented area, transmitting it to the data labeling module for labeling inspection and re-labeling; a data storage module, receiving and storing the accurately labeled data after inspection in the labeling inspection module to the data server.

[0014] In one implementation, the data annotation module further includes: photographing the image data according to the RGB pixel channel, converting the image data into a grayscale image; obtaining the contour edge data in the grayscale image according to the edge detection algorithm; obtaining the closed contour frame of the contour edge data according to the contour edge data, and selecting the frame with the largest area as the result contour of the photographed document segmentation; selecting four points on the periphery of the contour edge data according to the image data after traditional image processing as the matrix position coordinates of the unsupervised image segmentation. According to the unsupervised target segmentation model, the segmentation position coordinates are used as prior knowledge to perform unsupervised image annotation on the annotated image to obtain the result contour of the photographed document segmentation.

[0015] In one of the embodiments, the annotation inspection module also includes: defining image segmentation pixel points, setting the image segmentation pixel points after traditional image processing as a first pixel range, and setting the image segmentation pixel points after unsupervised image processing as a second pixel range; using an algorithm to calculate the pixel intersection and union ratio of the first pixel range and the second pixel range; judging the annotation results, judging whether there are abnormalities in the annotation, and selecting the annotation results between the first pixel range and the second pixel range; obtaining a new image segmentation result, and using an image annotation tool to optimize the annotation of the image segmentation result.

[0016] In one embodiment, a model training module is included, which is used to receive the training image data in the traditional image processing data and the unsupervised image processing data; perform image enhancement operations on the read training data; and perform training operations on the model according to the image segmentation model, input the training image data after image enhancement processing into the selected image segmentation model, and adjust the parameters of the model.

[0017] The above embodiments provide a photographed document image segmentation and annotation system and method combined with unsupervised learning, which has the following beneficial effects:

[0018] The present invention can realize fast and accurate data annotation, greatly saving manual annotation work. By using unsupervised learning algorithms, the system can automatically analyze image features and quickly determine the segmentation annotation of document images without manual annotation operations one by one. This not only improves the annotation efficiency, but also reduces the annotation errors caused by human factors, providing a high-quality data basis for subsequent data analysis and model training. It provides the possibility of processing more complex data annotation. When faced with complex document image scenes, traditional annotation methods may encounter difficulties, while unsupervised learning can automatically mine the potential patterns and structures in the image to achieve more accurate segmentation annotation. This can effectively improve the document image segmentation ability and effect of the training model, so that it can better adapt to various complex document image situations. Whether it is a document with different fonts, sizes, and layouts, or an image with interference factors, the system can effectively and accurately annotate, bringing new breakthroughs and development opportunities to the field of document image analysis and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0020] Figure 1 A working step diagram of a photographed document image segmentation and annotation method combined with unsupervised learning provided in the first embodiment of the present invention;

[0021] Figure 2 The data annotation original image provided by the first embodiment of the present invention using a photographed document image segmentation and annotation method combined with unsupervised learning;

[0022] Figure 3 A conventional image processing image segmentation result diagram using a photographed document image segmentation and annotation method combined with unsupervised learning provided in the first embodiment of the present invention;

[0023] Figure 4 An unsupervised image segmentation result diagram of a photographed document image segmentation and annotation method combined with unsupervised learning provided in the first embodiment of the present invention;

[0024] Figure 5 A module diagram of a photographed document image segmentation and annotation system using unsupervised learning provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indications will also change accordingly.

[0027] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited to "first" and "second" may explicitly or implicitly include at least one of the features. In addition, if "and / or" or "and / or" appears in the full text, its meaning includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or solutions that satisfy both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0028] The current methods for photographed document image segmentation have the problem of low intelligence. For complex images, it is difficult to find a suitable global threshold. They are sensitive to noise. If the image is interfered by noise, the threshold selection may be inaccurate, thus affecting the segmentation effect. Alternatively, the computational complexity is high, requiring a large number of matrix operations and optimization algorithms, and the processing time is long. They are also sensitive to noise and outliers, which may affect the construction of the graph and the segmentation results.

[0029] The present invention provides a photographed document image segmentation model training system and method combined with unsupervised learning, which solves the problem that the document image segmentation training data is too single and the labor cost of labeling complex labeling data is too high and difficult. It is conducive to training a more effective photographed document recognition model.

[0030] Example 1

[0031] refer to Figure 1-Figure 4 One embodiment of the present invention provides a method for image segmentation and annotation of photographed documents combined with unsupervised learning, comprising the following steps: S100: collecting and reading image segmentation annotation data from a data source; S200: after extracting the image segmentation annotation data, grayscale conversion is performed through traditional image processing, unsupervised image processing is performed on the data after traditional image processing according to an unsupervised target segmentation model, matrix position coordinate data of unsupervised image segmentation is obtained, the segmentation position coordinate data is input into the model as a preset point, unsupervised image annotation is performed on the annotated image, and the result contour data of the photographed document segmentation is obtained; S300: receiving the traditional image processing data and the result contour data of the photographed document segmentation, comparing the traditional image processing data and the result contour data of the photographed document segmentation according to a pixel intersection-over-union algorithm, and traversing each pixel point in the image matrix corresponding to the two data. For each pixel point, it is determined whether it belongs to the segmentation area in the traditional image processing data and the result contour data of the photographed document segmentation, and if it does not belong to the segmentation area, the annotation inspection process is performed and re-annotated; S400: receiving and storing the accurate data of the annotation inspection module after inspection to the data server.

[0032] In one embodiment, the traditional image processing steps include: photographing the image data according to the RGB pixel channel, and converting the image data into a grayscale image; obtaining contour edge data in the grayscale image according to an edge detection algorithm; obtaining a closed contour frame of the contour edge data according to the contour edge data, and selecting the frame with the largest area as the result contour of the photographed document segmentation.

[0033] The specific steps are as follows: Use the RGB pixel channel to capture the image data and convert it into a grayscale image according to the following formula, where R, G, and B respectively take the pixel data of the red, green, and blue channels of the image. The formula used is: gray = 0.299 × R + 0.587 × G + 0.114 × B. Image edge detection algorithm, using edge detection algorithms such as Canny, to obtain the contour edge data threshold in the gray image. Use the findContours function of opencv to obtain all closed contour boxes, and select the box with the largest area as the result contour of the photographed document segmentation.

[0034] In one embodiment, the unsupervised image processing step includes: selecting four peripheral points in the contour edge data as matrix position coordinates for unsupervised image segmentation based on the image data after traditional image processing. According to the unsupervised target segmentation model, the segmentation position coordinates are used as prior knowledge to perform unsupervised image annotation on the annotated image to obtain the result contour of the photographed document segmentation.

[0035] The specific steps are to use the traditional image processing results, select four points on the periphery of the contour edge data threshold as the matrix position coordinates of unsupervised image segmentation, use the SAM unsupervised target segmentation model, use the segmentation position coordinates as prior knowledge, perform unsupervised image annotation on the annotated image, and obtain the result contour of the photographed document segmentation.

[0036] In one of the embodiments, the labeling inspection processing step also includes: defining image segmentation pixel points, setting the image segmentation pixel points that have undergone traditional image processing as a first pixel range, and setting the image segmentation pixel points that have undergone unsupervised image processing as a second pixel range; using an algorithm to calculate the pixel intersection and union ratio of the first pixel range and the second pixel range; judging the labeling results, judging whether there are abnormalities in the labeling, and selecting the labeling results between the first pixel range and the second pixel range; obtaining a new image segmentation result, and using an image labeling tool to optimize the labeling of the image segmentation result.

[0037] In one embodiment, the image annotation step also includes a model training step: receiving the training image data in the traditional image processing data and the unsupervised image processing data; performing image enhancement operation on the read training data; training the model according to the image segmentation model, inputting the training image data after image enhancement processing into the selected image segmentation model, and adjusting the parameters of the model.

[0038] In one of the embodiments, the unsupervised target segmentation model in the unsupervised image processing step may be a sam open source unsupervised image segmentation model;

[0039] Sam's open source unsupervised image segmentation model is able to generate masks for any object in any image or video, even objects and image types that have not been encountered during training. It breaks the traditional model's reliance on specific tasks and has strong versatility and generalization capabilities, allowing users to prompt the model for segmentation in a variety of ways, such as single-click, interactive click, or using bounding box prompts. And when faced with ambiguity about the object being segmented, multiple valid masks can be output, which is very important for solving complex segmentation problems in the real world. It can automatically find and mask all objects in the image, greatly improving the efficiency of image segmentation. After pre-calculating the image embedding, real-time segmentation masks can be generated for any prompt, enabling real-time interaction with the model.

[0040] Example 2

[0041] refer to Figure 5 The present invention also provides an embodiment, including a photographed document image segmentation and annotation system combined with unsupervised learning, which is applied to the above-mentioned photographed document image segmentation and annotation method combined with unsupervised learning, and includes the following modules: a data reading module 1, used to collect and read image segmentation and annotation data from a data source; a data annotation module 2, used to extract the image segmentation and annotation data, perform grayscale conversion through traditional image processing, perform unsupervised image processing on the data after traditional image processing according to an unsupervised target segmentation model, obtain matrix position coordinate data of unsupervised image segmentation, input the segmentation position coordinate data into the model as a preset point, perform unsupervised image annotation on the annotated image, and obtain the result contour data of the photographed document segmentation; an annotation checking module 3, used to receive the traditional image processing data and the result contour data of the photographed document segmentation, compare the traditional image processing data and the result contour data of the photographed document segmentation according to a pixel intersection-over-union algorithm, and traverse each pixel point in the image matrix corresponding to the two types of data. For each pixel point, determine whether it belongs to the segmented area in the traditional image processing data and the contour data of the photographed document segmentation result. If it does not belong to the segmented area, it is transmitted to the data annotation module for annotation inspection and re-annotation; the data storage module 4 receives and stores the accurately annotated data after inspection in the annotation inspection module to the data server.

[0042] In one implementation, the data annotation module further includes: photographing the image data according to the RGB pixel channel, converting the image data into a grayscale image; obtaining the contour edge data in the grayscale image according to the edge detection algorithm; obtaining the closed contour frame of the contour edge data according to the contour edge data, and selecting the frame with the largest area as the result contour of the photographed document segmentation; selecting four points on the periphery of the contour edge data according to the image data after traditional image processing as the matrix position coordinates of the unsupervised image segmentation. According to the unsupervised target segmentation model, the segmentation position coordinates are used as prior knowledge to perform unsupervised image annotation on the annotated image to obtain the result contour of the photographed document segmentation.

[0043] In one of the embodiments, the annotation inspection module also includes: defining image segmentation pixel points, setting the image segmentation pixel points after traditional image processing as a first pixel range, and setting the image segmentation pixel points after unsupervised image processing as a second pixel range; using an algorithm to calculate the pixel intersection and union ratio of the first pixel range and the second pixel range; judging the annotation results, judging whether there are abnormalities in the annotation, and selecting the annotation results between the first pixel range and the second pixel range; obtaining a new image segmentation result, and using an image annotation tool to optimize the annotation of the image segmentation result.

[0044] In one embodiment, a model training module is included, which is used to receive the training image data in the traditional image processing data and the unsupervised image processing data; perform image enhancement operations on the read training data; and perform training operations on the model according to the image segmentation model, input the training image data after image enhancement processing into the selected image segmentation model, and adjust the parameters of the model.

[0045] In some embodiments, a histogram equalization technique is generally used to adjust the histogram distribution of an image so that the grayscale of the image is more evenly distributed in the entire grayscale range, thereby improving the contrast of the image. Specifically, the histogram of the image may be calculated first, and then the grayscale values ​​may be remapped according to the histogram to make the dark area darker and the bright area brighter. In addition, adaptive histogram equalization may be used to perform histogram equalization on different local areas of the image to better adapt to local contrast changes of the image.

[0046] Before model training, the training data needs to be preprocessed. This includes image normalization, resizing, and other operations. Normalization can adjust the pixel value range of an image to a specific interval, such as [0, 1] or [-1, 1], to improve the training effect and stability of the model. Resizing can adjust the image to the input size required by the model to avoid calculation problems caused by inconsistent sizes.

[0047] During model training, image enhancement techniques can be combined to improve the generalization ability of the model. For example, image contrast enhancement, affine distortion, random inversion and other operations can be randomly applied in each training batch so that the model can learn image data with different changes. In addition, data enhancement methods such as random cropping and flipping can be used to further increase the diversity of training data.

[0048] During the model training process, the model needs to be evaluated regularly to understand the model's performance and improvement direction. You can use the validation set or cross-validation method to evaluate the model's performance and calculate indicators such as accuracy, recall, intersection-over-union, etc. Based on the evaluation results, you can optimize the model, such as adjusting hyperparameters, increasing the number of training rounds, and improving the model architecture.

[0049] The above embodiments provide a photographed document image segmentation and annotation system and method combined with unsupervised learning, which has the following beneficial effects:

[0050] In today's big data era, data labeling often consumes a lot of manpower and time. The system can achieve fast and accurate data labeling, greatly saving manual labeling work. By using unsupervised learning algorithms, the system can automatically analyze image features and quickly determine the segmentation and labeling of document images without manual labeling operations one by one. This not only improves labeling efficiency, but also reduces labeling errors caused by human factors, providing a high-quality data foundation for subsequent data analysis and model training. It provides the possibility of processing more complex data labeling. When faced with complex document image scenes, traditional labeling methods may encounter difficulties, while unsupervised learning can automatically mine the potential patterns and structures in the image to achieve more accurate segmentation and labeling. This can effectively improve the document image segmentation ability and effect of the training model, enabling it to better adapt to various complex document image situations. Whether it is documents with different fonts, sizes, layouts, or images with interference factors, the system can effectively and accurately label, bringing new breakthroughs and development opportunities to the field of document image analysis and processing.

[0051] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for image segmentation and annotation of photographed documents combined with unsupervised learning, characterized in that: The following steps are involved: Collect and read image segmentation annotation data from the data source; After extracting the image segmentation and annotation data, grayscale conversion is performed through traditional image processing to obtain traditional image processing data, unsupervised image processing is performed on the traditional image processing data according to the unsupervised target segmentation model to obtain matrix position coordinate data of unsupervised image segmentation, the matrix position coordinate data of unsupervised image segmentation is input into the model as preset points, unsupervised image annotation is performed on the annotated image, and contour data of the result of photographed document segmentation is obtained; wherein the traditional image processing step includes: photographing the image data according to the RGB pixel channel, converting the image data using the formula gray=0.299×R+0.587×G+0.114×B into a grayscale image; according to an edge detection algorithm, obtaining contour edge data in the grayscale image; according to the contour edge data, using the findContours function of opencv to obtain a closed contour frame of the contour edge data, and selecting the frame with the largest area as the result contour of the photographed document segmentation; the unsupervised image processing step includes: according to the image data after traditional image processing, selecting four points on the periphery of the contour edge data as the matrix position coordinates of the unsupervised image segmentation; according to the unsupervised target segmentation model, using the segmentation position coordinates as prior knowledge, performing unsupervised image annotation on the annotated image to obtain the result contour of the photographed document segmentation; According to the pixel intersection-and-combination ratio algorithm, the traditional image processing data and the result contour data of the photographed document segmentation are compared, and each pixel point in the image matrix corresponding to the two types of data is traversed. For each pixel point, it is determined whether it belongs to the segmented area in the traditional image processing data and the result contour data of the photographed document segmentation. If it does not belong to the segmented area, a labeling check process is performed and re-labeled; the labeling check process step also includes: defining image segmentation pixel points, setting the image segmentation pixel points processed by the traditional image processing as the first pixel range, and setting the image segmentation pixel points processed by the unsupervised image processing as the second pixel range; using an algorithm to calculate the pixel intersection-and-combination ratio of the first pixel range and the second pixel range; judging the labeling result, judging whether there is an abnormality in the labeling, and selecting the labeling result in the first pixel range and the second pixel range; taking a new image segmentation result, and using an image labeling tool to optimize the labeling of the image segmentation result; Receive and store the accurate data after labeling inspection to the data server.

2. The method for image segmentation and annotation of photographed documents combined with unsupervised learning as claimed in claim 1, characterized in that: Also includes the model training steps: Receiving the traditional image processing data and the training image data in the unsupervised image processing data; Perform image enhancement operations on the read training data; According to the image segmentation model, the model is trained, the training image data after image enhancement processing is input into the selected image segmentation model, and the parameters of the model are adjusted.

3. The method for photographed document image segmentation and annotation combined with unsupervised learning as claimed in claim 1, characterized in that: The unsupervised object segmentation model is the SAM open source unsupervised image segmentation model.

4. A photographed document image segmentation and annotation system combined with unsupervised learning, applied to a photographed document image segmentation and annotation method combined with unsupervised learning, characterized in that: Includes the following modules: A data reading module is used to collect and read image segmentation and annotation data from a data source; The data annotation module is used to convert the extracted image segmentation annotation data into grayscale through traditional image processing, perform unsupervised image processing on the data after traditional image processing according to the unsupervised target segmentation model, obtain the matrix position coordinate data of the unsupervised image segmentation, input the matrix position coordinate data of the unsupervised image segmentation into the model as preset points, perform unsupervised image annotation on the annotated image, and obtain the result contour data of the photographed document segmentation; wherein, the traditional image processing step includes: photographing the image data according to the RGB pixel channel, and converting the image data into a grayscale image using the formula gray=0.299×R+0.587×G+0.114×B; obtaining the contour edge data in the grayscale image according to the edge detection algorithm; according to the contour edge data, using the findContours function of opencv to obtain the closed contour frame of the contour edge data, and selecting the frame with the largest area as the result contour of the photographed document segmentation; the unsupervised image processing step includes: according to the image data after traditional image processing, selecting four outer points in the contour edge data as the matrix position coordinates of the unsupervised image segmentation; according to the unsupervised target segmentation A segmentation model is provided, and the segmentation position coordinates are used as prior knowledge to perform unsupervised image annotation on the annotated image to obtain the result contour of the photographed document segmentation; a labeling inspection module is used to receive the result contour data of the traditional image processing data and the photographed document segmentation, and compare the result contour data of the traditional image processing data and the photographed document segmentation according to the pixel intersection and union ratio algorithm, and traverse each pixel point in the image matrix corresponding to the two types of data. For each pixel point, it is determined whether it belongs to the segmentation area in the traditional image processing data and the result contour data of the photographed document segmentation. If it does not belong to the segmentation area, it is transmitted to the data labeling module for labeling inspection processing and re-labeling; the labeling inspection processing step also includes: defining image segmentation pixels, setting the image segmentation pixels processed by traditional image processing as the first pixel range, and setting the image segmentation pixels processed by unsupervised image processing as the second pixel range; using an algorithm to calculate the pixel intersection and union ratio of the first pixel range and the second pixel range; judging the labeling result, judging whether there is an abnormality in the labeling, and selecting the labeling result in the first pixel range and the second pixel range; taking a new image segmentation result, and using an image labeling tool to optimize the labeling of the image segmentation result; The data storage module is used to receive and store the accurate data of the annotations after the annotation inspection to the data server.

5. The photographed document image segmentation and annotation system combined with unsupervised learning as claimed in claim 4, characterized in that: It also includes a model training module, which is used to receive the training image data in the traditional image processing data and the unsupervised image processing data; perform image enhancement operations on the read training data; and perform training operations on the model according to the image segmentation model, input the training image data after image enhancement processing into the selected image segmentation model, and adjust the parameters of the model.

Images