Content extraction method, device and equipment, and storage medium

Abstract

The application discloses a writing content extraction method, device and equipment and a storage medium. The method comprises the following steps: acquiring an image of a writing medium; performing a grayscale processing on the image of the writing medium to obtain a writing content image; using a statistical method to perform an adaptive threshold binaryzation on the writing content image to obtain a mask image of background and writing content binaryzation; extracting writing content in the image of the writing medium according to the mask image and outputting a result image containing only the writing content; and obtaining a binaryzation threshold of a target region based on a grayscale value clustering of the target region. The application optimizes the process of writing extraction for the scene of a blackboard or a whiteboard, reduces the probability of writing false detection or omission, improves the precision of writing content extraction, and quickly obtains a binaryzation threshold grayscale, thereby improving the speed of writing extraction.

Description

Technical Field

[0001] This invention relates to the technical field of artificial intelligence visual recognition, and more specifically, to a method, apparatus, device, and storage medium for extracting written content. Background Technology

[0002] Blackboard notes, written on whiteboards or blackboards for meetings, speeches, and teaching, are an important medium for information transmission. Participants, audiences, or students can better understand the information the speaker wants to convey by viewing the blackboard notes. Therefore, in remote meetings or teaching, videos of the blackboard notes are often recorded so that participants can see the speaker's notes. However, when recording blackboard notes, the speaker often obscures the content while explaining, preventing participants from seeing the entire blackboard.

[0003] Currently, to address the issue of obscured whiteboard content in videos, the whiteboard content is typically extracted and then overlaid onto a current image frame containing the speaker, resulting in a target video frame where both the speaker and the whiteboard content are clearly visible. However, the extraction process is inevitably affected by external factors such as uneven lighting, inconsistent handwriting pressure, and incomplete erasure of the whiteboard. This often leads to false positives or false negatives, and the computational complexity of the entire extraction process results in slow extraction speed. Summary of the Invention

[0004] To address the technical problem of reducing false positives or false negatives during the extraction of written content, this invention provides a method for extracting written content, a video switching method, an apparatus, a device, and a storage medium. The technical solution adopted by this invention to solve the above-mentioned technical problem is as follows.

[0005] In a first aspect, the present invention provides a method for extracting written content, comprising:

[0006] Acquire an image of the writing medium;

[0007] The image of the writing medium is converted to grayscale to obtain the writing content image, wherein the writing content image is a grayscale image;

[0008] Using statistical methods, adaptive threshold binarization is performed on the image of the written content to obtain a mask image for binary classification of background and written content;

[0009] Based on the mask image, the writing content is extracted from the image of the writing medium, and a result image containing only the writing content is output.

[0010] The process of using statistical methods to adaptively threshold binarize the written content image to obtain a mask image for binary classification of background and written content includes:

[0011] Clustering based on the gray values ​​of the target region yields the binarization threshold of the target region.

[0012] In one embodiment, the process of grayscale conversion of the image of the writing medium includes:

[0013] The maximum or minimum value of the RGB components in the image of the writing medium is used as the grayscale value of the grayscale image.

[0014] In one embodiment, the process of grayscale conversion of the image of the writing medium includes:

[0015] When the background color of the writing medium is one of the preset light colors, the minimum value of the RGB three components in the image of the writing medium is used as the gray value of the grayscale image.

[0016] When the background color of the writing medium is one of the preset dark colors, the maximum value of the RGB three components in the image of the writing medium is used as the grayscale value of the grayscale image.

[0017] In one embodiment, the process of grayscale conversion of the image of the writing medium includes:

[0018] The average of the RGB values ​​in the image of the writing medium is used as the grayscale value of the grayscale image.

[0019] In one embodiment, the process of grayscale conversion of the image of the writing medium to obtain the image of the written content includes:

[0020] The image of the writing medium is converted to grayscale to obtain a grayscale image of the writing medium.

[0021] Morphological processing is performed on the grayscale image to obtain an image with blurred written content;

[0022] The blurred image of the written content is compared with the image of the writing medium or the grayscale image, and the image of the written content is generated based on the result of the comparison operation.

[0023] In one implementation, the process of performing morphological processing on the grayscale image to obtain an image with blurred written content includes:

[0024] When the background color of the writing medium is one of the preset light colors, morphological processing is performed using a closing operation.

[0025] And / or, when the background color of the writing medium is one of the preset dark colors, morphological processing is performed using an opening operation.

[0026] In one embodiment, the process of using statistical methods to adaptively threshold binarize the written content image to obtain a mask image for binary classification of background and written content further includes:

[0027] Calculate the feature values ​​of the image of the written content;

[0028] When the feature value is greater than a specified threshold, local adaptive threshold binarization is used;

[0029] When the feature value is not greater than a specified threshold, global adaptive threshold binarization is used.

[0030] Secondly, the present invention provides a written content extraction device, characterized in that it comprises:

[0031] The acquisition module is used to acquire images of the writing medium;

[0032] The first processing module is used to process the image of the writing medium to obtain a writing content image, wherein the writing content image is a grayscale image;

[0033] The second processing module is used to perform adaptive threshold binarization on the written content image using statistical methods to obtain a mask image that classifies the background and the written content into two categories.

[0034] The extraction module is used to extract the writing content from the image of the writing medium based on the mask image, and output a result image containing only the writing content;

[0035] The second processing module includes:

[0036] The threshold determination unit is used to cluster based on the gray values ​​of the target region to obtain the binarization threshold of the target region.

[0037] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any of the above embodiments.

[0038] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method of any of the above embodiments.

[0039] In this invention, the handwriting extraction process is optimized for blackboard or whiteboard scenarios. The maximum / minimum value method of RGB is used to form a grayscale image, which reduces the probability of false or missed handwriting detection and improves the accuracy of handwriting extraction. Furthermore, an adaptive threshold selection method based on grayscale value clustering is used to determine the threshold, which quickly obtains the binarized threshold grayscale and improves the speed of handwriting extraction. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating Embodiment 1 of the present invention.

[0041] Figure 2 This is a mask image for the binary classification of written content in Embodiment 1 of the present invention.

[0042] Figure 3 This is a grayscale image obtained from one embodiment of Embodiment 1 of the present invention.

[0043] Figure 4 This is a flowchart illustrating a sub-step of Embodiment 1 of the present invention.

[0044] Figure 5 It is a binary classification mask image obtained through existing technology.

[0045] Figure 6 This is an image of blurred handwriting obtained from one embodiment of the present invention.

[0046] Figure 7 This is an image of the written content from Embodiment 1 of the present invention.

[0047] Figure 8 This is a schematic diagram of the overall structure of Embodiment 2 of the present invention. Detailed Implementation

[0048] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0049] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, ..." can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first, second, ..." can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0050] Example 1

[0051] Please see Figure 1 , Figure 1This is a flowchart illustrating a method for extracting written content according to Embodiment 1 of the present invention. The method includes steps S110, S120, S130, and S140. It should be noted that steps S110, S120, S130, and S140 are merely reference numerals used to clearly explain the embodiment and the accompanying drawings. Figure 1 The correspondence does not imply a limitation on the order of the method steps in this embodiment.

[0052] Step S110: Obtain an image of the writing medium;

[0053] Step S120: The image of the writing medium is converted to grayscale to obtain the writing content image, wherein the writing content image is a grayscale image;

[0054] Step S130: Using statistical methods, adaptive threshold binarization is performed on the written content image to obtain a mask image for binary classification of background and written content;

[0055] Step S140: Extract the writing content from the image of the writing medium according to the mask image, and output a result image containing only the writing content.

[0056] In step S110, an image of the writing medium is acquired. This writing medium can be a blackboard, whiteboard, or electronic blackboard used for recording writing, or a notebook, tablet, or other medium on which writing can be done. To facilitate the explanation of the invention's concept, a blackboard or whiteboard will be used as an example below. Generally, there is a camera that records information from a blackboard or whiteboard; this method acquires a frame of image captured by such a camera.

[0057] In step S120, the image of the writing medium is converted to grayscale to obtain a grayscale image of the writing medium. This grayscale image can then be used directly as the image of the writing content, or it can be adjusted appropriately to obtain the image of the writing content.

[0058] There are many existing methods for grayscale processing of images, including: component method, maximum / minimum method, average method, and weighted average method.

[0059] 1. Component method:

[0060] The brightness of three parts of the color image is used as the grayscale values ​​of the three grayscale images. One grayscale image can be selected according to the application requirements.

[0061] f1(i,j)=R(i,j), f2(i,j)=G(i,j), f3(i,j)=B(i,j);

[0062] Where fk(i,j) (k=1,2,3) is the gray value of the converted grayscale image at (i,j).

[0063] 2. Maximum / Minimum Method:

[0064] The maximum / minimum values ​​of the three brightness components in the color image are used as the grayscale values ​​of the grayscale image.

[0065] f(i,j)=max(R(i,j),G(i,j),B(i,j));

[0066] Or f(i,j)=min(R(i,j),G(i,j),B(i,j)).

[0067] 3. Average value method

[0068] A grayscale image is obtained by averaging the three brightness components in a color image.

[0069] f(i,j)=(R(i,j)+G(i,j)+B(i,j)) / 3.

[0070] 4. Weighted average method

[0071] Based on importance and other indicators, the three components are weighted and averaged with different values. Since the human eye is most sensitive to green and least sensitive to blue, a more reasonable grayscale image can be obtained by weighting the RGB components according to the following formula.

[0072] f(i,j)=0.299*R(i,j)+0.587*G(i,j)+0.114*B(i,j).

[0073] This method can use one or more existing technologies for grayscale conversion.

[0074] It should be noted that before executing step S120, the image can be preprocessed, including brightness equalization to reduce the influence of external light; downsampling of images that are too large to reduce the amount of processing computation in subsequent processing steps and speed up the processing.

[0075] Step S130 is to obtain a mask image for binary classification of background and written content through adaptive threshold binarization. In this invention, the process of obtaining the binarization threshold of the target region has been optimized.

[0076] Specifically, the process of step S130 includes: step S131.

[0077] Step S131: Based on the gray value clustering of the target region, obtain the binarization threshold of the target region.

[0078] This method uses clustering to obtain an adaptive binarization threshold, which is suitable for both global and local adaptation. For global adaptation, the target region is the global region, and for local adaptation, the target region is each small region.

[0079] Using clustering to obtain an adaptive binarization threshold is suitable for scenarios involving the extraction of written content. This is because the grayscale values ​​between the writing medium and the written content are highly distinguishable. For example, on a blackboard, the blackboard itself has a darker grayscale value, while the handwriting has a brighter grayscale value; similarly, on a whiteboard, the whiteboard has a brighter grayscale value, while the handwriting has a darker grayscale value. Both maintain a binary data distribution, but the threshold selection changes due to variations in the content. Specifically, obtaining the binarization threshold for the target region using grayscale value clustering includes the following steps:

[0080] 1. First, perform grayscale histogram statistics on the input grayscale image to count the distribution of grayscale values ​​of all pixels in the entire image and the number of pixels contained in each grayscale value.

[0081] 2. Set the initial grayscale values ​​C1 and C2 for the centers of the two categories.

[0082] 3. Calculate the distances d1 and d2 from each gray value v_i to the centers of the two categories; compare d1 and d2 to classify the gray value v_i into different category sets. Iterate through all gray values ​​in the range [0, 255] to obtain two gray value sets.

[0083] -d1 = alpha1 * |v_i - c1|;

[0084] -d2 = alpha2 * |v_i - c2|;

[0085] Note: |ab| is the absolute value of ab; alpha1 and alpha2 are distance adjustment weight coefficients, which need to be set according to the whiteboard scenario.

[0086] 4. Based on the two sets of gray values, update the gray values ​​C1 and C2 of the two category centers;

[0087] C = 1 / N*sum(n_i*v_i);

[0088] Note: N is the total number of pixels in the category set, n_i is the grayscale value, and v_i contains the number of pixels;

[0089] 5. Repeat steps 3 and 4 until the new category center gray value is equal to the old category center gray value, thus producing a stable binary classification gray set.

[0090] 6. Take the gray value v_threshold at the intersection of the two sets and define it as the optimal binary classification threshold for the output.

[0091] After obtaining the optimal binary classification threshold, the input grayscale values ​​are then classified according to the grayscale threshold v_threshold. Figure 2 Classify and output a binary classification binary image.

[0092] Compared to Otsu's thresholding method, the adaptive threshold selection method based on grayscale clustering does not require traversing all grayscale levels to calculate inter-class variance. Utilizing grayscale histograms and pixel count information, it can quickly cluster grayscale values ​​in a grayscale image. Taking advantage of the scenario's specific characteristics, the input data distribution is presupposed to have potential bipartiteness, defining two cluster centers. The grayscale value at the boundary between the two clustered categories is the target binarization threshold grayscale value.

[0093] Through the processing in step S130, the following is obtained: Figure 2 The image shows a mask for binary classification of background and written content. Finally, based on the results of the binary classification mask, background information is represented by values ​​of 0, while written content information is represented by non-zero values. The written content information is extracted from the same location in the image of the writing medium (i.e., the original image), and the resulting image contains only the written content. Typically, when a presenter writes on a blackboard or whiteboard, different colors of ink are used to distinguish content and key points. This method uses a mask to determine the location of the written content in the image of the writing medium, and then extracts the written content information from the same location in the image of the writing medium. The resulting written content information is naturally the written content information in the original handwriting color.

[0094] In one embodiment, the process of grayscale processing of the image of the writing medium includes: step S1201.

[0095] Step S1201: Use the maximum or minimum value of the RGB three components in the image of the writing medium as the grayscale value of the grayscale image.

[0096] This implementation uses the maximum / minimum value method. In the traditional weighted average method for converting to grayscale, the R and B channel pixels are suppressed, and the grayscale image cannot well represent colors with blue and red as the main primary colors. For example, if the input RGB pixel is (1,1,255), its output grayscale value will become (30), and the brightness is similar to that of a blackboard, making subsequent handwriting extraction very difficult. However, using the maximum value method, with the same input RGB pixel (1,1,255), its output grayscale value will become (255), and the brightness can well restore the information of the red channel. Figure 3 The grayscale image obtained above is based on blue handwriting on a blackboard. Figure 3 It is a grayscale image obtained by taking the maximum value of the RGB values.

[0097] In one embodiment, the process of grayscale processing of the image of the writing medium includes steps S1202 and S1203.

[0098] Step S1202: When the background color of the writing medium is one of the preset light colors, the minimum value of the RGB three components in the image of the writing medium is used as the gray value of the grayscale image.

[0099] Step S1203: When the background color of the writing medium is one of the preset dark colors, the maximum value of the RGB three components in the image of the writing medium is used as the gray value of the grayscale image.

[0100] This implementation uses the maximum / minimum value method, and appropriately selects whether to use the maximum or minimum value based on the background color of the writing medium. When the background color is relatively light, and is one of the preset light background colors, such as when the writing medium is a whiteboard, the minimum value is selected; when the background color is relatively dark, and is one of the preset dark background colors, such as when the writing medium is a blackboard, the maximum value is selected.

[0101] Continuing with the example of blue text above, in the case of a blackboard, its output grayscale value is (255), which is quite different from the brightness of the blackboard. In the case of a whiteboard, its output grayscale value is (1), which is also quite different from the brightness of the whiteboard. This effectively distinguishes the written content from the background.

[0102] In one embodiment, the process of grayscale conversion of the image of the writing medium includes step S1204.

[0103] Step S1204: Use the average value of the RGB three components in the image of the writing medium as the grayscale value of the grayscale image.

[0104] This implementation uses the average value method. As in the example above, if the average value method is used, the output grayscale value (86) will have a certain difference from the brightness of the blackboard or whiteboard.

[0105] In one implementation, such as Figure 4 As shown, Figure 4 This is a flowchart of step S120. The process of step S120 includes: step S121, step S122 and step S123.

[0106] Step S121: Perform grayscale processing on the image of the writing medium to obtain a grayscale image of the writing medium;

[0107] Step S122: Perform morphological processing on the grayscale image to obtain an image with blurred written content;

[0108] Step S123: Compare the blurred image of the writing content with the image of the writing medium or the grayscale image, and generate the image of the writing content based on the result of the comparison operation.

[0109] In existing technologies, a grayscale image of the writing medium is typically used as the image of the writing content, followed by adaptive threshold binary classification to obtain a mask image for binary classification of the background and the handwriting. However, the grayscale image obtained by grayscale processing alone may be affected by factors such as uneven lighting distribution, varying handwriting pressure, and incomplete erasure of the writing board, leading to suboptimal results in the binary classification mask image generated by subsequent binarization. Figure 5 As shown, Figure 5 This is a binary classification mask image obtained by performing only grayscale processing. It can be seen that... Figure 5 The light distribution is uneven near the "corresponding angles, alternate interior angles, and same-side interior angles" in the diagram. Utilizing... Figure 3 The binary classification mask image directly generated from the grayscale image is as follows: Figure 5 As shown, the words "corresponding angles, alternate interior angles, and same-side interior angles" cannot be displayed, which leads to false detections or missed detections during the handwriting extraction process. Therefore, this invention further processes the grayscale image to improve the accuracy of the handwriting extraction.

[0110] Specifically, after obtaining the grayscale image of the writing medium, step S122 is performed. The purpose of step S122 is to eliminate the detail information of the image and obtain an image with blurred writing content, such as... Figure 6 As shown.

[0111] After obtaining the blurred image, proceed to step S123 to... Figure 6 The image shown is blurry compared to the image of the writing medium (i.e., the original image), or as... Figure 3 The grayscale images shown are compared and processed to obtain the following results: Figure 7 The image of the written content shown ( Figure 6 (This is the result of difference normalization between the original image and the morphologically processed image). It can be seen that... Figure 7 There is none left in the middle. Figure 5 The uneven distribution of light in the image.

[0112] Specifically, the comparison operation can be either division or absolute value subtraction. This allows the handwriting content to be obtained through comparison. The result of the comparison operation is normalized to its maximum and minimum values, so that the result is mapped to the range [0, 255]. The mapped result is then the image of the written content.

[0113] In this embodiment, the grayscale image processing is optimized. By blurring the grayscale image, a blurred image of the written content is obtained. This blurred image is then compared with the original image or the grayscale image to obtain the written content image. Compared to a regular grayscale image, this written content image eliminates some outlines outside the handwriting that are not part of the handwriting itself, which is beneficial for subsequent binary classification processing. This reduces the probability of false positives or false negatives in handwriting detection and improves the accuracy of written content extraction.

[0114] In one embodiment, the morphological processing of step S122 includes: step S1221.

[0115] Step S1221: Perform morphological processing using either closing or opening operations.

[0116] In one embodiment, the morphological processing of step S122 includes: step S1222 and / or step S1223.

[0117] Step S1222: When the background color of the writing medium is one of the preset light colors, morphological processing is performed using a closing operation.

[0118] Step S1223: When the background color of the writing medium is one of the preset dark colors, morphological processing is performed using an opening operation.

[0119] The handwriting extraction method selects a morphological approach based on the background color. When the background color is relatively dark, belonging to one of the preset dark background color schemes, such as when the writing medium is a blackboard, the morphological processing method is an opening operation, yielding results like... Figure 5 The image shown has blurred handwriting. When the background color is relatively light, and is one of the preset light background colors, such as when the writing medium is a whiteboard, the morphological processing method is a closing operation.

[0120] Using morphological processing methods can better restore background and handwriting information without causing too much interference to subsequent normalization processing.

[0121] For example, in the case of a blackboard, the opening operation is used, which involves erosion followed by dilation. Erosion in grayscale images is a local minima processing operation. When there is a significant difference between the blackboard and the text content, the erosion operation can better target the blackboard pixel values ​​than mean blur or Gaussian blur. When comparing the original image and the processed result, the difference is more significant.

[0122] In one implementation, step S130 includes steps S131, S132, and S133.

[0123] Step S131: Calculate the feature values ​​of the written content image;

[0124] Step S132: When the feature value is greater than a specified threshold, local adaptive threshold binarization is used;

[0125] Step S133: When the feature value is not greater than a specified threshold, global adaptive threshold binarization is used.

[0126] After obtaining the written content image output in step S120, calculate the feature information value of the grayscale image of the written content image (which may be global variance, standard deviation, brightness mean, etc.); if the feature value sigma_G is greater than the specified threshold T, then select to use local adaptive threshold binarization; if the feature value sigma_G is less than the specified threshold T, then use global adaptive threshold binarization.

[0127] Among them, global adaptive binarization processes the grayscale image using an adaptive binarization method; local adaptive binarization divides the grayscale image into N*N small regions and then performs adaptive binarization processing on each region.

[0128] In this method, the handwriting extraction process is optimized for blackboard or whiteboard scenarios. The maximum / minimum value method of RGB is used to form a grayscale image, which reduces the probability of false detection or missed detection of handwriting and improves the accuracy of handwriting extraction. Furthermore, an adaptive threshold selection method based on grayscale value clustering is used to determine the threshold and quickly obtain the binarized threshold grayscale, which improves the speed of handwriting extraction.

[0129] Example 2

[0130] Corresponding to the method for extracting written content in Example 1, such as Figure 8 As shown, the present invention also provides a written content extraction device 8, including: an acquisition module 810, a first processing module 820, a second processing module 830, and an extraction module 840.

[0131] The acquisition module 810 is used to acquire an image of the writing medium;

[0132] The first processing module 820 is used to process the image of the writing medium to obtain a writing content image, wherein the writing content image is a grayscale image;

[0133] The second processing module 830 is used to perform adaptive threshold binarization on the written content image using statistical methods to obtain a mask image that classifies the background and the written content into two categories.

[0134] The extraction module 840 is used to extract the writing content from the image of the writing medium based on the mask image, and output a result image containing only the writing content.

[0135] The second processing module includes a threshold determination unit.

[0136] The threshold determination unit is used to cluster based on the gray values ​​of the target region to obtain the binarization threshold of the target region.

[0137] In one implementation, the first processing module includes: grayscale units.

[0138] The grayscale unit uses the maximum or minimum value of the RGB components in the image of the writing medium as the grayscale value of the grayscale image.

[0139] In one implementation, the first processing module includes: grayscale units.

[0140] When the background color of the writing medium is one of the preset light colors, the grayscale unit uses the minimum value of the RGB three components in the image of the writing medium as the grayscale value of the grayscale image.

[0141] When the background color of the writing medium is one of the preset dark colors, the grayscale unit uses the maximum value of the RGB three components in the image of the writing medium as the grayscale value of the grayscale image.

[0142] In one implementation, the first processing module includes: grayscale units.

[0143] The grayscale unit uses the average of the RGB values ​​in the image of the writing medium as the grayscale value of the grayscale image.

[0144] In one embodiment, the first processing module includes: a grayscale unit, a blur unit, and a comparison unit.

[0145] A grayscale unit is used to perform grayscale processing on the image of the writing medium to obtain a grayscale image of the writing medium.

[0146] A blurring unit is used to perform morphological processing on the grayscale image to obtain an image with blurred written content;

[0147] The comparison unit is used to compare the blurred image of the writing content with the image of the writing medium or the grayscale image, and generate the image of the writing content based on the result of the comparison operation.

[0148] In one implementation, the process by which the blurring unit performs morphological processing on the grayscale image to obtain an image with blurred written content includes:

[0149] When the background color of the writing medium is one of the preset light colors, morphological processing is performed using a closing operation.

[0150] And / or, when the background color of the writing medium is one of the preset dark colors, morphological processing is performed using an opening operation.

[0151] In one implementation, the second processing module performs the process of adaptive threshold binarization of the written content image using statistical methods to obtain a mask image for binary classification of background and written content, including:

[0152] Calculate the feature values ​​of the image of the written content;

[0153] When the feature value is greater than a specified threshold, local adaptive threshold binarization is used;

[0154] When the feature value is not greater than a specified threshold, global adaptive threshold binarization is used.

[0155] In this device, the handwriting extraction process is optimized for blackboard or whiteboard scenarios. The maximum / minimum value method of RGB is used to form a grayscale image, which reduces the probability of false detection or missed detection of handwriting and improves the accuracy of handwriting extraction. Furthermore, an adaptive threshold selection method based on grayscale value clustering is used to determine the threshold and quickly obtain the binarized threshold grayscale, which improves the speed of handwriting extraction.

[0156] Example 3

[0157] This invention also provides a storage medium storing computer instructions that, when executed by a processor, implement the written content extraction method of any of the above embodiments.

[0158] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disks, or optical disks.

[0159] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, RAM, ROM, magnetic disks, or optical disks.

[0160] Corresponding to the computer storage medium described above, one embodiment also provides a computer device, which includes a memory, an encoder, and a computer program stored in the memory and executable on the encoder, wherein the encoder executes the program to implement any of the written content extraction methods described in the above embodiments.

[0161] The aforementioned computer equipment optimizes the handwriting extraction process for blackboard or whiteboard scenarios. It uses the maximum / minimum value method of RGB to form a grayscale image, reducing the probability of false or missed handwriting detection and improving the accuracy of handwriting extraction. Furthermore, it adopts an adaptive threshold selection method based on grayscale value clustering to determine the threshold, quickly deriving the binarized threshold grayscale, and improving the speed of handwriting extraction.

[0162] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting written content, characterized in that, include: Acquire an image of the writing medium; The image of the writing medium is converted to grayscale to obtain the writing content image, wherein the writing content image is a grayscale image; Using statistical methods, adaptive threshold binarization is performed on the image of the written content to obtain a mask image for binary classification of background and written content; Based on the mask image, the writing content is extracted from the image of the writing medium, and a result image containing only the writing content is output. Specifically, the statistical method used is to perform adaptive threshold binarization on the image of the written content. The process of obtaining a mask image for binary classification of background and written content includes: Calculate the feature values ​​of the written content image; wherein, the feature values ​​include: global variance, standard deviation, or mean brightness; When the feature value is greater than a specified threshold, local adaptive threshold binarization is used; wherein, local Adaptive binarization is used to divide the grayscale image into several small regions, and then adaptive binarization is performed on each small region separately. When the feature value is not greater than a specified threshold, global adaptive threshold binarization is used; wherein, Global adaptive binarization is a process that uses adaptive binarization methods to process grayscale images. The process of the adaptive binarization method includes: Based on grayscale value clustering of the target region, the binarization threshold of the target region is obtained, where, for the full... For local adaptive threshold binarization, the target region is the global region; for local adaptive threshold binarization, the target region is each small region. The process of obtaining the binarization threshold of the target region based on grayscale value clustering includes: S1. First, perform grayscale histogram statistics on the input grayscale image to count the distribution of grayscale values ​​of all pixels in the entire image and the number of pixels contained in each grayscale value; S2. Set the initial grayscale values ​​for the centers of the two categories; S3. Adjust the weighting coefficients based on the distance to the centers of the two categories, and calculate the distance of each grayscale value to the centers of the two categories. The distances between the centers are denoted as d1 and d2; by comparing d1 and d2, each gray value is assigned to a different category set. In the middle; iterate through all gray values ​​in [0, 255] to obtain two sets of gray values; S4. Update the gray values ​​of the two category centers based on the two gray value sets; S5. Repeat S3 and S4 until the new category center gray value equals the old category center gray value; S6. Take the gray value at the intersection of the two sets and define it as the binarization threshold.

2. The method for extracting written content according to claim 1, characterized in that, The book The process of converting an image of a writing medium to grayscale includes: using the maximum or minimum value of the RGB components in the image of the writing medium as the grayscale value of the grayscale image.

3. The method for extracting written content according to claim 1, characterized in that, The book The process of converting an image on a writing medium to grayscale includes: When the background color of the writing medium is one of the preset light colors, the minimum value of the RGB three components in the image of the writing medium is used as the gray value of the grayscale image. When the background color of the writing medium is one of the preset dark colors, the maximum value of the RGB three components in the image of the writing medium is used as the grayscale value of the grayscale image.

4. The method for extracting written content according to claim 1, characterized in that, The process of grayscale conversion of the image of the writing medium includes: The average of the RGB values ​​in the image of the writing medium is used as the grayscale value of the grayscale image.

5. The method for extracting written content according to claim 1, characterized in that, The book The process of converting the image of the writing medium to grayscale to obtain the image of the written content includes: The image of the writing medium is converted to grayscale to obtain a grayscale image of the writing medium. Morphological processing is performed on the grayscale image to obtain an image with blurred written content; The blurred image of the written content is compared with the image of the writing medium or the grayscale image, and the image of the written content is generated based on the result of the comparison operation.

6. The method for extracting written content according to claim 5, characterized in that, For the grayscale image The process of performing morphological processing to obtain an image with blurred written content includes: When the background color of the writing medium is one of the preset light colors, morphological processing is performed using a closing operation. And / or, when the background color of the writing medium is one of the preset dark colors, morphological processing is performed using an opening operation.

7. A device for extracting written content, characterized in that, include: The acquisition module is used to acquire images of the writing medium; The first processing module is used to process the image of the writing medium to obtain an image of the written content. The image of the written content is a grayscale image; The second processing module is used to perform adaptive threshold binarization on the written content image using statistical methods to obtain a mask image that classifies the background and the written content into two categories. The extraction module is used to extract the writing content from the image of the writing medium based on the mask image, and output a result image containing only the writing content; The second processing module performs the statistical method to analyze the written content diagram. The process of performing adaptive threshold binarization to obtain a mask image for binary classification of background and written content includes: Calculate the feature values ​​of the written content image; wherein, the feature values ​​include: global variance, standard deviation, or mean brightness; When the feature value is greater than a specified threshold, local adaptive threshold binarization is used; wherein, local Adaptive binarization is used to divide the grayscale image into several small regions, and then adaptive binarization is performed on each small region separately. When the feature value is not greater than a specified threshold, global adaptive threshold binarization is used; wherein, Global adaptive binarization is a process that uses adaptive binarization methods to process grayscale images. The process of the adaptive binarization method includes: Based on grayscale value clustering of the target region, the binarization threshold of the target region is obtained, where, for the full... For local adaptive threshold binarization, the target region is the global region; for local adaptive threshold binarization, the target region is each small region. The process of obtaining the binarization threshold of the target region based on grayscale value clustering includes: S1. First, perform grayscale histogram statistics on the input grayscale image to count the distribution of grayscale values ​​of all pixels in the entire image and the number of pixels contained in each grayscale value; S2. Set the initial grayscale values ​​for the centers of the two categories; S3. Adjust the weight coefficients based on the distance to the two category centers. Calculate the distance from each gray value to the two category centers, denoted as d1 and d2. Compare d1 and d2 and assign each gray value to a different category set. Iterate through all gray values ​​in [0, 255] to obtain two gray value sets. S4. Update the gray values ​​of the two category centers based on the two gray value sets; S5. Repeat S3 and S4 until the new category center gray value equals the old category center gray value; S6. Take the gray value at the intersection of the two sets and define it as the binarization threshold.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images