Method and device for extracting movie subtitles in background complex change scene based on OCR and color preprocessing

Abstract

The invention discloses a film subtitle extraction method and device under a background complex change scene based on OCR (optical character recognition) and color preprocessing, and the method comprises the steps: intercepting and positioning a film subtitle position in advance, extracting film subtitle color information features, and extracting a subtitle text according to subtitle color preprocessing; the method improves and optimizes the poor subtitle extraction effect caused by background element confusion, color change and the like in the traditional film subtitle extraction method. According to the method, the recognition precision of the OCR in a movie scene can be improved, and challenges caused by background complexity are overcome while the efficiency is ensured.

Description

Technical field:

[0001] The invention relates to optical character recognition technology, and in particular to a method for extracting movie subtitles in a scene with complex background changes based on OCR and specific color preprocessing. Background technology:

[0002] In the text recognition technology for movie subtitles, there are often complex scene changes. Movie scenes often contain rich and varied background elements, such as fast-moving objects, light changes, special effects, etc. Strong light and shadow effects may directly interfere with the clarity of subtitles. These factors make subtitle extraction in movies extremely complicated.

[0003] Traditional movie subtitle extraction methods are based solely on optical character recognition (OCR) technology, but often perform poorly because subtitles may be confused by background elements or even affected by color changes, making it difficult to accurately recognize them. Therefore, it is difficult for pure optical character recognition technology to accurately extract subtitles in movie scenes.

[0004] The above problems need to be solved urgently. Summary of the invention:

[0005] The present invention aims to overcome the above-mentioned disadvantages of the prior art, and based on the advantages of OCR and preprocessing, provides a method and device for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing.

[0006] In order to solve the above technical problems, a method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing is proposed, which includes the following steps:

[0007] S110, sampling movie frames to obtain specific locations of text boxes, randomly sampling multiple picture frames, and obtaining locations of all text boxes from these picture frames using an OCR method;

[0008] S120, using a clustering algorithm to obtain subtitle position information, for text frame clustering, obtaining the class with the largest number of clusters in the category, taking the coordinates of the center point of the cluster as the center point of the subtitle position, counting the mode of the text frame heights in the class to which the center point coordinate belongs as the height of the subtitle position, and taking the width of the movie as the width of the subtitle position;

[0009] S130, calculating subtitle color features according to the majority of pixel points;

[0010] S140, extracting subtitle text according to subtitle color preprocessing.

[0011] Furthermore, the sampling of the movie frame to obtain the specific position of the text frame in step S110 specifically includes:

[0012] S1101, for a movie, randomly sample n picture frames from it;

[0013] S1102, for n picture frames, calculate the positions of all text boxes using the OCR method, and record the number of text boxes as S n ;

[0014] S1103, use {X, Y, H} to represent the specific position of the text box, where X and Y are the coordinates of the center point of the text box, and H is the height of the text box.

[0015] Furthermore, the step S120 of obtaining the subtitle position information by using a clustering algorithm specifically includes:

[0016] S1201, for S n Perform Kmeans clustering on the X and Y positions of the text boxes to obtain the class with the largest number of categories, and use the coordinates of its center point as the center point of the subtitle position;

[0017] S1202, counting the mode of the text box height H in the category of the center point coordinate, and using it as the height of the subtitle position;

[0018] S1203: Using the width of the movie as the width of the subtitle position.

[0019] Furthermore, the step S130 of calculating the subtitle color feature according to the majority of pixels specifically includes:

[0020] S1301, for the picture frame sampled by S1101, use the subtitle position to intercept the movie picture, and use Img i,j Indicates the pixel value of the captured image;

[0021] S1302, for each column of pixels in the captured image, count the pixel values ​​with the largest number, that is, the mode of the pixel values ​​in each column, and use C j Represents the mode of pixel values ​​in each column;

[0022] S1303, calculate {C1,…,C j ,…,C w}, denoted as M, where w is the number of columns of image pixels;

[0023] S1304: Repeat the above operation for all captured image frames, and count the majority of pixel values ​​in {M} as subtitle color information features.

[0024] Furthermore, in step S130, the subtitle color feature storage format is required to be: RGB three-channel mode.

[0025] Furthermore, the extraction of subtitle text according to subtitle color preprocessing in step S140 specifically includes:

[0026] S1401, extracting all picture frames in the movie and capturing pictures according to the subtitle positions;

[0027] S1402, performing a binarization operation on the captured image, setting the pixel value equal to the subtitle color feature to 255, and setting the pixel value not equal to the subtitle color feature to 0;

[0028] S1403: Send the binarized image to OCR for subtitle text extraction.

[0029] The second aspect of the present invention relates to a device for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing of the present invention.

[0030] The third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing as described in any one of claims 1 to 6.

[0031] The beneficial effect of the present invention is that the present invention provides a subtitle extraction method based on OCR and color preprocessing technology in a scene with complex changes in the movie background, wherein the method uses OCR technology to count all text boxes in the picture frame, clusters these text box information and takes the majority operation to determine the subtitle position. According to the order of column statistics, row statistics, and movie frame statistics, the color characteristics of the subtitles are clarified, and the color information is preprocessed by a binary operation and then put into the OCR technology to extract the subtitles, which improves the problem that the existing simple OCR technology cannot recognize the complex scene changes in the movie subtitles, thereby improving the accuracy of movie subtitle extraction. Description of the drawings:

[0032] Figure 1 It is a flowchart of a subtitle extraction method in a scene with complex and changing movie backgrounds based on OCR and color preprocessing technology provided by an embodiment of the present invention.

[0033] Figure 2 It is a schematic diagram of the device of the present invention.

[0034] Figure 3 It is a diagram showing the effect of the present invention after the binarization operation preprocessing. Specific implementation method:

[0035] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0036] Example 1

[0037] like Figure 1 As shown, this embodiment 1 provides a method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing. The method improves the adaptability of the existing technology to complex background changes in movies, thereby helping to improve the accuracy of movie subtitle extraction.

[0038] Specifically, the method includes:

[0039] S110, sampling the movie frame to obtain the specific position of the text box;

[0040] Specifically, for a movie, n picture frames are randomly sampled from it; for n picture frames, the positions of all text boxes are calculated by the OCR method, and the number of text boxes is recorded as S n ; Use {X,Y,H} to represent the specific location of the text box, where X and Y are the coordinates of the center point of the text box, and H is the height of the text box.

[0041] S120, obtaining subtitle position information using a clustering algorithm;

[0042] Specifically, for S n Perform Kmeans clustering on the X and Y positions of the text boxes to obtain the class with the largest number of categories, and use its center point coordinates as the center point of the subtitle position; count the mode of the text box height H in the class to which the center point coordinate belongs, and use it as the height of the subtitle position; use the width of the movie as the width of the subtitle position.

[0043] S130, calculating subtitle color features according to the majority of pixel points;

[0044] Specifically, for the picture frame sampled by S1101, the movie picture is intercepted using the subtitle position, and Img i,j Represents the pixel value of the intercepted image; for each column of pixels in the intercepted image, count the pixel value with the largest number, that is, the mode of the pixel values ​​in each column, and use C j Represents the mode of pixel values ​​in each column; calculate {C1,…,C j ,…,C w}, denoted as M, where w is the number of columns of image pixels; for all captured image frames, repeat the above operation and count the mode of pixel values ​​in {M} as the subtitle color information feature.

[0045] S140, extracting subtitle text according to subtitle color preprocessing.

[0046] Specifically, all the picture frames in the movie are extracted, and pictures are cut according to the subtitle position; the cut pictures are binarized, and the pixel values ​​equal to the subtitle color feature are set to 255, and the pixel values ​​not equal to the subtitle color feature are set to 0; the pictures after the binarization operation are sent to OCR for subtitle text extraction.

[0047] In summary, the present invention provides a method for extracting subtitles in a complex and changing movie background based on OCR and color preprocessing technology. The method uses OCR technology to count all text boxes in a picture frame, and clusters and takes the majority of these text box information to determine the subtitle position. According to the order of column statistics, row statistics, and movie frame statistics, the color characteristics of the subtitles are clarified, and the color information is preprocessed by binarization operation and then put into OCR technology to extract subtitles. The effect of the binarization operation preprocessing is as follows: Figure 3 As shown in the figure, the problem that the existing simple OCR technology cannot recognize complex scene changes in movie subtitles is improved, thereby improving the accuracy of movie subtitle extraction.

[0048] Example 2

[0049] This embodiment relates to a device for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing, including a memory and one or more processors, such as Figure 2 As shown, the memory stores executable codes, and when the one or more processors execute the executable codes, they are used to implement the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing of Example 1.

[0050] Example 3

[0051] The present embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing of Embodiment 1 is implemented.

[0052] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for extracting movie subtitles in complex background changing scenes based on OCR and color preprocessing, characterized in that: The following steps are involved: S110, sampling movie frames to obtain specific locations of text boxes, randomly sampling multiple picture frames, and obtaining locations of all text boxes from these picture frames using an OCR method; S120, using a clustering algorithm to obtain subtitle position information, for text frame clustering, obtaining the class with the largest number of clusters in the category, taking the coordinates of the center point of the cluster as the center point of the subtitle position, counting the mode of the text frame heights in the class to which the center point coordinate belongs as the height of the subtitle position, and taking the width of the movie as the width of the subtitle position; S130, calculating subtitle color features according to the majority of pixel points; S140, extracting subtitle text according to subtitle color preprocessing.

2. The method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: The sampling of the movie frame to obtain the specific position of the text frame in step S110 specifically includes: S1101, randomly sampling n picture frames from a movie; S1102, for n picture frames, calculate the positions of all text boxes using the OCR method, and record the number of text boxes as S n ; S1103, use {X, Y, H} to represent the specific position of the text box, where X and Y are the coordinates of the center point of the text box, and H is the height of the text box.

3. The method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: The step S120 of obtaining the subtitle position information by using a clustering algorithm specifically includes: S1201, for S n Perform Kmeans clustering on the X and Y positions of the text boxes to obtain the class with the largest number of categories, and use the coordinates of its center point as the center point of the subtitle position; S1202, counting the mode of the text box height H in the category of the center point coordinate, and using it as the height of the subtitle position; S1203: Using the width of the movie as the width of the subtitle position.

4. The method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: The step S130 of calculating the subtitle color feature according to the majority of pixels specifically includes: S1301, for the picture frame sampled by S1101, use the subtitle position to intercept the movie picture, and use Img i,j Indicates the pixel value of the captured image; S1302, for each column of pixels in the captured image, count the pixel values ​​with the largest number, that is, the mode of the pixel values ​​in each column, and use C j Represents the mode of pixel values ​​in each column; S1303, calculate {C1,…,C j ,…,C w }, denoted as M, where w is the number of columns of image pixels; S1304: Repeat the above operation for all captured image frames, and count the majority of pixel values ​​in {M} as subtitle color information features.

5. The method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: In the step S130, the subtitle color feature storage format is required to be: RGB three-channel mode.

6. The method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: The extraction of subtitle text according to subtitle color preprocessing in step S140 specifically includes: S1401, extracting all picture frames in the movie and capturing pictures according to the subtitle positions; S1402, performing a binarization operation on the captured image, setting the pixel value equal to the subtitle color feature to 255, and setting the pixel value not equal to the subtitle color feature to 0; S1403: Send the binarized image to OCR for subtitle text extraction.

7. A device for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing as described in any one of claims 1 to 6 is implemented.

Images