Method and device for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing

Abstract

The present invention discloses a method and device for extracting movie subtitles in complex and changing backgrounds based on optical character recognition (OCR) and color preprocessing. This method pre-locates the subtitles, extracts color information, and extracts the subtitle text based on the subtitle color preprocessing. This method improves and optimizes traditional subtitle extraction methods, which often suffer from poor subtitle extraction due to background confusion and color variations. The present invention helps improve OCR recognition accuracy in movie scenes and overcomes the challenges posed by complex backgrounds while maintaining efficiency.

Description

Technical field:

[0001] The present invention relates to optical character recognition technology, and in particular to a method for extracting movie subtitles in scenes with complex background changes based on OCR and specific color preprocessing. Background technology:

[0002] In text recognition technology for movie subtitles, complex scene variations often occur. Movie scenes often contain a variety of background elements, such as fast-moving objects, changing lighting, and special effects. Strong lighting and shadow effects can directly interfere with the clarity of subtitles, making subtitle extraction from movies extremely complex.

[0003] Traditional methods for extracting movie subtitles, based solely on optical character recognition (OCR) technology, often perform poorly because subtitles can be obscured by background elements or even affected by color variations, making accurate recognition difficult. Therefore, OCR alone struggles to accurately extract subtitles from movie scenes.

[0004] The above problems are in urgent need of resolution. Summary of the invention:

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and, based on the advantages of OCR and preprocessing, provides a method and device for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing.

[0006] To solve the above technical problems, a method for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing is proposed, which includes the following steps:

[0007] S110, sampling movie frames to obtain specific locations of text boxes, randomly sampling multiple picture frames from the frames, and using an OCR method to obtain locations of all text boxes from these picture frames;

[0008] S120, using a clustering algorithm to obtain subtitle position information, for text frame clustering, obtaining the largest cluster among the categories, using the center coordinates of the cluster as the center point of the subtitle position, counting the mode of the text frame heights in the cluster to which the center coordinates belong, and using the mode of the text frame heights as the height of the subtitle position, and using the width of the movie as the width of the subtitle position;

[0009] S130, calculating subtitle color features based on the majority of pixels;

[0010] S140, extracting subtitle text according to subtitle color preprocessing.

[0011] Furthermore, the sampling of the movie frame to obtain the specific position of the text box in step S110 specifically includes:

[0012] S1101, for a movie, randomly sample n picture frames from it;

[0013] S1102: For n picture frames, calculate the positions of all text boxes using the OCR method. The number of text boxes is S. n ;

[0014] S1103, use {X, Y, H} to represent the specific position of the text box, where X and Y are the coordinates of the center point of the text box, and H is the height of the text box.

[0015] Furthermore, the step S120 of obtaining subtitle position information by using a clustering algorithm specifically includes:

[0016] S1201, for S n Perform Kmeans clustering on the X and Y positions of the text boxes to obtain the class with the largest number of categories, and use the coordinates of its center point as the center point of the subtitle position;

[0017] S1202, counting the mode of the text box height H in the category of the center point coordinate, and using it as the height of the subtitle position;

[0018] S1203: Using the width of the movie as the width of the subtitle position.

[0019] Furthermore, the step S130 of calculating the subtitle color features based on the pixel mode specifically includes:

[0020] S1301, for the picture frame sampled by S1101, use the subtitle position to intercept the movie picture, and use Img i,j Indicates the pixel value of the captured image;

[0021] S1302: For each column of pixels in the captured image, count the pixel values ​​with the largest number, that is, the mode of the pixel values ​​in each column, and use C j Represents the mode of pixel values ​​in each column;

[0022] S1303, calculate {C1,…,C j ,…,C w}, denoted as M, where w is the number of columns of image pixels;

[0023] S1304: Repeat the above operation for all captured image frames, and count the mode of the pixel values ​​in {M} as the subtitle color information feature.

[0024] Furthermore, in step S130, the subtitle color feature storage format is required to be: RGB three-channel mode.

[0025] Furthermore, the extraction of subtitle text according to subtitle color preprocessing in step S140 specifically includes:

[0026] S1401, extracting all picture frames in the movie and capturing pictures according to the subtitle position;

[0027] S1402, performing a binarization operation on the captured image, setting pixel values ​​equal to the subtitle color feature to 255, and setting pixel values ​​not equal to the subtitle color feature to 0;

[0028] S1403: Send the binarized image to OCR for subtitle extraction.

[0029] A second aspect of the present invention relates to a device for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing, comprising a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing of the present invention.

[0030] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing as described in any one of claims 1 to 6.

[0031] The present invention provides a method for extracting subtitles from complex and changing movie backgrounds based on OCR and color preprocessing techniques. The method uses OCR to count all text boxes within an image frame, clustering and taking the mode of these text boxes to determine the subtitle locations. Following a sequence of column statistics, row statistics, and movie frame statistics, the color characteristics of the subtitles are determined. This color information is then preprocessed using a binarization operation before being used in OCR to extract the subtitles. This overcomes the problem that existing OCR techniques alone are unable to recognize complex scene changes in movie subtitles, thereby improving the accuracy of subtitle extraction. Description of the drawings:

[0032] Figure 1 This is a flowchart of a subtitle extraction method in a scene with complex and changing movie backgrounds based on OCR and color preprocessing technology provided by an embodiment of the present invention.

[0033] Figure 2 It is a schematic diagram of the device of the present invention.

[0034] Figure 3 This is a diagram showing the effect of the present invention after the binarization operation preprocessing. Specific implementation method:

[0035] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0036] Example 1

[0037] like Figure 1 As shown, this embodiment 1 provides a method for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing. The method improves the adaptability of existing technologies to complex background changes in movies, thereby helping to improve the accuracy of movie subtitle extraction.

[0038] Specifically, the method includes:

[0039] S110, sampling the movie frame to obtain the specific position of the text box;

[0040] Specifically, for a movie, n picture frames are randomly sampled from it; for n picture frames, the positions of all text boxes are calculated by OCR method, and the number of text boxes is recorded as S n ; Use {X, Y, H} to represent the specific location of the text box, where X and Y are the coordinates of the center point of the text box, and H is the height of the text box.

[0041] S120, obtaining subtitle position information using a clustering algorithm;

[0042] Specifically, for S n Perform Kmeans clustering on the X and Y positions of the text boxes to obtain the class with the largest number of categories, and use its center point coordinates as the center point of the subtitle position; count the mode of the text box height H in the class to which the center point coordinate belongs, and use it as the height of the subtitle position; use the width of the movie as the width of the subtitle position.

[0043] S130, calculating subtitle color features based on the majority of pixels;

[0044] Specifically, for the picture frame sampled by S1101, the movie picture is intercepted using the subtitle position, and Img i,j Represents the pixel value of the intercepted image; for each column of pixels in the intercepted image, count the pixel value with the largest number, that is, the mode of the pixel values ​​in each column, and use C j Represents the mode of pixel values ​​in each column; calculate {C1,…,C j ,…,C w}, denoted as M, where w is the number of columns of image pixels; for all captured image frames, repeat the above operation and count the mode of pixel values ​​in {M} as the subtitle color information feature.

[0045] S140, extracting subtitle text according to subtitle color preprocessing.

[0046] Specifically, all the picture frames in the movie are extracted and the pictures are cut out according to the subtitle position; the cut out pictures are binarized, and the pixel values ​​equal to the subtitle color feature are set to 255, and the pixel values ​​not equal to the subtitle color feature are set to 0; the pictures after the binarization operation are sent to OCR for subtitle text extraction.

[0047] In summary, the present invention provides a subtitle extraction method based on OCR and color preprocessing technology in a complex and changing movie background scene. The method uses OCR technology to count all text boxes in the picture frame, clusters these text box information and takes the mode operation to determine the subtitle position. According to the order of column statistics, row statistics, and movie frame statistics, the color characteristics of the subtitles are clarified, and the color information is preprocessed by binarization operation and then put into OCR technology to extract subtitles. The effect of the binarization operation preprocessing is as follows: Figure 3 As shown in the figure, the problem that the existing simple OCR technology cannot recognize complex scene changes in movie subtitles is improved, thereby improving the accuracy of movie subtitle extraction.

[0048] Example 2

[0049] This embodiment relates to a device for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing, including a memory and one or more processors, such as Figure 2 As shown, the memory stores executable codes, and when the one or more processors execute the executable codes, they are used to implement the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing according to embodiment 1.

[0050] Example 3

[0051] This embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing of embodiment 1 is implemented.

[0052] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A method for extracting movie subtitles in complex background changing scenes based on OCR and color preprocessing, characterized by: The following steps are involved: S110, sampling movie frames to obtain specific locations of text boxes, randomly sampling multiple picture frames, and obtaining the locations of all text boxes from these picture frames using an OCR method; the sampling movie frames to obtain specific locations of text boxes specifically includes: S1101, randomly sampling n picture frames from a movie; S1102: For n picture frames, calculate the positions of all text boxes using the OCR method. The number of text boxes is recorded as ; S1103, use Indicates the specific location of the text box, X and Y are the coordinates of the center point of the text box, and H is the height of the text box; S120, using a clustering algorithm to obtain subtitle position information, for text frame clustering, obtaining the largest cluster among the categories, using the center coordinates of the cluster as the center point of the subtitle position, counting the mode of the text frame heights in the cluster to which the center coordinates belong, and using the mode of the text frame heights as the height of the subtitle position, and using the width of the movie as the width of the subtitle position; S130, calculating subtitle color features based on the majority of pixels; specifically, including: S1301, for the picture frame sampled by S1101, use the subtitle position to intercept the movie picture, and use Img i,j Indicates the pixel value of the captured image; S1302: For each column of pixels in the captured image, count the pixel values ​​with the largest number, that is, the mode of the pixel values ​​in each column, and use C j Indicates the mode of pixel values ​​in each column; S1303, calculate {C1,…,C j ,…,C w }, denoted as M, where w is the number of columns of image pixels; S1304: Repeat the above operation for all captured image frames, and count the mode of the pixel values ​​in {M} as the subtitle color information feature; S140, extracting subtitle text based on subtitle color preprocessing; specifically including: S1401, extract all picture frames in the movie and capture pictures according to the subtitle position; S1402, performing a binarization operation on the captured image, setting pixel values ​​equal to the subtitle color feature to 255, and setting pixel values ​​not equal to the subtitle color feature to 0; S1403: Send the binarized image to OCR for subtitle extraction.

2. The method for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: The step S120 of obtaining subtitle position information by using a clustering algorithm specifically includes: S1201, yes Text box positions Perform Kmeans clustering to obtain the class with the largest number of clusters, and use the coordinates of its center point as the center point of the subtitle position; S1202, count the center point coordinates of the text box height The mode of , and used as the height of the subtitle position; S1203: Using the width of the movie as the width of the subtitle position.

3. The method for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing according to claim 1, characterized in that: In step S130, the subtitle color feature storage format is required to be: RGB three-channel mode.

4. A device for extracting movie subtitles in scenes with complex background changes based on OCR and color preprocessing, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the method for extracting movie subtitles in a scene with complex background changes based on OCR and color preprocessing as described in any one of claims 1 to 3 is implemented.

Images