Position recognition method, device, electronic device and storage medium for text area

Abstract

An embodiment of the present invention provides a method and apparatus for identifying the position of a text area. The method includes: obtaining a frame image to be identified, where the frame image includes a text area; extracting the feature values of each pixel point in the frame image; calculating the gradient of each pixel point according to the feature values, and identifying the position of the frame image corresponding to the gradient peak among each gradient from the frame image; counting the number of pixel points corresponding to the gradient peak in the frame image position; and based on the comparison result between the number of pixel points and a preset threshold, confirming that the frame image position is the position of the text area. The embodiment of the present invention preliminarily determines the row position or column position of the text area, and then based on the comparison result between the number of pixel points corresponding to the row position or column position and the preset threshold, finally confirms that the frame image position is the position of the text area, which can provide a filtering basis for filtering the text in the subsequent text area, achieve the effect of narrowing the scope of text filtering, reducing the calculation amount of text filtering, and optimizing the text filtering effect.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a method and device for identifying the position of a text area, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of the Internet and computer technology, there is a large amount of video content on traditional television stations and the Internet. In order to assist users in understanding the video content, corresponding text can usually be set for the video content. For example, in variety shows, line texts and non-line texts are displayed.

[0003] However, when there is a large amount of text in the video content, interference may occur between the texts. For example, non-line texts in variety shows interfere with line texts. Therefore, it is necessary to filter the texts in the video content to display more valuable texts. Before text filtering, it is necessary to determine the scope of each text. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method and device for identifying the position of a text area, an electronic device, and a computer-readable storage medium, which solve the problem of how to determine the scope of the text. The specific technical solutions are as follows:

[0005] In the first aspect of the present invention, a method for identifying the position of a text area is first provided, including: obtaining a frame image to be recognized, where the frame image includes a text area; extracting the feature values of each pixel point in the frame image; calculating the gradient of each pixel point according to the feature values, and identifying the frame image position corresponding to the gradient peak in each gradient from the frame image; counting the number of pixel points corresponding to the gradient peak in the frame image position; and based on the comparison result between the number of pixel points and a preset threshold, confirming the frame image position as the position of the text area.

[0006] Optionally, the identifying the frame image position corresponding to the gradient peak in each gradient from the frame image includes: when the frame image includes at least one text area, for at least one text area with a horizontal text display direction, identifying at least one row position corresponding to the gradient peak in each gradient; when the frame image includes at least one text area, for at least one text area with a vertical text display direction, identifying at least one column position corresponding to the gradient peak in each gradient.

[0007] Optionally, the identifying at least one row position corresponding to a gradient peak in each of the gradients includes: when the gradient peak is a positive gradient peak, identifying the upper boundary of a text area with a horizontal text display direction; when the gradient peak is a negative gradient peak, identifying the lower boundary of a text area with a horizontal text display direction; the identifying at least one column position corresponding to a gradient peak in each of the gradients includes: when the gradient peak is a positive gradient peak, identifying the right boundary of a text area with a vertical text display direction; when the gradient peak is a negative gradient peak, identifying the left boundary of a text area with a vertical text display direction.

[0008] Optionally, the counting the number of pixel points corresponding to a gradient peak in the frame image position includes: counting the number of pixel points within a preset surrounding area of the pixel points corresponding to the gradient peak.

[0009] Optionally, the confirming that the frame image position is the position of the text area based on the comparison result between the number of pixel points and a preset threshold includes: if the comparison result indicates that the number of pixel points is greater than the preset threshold, then confirming that the frame image position is the position of the text area.

[0010] Optionally, the confirming that the frame image position is the position of the text area based on the comparison result between the number of pixel points and a preset threshold includes: calculating the difference between the number of pixel points and the number of pixel points corresponding to other frame image positions in the frame image; if the comparison result indicates that the difference is greater than the preset threshold, then confirming that the frame image position is the position of the text area containing single-line text or single-column text; if the comparison result indicates that the difference is less than or equal to the preset threshold, then confirming that the frame image position is the position of the text area containing multi-line text or multi-column text.

[0011] Optionally, the extracting the feature value of each pixel point in the frame image includes: extracting the gray-scale feature value of each pixel point in the frame image.

[0012] In a second aspect of the implementation of the present invention, there is also provided a device for identifying the position of a text area, including: an image acquisition module for acquiring a frame image to be identified, the frame image containing a text area; a feature value extraction module for extracting the feature value of each pixel point in the frame image; a position identification module for calculating the gradient of each pixel point according to the feature value and identifying the frame image position corresponding to a gradient peak in each of the gradients from the frame image; a quantity statistics module for counting the number of pixel points corresponding to the gradient peak in the frame image position; a position confirmation module for confirming that the frame image position is the position of the text area based on the comparison result between the number of pixel points and a preset threshold.

[0013] Optionally, the position recognition module is configured to, when the frame image includes at least one of the text regions, identify at least one row position corresponding to the gradient peak in each of the gradients for at least one of the text regions with a horizontal text display direction; when the frame image includes at least one of the text regions, identify at least one column position corresponding to the gradient peak in each of the gradients for at least one of the text regions with a vertical text display direction.

[0014] Optionally, the position recognition module is configured to, when the gradient peak is a positive gradient peak, identify the upper boundary of the text region with a horizontal text display direction; when the gradient peak is a negative gradient peak, identify the lower boundary of the text region with a horizontal text display direction; the position recognition module is configured to, when the gradient peak is a positive gradient peak, identify the right boundary of the text region with a vertical text display direction; when the gradient peak is a negative gradient peak, identify the left boundary of the text region with a vertical text display direction.

[0015] Optionally, the quantity statistics module is configured to count the number of pixel points within a preset surrounding region of the pixel points corresponding to the gradient peak.

[0016] Optionally, the position confirmation module is configured to, if the comparison result indicates that the number of pixel points is greater than the preset threshold, confirm that the position of the frame image is the position of the text region.

[0017] Optionally, the position confirmation module includes: a difference calculation module configured to calculate the difference between the number of pixel points and the number of pixel points corresponding to other frame image positions in the frame image; a text position confirmation module configured to, if the comparison result indicates that the difference is greater than the preset threshold, confirm that the position of the frame image is the position of the text region including single-line text or single-column text; if the comparison result indicates that the difference is less than or equal to the preset threshold, confirm that the position of the frame image is the position of the text region including multi-line text or multi-column text.

[0018] Optionally, the eigenvalue extraction module is configured to extract the gray-scale eigenvalue of each pixel point in the frame image.

[0019] In another aspect of the implementation of the present invention, there is also provided a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to execute the position recognition method of the text region described in any one of the above.

[0020] In another aspect of the implementation of the present invention, there is also provided a computer program product including instructions that, when run on a computer, cause the computer to execute the position recognition method of the text region described in any one of the above.

[0021] The position recognition solution for the text area provided by the embodiments of the present invention extracts the feature values of each pixel point from the frame image containing the text area, then calculates the gradient of each pixel point according to the feature values, and identifies the position of the frame image corresponding to the gradient peak among each gradient from the frame image, and then counts the number of pixel points corresponding to the gradient peak in the frame image position. Based on the comparison result between the number of pixel points and the preset threshold, the technical means of confirming that the frame image position is the position of the text area is realized. Based on the feature values of the pixel points in the frame image, the row position or column position of the text area is initially determined, and then based on the comparison result between the number of pixel points corresponding to the row position or column position and the preset threshold, it is finally confirmed that the frame image position is the position of the text area, that is, the range of the text area is finally confirmed, which can provide a filtering basis for filtering the text in the subsequent text area, and can solve the technical problem that the range of each text area cannot be determined before the existing text filtering, achieving the effect of narrowing the range of text filtering, reducing the calculation amount of text filtering, and optimizing the text filtering effect. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art.

[0023] Figure 1 It is a step flowchart of a method for recognizing the position of a text area according to an embodiment of the present invention.

[0024] Figure 2 It is a step flowchart of a method for preprocessing the line filtering in the variety show video data according to an embodiment of the present invention.

[0025] Figure 3 It is a schematic structural diagram of a device for recognizing the position of a text area according to an embodiment of the present invention.

[0026] Figure 4 It is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0027] Figure 5 It is a schematic working process diagram of a recognition system for line subtitles in an audio-visual file according to an embodiment of the present invention. Detailed Embodiments

[0028] The following will describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention.

[0029] An embodiment of the present invention proposes a position recognition solution for a text region. First, the feature values of each pixel point in the frame image containing the text region are extracted. Then, the gradients of each pixel point are calculated based on the feature values, and the position of the frame image corresponding to the gradient peak is recognized from the frame image. Furthermore, the number of pixel points corresponding to the gradient peak in the frame image is counted. Finally, based on the comparison result between the number of pixel points and a preset threshold, the position of the frame image is confirmed as the position of the text region, providing a filtering basis for filtering the text within the text region in the subsequent process.

[0030] As Figure 1 shown, a flowchart of the steps of a method for recognizing the position of a text region according to an embodiment of the present invention is shown. This method for recognizing the position of a text region can be applied to a terminal or a server. The method for recognizing the position of a text region specifically may include the following steps.

[0031] Step 101, obtain a frame image to be recognized.

[0032] In an embodiment of the present invention, the frame image may be sourced from video data or audio - video data. In practical applications, the frame image may be a frame image of video data or audio - video data. Moreover, the frame image may contain a text region. The text region can be understood as the region where the text content in a frame image is located. In one embodiment, the frame image may contain one text region or multiple text regions.

[0033] Step 102, extract the feature values of each pixel point in the frame image.

[0034] In an embodiment of the present invention, the frame image may be input into a pre - trained network model, and then the network model is used to extract the feature values of each pixel point in the frame image.

[0035] Step 103, calculate the gradients of each pixel point according to the feature values, and recognize the position of the frame image corresponding to the gradient peak among each gradient from the frame image.

[0036] In an embodiment of the present invention, the gradients of each pixel point in the positive direction and the negative direction can be calculated according to the feature values of each pixel point. Then, the gradient peaks in their respective directions are determined, and then the position of the frame image corresponding to the gradient peak is recognized from the frame image.

[0037] Step 104, count the number of pixel points corresponding to the gradient peak in the frame image position.

[0038] In an embodiment of the present invention, the number of pixel points within a preset surrounding area centered on the pixel point corresponding to the gradient peak in the frame image position can be counted.

[0039] Step 105: Based on the comparison result between the number of pixel points and a preset threshold, confirm that the position of the frame image is the position of the text area.

[0040] In an embodiment of the present invention, if it is confirmed that the position of the identified frame image is the position of the text area, the position of the identified frame image needs to meet certain basic conditions. The basic condition is that the number of pixel points corresponding to the gradient peak in the position of the frame image needs to be greater than a certain quantity threshold. That is to say, when the comparison result indicates that the number of pixel points corresponding to the gradient peak in the position of the frame image is greater than the preset threshold, confirm that the position of the frame image is the position of the text area.

[0041] The position recognition solution for the text area provided by the embodiment of the present invention extracts the feature values of each pixel point from the frame image containing the text area, then calculates the gradient of each pixel point according to the feature values, and identifies the position of the frame image corresponding to the gradient peak in each gradient from the frame image, and then counts the number of pixel points corresponding to the gradient peak in the position of the frame image. Based on the comparison result between the number of pixel points and the preset threshold, the technical means of confirming that the position of the frame image is the position of the text area is adopted. It realizes initially determining the row position or column position of the text area based on the feature values of the pixel points in the frame image, and then finally confirming that the position of the frame image is the position of the text area, that is, finally confirming the range of the text area, based on the comparison result between the number of pixel points corresponding to the row position or column position and the preset threshold. It can provide a filtering basis for filtering the text within the subsequent text area, and can solve the technical problem that the range of each text area cannot be determined before the existing text filtering, achieving the effect of narrowing the range of text filtering, reducing the calculation amount of text filtering, and optimizing the text filtering effect.

[0042] In an exemplary embodiment of the present invention, the frame image may contain at least one text area, and moreover, the text display direction in the text area can generally be divided into two directions: horizontal and vertical. Therefore, one implementation manner of identifying the position of the frame image corresponding to the gradient peak in each gradient from the frame image is that when the frame image contains at least one text area, for at least one text area with a horizontal text display direction, identify at least one row position corresponding to the gradient peak in each gradient. It should be noted that the row position in this implementation manner can be a horizontal row position or a row position with a certain angle to the horizontal row position. Another implementation manner of identifying the position of the frame image corresponding to the gradient peak in each gradient from the frame image is that when the frame image contains at least one text area, for at least one text area with a vertical text display direction, identify at least one column position corresponding to the gradient peak in each gradient. It should be noted that the column position in this implementation manner can be a vertical column position or a column position with a certain angle to the vertical column position.

[0043] In an exemplary embodiment of the present invention, since the gradient is a vector, it includes the gradient in the positive direction and the gradient in the negative direction. In practical applications, if the text display direction in the text area is horizontal, one implementation of identifying at least one line position corresponding to the gradient peak among each gradient is that when the gradient peak is a positive gradient peak, the upper boundary of the text area with a horizontal text display direction is identified; when the gradient peak is a negative gradient peak, the lower boundary of the text area with a horizontal text display direction is identified. If the text display direction in the text area is vertical, one implementation of identifying at least one column position corresponding to the gradient peak among each gradient is that when the gradient peak is a positive gradient peak, the right boundary of the text area with a vertical text display direction is identified; when the gradient peak is a negative gradient peak, the left boundary of the text area with a vertical text display direction is identified.

[0044] In an exemplary embodiment of the present invention, one implementation of counting the number of pixel points corresponding to the gradient peak in the frame image position is to count the number of pixel points in the preset surrounding area of the pixel points corresponding to the gradient peak in the frame image position. As can be seen from the above, the frame image position can be the upper boundary, lower boundary, left boundary or right boundary of the text area. For example, the frame image position U is the upper boundary of the text area. For the pixel point d corresponding to the gradient peak of the frame image position U, within the preset surrounding area [d xy -dis, d xy +dis] of the pixel point d, the number of pixel points UX is counted. Wherein, d xy represents the abscissa value and ordinate value of the pixel point d, and dis represents the preset coordinate threshold.

[0045] In an exemplary embodiment of the present invention, one implementation of confirming the position of the frame image as the position of the text area based on the comparison result between the number of pixel points and the preset threshold is that if the comparison result indicates that the number of pixel points is greater than the preset threshold, it is confirmed that the frame image position is the position of the text area; if the comparison result indicates that the number of pixel points is less than or equal to the preset threshold, it is confirmed that the frame image position is not the position of the text area.

[0046] The above preset threshold can be an empirically set value, or the above preset threshold can be the difference between the number of pixel points corresponding to the gradient peak in the current frame image position and the number of pixel points corresponding to the gradient peak in the first or previous frame image position after sorting the number of pixel points corresponding to the gradient peak in each frame image position.

[0047] In an exemplary embodiment of the present invention, in addition to being able to confirm that the position of the frame image is the position of the text area, it is also possible to further confirm whether the text area contains single-line text or single-column text, or multi-line text or multi-column text. In practical applications, the difference between the number of pixel points corresponding to the current frame image position and the number of pixel points corresponding to other frame image positions can be calculated, and then the difference is compared with a preset threshold to obtain a comparison result. If the comparison result indicates that the difference is greater than the preset threshold, it is confirmed that the text area contains single-line text or single-column text, and the current frame image position is confirmed as the position of the text area containing single-line text or single-column text. If the comparison result indicates that the difference is less than or equal to the preset threshold, it is confirmed that the text area contains multi-line text or multi-column text, and the current frame image position is confirmed as the position of the text area containing multi-line text or multi-column text.

[0048] In an exemplary embodiment of the present invention, one implementation manner of extracting the feature values of each pixel point in the frame image is to extract the gray-scale feature values of each pixel point in the frame image. In practical applications, the frame image can be a grayscale image or an RGB image. If the frame image is an RGB image, it can be further converted into a grayscale image, and then the gray-scale feature values of each pixel point in the frame image are extracted.

[0049] Based on the above relevant description of the embodiment of the method for identifying the position of the text area, a method for preprocessing the filtering of lines in variety show video data will be introduced below. As Figure 2 shown, Figure 2 shows a flowchart of the steps of a method for preprocessing the filtering of lines in variety show video data.

[0050] Obtain a text detection image containing a text area. The text detection image can be the frame image in the above embodiments. The text detection image contains a line text area. The purpose of this line filtering preprocessing method is to identify the approximate range of the line text area in the text detection image, providing a filtering basis for subsequent line filtering. It should be noted that the text display direction within the line text area in the text detection image is horizontal. Obtain the grayscale feature values of each pixel point in the text detection image, calculate the gradient of each pixel point based on the grayscale feature values, and then identify multiple frame image positions from the text detection image. Specifically, the multiple frame image positions can correspond to the upper boundary, lower boundary, and the row where the midpoint of each line text area is located. If the frame image positions B1, B2, and B3 are obtained through the above identification, then respectively count the number of pixel points Bs1, Bs2, and Bs3 corresponding to the gradient peaks at the frame image positions B1, B2, and B3. Respectively compare the pixel point numbers Bs1, Bs2, and Bs3 with a preset threshold BM to obtain the respective comparison results D1, D2, and D3. If the comparison result D1 indicates that Bs1 is greater than BM, then confirm that the frame image position B1 is the position of the line text area; if the comparison result D2 indicates that Bs2 is greater than BM, then confirm that the frame image position B2 is the position of the line text area; if the comparison result D3 indicates that Bs3 is less than BM, then confirm that the frame image position B3 is not the position of the line text area. Finally, the position of the line text area in the text detection image is determined by the frame image positions B1 and B2. If the ordinate of the frame image position B1 is greater than the ordinate of the frame image position B2, then the frame image position B1 is the upper boundary of the line text area, and the frame image position B2 is the lower boundary of the line text area.

[0051] As Figure 3 shown, a schematic structural diagram of a position recognition device for a text area according to an embodiment of the present invention is shown. The position recognition device for the text area may include the following modules.

[0052] An image acquisition module 31, configured to acquire a frame image to be recognized, where the frame image contains a text area;

[0053] A feature value extraction module 32, configured to extract the feature values of each pixel point in the frame image;

[0054] A position recognition module 33, configured to calculate the gradient of each pixel point according to the feature value, and identify the frame image position corresponding to the gradient peak in each of the gradients from the frame image;

[0055] A quantity statistics module 34, configured to count the number of pixel points corresponding to the gradient peak in the frame image position;

[0056] A position confirmation module 35, configured to confirm that the position of the frame image is the position of the text area based on the comparison result between the number of pixel points and a preset threshold.

[0057] In an exemplary embodiment of the present invention, the position recognition module 33 is configured to, when the frame image includes at least one of the text areas, for at least one of the text areas with a horizontal text display direction, recognize at least one row position corresponding to the gradient peak in each of the gradients; when the frame image includes at least one of the text areas, for at least one of the text areas with a vertical text display direction, recognize at least one column position corresponding to the gradient peak in each of the gradients.

[0058] In an exemplary embodiment of the present invention, the position recognition module 33 is configured to, when the gradient peak is a positive gradient peak, recognize the upper boundary of the text area with a horizontal text display direction; when the gradient peak is a negative gradient peak, recognize the lower boundary of the text area with a horizontal text display direction;

[0059] The position recognition module 33 is configured to, when the gradient peak is a positive gradient peak, recognize the right boundary of the text area with a vertical text display direction; when the gradient peak is a negative gradient peak, recognize the left boundary of the text area with a vertical text display direction.

[0060] In an exemplary embodiment of the present invention, the quantity statistics module 34 is configured to count the number of pixel points in a preset surrounding area of the pixel points corresponding to the gradient peak.

[0061] In an exemplary embodiment of the present invention, the position confirmation module 35 is configured to, if the comparison result indicates that the number of pixel points is greater than the preset threshold, confirm that the position of the frame image is the position of the text area.

[0062] In an exemplary embodiment of the present invention, the position confirmation module 35 includes:

[0063] A difference calculation module, configured to calculate the difference between the number of pixel points and the number of pixel points corresponding to other frame image positions in the frame image;

[0064] A text position confirmation module, configured to, if the comparison result indicates that the difference is greater than the preset threshold, confirm that the position of the frame image is the position of the text area including single-line text or single-column text; if the comparison result indicates that the difference is less than or equal to the preset threshold, confirm that the position of the frame image is the position of the text area including multi-line text or multi-column text.

[0065] In an exemplary embodiment of the present invention, the eigenvalue extraction module 32 is configured to extract the grayscale eigenvalue of each pixel point in the frame image.

[0066] An embodiment of the present invention also provides an electronic device, as Figure 4 shown, including a processor 41, a communication interface 42, a memory 43, and a communication bus 44. Among them, the processor 41, the communication interface 42, and the memory 43 complete mutual communication through the communication bus 44.

[0067] The memory 43 is used to store computer programs;

[0068] When the processor 41 is used to execute the program stored in the memory 43, the following steps are implemented:

[0069] Obtain a frame image to be recognized, where the frame image includes a text area;

[0070] Extract the eigenvalue of each pixel point in the frame image;

[0071] Calculate the gradient of each pixel point according to the eigenvalue, and identify the position of the frame image corresponding to the gradient peak in each gradient from the frame image;

[0072] Count the number of pixel points corresponding to the gradient peak in the frame image position;

[0073] Based on the comparison result between the number of pixel points and a preset threshold, confirm that the frame image position is the position of the text area.

[0074] The identifying the position of the frame image corresponding to the gradient peak in each gradient from the frame image includes:

[0075] When the frame image includes at least one text area, for at least one text area with a horizontal text display direction, identify at least one row position corresponding to the gradient peak in each gradient;

[0076] When the frame image includes at least one text area, for at least one text area with a vertical text display direction, identify at least one column position corresponding to the gradient peak in each gradient.

[0077] The identifying at least one row position corresponding to the gradient peak in each gradient includes:

[0078] When the gradient peak is a positive gradient peak, identify the upper boundary of the text area with a horizontal text display direction;

[0079] When the gradient peak is a negative gradient peak, identify the lower boundary of the text area with a horizontal text display direction;

[0080] Identifying at least one column position corresponding to the peak value of the gradient in each of the gradients includes:

[0081] When the peak value of the gradient is a positive gradient peak value, identifying the right boundary of the text area where the text display direction is vertical;

[0082] When the peak value of the gradient is a negative gradient peak value, identifying the left boundary of the text area where the text display direction is vertical.

[0083] Counting the number of pixel points corresponding to the peak value of the gradient in the position of the frame image includes:

[0084] Counting the number of pixel points within a preset surrounding area of the pixel points corresponding to the peak value of the gradient.

[0085] Based on the comparison result between the number of pixel points and a preset threshold value, confirming that the position of the frame image is the position of the text area includes:

[0086] If the comparison result indicates that the number of pixel points is greater than the preset threshold value, then confirming that the position of the frame image is the position of the text area.

[0087] Based on the comparison result between the number of pixel points and a preset threshold value, confirming that the position of the frame image is the position of the text area includes:

[0088] Calculating the difference between the number of pixel points and the number of pixel points corresponding to other frame image positions in the frame image;

[0089] If the comparison result indicates that the difference is greater than the preset threshold value, then confirming that the position of the frame image is the position of the text area containing single-line text or single-column text;

[0090] If the comparison result indicates that the difference is less than or equal to the preset threshold value, then confirming that the position of the frame image is the position of the text area containing multi-line text or multi-column text.

[0091] Extracting the feature values of each pixel point in the frame image includes:

[0092] Extracting the gray-scale feature values of each pixel point in the frame image.

[0093] Based on the description of the above embodiments such as the position recognition method and device of the text area, a subtitle recognition system in an audio-visual file is introduced below. This recognition system can be composed of hardware devices such as a personal computer or a server, a server cluster, etc. A subtitle detection framework is deployed in this recognition system, and this subtitle detection framework mainly includes a subtitle detection model, a subtitle filtering model, a subtitle tracking model, and a text classification model.

[0094] Reference Figure 5 As shown in Figure 5 , a schematic diagram of the working process of a subtitle recognition system in an audio - video file according to an embodiment of the present invention is shown. In the actual application process, a complete audio - video file is input into the recognition system. Using the Graphics Processing Unit (GPU) of the hardware device, video frame results are extracted from the audio - video file. The specific video frame results can be partial frame images and full - volume frame images. Among them, the partial frame images can be three frame images extracted from a 1 - second time period of the audio - video file. In order to reduce the computational amount of video frame result extraction, extraction processing can be performed on partial video frame images of the audio - video file. For example, extraction processing is only performed on the lower 1 / 3 part of the video frame image. The extracted partial frame images can be continuously written into a memory queue. The extracted full - volume frame images can be stored on the hard disk.

[0095] Based on multi - threads, multiple subtitle detection models are started to continuously read partial frame images from the memory queue and splice the read - out partial frame images. For example, three consecutive partial frame images are spliced into one frame image. The subtitle detection model locates the positions of all text boxes from the spliced frame image.

[0096] The subtitle filtering model counts the frequencies of all text boxes appearing in the time axis, determines a high - frequency heat - map area according to the frequencies, and then uses this high - frequency heat - map area as the filtering standard area for subtitles and non - subtitles, and filters all text boxes into subtitle text boxes and non - subtitle text boxes using this filtering standard area. Among them, the non - subtitle text boxes are discarded and do not participate in subsequent processing.

[0097] The subtitle tracking model extracts the deep features of Optical Character Recognition (OCR) from the subtitle text boxes and performs tracking processing based on the full - volume frame images stored on the hard disk to obtain the start - end time information of each subtitle text box appearing in the audio - video file.

[0098] The text classification model determines the language information corresponding to each subtitle text box, and transmits the deep features to the corresponding OCR prediction network according to the language information. The OCR prediction network uses its respective text recognition algorithms to recognize the subtitles in the subtitle text boxes to obtain subtitle results. For example, the language information can be Chinese language and English language. The Chinese language corresponds to the Chinese OCR prediction network, and the English language corresponds to the English OCR prediction network.

[0099] The recognition system provided by the embodiments of the present invention constructs a software development kit (SDK) for line subtitle frame-level recognition. For the same audio-visual file, the ratio of the time for recognizing line subtitles on the GPU to the time for recognizing line subtitles through this SDK is approximately 1:0.11, which greatly improves the recognition rate and accuracy of line subtitles. According to the recognized line results, external subtitles for the audio-visual file can be generated, and the embedded subtitles of the audio-visual file can be quickly converted into external subtitles.

[0100] When the recognition system provided by the embodiments of the present invention filters text boxes, it uses the time-domain information of the text boxes to filter the text boxes into line subtitle text boxes and non-line subtitle text boxes, and the accuracy of text box filtering is very high. When the audio-visual file is a film and television work, the filtering accuracy exceeds 99%; when the audio-visual file is a variety show, the filtering accuracy exceeds 98.5%.

[0101] The recognition system provided by the embodiments of the present invention can also extract video frames for specified areas such as name bars, lyric versions, and lyrics in variety shows, and then execute processes such as positioning, filtering, tracking, classification, and recognition of names, lyrics, etc., realizing the character recognition of names, lyrics, etc.

[0102] The recognition system provided by the embodiments of the present invention can not only recognize line subtitles in Chinese and English languages, but also realize the intelligent recognition of line subtitles in Chinese-English, Chinese-Japanese, Chinese-Korean, etc. in bilingual audio-visual files, supporting the recognition of line subtitles in multi-language audio-visual files.

[0103] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0104] The communication interface is used for communication between the above terminal and other devices.

[0105] The memory can include a Random Access Memory (RAM), and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located far from the aforementioned processor.

[0106] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU for short), a Network Processor (NP for short), etc.; it may also be a Digital Signal Processor (DSP for short), an Application Specific Integrated Circuit (ASIC for short), a Field-Programmable Gate Array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0107] In another embodiment provided by the present invention, there is also provided a computer-readable storage medium, in which instructions are stored. When it runs on a computer, it enables the computer to execute the position recognition method of any text area described in the above embodiments.

[0108] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions. When it runs on a computer, it enables the computer to execute the position recognition method of any text area described in the above embodiments.

[0109] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that the computer can access, or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a Solid State Disk (SSD)).

[0110] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0111] Each embodiment in this specification is described in a related manner. For the parts that are the same or similar among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the description of the method embodiment.

[0112] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for identifying the position of a text area, characterized in that, Including: Obtain a frame image to be recognized, where the frame image contains a text area; Extract the feature values of each pixel point in the frame image; Calculate the gradient of each pixel point according to the feature value, and identify the frame image position corresponding to the gradient peak in each of the gradients from the frame image; Count the number of pixel points corresponding to the gradient peak in the frame image position; Based on the comparison result between the number of pixel points and a preset threshold, confirm that the frame image position is the position of the text area; The counting the number of pixel points corresponding to the gradient peak in the frame image position includes: Count the number of pixel points within a preset surrounding area of the pixel points corresponding to the gradient peak.

2. The method according to claim 1, wherein The identifying the frame image position corresponding to the gradient peak in each of the gradients from the frame image includes: When the frame image contains at least one text area, for at least one text area with a horizontal text display direction, identify at least one row position corresponding to the gradient peak in each of the gradients; When the frame image contains at least one text area, for at least one text area with a vertical text display direction, identify at least one column position corresponding to the gradient peak in each of the gradients.

3. The method according to claim 2, characterized in that, The identifying at least one row position corresponding to the gradient peak in each of the gradients includes: When the gradient peak is a positive gradient peak, identify the upper boundary of the text area with a horizontal text display direction; When the gradient peak is a negative gradient peak, identify the lower boundary of the text area with a horizontal text display direction; The identifying at least one column position corresponding to the gradient peak in each of the gradients includes: When the gradient peak is a positive gradient peak, identify the right boundary of the text area with a vertical text display direction; When the gradient peak is a negative gradient peak, identify the left boundary of the text area with a vertical text display direction.

4. The method according to claim 1, characterized in that, The based on the comparison result between the number of pixel points and a preset threshold, confirm that the frame image position is the position of the text area includes: If the comparison result indicates that the number of pixel points is greater than the preset threshold, confirm that the frame image position is the position of the text area.

5. The method according to claim 1, characterized in that, The based on the comparison result between the number of pixel points and a preset threshold, confirm that the frame image position is the position of the text area includes: Calculate the difference between the number of pixel points and the number of pixel points corresponding to other frame image positions in the frame image; If the comparison result indicates that the difference is greater than the preset threshold, confirm that the frame image position is the position of the text area containing single-line text or single-column text; If the comparison result indicates that the difference is less than or equal to the preset threshold, confirm that the frame image position is the position of the text area containing multi-line text or multi-column text.

6. The method according to claim 1, wherein The extracting the feature values of each pixel point in the frame image includes: Extract the gray-scale feature values of each pixel point in the frame image.

7. A position recognition device for a text area, characterized in that Including: An image acquisition module for acquiring a frame image to be recognized, where the frame image contains a text area; A feature value extraction module for extracting the feature values of each pixel point in the frame image; A position recognition module, configured to calculate gradients of each of the pixel points according to the eigenvalue, and recognize a frame image position corresponding to a gradient peak among the gradients from the frame image; A quantity statistics module, configured to count the number of pixel points corresponding to the gradient peaks in the frame image positions; A position confirmation module, configured to confirm the frame image position as the position of the text area based on a comparison result between the number of pixel points and a preset threshold; Specifically, the quantity statistics module is configured to count the number of pixel points within a preset surrounding area of the pixel points corresponding to the gradient peaks.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store a computer program; The processor is configured to implement the method steps described in any one of claims 1-6 when executing the program stored on the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1-6.

Images