Video subtitle detection method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the field of computer technology, and provides a video subtitle detection method, apparatus, terminal device and storage medium, wherein the method comprises: obtaining the pixel difference between two adjacent frames of images in the video data; when the pixel difference is greater than a first preset threshold, edge detection is performed on the target frame image in the two adjacent frames of images; wherein the target frame image is the latter frame image in the two adjacent frames of images; obtaining the connected area in the target frame image after edge detection; and filtering the connected area. In the present application, when the pixel difference between two adjacent frames of images in the video data is greater than a first preset threshold, the latter frame image in the two adjacent frames of images is used as the target frame image, and the target frame image is first edge detected, then the connected area is detected, and after the detected connected area is filtered, the filtered connected area is used as the subtitle area, so that the subtitle area in the video data can be detected quickly and accurately.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a video subtitle detection method, apparatus, terminal device and storage medium. Background Art

[0002] With the continuous development of computer and multimedia technology, video has become an important carrier of information dissemination. Various types of subtitles are often superimposed on the disseminated videos through post-production. When the video needs to be processed, it is sometimes necessary to detect the subtitle area of ​​the video.

[0003] The current video subtitle detection technology usually detects the subtitles of each frame of the video, but the detection speed and accuracy are not high. Summary of the invention

[0004] The embodiments of the present application provide a video subtitle detection method, apparatus, terminal device and storage medium, which can quickly and accurately detect subtitle areas in video data.

[0005] In a first aspect, an embodiment of the present application provides a video subtitle detection method, comprising:

[0006] Obtain the pixel difference between two adjacent frames of images in the video data;

[0007] When the pixel difference is greater than a first preset threshold, edge detection is performed on a target frame image in the two adjacent frame images; wherein the target frame image is the latter frame image in the two adjacent frame images;

[0008] Acquire a connected area in the target frame image after edge detection;

[0009] The connected regions are screened, and the screened connected regions are used as subtitle regions.

[0010] In a second aspect, an embodiment of the present application provides a video subtitle detection device, including:

[0011] A first acquisition module is used to acquire the pixel difference between two adjacent frames of images in the video data;

[0012] A detection module, configured to perform edge detection on a target frame image in the two adjacent frame images when the pixel difference is greater than a first preset threshold; wherein the target frame image is the latter frame image in the two adjacent frame images;

[0013] A second acquisition module, used for acquiring a connected area in the target frame image after edge detection;

[0014] The screening module is used to screen the connected areas and use the screened connected areas as subtitle areas.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned video subtitle detection method when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video subtitle detection method are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of the above-mentioned video subtitle detection method.

[0018] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application can obtain the pixel difference between two adjacent frames of images in a video, and when the pixel difference between the two adjacent frames of images is large, that is, when the change between the two adjacent frames of images is relatively large, edge detection is performed on the target frame image in the two adjacent frames of images; wherein the target frame image is the latter frame image in the two adjacent frames of images; the connected area in the target frame image after edge detection is obtained; the connected area is screened, and the screened connected area is used as the subtitle area. Since when the pixel difference between the two adjacent frames of images in the video data is greater than the first preset threshold, the latter frame image in the two adjacent frames of images is used as the target frame image, and the target frame image is first edge detected, then the connected area is detected, and after the detected connected area is screened, the screened connected area is used as the subtitle area, so that the subtitle area in the video data can be quickly and accurately detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a flowchart of a video subtitle detection method provided in Example 1 of the present application;

[0021] Figure 2 It is a flowchart of a video subtitle detection method provided in Embodiment 2 of the present application;

[0022] Figure 3 It is a flowchart of a video subtitle detection method provided in Embodiment 3 of the present application;

[0023] Figure 4 is a structural diagram of a video subtitle detection device provided in Embodiment 4 of the present application;

[0024] Figure 5 It is a structural diagram of the terminal device provided in Example 5 of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] The video subtitle detection method provided in the embodiment of the present application can be applied to terminal devices such as servers, mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiment of the present application does not impose any restrictions on the specific types of terminal devices.

[0032] In order to illustrate the technical solution described in this application, the following embodiments are used for illustration.

[0033] Embodiment 1

[0034] See also Figure 1 , a video subtitle detection method provided by an embodiment of the present application includes:

[0035] Step S101, obtaining the pixel difference between two adjacent frames of images in the video data.

[0036] In applications, a video is composed of multiple frame images in sequence. When multiple frames are played at a rate exceeding a certain number per second, the human eye cannot distinguish that a single static image is being played, and the visual effect looks smooth and continuous, thus forming a video. When detecting subtitles in a certain video, the pixel change difference between two adjacent frames in the video data is obtained.

[0037] In one embodiment, obtaining the pixel difference between two adjacent frames of images in video data includes: calculating a first average pixel value of a previous frame of images among the two adjacent frames of images; calculating a second average pixel value of a subsequent frame of images among the two adjacent frames of images, and determining the pixel difference between the two adjacent frames of images based on a first difference between the first average pixel value and the second average pixel value.

[0038] In the application, the average pixel value of all pixels in the previous frame of two adjacent frames can be calculated and used as the first average pixel value; the average pixel value of all pixels in the next frame of two adjacent frames can be calculated and used as the second average pixel value. The difference between the first average pixel value and the second average pixel value is used as the pixel difference between the two adjacent frames, and the pixel difference can reflect the pixel change between the two adjacent frames.

[0039] Step S102: when the pixel difference is greater than a first preset threshold, edge detection is performed on a target frame image in the two adjacent frame images; wherein the target frame image is the latter frame image in the two adjacent frame images.

[0040] In the application, when the pixel difference between two adjacent frames of images in the video data is greater than a first preset threshold, it indicates that a relatively large change has occurred between the two adjacent frames of images. At this time, the position of the subtitle area in the video is likely to change. Edge detection is performed on the next frame of the two adjacent frames of images that have undergone a relatively large change to obtain the position of the subtitle area in the next frame of the two adjacent frames of images. For ease of understanding, the next frame of the two adjacent frames of images is defined as the target frame image.

[0041] In one embodiment, the performing edge detection on the target frame image in the two adjacent frames includes: performing edge detection on the target frame image in the two adjacent frames according to a Sobel edge detection algorithm. Of course, edge detection can also be achieved by a Laplacian edge detection algorithm or a Canny edge detection algorithm.

[0042] Step S103, obtaining the connected area in the target frame image after edge detection.

[0043] In the application, the target frame image after edge detection can be understood as the edge image obtained after edge extraction of the target frame image, that is, the edge image of the target frame image mentioned above. The edge is where the regional attributes suddenly change, where the uncertainty in the image is the greatest, and where the image information is most concentrated. The edge of the image contains rich information. Since subtitles generally have a high contrast characteristic with the background in order to distinguish them from the background, this characteristic is manifested on the edge, that is, the subtitle area has rich edge characteristics. Therefore, the position of the subtitle area in the target frame image can be determined based on the connected area of ​​the edge image of the target frame image detected. Obtaining the connected area in the target frame image after edge detection can be: first, according to the connected area marking algorithm, the connected area detection is performed on the target frame image after edge detection, and then the position information of the detected connected area is obtained.

[0044] In one embodiment, the obtaining of the connected area in the target frame image after edge detection includes: preprocessing the target frame image after edge detection; wherein the preprocessing is used to denoise the target frame image after edge detection; and obtaining the connected area in the target frame image after edge detection and the preprocessing.

[0045] In one embodiment, obtaining the connected area in the target frame image after edge detection and preprocessing includes: obtaining areas with the same pixel values ​​and adjacent positions in the target frame image after edge detection and preprocessing.

[0046] In the application, before the connected area detection is performed on the edge image of the target frame image, in order to suppress the background area noise and highlight the text, preprocessing can be performed to remove the background noise and highlight the text. The preprocessing can be performed by morphological filtering algorithm. The edge image of the target frame image is preprocessed by morphological filtering algorithm, which is to apply morphology to the edge image to eliminate noise. Specifically, mathematical morphology can be used as a tool to extract edge image components useful for expressing and depicting the shape of the region from the image, and the structural element method is used to realize its edge detection, which can distinguish the noise and edge from the morphology, thereby suppressing the noise in the background area and highlighting the text.

[0047] In one embodiment, the preprocessing of the target frame image after edge detection includes: preprocessing the target frame image after edge detection through a morphological filtering algorithm; wherein the morphological filtering algorithm includes a morphological filtering algorithm based on an opening operation, a morphological filtering algorithm based on a closing operation, or a morphological filtering algorithm based on a morphological gradient.

[0048] In applications, a morphological filtering algorithm based on a closing operation may be used for preprocessing, wherein the closing operation performs an operation of first dilation and then erosion through a preset structural element.

[0049] Step S104, screening the connected areas, and using the screened connected areas as subtitle areas.

[0050] In the application, the detected connected areas are screened, specifically, they can be screened according to the characteristics of the subtitle area, and / or according to the pixel difference between the target frame image and the connected area in the next frame image, and the screened connected areas are used as subtitle areas.

[0051] In one embodiment, after the connected areas are screened and the screened connected areas are used as subtitle areas, the method further includes: obtaining and outputting position information of the subtitle areas in the target frame image, and returning to execute the step of obtaining the pixel difference between two adjacent frame images in the video data until the position information of all subtitle areas in the video data is output.

[0052] In the application, after connected area detection and screening of the edge image of the target frame image, the screened connected area is used as the subtitle area, and the position information of the subtitle area is obtained. The obtained position is the position information of the subtitle area in the target frame image, and then the position information of the subtitle area in the target frame image can be obtained, and the position information is output, and the above step S101 is continued to execute the pixel difference of the next two adjacent frames of images in the video data and the subsequent steps, and the next two adjacent frames of images in the video are continued to perform corresponding processing until the above step of obtaining the pixel difference of the next two adjacent frames of images in the video data and the subsequent steps are completed for the last two frames of images in the video data, and the position information of all subtitle areas in the video data is output.

[0053] In a specific application, outputting the position information of all subtitle areas in the video data can be understood as outputting the subtitle areas in all target frame images and the subtitle areas in all intermediate video frames, wherein the intermediate video frame is a video frame between two adjacent target frame images. The position of the subtitle area of ​​the previous target frame image in the two adjacent target frame images corresponding to the intermediate video frame is used as the position of the subtitle area in the intermediate video frame.

[0054] It can be seen that in the embodiment of the present application, when the pixel difference between two adjacent frame images in the video data is greater than the first preset threshold value, the latter frame image of the two adjacent frame images is used as the target frame image, and the target frame image is first edge detected, then the connected area is detected, and after the detected connected area is filtered, the filtered connected area is used as the subtitle area, so that the subtitle area in the video data can be detected quickly and accurately.

[0055] Embodiment 2

[0056] The embodiment of the present application provides a video subtitle detection method. The video subtitle detection of this embodiment includes the steps in the first embodiment. For the same or similar parts as the first embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here. Figure 2 As shown, the edge detection of the target frame image in the two adjacent frame images in the above step S102 specifically includes:

[0057] Step S201 , respectively calculating the gradient value of each pixel point in the target frame image and the next two frames of the target frame image.

[0058] In the application, the gradient value of each pixel in the target frame image is calculated, and the gradient value of each pixel in the last two frames of the target frame image is calculated respectively, that is, the corresponding gradient value is calculated for each pixel in the three images, so there are three gradient values ​​for the same position of the three images, and the gradient can be used to indicate the speed of the change of the pixel at the position relative to the adjacent position, and the magnitude of the gradient value specifically indicates the degree of change, so the three gradient values ​​respectively indicate the degree of change of the pixel at the same position of the three images, because when the pixel difference between two consecutive frames in the video is greater than the first threshold, the target frame image is the latter frame of the two consecutive frames, which can be understood as the video screen sending a sudden change, at this time, in order to meet the principle of visual persistence, 30 to 60 frames after the target frame image will be video frames with a relatively large correlation with the target frame, and the correlation between the target frame image and the last two frames of the target frame image is relatively large, and the median of the three gradients corresponding to the target frame image and the last two frames of the target frame image can more stably represent the rate of change of the image edge, which can prevent inaccurate edge detection of a certain frame image due to jitter or noise, so that the target frame image can be more accurately determined for edge detection.

[0059] Step S202, obtaining the median of the gradient values ​​of the pixel points at the same position in the target frame image and the next two frame images as the pixel value of the pixel point at the corresponding position in the target frame image after edge detection.

[0060] In the application, the median of the gradient values ​​of the pixels at the same position in the target frame image and the two frames after the target frame image is obtained. Each pixel at the same position will have three gradient values ​​corresponding to it, and the median of these three gradient values ​​is taken as the pixel value of the pixel at the corresponding position of the edge image of the target frame image. After calculating the gradient value of each pixel in the target frame image and the two frames after the target frame image, three gradient values ​​corresponding to each pixel will be obtained, and the gradient value of the middle size of the three gradient values ​​corresponding to each pixel is taken as the pixel value of the pixel at the corresponding position of the target frame image after edge detection.

[0061] It can be seen that in the embodiment of the present application, the gradient values ​​of each pixel point in the target frame image and the two frames after the target frame image are calculated respectively, and the median of the gradient values ​​of the pixel points at the same position in the target frame image and the two frames after the target frame image is obtained, which is used as the pixel value of the pixel point at the corresponding position in the target frame image after edge detection, so that edge detection can be accurately performed on the target frame image in the two adjacent frames.

[0062] Embodiment 3

[0063] The embodiment of the present application provides a video subtitle detection method. The video subtitle detection of this embodiment includes the steps in Embodiment 1 and / or Embodiment 2. For details of the same or similar parts as those in Embodiment 1 and / or Embodiment 2, please refer to the relevant description of Embodiment 1 and / or Embodiment 2, which will not be repeated here. Figure 3 As shown, the above step S104 specifically includes:

[0064] Step S301 : screening out connected regions whose sizes are within a preset size range from the connected regions to obtain target connected regions.

[0065] In applications, in order to meet the visual requirements of the human eye, the size of the subtitles in the video is usually not less than a certain minimum value, nor more than half of the video frame height, and based on the structural characteristics of the text itself, it is also necessary to meet certain aspect ratio constraints. Therefore, according to the preset size range attribute and aspect ratio value range attribute of the text attribute, the connected areas that meet the preset size range attribute and aspect ratio value range attribute are screened out from the detected connected areas to obtain the target connected area;

[0066] Step S302: Obtain a third average pixel value of all pixels in the target frame image located in the connected area.

[0067] In the application, the average pixel value of all pixels in the connected area in the target frame image is calculated as the third average pixel value. When there are multiple connected areas, the average pixel value of all pixels in each connected area is calculated respectively.

[0068] Step S303, obtaining a fourth average pixel value corresponding to the target connected area in a frame image subsequent to the target frame image.

[0069] In the application, the average pixel value of all pixels in the connected area in the next frame image of the target frame image is calculated as the fourth average pixel value. Similarly, when there are multiple connected areas, the average pixel value of all pixels in each connected area is calculated respectively.

[0070] Step S304: According to the second difference between the third average pixel value and the fourth average pixel value, a region in the first connected region where the second difference is less than a preset second threshold is selected as a subtitle region.

[0071] In applications, in order to meet visual characteristics, the text superimposed on the video later is generally kept unchanged in 30 to 60 consecutive frames or more. Therefore, the connected areas that are not subtitle areas can be further eliminated based on the pixel difference between the position of the connected area corresponding to the target frame image and the frame image after the target frame image. From the first connected area, the area with the second difference less than the preset second threshold is selected as the subtitle area, which can more accurately detect the subtitle area, thereby obtaining the position of the subtitle area more accurately.

[0072] It can be seen that in the embodiment of the present application, the connected area is screened twice, and the screened connected area is used as the subtitle area, which can more accurately detect the subtitle area.

[0073] Embodiment 4

[0074] Corresponding to the subtitle detection method described in the above embodiment, Figure 4 The structural block diagram of the subtitle detection device provided in the embodiment of the present application is shown. For the convenience of explanation, only the part related to the embodiment of the present application is shown.

[0075] Reference Figure 4 , the video subtitle detection device 400 includes:

[0076] The first acquisition module 401 is used to acquire the pixel difference between two adjacent frames of images in the video data;

[0077] A detection module 402, configured to perform edge detection on a target frame image in the two adjacent frame images when the pixel difference is greater than a first preset threshold; wherein the target frame image is the latter frame image in the two adjacent frame images;

[0078] A second acquisition module 403 is used to acquire a connected area in the target frame image after edge detection;

[0079] A screening module 404 is used to screen the connected areas and use the screened connected areas as subtitle areas;

[0080] In one embodiment, the video subtitle detection apparatus 400 further includes:

[0081] The output module is used to obtain and output the position information of the subtitle area in the target frame image, and return to execute the step of obtaining the pixel difference between two adjacent frame images in the video data until the position information of all subtitle areas in the video data is output.

[0082] In one embodiment, the first acquisition module 401 includes:

[0083] A first calculation unit, used for calculating a first average pixel value of a previous frame of the image in the two adjacent frames of the image;

[0084] A second calculation unit, used for calculating a second average pixel value of a later frame of the two adjacent frames of images;

[0085] A determining unit is used to determine the pixel difference between the two adjacent frames of images according to a first difference between the first average pixel value and the second average pixel value.

[0086] In one embodiment, the detection module 402 is specifically used to: when the pixel difference is greater than a first preset threshold, respectively calculate the gradient value of each pixel point in the target frame image and the next two frames of the target frame image;

[0087] The median of the gradient values ​​of the pixel points at the same position in the target frame image and the latter two frame images is obtained as the pixel value of the pixel point at the corresponding position in the target frame image.

[0088] In one embodiment, the detection module 402 is specifically configured to perform edge detection on the target frame image in the two adjacent frame images according to a Sobel edge detection algorithm.

[0089] In one embodiment, the second acquisition module 403 includes:

[0090] A preprocessing unit, used for preprocessing the target frame image after edge detection;

[0091] The connected region acquisition unit is used to acquire the connected region in the target frame image after edge detection and preprocessing.

[0092] In one embodiment, the obtaining of connected areas in the target frame image after edge detection and preprocessing includes: obtaining areas with the same pixel values ​​and adjacent positions in the target frame image after edge detection and preprocessing.

[0093] In one embodiment, the preprocessing unit is specifically used to: preprocess the target frame image after edge detection by using a morphological filtering algorithm.

[0094] In one embodiment, the screening module 404 includes:

[0095] A first screening unit, used to screen out connected areas whose sizes are within a preset size range from the connected areas to obtain target connected areas;

[0096] A first acquisition unit, used to acquire a third average pixel value of all pixel points located in the target connected area in the target frame image;

[0097] A second acquisition unit, used for acquiring a fourth average pixel value corresponding to the target connected area in a frame image subsequent to the target frame image;

[0098] The second screening unit is used to screen out an area in the first connected area where the second difference is less than a preset second threshold value according to a second difference between the third average pixel value and the fourth average pixel value as a subtitle area.

[0099] It can be seen that in the embodiment of the present application, when the pixel difference between two adjacent frame images in the video data is greater than the first preset threshold, the latter frame image of the two adjacent frame images is used as the target frame image, and the target frame image is first edge detected, then the connected area is detected, and after the detected connected area is filtered, the filtered connected area is used as the subtitle area, so that the subtitle area in the video data can be detected quickly and accurately.

[0100] Embodiment 5

[0101] like Figure 5 As shown, an embodiment of the present invention further provides a terminal device 500 including: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501, such as a video subtitle detection program. When the processor 501 executes the computer program 503, the steps in the above-mentioned video subtitle detection method embodiments are implemented, such as the method steps in Embodiment 1, Embodiment 2, and / or Embodiment 3. When the processor 501 executes the computer program 503, the functions of the modules in the above-mentioned device embodiments are implemented, such as Figure 4 Functions of modules 401 to 404 are shown.

[0102] Exemplarily, the computer program 503 may be divided into one or more modules, which are stored in the memory 502 and executed by the processor 501 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 503 in the terminal device 500. For example, the computer program 503 may be divided into a first acquisition module, a detection module, a second acquisition module, and a screening module. The specific functions of each module have been described in the above-mentioned embodiment 4 and will not be repeated here.

[0103] The terminal device 500 may be a computing device such as a server, a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 501 and a memory 502. Those skilled in the art will appreciate that Figure 5It is only an example of the terminal device 500 and does not constitute a limitation of the terminal device 500. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0104] The processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0105] The memory 502 may be an internal storage unit of the terminal device 500, such as a hard disk or memory of the terminal device 500. The memory 502 may also be an external storage device of the terminal device 500, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 500. Further, the memory 502 may also include both an internal storage unit of the terminal device 500 and an external storage device. The memory 502 is used to store the computer program and other programs and data required by the terminal device. The memory 502 may also be used to temporarily store data that has been output or is to be output.

[0106] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0109] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0112] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0113] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A video subtitle detection method, It is characterized in that include: Obtain the pixel difference between two adjacent frames of images in the video data; When the pixel difference is greater than a first preset threshold, edge detection is performed on a target frame image in the two adjacent frame images; wherein the target frame image is the latter frame image in the two adjacent frame images; Acquire a connected area in the target frame image after edge detection; Screening the connected area, and using the screened connected area as a subtitle area; The position information of the subtitle area in the target frame image is obtained and output, and the step of obtaining the pixel difference between two adjacent frame images in the video data is returned to execute until the position information of all subtitle areas in the video data is output; the position information of all subtitle areas in the output video data is the output of the subtitle areas in all target frame images and the subtitle areas in all intermediate video frames, wherein the intermediate video frame is the video frame between two adjacent target frame images, wherein the position of the subtitle area of ​​the previous target frame image in the two adjacent target frame images corresponding to the intermediate video frame is used as the position of the subtitle area in the intermediate video frame.

2. The video subtitle detection method according to claim 1, It is characterized in that The step of obtaining the pixel difference between two adjacent frames of images in the video data includes: Calculating a first average pixel value of a previous frame of image in the two adjacent frames of image; Calculating a second average pixel value of a later frame of the image between the two adjacent frames of the image; The pixel difference between the two adjacent frames of images is determined according to a first difference between the first average pixel value and the second average pixel value.

3. The video subtitle detection method according to claim 1, It is characterized in that The performing edge detection on the target frame image in the two adjacent frame images includes: Respectively calculating the gradient value of each pixel point in the target frame image and the two frames following the target frame image; The median of the gradient values ​​of the pixel points at the same position in the target frame image and the latter two frame images is obtained as the pixel value of the pixel point at the corresponding position in the target frame image after edge detection.

4. The video subtitle detection method according to claim 1, It is characterized in that The performing edge detection on the target frame image in the two adjacent frame images includes: According to the Sobel edge detection algorithm, edge detection is performed on the target frame image in the two adjacent frame images.

5. The video subtitle detection method according to claim 1, It is characterized in that The step of obtaining a connected area in the target frame image after edge detection comprises: Preprocessing the target frame image after edge detection; Obtaining a connected area in the target frame image after edge detection and preprocessing.

6. The video subtitle detection method according to claim 5, It is characterized in that The obtaining of the connected region in the target frame image after edge detection and preprocessing comprises: Acquire regions having the same pixel values ​​and adjacent positions in the target frame image after edge detection and preprocessing.

7. The video subtitle detection method according to claim 5, It is characterized in that The preprocessing of the target frame image after edge detection comprises: The target frame image after edge detection is preprocessed by a morphological filtering algorithm.

8. The video subtitle detection method according to claim 1, It is characterized in that The step of screening the connected area and using the screened connected area as a subtitle area includes: Selecting connected areas whose sizes are within a preset size range from the connected areas to obtain target connected areas; Obtaining a third average pixel value of all pixels in the target frame image located in the target connected area; Obtaining a fourth average pixel value corresponding to the target connected area in a frame image subsequent to the target frame image; According to the second difference between the third average pixel value and the fourth average pixel value, an area in the first connected area where the second difference is less than a preset second threshold is screened out as a subtitle area.

9. A video subtitle detection device, It is characterized in that include: A first acquisition module is used to acquire the pixel difference between two adjacent frames of images in the video data; A detection module, configured to perform edge detection on a target frame image in the two adjacent frame images when the pixel difference is greater than a first preset threshold; wherein the target frame image is the latter frame image in the two adjacent frame images; A second acquisition module, used for acquiring a connected area in the target frame image after edge detection; A screening module, used for screening the connected areas and using the screened connected areas as subtitle areas; An output module is used to obtain and output the position information of the subtitle area in the target frame image, and return to execute the step of obtaining the pixel difference between two adjacent frame images in the video data until the position information of all subtitle areas in the video data is output; the position information of all subtitle areas in the output video data is the output of the subtitle areas in all target frame images and the subtitle areas in all intermediate video frames, the intermediate video frame is the video frame between two adjacent target frame images, wherein the position of the subtitle area of ​​the previous target frame image in the two adjacent target frame images corresponding to the intermediate video frame is used as the position of the subtitle area in the intermediate video frame.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images