A title detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a title detection method and device, electronic equipment and storage medium, and relate to the technical field of intelligent analysis, including: obtaining at least two video frames in a to-be-processed video, then for each video frame, detecting the position of the title region in the video frame by using a pre-trained title detection model, and then calculating the coincidence degree of the title regions of the at least two video frames. If the coincidence degree is greater than or equal to a preset coincidence degree threshold, a coincidence region of the title regions of the at least two video frames is determined. If the similarity between the edge pixel gray value sequences of the coincidence region of the at least two video frames is greater than or equal to a first similarity threshold, a target text is extracted from the title region of any video frame of the at least two video frames, and the target text is determined as the title content of the to-be-processed video. In this way, the title detection accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent analysis technology, and in particular to a title detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] A typical news segment consists of multiple news items. The entire news segment can be broken down into multiple news videos, with each video corresponding to one news item.

[0003] After breaking down a full news video into segments, the titles of the segments need to be extracted and used as the titles for each segment. Currently, fixed title bar positions can be pre-set manually based on experience, and then the content in those fixed title bar positions within the video frames can be extracted as the titles.

[0004] However, the position and style of the title bar vary from news column to news column, and the style and position of the title bar for the same news column may also change from time to time. This may result in the fixed position of the title bar not matching the actual position of the title bar, which in turn leads to a low accuracy rate in news headline detection. Summary of the Invention

[0005] The purpose of this invention is to provide a title detection method, apparatus, electronic device, and storage medium to improve the accuracy of title detection. The specific technical solution is as follows:

[0006] A first aspect of this application provides a title detection method, the method comprising:

[0007] Obtain at least two video frames from the video to be processed;

[0008] For each video frame, the position of the title region in the video frame is detected using a pre-trained title detection model;

[0009] Calculate the overlap of the title regions of the at least two video frames;

[0010] If the overlap is greater than or equal to a preset overlap threshold, then the overlapping area of ​​the title region of the at least two video frames is determined.

[0011] If the first similarity between the edge pixel grayscale value sequences of the overlapping regions of the at least two video frames is greater than or equal to the first similarity threshold, then the target text is extracted from the title region of any one of the at least two video frames, and the target text is determined as the title content of the video to be processed.

[0012] Optionally, before extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed, the method further includes:

[0013] Determine whether the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than a second similarity threshold;

[0014] If so, then the step of extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed is performed.

[0015] Optionally, after determining the overlapping area of ​​the title regions of the at least two video frames, the method further includes:

[0016] By extracting images of the overlapping regions in each video frame, an image of the overlapping region in each video frame is obtained.

[0017] Edge extraction is performed on each overlapping region image to obtain the gray value sequence of edge pixels for each overlapping region image;

[0018] Establish a comparison array. For each element in the comparison array, if the element at that position has the same value in each edge pixel grayscale value sequence, then the element at that position in the comparison array is set to the first value. If the element at that position has different values ​​in two edge pixel grayscale value sequences, then the element at that position in the comparison array is set to the second value.

[0019] The first similarity is calculated based on the comparison array and the edge pixel gray value sequence of each overlapping region image.

[0020] Optionally, calculating the first similarity based on the comparison array and the edge pixel grayscale value sequence of each overlapping region image includes:

[0021] Calculate the sum of the values ​​of each element in the grayscale sequence of edge pixels in each overlapping region image;

[0022] Calculate the ratio between the sum of the elements in the comparison array and the total sum to obtain a first ratio;

[0023] The difference between 1 and the first ratio is taken as the first similarity.

[0024] Optionally, determining whether the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than a second similarity threshold includes:

[0025] For each pair of video frames comprising the at least two video frames, the following processing is performed:

[0026] Images are cropped from the first video frame according to the first title region and the second title region respectively to obtain the first image and the second image. The first title region is the title region detected from the first video frame, and the second title region is the title region detected from the second video frame. The first video frame and the second video frame are two video frames included in the at least two video frames.

[0027] In the second video frame, images are cropped according to the first title region and the second title region, respectively, to obtain the third image and the fourth image;

[0028] If the feature distance between the hash value of the first image and the hash value of the second image is less than a preset feature distance threshold, and the feature distance between the hash value of the third image and the hash value of the fourth image is less than the preset feature distance threshold, then it is determined that the feature similarity of the text content in the overlapping area of ​​the first video frame and the second video frame is greater than the second similarity threshold.

[0029] If the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than the second similarity threshold, then it is determined that the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than the second similarity threshold.

[0030] Optionally, each video frame contains at least two title regions, and at least two video frames contain overlapping regions of at least two title regions, and the similarity between the edge pixel grayscale value sequences of each overlapping region of the at least two video frames is greater than or equal to the first similarity threshold.

[0031] Extracting target text from the title region of any one of the at least two video frames, and determining the target text as the title content of the video to be processed, includes:

[0032] Add the position of the title region corresponding to the same overlapping area in each video frame to a title position sequence to obtain at least two title position sequences;

[0033] Calculate the average title area position for each title position sequence;

[0034] Calculate the distribution score for each average title region position, and calculate the mean of the distribution scores for each average title region position. The distribution score is used to characterize the position distribution of each average title region position in the video to be processed.

[0035] Determine the location of the target average title region where the distribution score is greater than the mean;

[0036] Extract the target text from the target average title region position of any one of the at least two video frames, and determine the target text as the title content of the video to be processed.

[0037] Optionally, calculating the distribution score for each average title region position includes:

[0038] For each average title region position, obtain the number of times each pixel position within that average title region position appears in the corresponding title position sequence, and generate an array corresponding to that average title region position.

[0039] The distribution score of the average title area position is calculated using the following formula:

[0040]

[0041] Where score represents the distribution score of the average title area position, and D is the array corresponding to the average title area position. The x-coordinate of the bottom right corner of the average title area position. The x-coordinate of the top-left corner of the average title area position. This represents the bottom right ordinate of the average title area position. The top-left y-coordinate of the average title area position is given by , and i and j represent the positions of the elements in the array corresponding to the average title area position.

[0042] A second aspect of this application provides a title detection device, the device comprising:

[0043] The acquisition module is used to acquire at least two video frames from the video to be processed.

[0044] The detection module is used to detect the position of the title region in each video frame acquired by the acquisition module using a pre-trained title detection model.

[0045] A calculation module is used to calculate the overlap of the title regions of the at least two video frames detected by the detection module;

[0046] The determination module is used to determine the overlapping area of ​​the title regions of the at least two video frames if the overlap degree calculated by the calculation module is greater than or equal to a preset overlap degree threshold.

[0047] An extraction module is configured to extract target text from the title region of any one of the at least two video frames if the similarity between the edge pixel grayscale value sequences of the overlapping regions of the at least two video frames determined by the determining module is greater than or equal to a first similarity threshold, and to determine the target text as the title content of the video to be processed.

[0048] Optionally, the device further includes:

[0049] The judgment module is used to determine whether the feature similarity of the text content in the overlapping area of ​​the at least two video frames is greater than a second similarity threshold before extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed.

[0050] The calling module is used to, if the judgment result of the judgment module is yes, call the extraction module to perform the step of extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed.

[0051] Optionally, the device further includes:

[0052] The cropping module is used to crop an image of the position of the overlapping region in each video frame after determining the overlapping region of the title region of the at least two video frames, so as to obtain an image of the overlapping region in each video frame.

[0053] The edge extraction module is used to extract edges from each overlapping region image to obtain the edge pixel gray value sequence of each overlapping region image.

[0054] A module is established to create a comparison array. For each element in the comparison array, if the element at that position has the same value in each edge pixel grayscale value sequence, then the element at that position in the comparison array is set to the first value. If the element at that position has different values ​​in two edge pixel grayscale value sequences, then the element at that position in the comparison array is set to the second value.

[0055] The calculation module is used to calculate the first similarity based on the comparison array and the edge pixel gray value sequence of each overlapping region image.

[0056] Optionally, the computing module is specifically used for:

[0057] Calculate the sum of the values ​​of each element in the grayscale sequence of edge pixels in each overlapping region image;

[0058] Calculate the ratio between the sum of the elements in the comparison array and the total sum to obtain a first ratio;

[0059] The difference between 1 and the first ratio is taken as the first similarity.

[0060] Optionally, the determination module is specifically used for:

[0061] For each pair of video frames comprising the at least two video frames, the following processing is performed:

[0062] Images are cropped from the first video frame according to the first title region and the second title region respectively to obtain the first image and the second image. The first title region is the title region detected from the first video frame, and the second title region is the title region detected from the second video frame. The first video frame and the second video frame are two video frames included in the at least two video frames.

[0063] In the second video frame, images are cropped according to the first title region and the second title region, respectively, to obtain the third image and the fourth image;

[0064] If the feature distance between the hash value of the first image and the hash value of the second image is less than a preset feature distance threshold, and the feature distance between the hash value of the third image and the hash value of the fourth image is less than the preset feature distance threshold, then it is determined that the feature similarity of the text content in the overlapping area of ​​the first video frame and the second video frame is greater than the second similarity threshold.

[0065] If the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than the second similarity threshold, then it is determined that the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than the second similarity threshold.

[0066] Optionally, each video frame contains at least two title regions, and at least two video frames contain overlapping regions of at least two title regions, and the similarity between the edge pixel grayscale value sequences of each overlapping region of the at least two video frames is greater than or equal to the first similarity threshold.

[0067] The extraction module is specifically used for:

[0068] Add the position of the title region corresponding to the same overlapping area in each video frame to a title position sequence to obtain at least two title position sequences;

[0069] Calculate the average title area position for each title position sequence;

[0070] Calculate the distribution score for each average title region position, and calculate the mean of the distribution scores for each average title region position. The distribution score is used to characterize the position distribution of each average title region position in the video to be processed.

[0071] Determine the location of the target average title region where the distribution score is greater than the mean;

[0072] Extract the target text from the target average title region position of any one of the at least two video frames, and determine the target text as the title content of the video to be processed.

[0073] Optionally, the extraction module is specifically used for:

[0074] For each average title region position, obtain the number of times each pixel position within that average title region position appears in the corresponding title position sequence, and generate an array corresponding to that average title region position.

[0075] The distribution score of the average title area position is calculated using the following formula:

[0076]

[0077] Where score represents the distribution score of the average title area position, and D is the array corresponding to the average title area position. The x-coordinate of the bottom right corner of the average title area position. The x-coordinate of the top-left corner of the average title area position. This represents the bottom right ordinate of the average title area position. The top-left y-coordinate of the average title area position is given by , and i and j represent the positions of the elements in the array corresponding to the average title area position.

[0078] A third aspect of this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0079] Memory, used to store computer programs;

[0080] A processor, when executing a program stored in memory, implements the method described in the first aspect.

[0081] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0082] Using the above method, at least two video frames from the video to be processed can be obtained. A title detection model is then used to detect the position of the title region in each video frame. Compared to manually setting a fixed title bar position, the title detection model can identify title bars in different positions and styles within video frames, improving the accuracy of title detection and exhibiting high universality. Then, the overlap degree of the title regions of at least two video frames is calculated. If the overlap degree is greater than or equal to a preset overlap degree threshold, it indicates that the positions of the titles in at least two video frames are highly similar. Simultaneously, if the similarity between the edge pixel grayscale value sequences of the overlapping regions of at least two video frames is greater than or equal to a first similarity threshold, it indicates that the text content within the titles of at least two video frames is highly similar. Therefore, target text can be extracted from the title region of any one of the at least two video frames, and this target text is identified as the title content of the video to be processed, improving the accuracy of title detection. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0084] Figure 1 A flowchart of a title detection method provided in an embodiment of this application;

[0085] Figure 2a An exemplary schematic diagram of a first type of title region provided in the embodiments of this application;

[0086] Figure 2b An exemplary schematic diagram of a second type of title region provided in the embodiments of this application;

[0087] Figure 3 A flowchart of a method for calculating a first similarity provided in an embodiment of this application;

[0088] Figure 4a An exemplary schematic diagram of a third type of title region provided in the embodiments of this application;

[0089] Figure 4b An exemplary schematic diagram of an edge image provided in an embodiment of this application;

[0090] Figure 4c An exemplary schematic diagram of an edge image comparison result provided in an embodiment of this application;

[0091] Figure 5 A flowchart illustrating a method for determining whether a feature similarity is greater than a second similarity threshold, provided in an embodiment of this application;

[0092] Figure 6 An exemplary schematic diagram of the fourth type of title region provided in the embodiments of this application;

[0093] Figure 7 An exemplary schematic diagram of a title detection process provided in an embodiment of this application;

[0094] Figure 8 This is a schematic diagram of the structure of a title detection device provided in an embodiment of this application;

[0095] Figure 9 This application provides a block diagram of an electronic device structure. Detailed Implementation

[0096] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0097] To address the aforementioned problems, this application provides a title detection method. This method is applied to an electronic device, which can be a terminal or a server, such as... Figure 1 As shown, the method includes:

[0098] S101. Obtain at least two video frames from the video to be processed.

[0099] S102. For each video frame, use a pre-trained title detection model to detect the location of the title region in the video frame.

[0100] S103. Calculate the overlap of the title regions of at least two video frames.

[0101] S104. If the overlap is greater than or equal to the preset overlap threshold, then determine the overlapping area of ​​the title region of at least two video frames.

[0102] S105. If the first similarity between the edge pixel grayscale value sequences of the overlapping regions of at least two video frames is greater than or equal to the first similarity threshold, then the target text is extracted from the title region of any one of the at least two video frames, and the target text is determined as the title content of the video to be processed.

[0103] Using the above method, at least two video frames from the video to be processed can be obtained. A title detection model is then used to detect the position of the title region in each video frame. Compared to manually setting a fixed title bar position, the title detection model can identify title bars in different positions and styles within video frames, improving the accuracy of title detection and exhibiting high universality. Then, the overlap degree of the title regions of at least two video frames is calculated. If the overlap degree is greater than or equal to a preset overlap degree threshold, it indicates that the positions of the titles in at least two video frames are highly similar. Simultaneously, if the similarity between the edge pixel grayscale value sequences of the overlapping regions of at least two video frames is greater than or equal to a first similarity threshold, it indicates that the text content within the titles of at least two video frames is highly similar. Therefore, target text can be extracted from the title region of any one of the at least two video frames, and this target text is identified as the title content of the video to be processed, improving the accuracy of title detection.

[0104] For step S101, at least two video frames are acquired from the video to be processed. As an example, the video to be processed can be a complete news video or a segmented news video. Alternatively, the video to be processed can be a titled video such as an educational video or a short entertainment video; this embodiment does not specifically limit the content type of the video to be processed.

[0105] Optionally, the electronic device can uniformly extract frames from the video to be processed at a preset frame extraction frequency to obtain at least two video frames. For example, the preset frame extraction frequency is 5 frames per second (FPS), meaning that the electronic device can uniformly extract 5 frames from the video to be processed every second.

[0106] Alternatively, the electronic device may randomly extract at least two video frames from the video to be processed. Alternatively, the electronic device may use other methods for frame extraction, which are not specifically limited in this embodiment.

[0107] For S102, for each video frame, the location of the title region in the video frame is detected using a pre-trained title detection model.

[0108] For example, the title detection model can be a pre-trained You Only Look Once X (YOLOX) network, which is a neural network model. Alternatively, the title detection model can be other pre-trained neural network models, such as YOLOv1 or YOLOv2, etc., and this application embodiment does not specifically limit it.

[0109] After an electronic device inputs a video frame into the title detection model, the model can output the position of the title region within that video frame. Optionally, the position of the title region can be represented by the coordinates of its top-left and bottom-right corners, with the origin located at the top-left corner of the video frame. Alternatively, the position of the title region can be represented by the coordinates of its top-right and bottom-left corners.

[0110] For example, Figure 2a The large rectangle in the diagram represents video frame 1, and the small rectangle represents the position of the title area of ​​video frame 1 detected by the YOLOX network, including the top-left corner coordinates (20, -20) and the bottom-right corner coordinates (50, -30), in pixels. It should be noted that the coordinate system units in subsequent embodiments can all be pixels.

[0111] Figure 2b The large rectangle in the image represents video frame 2, and the small rectangle represents the position of the title area of ​​video frame 2 detected by the YOLOX network, including the top left corner coordinates (21, -21) and the bottom right corner coordinates (51, -31).

[0112] For S103, the method for calculating the overlap of the title regions of at least two video frames may include the following steps:

[0113] Step 1: For each acquired video frame, determine the maximum width and minimum width based on the x-coordinate of the title region in that video frame and the x-coordinate of the title region in the previous video frame adjacent to that video frame (at least two acquired video frames). Here, the previous video frame adjacent to the current video frame refers to the video frame obtained after frame extraction. For example, if the current video frame is the fifth video frame extracted from the video to be processed at a preset extraction rate, the previous video frame adjacent to it refers to the fourth video frame extracted from the same video to be processed at the same preset extraction rate. These two video frames may not be adjacent in the actual video to be processed.

[0114] The maximum width is the difference between the maximum and minimum horizontal coordinates among the four horizontal coordinates; the minimum width is the absolute value of the difference between the other two horizontal coordinates.

[0115] For example, if the coordinates of the title region in the video frame are (20, -20) and (50, -30), and the coordinates of the title region in the previous video frame adjacent to the current video frame are (23, -25) and (53, -31), then the maximum width is 53 - 20 = 33, and the minimum width is 50 - 23 = 27.

[0116] Step 2: Determine the maximum height and minimum height based on the ordinate of the title region in the current video frame and the ordinate of the title region in at least two previous video frames adjacent to the current video frame. The maximum height is the difference between the maximum and minimum ordinates; the minimum height is the absolute value of the difference between the other two ordinates.

[0117] For example, if the coordinates of the title region in the video frame are (20, -20) and (50, -30), and the coordinates of the title region in the previous video frame adjacent to the current video frame are (23, -25) and (53, -31), then the maximum height is -20 - (-31) = 11, and the minimum height is -25 - (-30) = 5.

[0118] Step 3: Calculate the product of the minimum width and the minimum height to obtain the area of ​​the overlapping region.

[0119] Step 4: Calculate the product of the maximum width and the maximum width to obtain the maximum area.

[0120] The product of the maximum widths represents the smallest rectangular area that includes both the title area in the current video frame and the title area in the previous video frame.

[0121] Step 5: The ratio of the overlapping area to the largest area is taken as the degree of overlap.

[0122] The degree of overlap can be calculated using the following formula:

[0123]

[0124]

[0125] iou = inter / uion

[0126] Where, inter represents the overlapping area of ​​the title region in the current video frame and the title region in the previous video frame, union refers to the area of ​​the smallest rectangular region that simultaneously includes the title regions in the current video frame and the previous video frame, i.e., the maximum area mentioned above, iou represents the degree of overlap, min(A,B) means taking the minimum value of A and B, and max(A,B) means taking the maximum value of A and B. and Indicates the location of the title area of ​​the video frame, for example This represents the coordinates of the top-left corner of the title area of ​​the video frame. This represents the coordinates of the bottom right corner of the title area of ​​this video frame. and This indicates the position of the title region in the previous video frame adjacent to the current video frame out of at least two acquired video frames, for example... The coordinates of the top-left corner of the title area of ​​the previous video frame. This is the coordinate of the bottom right corner of the title area of ​​the previous video frame.

[0127] This means calculating the minimum width based on the four x-coordinates of the title region in the current video frame and the title region in the previous video frame. For example, The x-coordinate of the top-left corner of the title area in this video frame. This is the x-coordinate of the bottom right corner of the title area in this video frame. The x-coordinate of the top-left corner of the title area in the previous video frame. This is the x-coordinate of the bottom right corner of the title area in the previous video frame.

[0128] This means calculating the minimum height based on the coordinates of the title area in the current video frame and the four ordinates of the title area in the previous video frame. For example, The vertical coordinate of the top-left corner of the title area in this video frame. This is the y-coordinate of the bottom right corner of the title area in this video frame. The vertical coordinate of the top-left corner of the title area in the previous video frame. This is the ordinate of the bottom right corner of the title area in the previous video frame.

[0129] This means calculating the maximum width based on the coordinates of the title area in the current video frame and the four horizontal coordinates of the title area in the previous video frame.

[0130] This means calculating the maximum height based on the coordinates of the title area in the current video frame and the four vertical coordinates of the title area in the previous video frame.

[0131] For example, if the maximum width is 33, the minimum width is 27, the maximum height is 11, and the minimum height is 5, then the overlapping area is 27*5=135, the maximum area is 11*33=363, and the overlap is 135 / 363, which is approximately 0.37.

[0132] Using the above method, this embodiment of the application can calculate the overlap of the title regions in every two adjacent video frames. Since the time difference between adjacent video frames is small, the position of the same title is generally the same or changes only slightly. Therefore, this embodiment of the application can reflect whether the title regions of two adjacent video frames contain the same title content by calculating the overlap of the title regions of two adjacent video frames.

[0133] For S104, if the overlap is greater than or equal to the preset overlap threshold, then the overlapping areas of the title regions of at least two video frames are determined.

[0134] The preset overlap threshold can be set empirically; for example, the preset overlap threshold can be 0.7. When the overlap is greater than or equal to the preset overlap threshold and the area of ​​the overlapping region is greater than 0, it indicates that the positional error between the title region in the video frame and the title region in the previous video frame adjacent to the current video frame is small, so these two title regions may contain the same title.

[0135] After S103 above, when the overlap is less than the preset overlap threshold, it means that the positional error between the title region in the video frame and the title region in the previous video frame adjacent to the video frame among the multiple extracted video frames is large, so the two title regions may not contain the same title.

[0136] See Figure 3 After determining the overlapping regions of the title regions of at least two video frames in S104 above, the first similarity between the edge pixel grayscale value sequences of the overlapping regions of at least two video frames can be calculated in the following way:

[0137] S301. Extract the image of the overlapping region in each video frame to obtain the overlapping region image in each video frame.

[0138] For example, the location of the overlapping region can be represented by the coordinates of the upper left corner and the lower right corner of the overlapping region.

[0139] For each acquired video frame, an image of the overlapping region can be extracted from that video frame to obtain a first overlapping image. Then, from at least two acquired image frames, an image of the overlapping region can be extracted from the preceding video frame adjacent to the current video frame to obtain a second overlapping image.

[0140] As an example, if the coordinates of the overlapping region are (20, -10, 50, -20), where (20, -10) is the coordinate of the upper left corner of the overlapping region and (50, -20) is the coordinate of the lower right corner of the overlapping region, then the first overlapping image is the image within the rectangular area enclosed by the coordinates (20, -10, 50, -20) of the overlapping region in the video frame, and the second overlapping image is the image within the rectangular area enclosed by the coordinates (20, -10, 50, -20) of the overlapping region in the previous video frame adjacent to the video frame among at least two acquired image frames.

[0141] S302. Perform edge extraction on each overlapping region image to obtain the edge pixel gray value sequence of each overlapping region image.

[0142] In this context, the edge in an image refers to the region where the gray level of a pixel changes drastically, and edge extraction refers to identifying and highlighting the image contour.

[0143] Specifically, the edge extraction process is as follows: For each acquired video frame, the electronic device first converts the first overlapping image in that video frame and the second overlapping image in the previous video frame into grayscale images, then performs Gaussian blurring on each image. Finally, the Canny operator is used to calculate the first edge image from the Gaussian-blurred first image and the second edge image from the Gaussian-blurred second image. The Canny operator is an edge detection algorithm. The above edge extraction method is merely an example, and this application does not impose specific limitations on the edge extraction method.

[0144] For example, such as Figure 4a As shown, the first overlapping image captured by the electronic device is Figure 4a The image above, the second overlapping image captured by the electronic device, is Figure 4a The image below. The first edge image corresponding to the first overlapping image is... Figure 4b The second edge image corresponding to the second overlapping image shown above is... Figure 4b The images shown below. In the first edge image, white areas represent edges in the first overlapping image; in the second edge image, white areas represent edges in the second overlapping image.

[0145] After edge extraction, the pixel value of each pixel in the first edge image can be extracted to obtain the gray value sequence of the first edge pixels, and the pixel value of each pixel in the second edge image can be extracted to obtain the gray value sequence of the second edge pixels.

[0146] In the first edge image, each pixel has two possible values, which indicate whether the pixel is located in the edge region. The position of the pixel in the first edge image is used as the position of the corresponding element in the grayscale value sequence of the first edge pixels, and the pixel value of that pixel is used as the value of the corresponding element.

[0147] As an example, Figure 4b In the edge image, each pixel can take the value of 0 or 255, where 0 indicates that the pixel is in a non-edge region and 255 indicates that the pixel is in an edge region.

[0148] Suppose that pixel A is located at (10, -15) in the first edge image and has a value of 255. Then, the element A corresponding to pixel A is located in the 10th column and the 15th row in the gray value sequence of the first edge pixels, and the value of element A is 255.

[0149] Accordingly, each pixel in the second edge image has two possible values, representing whether the pixel is located in the edge region. The position of the pixel in the second edge image is used as the position of the corresponding element in the grayscale value sequence of the second edge pixels, and the pixel value of that pixel is used as the value of the corresponding element.

[0150] Suppose that pixel B is located at (11,-15) in the second edge image and has a value of 255. Then the element B corresponding to pixel B is located in the 11th column and 15th row of the second edge pixel gray value sequence, and the value of element B is 255.

[0151] S303. Establish a comparison array. For each element in the comparison array, if the element at that position has the same value in each edge pixel grayscale value sequence, then the element at that position in the comparison array is the first value. If the element at that position has different values ​​in two edge pixel grayscale value sequences, then the element at that position in the comparison array is the second value.

[0152] The comparison array represents the difference between the first edge pixel grayscale value sequence and the second edge pixel grayscale value sequence. The first and second values ​​can have a large difference. For example, the first value is 0 and the second value is 255.

[0153] That is, for each acquired video frame, the electronic device can establish a comparison array for the edge pixel grayscale value sequence obtained based on that video frame and the second edge pixel grayscale value sequence obtained for the previous video frame.

[0154] Specifically, the process of creating the comparison array can be represented by the following formula:

[0155]

[0156] Where `diff[i][j]` represents the value of the element in the i-th row and j-th column of the comparison array, 0 < i ≤ N1, 0 < j ≤ N2, N1 is the width of the overlapping region image, N2 is the height of the overlapping region image, and the units of N1 and N2 are pixels. `roi1_canndy` is the gray value sequence of the first edge pixel, and `roi2_canndy` is the gray value sequence of the second edge pixel. `if roi1_canndy[i][j] = roi2_canndy[i][j]` means that the elements at the same position in the first and second edge pixel gray value sequences have the same value. `if roi1_canndy[i][j] ! = roi2_canndy[i][j]` means that the elements at the same position in the first and second edge pixel gray value sequences have different values.

[0157] The visualization results corresponding to the comparison arrays are as follows: Figure 4c As shown, Figure 4c The white area includes pixels with a value of 255, which are pixels with different values ​​in the first edge pixel grayscale value sequence and the second edge pixel grayscale value sequence. Figure 4c The black area includes pixels with a value of 0, that is, pixels with the same value in the first edge pixel gray value sequence and the second edge pixel gray value sequence.

[0158] S304. Calculate the first similarity based on the comparison array and the edge pixel gray value sequence of each overlapping region image.

[0159] The specific calculation process for the first similarity is as follows: calculate the sum of the values ​​of each element in the gray value sequence of the edge pixels of each overlapping region image, then calculate the ratio between the sum of the elements in the comparison array and the total sum to obtain the first ratio, and take the difference between 1 and the first ratio as the first similarity.

[0160] That is, for each acquired video frame, the sum of the elements in the first edge pixel gray value sequence and the sum of the elements in the second edge pixel gray value sequence are calculated; the ratio between the sum of the elements in the comparison array and the total sum is calculated to obtain the first ratio; and the difference between 1 and the first ratio is taken as the first similarity.

[0161] The specific calculation process for the first similarity mentioned above can be expressed as the following formula:

[0162] canny similarity =1-sum(diff) / (sum(roi1_canndy)+sum(roi2_canndy))

[0163] Among them, canny similarity The first similarity is represented by sum(diff), the sum of the elements in the comparison array is represented by sum(roi1_canndy), the sum of the elements in the gray value sequence of the first edge pixel is represented by sum(roi2_canndy), and the sum of the elements in the gray value sequence of the second edge pixel is represented by sum(roi2_canndy).

[0164] It is understandable that scrolling text content may appear in videos, typically displayed in a fixed area, and could therefore be mistaken for the video title. To avoid mistaking scrolling text for the video title, this embodiment extracts overlapping regions from adjacent video frames and performs edge extraction. Based on the edge extraction results, it determines whether the content of the overlapping regions is the same. The pixel values ​​of the edge images highlight the text content in the overlapping images, reducing the influence of color and background texture, thus exhibiting high robustness and mitigating the impact of subtle differences in the position of the title regions between two adjacent video frames on determining whether the content of these two title regions is the same. Furthermore, by comparing the first edge pixel grayscale value sequence and the second edge pixel grayscale value sequence, the differences in the content of the first and second overlapping images can be obtained. Then, based on the comparison array, the first edge pixel grayscale value sequence, and the second edge pixel grayscale value sequence, a first similarity score of the text content in the first and second overlapping images can be calculated, whereby the first similarity score characterizes the degree of similarity between the text content in the first and second overlapping images. Subsequently, based on the first similarity, it can be determined whether the content in the first and second overlapping images is scrolling text, such as scrolling news. This can filter out the scrolling text areas that the title detection model mistakenly identifies as title areas, thus improving the accuracy of title detection.

[0165] After the above 304, if the first similarity is less than the first similarity threshold, it means that the title area of ​​the video frame and the title area of ​​the previous video frame adjacent to the video frame among the at least two acquired video frames may be scrolling text areas. Therefore, the positions of the title area of ​​the video frame and the title area of ​​the previous video frame can be deleted.

[0166] The first similarity threshold can be preset based on experience. For example, if the first similarity value ranges from 0 to 1, the first similarity threshold can be 0.9.

[0167] Understandably, the first and second overlapping images are obtained based on the positions of the overlapping regions. If the title region in the current video frame corresponds to different text than the title region in the previous video frame, then the first similarity of the text content in the first and second overlapping images is less than the first similarity threshold. That is, there are many differences in the text content in the first and second overlapping images. The positions detected as title regions in the current video frame and the previous video frame may be used to display scrolling text. Therefore, the electronic device deletes the positions of the title regions in both the current and previous video frames.

[0168] After S104 above, if the first similarity is greater than or equal to the preset first similarity threshold, S105 can be executed to extract the target text from the title region of any one of the at least two video frames and determine the target text as the title content of the video to be processed.

[0169] Understandably, if the title region in the current video frame corresponds to the same text as the title region in the previous video frame, then the first similarity of the text content in the first overlapping image and the second overlapping image is greater than or equal to the first similarity threshold. In other words, the text content in the first overlapping image and the second overlapping image has a high similarity, and the text content included at the location detected as the title region in the current video frame and the location detected as the title region in the previous video frame is highly likely to be the same title. Therefore, the electronic device can extract the target text from the title region of any video frame and use it as the title content of the video to be processed.

[0170] In some embodiments of this application, before extracting the target text from the title region of any one of the at least two video frames in step S105 and determining the target text as the title content of the video to be processed, the electronic device may further determine whether the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than a second similarity threshold. If so, step S105 is executed.

[0171] That is, for each acquired video frame, it can be determined whether the feature similarity of the text content in the overlapping area between the title region of the video frame and the title region of the previous video frame adjacent to the video frame is greater than the second similarity threshold, thereby determining whether the text content in the overlapping area between the video frame and the previous video frame has a high degree of similarity.

[0172] If yes, it means that the text content in the overlapping area of ​​the current video frame and the previous video frame has a high similarity, and they are likely to contain the same title. Therefore, after judging each acquired video frame and determining that the first similarity between the edge pixel grayscale value sequences of the overlapping area of ​​at least two video frames is greater than the first similarity threshold, and the similarity of the text content in the overlapping area of ​​the at least two video frames is greater than the second similarity threshold, S105 can be executed. If no, it means that the text content in the overlapping area of ​​the current video frame and the previous video frame has a low similarity, and they are likely to contain different titles. Therefore, S105 is not executed for these two video frames, thereby improving the accuracy of acquiring the title in the video.

[0173] In some embodiments of this application, such as Figure 5 As shown, in the specific implementation of S105 above, the method of determining whether the feature similarity of the text content in the overlapping regions of at least two video frames is greater than the second similarity threshold can be implemented by performing the following processing on every two video frames included in the at least two video frames:

[0174] S501. Extract images from the first video frame according to the first title area and the second title area respectively to obtain the first image and the second image.

[0175] The first title region is the title region detected from the first video frame, the second title region is the title region detected from the second video frame, and the first video frame and the second video frame are two video frames included in at least two video frames.

[0176] The first video frame can be taken as the first video frame and the second video frame as the second video frame, according to the order of the acquired video frames. The feature similarity of the text content in the overlapping area can be determined according to S501 to S503. Then, the second video frame can be taken as the first video frame and the third video frame as the second video frame, and S501 to S503 can be executed, and so on.

[0177] The first image is denoted as: The second image is denoted as:

[0178] S502. In the second video frame, images are cropped according to the first title area and the second title area respectively to obtain the third image and the fourth image.

[0179] The third image is denoted as: The fourth image is denoted as:

[0180] Taking S501 and S502 as examples: Assume the position of the first title area is (20, -10, 50, -20) and the position of the second title area is (30, -10, 55, -20). The first two elements of the title area position represent the coordinates of the upper left corner of the title area, and the last two elements of the title area position represent the coordinates of the lower right corner of the title area. Then, the rectangular area enclosed by (20, -10, 50, -20) in the first video frame is taken as the first image, the rectangular area enclosed by (30, -10, 55, -20) in the first video frame is taken as the second image, the rectangular area enclosed by (20, -10, 50, -20) in the second video frame is taken as the third image, and the rectangular area enclosed by (30, -10, 55, -20) in the second video frame is taken as the fourth image.

[0181] S503. If the feature distance between the hash value of the first image and the hash value of the second image is less than a preset feature distance threshold, and the feature distance between the hash value of the third image and the hash value of the fourth image is less than a preset feature distance threshold, then it is determined that the feature similarity of the text content in the overlapping area of ​​the first video frame and the second video frame is greater than a second similarity threshold.

[0182] Electronic devices can determine whether the text content in the title regions of the first and second images is the same by calculating the feature distance between the hash values ​​of the first and second images, thereby determining whether the title regions of the first and second video frames contain the same title.

[0183] Specifically, the hash values ​​of the first image and the second image can be calculated separately, and then the Hamming distance between the two hash values ​​can be calculated and used as the feature distance.

[0184] To reduce the amount of data and improve computational efficiency, the size and grayscale level of the first and second images can be reduced, and then the hash values ​​of the first and second images can be calculated.

[0185] Taking the first image as an example, the electronic device can reduce the size of the first image to 8*8 pixels, then reduce the grayscale level of the first image to 64 levels. The mean grayscale value of the reduced first image is calculated, and the grayscale value of each pixel in the reduced first image is compared with the mean grayscale value. If the grayscale value of a pixel is greater than or equal to the mean grayscale value, the pixel value is set to 1; if the grayscale value of a pixel is less than the mean grayscale value, the pixel value is set to 0. This results in a 64-dimensional vector, which is used as the hash value of the first image. This hash value calculation method is merely an example provided in this application embodiment; other hash value calculation methods can also be used, and this application embodiment does not limit the hash value calculation method.

[0186] Similarly, electronic devices can determine whether the text content in the title regions of the third and fourth images is the same by calculating the feature distance between the hash values ​​of the third and fourth images, thereby determining whether the title regions of the first and second video frames contain the same title.

[0187] Specifically, the hash values ​​of the second image and the fourth image can be calculated separately, and then the Hamming distance between the two hash values ​​can be calculated and used as the feature distance.

[0188] The preset feature distance threshold can be set empirically, for example, a preset feature distance threshold of 5. The preset feature distance threshold is used to characterize the maximum feature distance between the hash values ​​of images containing the same text content. If the feature distance between the hash values ​​of the first and second images, and the feature distance between the hash values ​​of the third and fourth images, are all less than the preset feature distance threshold, it indicates that the text content in the title regions of the first and second video frames is essentially the same. The electronic device can determine that the content in the title regions of these two video frames is the same title content, and subsequently, the electronic device can store the text content in the title regions of the first and second video frames in the same title position sequence.

[0189] Conversely, if any of the feature distances—between the hash values ​​of the first and second images, and between the hash values ​​of the third and fourth images—is greater than or equal to a preset feature distance threshold, it indicates that the text content in the title regions of the first and second video frames may differ. Subsequent electronic devices can then store the text content in the title regions of the first and second video frames in different title position sequences.

[0190] S504. If the feature similarity of the text content in the overlapping regions of at least two video frames is greater than the second similarity threshold, then it is determined that the feature similarity of the text content in the overlapping regions of at least two video frames is greater than the second similarity threshold.

[0191] The text content in the first to fourth images obtained using this method exhibits high robustness. By comprehensively utilizing the feature distance of hash values ​​between two images to determine whether the text content in the title regions of two video frames are different titles, different title content can be distinguished more accurately, and different title regions in the video to be processed can be identified more precisely, thus improving the accuracy of title detection.

[0192] In some embodiments of this application, each video frame contains at least two title regions, and at least two video frames contain overlapping regions of at least two title regions, and the similarity between the edge pixel grayscale value sequences of each overlapping region of at least two video frames is greater than or equal to a first similarity threshold.

[0193] Based on this, the above-mentioned step S105, which extracts the target text from the title region of any one of at least two video frames and determines the target text as the title content of the video to be processed, may include the following steps:

[0194] Step 1: Add the position of the title area corresponding to the same overlapping area in each video frame to a title position sequence to obtain at least two title position sequences.

[0195] Step 2: Calculate the average title area position for each title position sequence.

[0196] The average position of the heading area can be denoted as: in, and These are the coordinates of the two boundary points of the average title area. For example, The coordinates of the top left corner of the average title area. This represents the coordinates of the bottom right corner of the average title area.

[0197] For example, a sequence of heading positions includes: the position of heading region 1 (10, -10, 20, -20), the position of heading region 2 (11, -11, 21, -21), and the position of heading region 3 (9, -9, 22, -22). The average value of the elements at the same position in each title area of ​​the title sequence is calculated. Specifically, the average value for the first position is (10+11+9) / 3 = 10, the average value for the second position is (-10+(-11)+(-9)) / 3 = -10, the average value for the third position is (20+21+22) / 3 = 21, and the average value for the fourth position is (-20+(-21)+(-22)) / 3 = -21. Thus, the average title area position corresponding to this title position sequence is (10, -10, 21, -21), where (10, -10) is the upper left corner coordinate of the average title area position, and (21, -21) is the lower right corner coordinate of the average title area position.

[0198] Step 3: Calculate the distribution score for each average title area position, and calculate the mean of the distribution scores for each average title area position.

[0199] The distribution score characterizes the positional distribution of each average title region in the video being processed. Since the positions of multiple title regions within the same title position sequence may have slight differences in size and location, the average title region position can more closely approximate the actual region position of the title corresponding to the title position sequence in the video frame.

[0200] Step 4: Determine the location of the target average title area where the distribution score is greater than the mean.

[0201] The distribution score reflects the distribution of title regions in the video to be processed. If the distribution score of the average title region position is larger than the mean, it indicates that the average title region position is highly correlated with the positions of most title regions in the video to be processed, that is, the deviation between the average title region position and the positions of most average title regions is small.

[0202] Conversely, if the distribution score of the average title area position is less than or equal to the mean, it indicates that the average title area position has a low correlation with the position of most title areas in the video to be processed. In other words, the average title area position deviates significantly from the average title area position of most of the videos, which is likely due to false detection and needs to be filtered out.

[0203] Step 5: Extract the target text from the target average title region position of any one of the at least two video frames, and determine the target text as the title content of the video to be processed.

[0204] Using the above method, this application embodiment can filter out title regions that have low positional correlation with most title regions in the video to be processed. In the same video, it is rare for a certain title to deviate significantly from the position of other titles. Therefore, such title regions with low correlation are likely not actually title regions. Thus, this application embodiment can filter out such title regions, thereby improving the accuracy of determining title regions.

[0205] The following is a method for calculating the distribution score in step 3 of this application embodiment:

[0206] Step 1: For each average title region position, obtain the number of times each pixel position within that average title region position appears in the corresponding title position sequence, and generate an array corresponding to that average title region position.

[0207] Step 2: Calculate the distribution score of the average title area position using the following formula:

[0208]

[0209] Where score represents the distribution score of the average title area position, and D is the array corresponding to the average title area position. The x-coordinate of the bottom right corner of the average title area position. The x-coordinate of the top-left corner of the average title area position. This represents the bottom right ordinate of the average title area position. The top-left y-coordinate of the average title area position is given by , and i and j represent the positions of the elements in the array corresponding to the average title area position.

[0210] Using the above method, the value of each pixel in the average title region position can reflect the distribution of the titles in the corresponding title position sequence. Then, the distribution score of the average title region position is calculated using the above formula. Therefore, the average title region position with a distribution score greater than the mean is used as the title region position in the video to be processed. This allows for further filtering of title region positions detected by the title detection model, removing non-title content areas similar to the title bar style, and improving the accuracy of title detection.

[0211] For step one above, as an example, the position of pixel A in the average title area position is (10, -10). The title position sequence corresponding to this average title area position includes the positions of the title area as (10, -10, 20, -20), (11, -11, 21, -21), and (9, -9, 22, -22). The position of pixel A (10, -10) appears in (10, -10, 20, -20) and (9, -9, 22, -22) respectively. Therefore, in the array corresponding to this average title area position, the value of the element corresponding to pixel A is 2.

[0212] For ease of understanding, combined with Figure 6 This section explains how to generate a corresponding array based on a sequence of title positions. For example... Figure 6 As shown, Figure 6 For a given video frame, each of the four solid-line boxes at the bottom represents the position of a title region in title position sequence A, and each of the two dashed-line boxes at the top represents the position of a title region in title position sequence B. Then, calculate the average title position A for title position sequence A and the average title position B for title position sequence B. Count the number of times each pixel in the average title position A appears in the four solid-line boxes, generating array A for title position sequence A. Count the number of times each pixel in the average title position B appears in the two dashed-line boxes, generating array B for title position sequence B. Figure 6 The number of boxes shown is for illustrative purposes only. In actual applications, the number of positions of the title area in the title position sequence is not limited to this.

[0213] The following describes the complete process of the title detection method provided in the embodiments of this application, using specific examples, such as... Figure 7 As shown, the method specifically includes:

[0214] S701, Begin.

[0215] S702, Extract multiple video frames from the video to be processed.

[0216] Among them, electronic devices can perform uniform frame extraction on the video to be processed.

[0217] S703. Input a video frame according to the frame extraction order.

[0218] It should be noted that after the electronic device inputs a video frame, it will continue to input the next video frame until the electronic device completes S704-S709.

[0219] S704. Perform title region detection on the video frame.

[0220] In this context, title region detection refers to using a title detection model in the above embodiments to detect the location of the title region in the video frame.

[0221] It should be noted that if the input video frame is the first video frame among the extracted video frames, the electronic device can create a title position sequence after performing title region detection on the first video frame in S704, and directly store the position of the title region in the first video frame into the newly created title position sequence, and return to S703.

[0222] S705. Does the overlap between the title region of this video frame and the previous title region exceed 0.7? The previous title region refers to the title region of the video frame preceding this video frame from among the extracted video frames.

[0223] If yes, execute S706; otherwise, clear the current title position sequence from the electronic device's database and then store the current title position sequence into the database of the filtered title region set.

[0224] S706. Perform scrolling text filtering on the video frame.

[0225] The scrolling text filtering includes, in the above embodiment, calculating the first similarity between the edge pixel grayscale value sequences of the overlapping regions of the title region in the current video frame and the title region in the previous video frame, and determining whether the first similarity is greater than or equal to a first similarity threshold. If so, the filtering is deemed successful; otherwise, the filtering is deemed unsuccessful. The first similarity threshold can be 0.9.

[0226] S707, Whether to pass the filter.

[0227] If yes, execute S708; otherwise, clear the position of the title area in the current title position sequence.

[0228] S708. Perform title aggregation on the video frame.

[0229] The title aggregation includes the step of calculating the feature similarity of the text content in the overlapping areas of the video frame and the previous video frame in the above embodiments.

[0230] S709, whether to aggregate.

[0231] If the feature similarity of the text content in the overlapping area of ​​the current video frame and the previous video frame is greater than the second similarity threshold, then aggregation is determined, and the action of storing the position of the title area of ​​the current video frame into the current title position sequence is performed.

[0232] If the feature similarity of the text content in the overlapping area of ​​the current video frame and the previous video frame is less than or equal to the second similarity threshold, then aggregation is not allowed, and the action of clearing the current title position sequence from the database of the electronic device is performed. Then, the current title position sequence is stored in the database of the filtered title region set.

[0233] S710, Filter the title position of the video frame.

[0234] The title position filtering includes the electronic device in the above embodiment calculating the average title region position corresponding to each title position sequence and deleting target average title region positions whose distribution score is less than or equal to the mean.

[0235] S711, End.

[0236] Using the above method, a title detection model is employed to detect the position of the title region. Compared to manually setting a fixed title region position, this method can more flexibly detect title bars with different styles and positions in the video being processed. Then, through scrolling text filtering, it can detect whether the text content within similar title regions changes significantly, thereby filtering out scrolling content regions that are mistakenly detected as title regions by the title detection model, improving the accuracy of title detection. Next, through title aggregation, the hash value of the title regions in two adjacent video frames is determined, thereby judging whether the text content within the title regions of two adjacent video frames is similar, and different titles are distinguished based on the title aggregation results. In this way, titles in the video being processed can be identified more accurately. Finally, the average title region position of the title position sequence included in the video being processed is filtered out, removing non-title content regions that are similar to the title bar style, further improving the accuracy of title detection.

[0237] Based on the same inventive concept, corresponding to the above method embodiments, this application provides a title detection device, such as... Figure 8 As shown, the device includes: an acquisition module 801, a detection module 802, a calculation module 803, a determination module 804, and an extraction module 805;

[0238] The acquisition module 801 is used to acquire at least two video frames from the video to be processed.

[0239] The detection module 802 is used to detect the position of the title region in each video frame acquired by the acquisition module 801 using a pre-trained title detection model.

[0240] The calculation module 803 is used to calculate the overlap of the title regions of at least two video frames detected by the detection module 802;

[0241] The determination module 804 is used to determine the overlapping area of ​​the title region of at least two video frames if the overlap calculated by the calculation module 803 is greater than or equal to a preset overlap threshold.

[0242] The extraction module 805 is used to extract target text from the title region of any one of the at least two video frames if the similarity between the edge pixel gray value sequences of the overlapping regions of at least two video frames determined by the determination module 804 is greater than or equal to a first similarity threshold, and to determine the target text as the title content of the video to be processed.

[0243] In some embodiments of this application, the device may further include:

[0244] The judgment module is used to determine whether the feature similarity of the text content in the overlapping area of ​​at least two video frames is greater than a second similarity threshold before extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed.

[0245] The calling module is used to call the extraction module to perform the step of extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed if the judgment result of the judgment module is yes.

[0246] Optionally, the device may also include:

[0247] The cropping module is used to crop the image of the position of the overlapping area in the title region of at least two video frames after determining the overlapping area of ​​the title region, so as to obtain the overlapping area image in each video frame.

[0248] The edge extraction module is used to extract edges from each overlapping region image to obtain the edge pixel gray value sequence of each overlapping region image.

[0249] The module is used to create a comparison array. For each element in the comparison array, if the element at that position has the same value in the grayscale value sequences of each edge pixel, then the element at that position in the comparison array is set to the first value. If the element at that position has different values ​​in two edge pixel grayscale value sequences, then the element at that position in the comparison array is set to the second value.

[0250] The calculation module is used to calculate the first similarity based on the comparison array and the sequence of edge pixel gray values ​​of each overlapping region image.

[0251] Optional, a calculation module, specifically used for:

[0252] Calculate the sum of the values ​​of each element in the grayscale sequence of edge pixels in each overlapping region image;

[0253] Calculate the ratio between the sum of the elements in the comparison array and the total sum, and obtain the first ratio;

[0254] The difference between 1 and the first ratio is taken as the first similarity.

[0255] Optional, a judgment module, specifically used for:

[0256] For each pair of at least two video frames, perform the following processing:

[0257] Images are cropped from the first video frame according to the first title region and the second title region respectively to obtain the first image and the second image. The first title region is the title region detected from the first video frame, and the second title region is the title region detected from the second video frame. The first video frame and the second video frame are two video frames including at least two video frames.

[0258] In the second video frame, images are cropped according to the first title area and the second title area respectively to obtain the third image and the fourth image;

[0259] If the feature distance between the hash value of the first image and the hash value of the second image is less than a preset feature distance threshold, and the feature distance between the hash value of the third image and the hash value of the fourth image is less than a preset feature distance threshold, then it is determined that the feature similarity of the text content in the overlapping area of ​​the first video frame and the second video frame is greater than the second similarity threshold.

[0260] If the feature similarity of the text content in the overlapping regions of at least two video frames is greater than the second similarity threshold, then it is determined that the feature similarity of the text content in the overlapping regions of at least two video frames is greater than the second similarity threshold.

[0261] Optionally, each video frame contains at least two title regions, and at least two video frames contain overlapping regions of at least two title regions, and the similarity between the edge pixel grayscale value sequences of each overlapping region in at least two video frames is greater than or equal to a first similarity threshold.

[0262] Extraction module 805 is specifically used for:

[0263] Add the position of the title region corresponding to the same overlapping area in each video frame to a title position sequence to obtain at least two title position sequences;

[0264] Calculate the average title area position for each title position sequence;

[0265] Calculate the distribution score for each average title region position, and calculate the mean of the distribution scores for each average title region position. The distribution score is used to characterize the position distribution of each average title region position in the video to be processed.

[0266] Determine the location of the target average title area where the distribution score is greater than the mean;

[0267] Extract the target text from the target average title region position of any one of at least two video frames, and determine the target text as the title content of the video to be processed.

[0268] Optionally, the extraction module 805 is specifically used for:

[0269] For each average title region position, obtain the number of times each pixel position within that average title region position appears in the corresponding title position sequence, and generate an array corresponding to that average title region position.

[0270] The distribution score of the average title area position is calculated using the following formula:

[0271]

[0272] Where score represents the distribution score of the average title area position, and D is the array corresponding to the average title area position. The x-coordinate of the bottom right corner of the average title area position. The x-coordinate of the top-left corner of the average title area position. This represents the bottom right ordinate of the average title area position. The top-left y-coordinate of the average title area position is given by , and i and j represent the positions of the elements in the array corresponding to the average title area position.

[0273] This invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0274] Memory 903 is used to store computer programs;

[0275] The processor 901, when executing the program stored in the memory 903, implements the title detection method in the above embodiments.

[0276] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0277] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0278] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0279] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0280] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the title detection methods described in the above embodiments.

[0281] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the title detection methods described in the above embodiments.

[0282] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0283] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0284] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0285] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A title detection method, characterized in that, The method includes: Obtain at least two video frames from the video to be processed; For each video frame, the position of the title region in the video frame is detected using a pre-trained title detection model; Calculate the overlap of the title regions of the at least two video frames; If the overlap is greater than or equal to a preset overlap threshold, then the overlapping area of ​​the title region of the at least two video frames is determined. If the first similarity between the edge pixel grayscale value sequences of the overlapping regions of the at least two video frames is greater than or equal to the first similarity threshold, then the target text is extracted from the title region of any one of the at least two video frames, and the target text is determined as the title content of the video to be processed.

2. The method according to claim 1, characterized in that, Before extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed, the method further includes: Determine whether the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than a second similarity threshold; If so, then the step of extracting the target text from the title region of any one of the at least two video frames and determining the target text as the title content of the video to be processed is performed.

3. The method according to claim 1, characterized in that, After determining the overlapping area of ​​the title regions of the at least two video frames, the method further includes: By extracting images of the overlapping regions in each video frame, an image of the overlapping region in each video frame is obtained. Edge extraction is performed on each overlapping region image to obtain the gray value sequence of edge pixels for each overlapping region image; Establish a comparison array. For each element in the comparison array, if the element at that position has the same value in each edge pixel grayscale value sequence, then the element at that position in the comparison array is set to the first value. If the element at that position has different values ​​in two edge pixel grayscale value sequences, then the element at that position in the comparison array is set to the second value. The first similarity is calculated based on the comparison array and the edge pixel gray value sequence of each overlapping region image.

4. The method according to claim 3, characterized in that, The calculation of the first similarity based on the comparison array and the edge pixel gray value sequence of each overlapping region image includes: Calculate the sum of the values ​​of each element in the grayscale sequence of edge pixels in each overlapping region image; Calculate the ratio between the sum of the elements in the comparison array and the total sum to obtain a first ratio; The difference between 1 and the first ratio is taken as the first similarity.

5. The method according to claim 2, characterized in that, The step of determining whether the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than a second similarity threshold includes: For each pair of video frames comprising the at least two video frames, the following processing is performed: Images are cropped from the first video frame according to the first title region and the second title region respectively to obtain the first image and the second image. The first title region is the title region detected from the first video frame, and the second title region is the title region detected from the second video frame. The first video frame and the second video frame are two video frames included in the at least two video frames. In the second video frame, images are cropped according to the first title region and the second title region, respectively, to obtain the third image and the fourth image; If the feature distance between the hash value of the first image and the hash value of the second image is less than a preset feature distance threshold, and the feature distance between the hash value of the third image and the hash value of the fourth image is less than the preset feature distance threshold, then it is determined that the feature similarity of the text content in the overlapping area of ​​the first video frame and the second video frame is greater than the second similarity threshold. If the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than the second similarity threshold, then it is determined that the feature similarity of the text content in the overlapping regions of the at least two video frames is greater than the second similarity threshold.

6. The method according to claim 1, characterized in that, Each video frame contains at least two title regions, and the at least two video frames contain at least two overlapping title regions, and the similarity between the edge pixel grayscale value sequences of each overlapping region of the at least two video frames is greater than or equal to the first similarity threshold. Extracting target text from the title region of any one of the at least two video frames, and determining the target text as the title content of the video to be processed, includes: Add the position of the title region corresponding to the same overlapping area in each video frame to a title position sequence to obtain at least two title position sequences; Calculate the average title area position for each title position sequence; Calculate the distribution score for each average title region position, and calculate the mean of the distribution scores for each average title region position. The distribution score is used to characterize the position distribution of each average title region position in the video to be processed. Determine the location of the target average title region where the distribution score is greater than the mean; Extract the target text from the target average title region position of any one of the at least two video frames, and determine the target text as the title content of the video to be processed.

7. The method according to claim 6, characterized in that, The calculation of the distribution score for each average title region position includes: For each average title region position, obtain the number of times each pixel position within that average title region position appears in the corresponding title position sequence, and generate an array corresponding to that average title region position. The distribution score of the average title area position is calculated using the following formula: Where score represents the distribution score of the average title area position, and D is the array corresponding to the average title area position. The x-coordinate of the bottom right corner of the average title area position. The x-coordinate of the top-left corner of the average title area position. This represents the bottom right ordinate of the average title area position. The top-left y-coordinate of the average title area position is given by , and i and j represent the positions of the elements in the array corresponding to the average title area position.

8. A title detection device, characterized in that, The device includes: The acquisition module is used to acquire at least two video frames from the video to be processed. The detection module is used to detect the position of the title region in each video frame acquired by the acquisition module using a pre-trained title detection model. A calculation module is used to calculate the overlap of the title regions of the at least two video frames detected by the detection module; The determination module is used to determine the overlapping area of ​​the title regions of the at least two video frames if the overlap degree calculated by the calculation module is greater than or equal to a preset overlap degree threshold. An extraction module is configured to extract target text from the title region of any one of the at least two video frames if the similarity between the edge pixel grayscale value sequences of the overlapping regions of the at least two video frames determined by the determining module is greater than or equal to a first similarity threshold, and to determine the target text as the title content of the video to be processed.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images