Method, device, equipment and medium for extracting chapter identification text in video

Abstract

The present application provides a method, apparatus, device and computer-readable medium for extracting chapter marker text in a video. The application obtains a video frame sequence by extracting frames from a video file to be processed, and then performs chapter marker text recognition on each video frame in the video frame sequence based on the structural features of the chapter marker text to obtain a candidate text instance corresponding to each of the video frames. The application then performs inter-frame binding of the same candidate text instances between adjacent video frames through text tracking, and outputs the candidate text instances to an inter-frame binding output result set according to the binding relationship. The application then performs global processing on all candidate text instances in the inter-frame binding output result set, and determines the chapter marker text based on the globally processed candidate text instances, thereby accurately extracting the chapter marker text in the video.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a method, apparatus, device, and computer-readable medium for extracting chapter identification text from a video. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present application that are recited in the claims. Nothing herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] For your convenience, please refer to Figure 1 Currently, some videos published on video websites have added chapter identification progress bars. Chapter identification texts are set on the chapter identification progress bars corresponding to each time period of the video. The content of each time period of the video is identified by the chapter identification texts. The chapter identification texts contain relatively rich video content hierarchy information. If they can be extracted for video recommendation, the accuracy of video recommendation can be effectively improved.

[0004] In view of this, it is necessary to provide a technical solution that can automatically extract chapter identification text in a video. Summary of the Invention

[0005] Various aspects of the present application provide a method, device, and computer-readable storage medium for extracting chapter marker text from a video, for automatically extracting chapter marker text from a video.

[0006] In one aspect of the present application, a method for extracting chapter identifier text in a video is provided, comprising:

[0007] Extract frames from the video file to be processed to obtain a video frame sequence;

[0008] Performing chapter marker text recognition on each video frame in the video frame sequence according to preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame;

[0009] Binding the same candidate text instances between adjacent video frames through text tracking, and outputting the candidate text instances to an inter-frame binding output result set according to the binding relationship;

[0010] Global processing is performed on all candidate text instances in the inter-frame binding output result set, and chapter identification text is determined based on the globally processed candidate text instances, wherein the global processing includes at least one of global binding, global filtering, row-by-row association, and row-by-row filtering.

[0011] Optionally, performing chapter marker text recognition on each video frame in the video frame sequence according to preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame includes:

[0012] Performing text recognition on each video frame in the video frame sequence to obtain a text instance corresponding to each video frame;

[0013] Candidate text instances that meet preset chapter identification text structure features are screened out from the text instances.

[0014] Optionally, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0015] The step of selecting candidate text instances that meet preset chapter identification text structure features from the text instances includes:

[0016] According to the display position information and format information of each text instance, candidate text instances that meet the display position feature and the format feature are matched from the text instances.

[0017] Optionally, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0018] The step of performing chapter identification text recognition on each video frame in the video frame sequence according to preset chapter identification text structural features to obtain a candidate text instance corresponding to each video frame includes:

[0019] determining a chapter identification text recognition region of each video frame in the video frame sequence according to the display position feature;

[0020] Performing text recognition on the chapter identification text recognition area of ​​each video frame to obtain a regional text instance;

[0021] The regional text instances are screened according to the format features to obtain candidate text instances.

[0022] Optionally, the step of inter-frame binding the same candidate text instances between adjacent video frames through text tracking, and outputting the candidate text instances to an inter-frame binding output result set according to the binding relationship, includes:

[0023] Traverse the candidate text instances of each video frame in the video frame sequence, add the candidate text instance of the first video frame to the cache area corresponding to the video frame and output it to the inter-frame binding output result set, and for each subsequent candidate text instance of the video frame, determine whether the same candidate text instance exists in the cache area of ​​the previous frame. If so, inter-frame bind the candidate text instance with the same candidate text instance of the previous frame and output it to the inter-frame binding output result set; if not, write the candidate text instance into the cache area corresponding to the current video frame and output it to the inter-frame binding output result set.

[0024] Optionally, the global processing includes global binding;

[0025] The global binding includes:

[0026] All candidate text instances are compared pairwise, and the candidate text instances with the same comparison results are bound.

[0027] Optionally, the global processing further includes global screening;

[0028] The global screening includes:

[0029] From all candidate text instances, candidate text instances having an appearance frequency greater than a preset threshold are screened out; wherein the appearance frequency includes a ratio of the number of video frames in which the candidate text instance appears to the total number of frames in the video frame sequence.

[0030] Optionally, the global processing further includes row-by-row association;

[0031] The row-by-row association includes:

[0032] For all candidate text instances, dividing the candidate text instances into rows according to their ordinate values ​​to obtain at least one row of candidate text instances;

[0033] For each row of candidate text instances, multiple candidate text instances in the row are sorted according to the horizontal coordinate values ​​of the candidate text instances and sequentially associated to obtain candidate text instances associated by row.

[0034] Optionally, the global processing further includes filtering by row;

[0035] The row filtering includes:

[0036] According to whether the number of candidate text instances in each line meets a preset threshold condition, it is determined whether to retain or delete the candidate text instances on a line-by-line basis.

[0037] Another aspect of the present application provides a device for extracting chapter identification text in a video, comprising:

[0038] The frame extraction module is used to extract frames from the video file to be processed to obtain a video frame sequence;

[0039] a chapter text recognition module, configured to perform chapter identification text recognition on each video frame in the video frame sequence according to preset chapter identification text structural features, and obtain a candidate text instance corresponding to each video frame;

[0040] An inter-frame binding module is used to inter-frame bind the same candidate text instances between adjacent video frames through text tracking, and output the candidate text instances to an inter-frame binding output result set according to the binding relationship;

[0041] A global processing module is used to output chapter identification text after performing global processing on all candidate text instances in the inter-frame binding output result set, wherein the global processing includes at least one of global deduplication, global screening, global binding and row-by-row association.

[0042] Optionally, the chapter text recognition module includes:

[0043] a text recognition unit, configured to perform text recognition on each video frame in the video frame sequence to obtain a text instance corresponding to each video frame;

[0044] The text instance screening unit is used to screen out candidate text instances that meet preset chapter identification text structure features from the text instances.

[0045] Optionally, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0046] The text instance screening unit includes:

[0047] The text instance screening subunit is used to match the text instances according to the display position information and format information of each text instance to obtain candidate text instances that meet the display position characteristics and the format characteristics.

[0048] Optionally, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0049] The chapter text recognition module includes:

[0050] a recognition region determining unit, configured to determine a chapter identification text recognition region of each video frame in the video frame sequence according to the display position feature;

[0051] a regional text recognition unit, configured to perform text recognition on the chapter identification text recognition region of each video frame to obtain a regional text instance;

[0052] The format screening unit is used to screen the regional text instances according to the format features to obtain candidate text instances.

[0053] Optionally, the inter-frame binding module includes:

[0054] An inter-frame binding unit is used to traverse the candidate text instances of each video frame in the video frame sequence, add the candidate text instance of the first video frame to the buffer area corresponding to the video frame and output it to the inter-frame binding output result set, and for each subsequent candidate text instance of the video frame, determine whether the same candidate text instance exists in the buffer area of ​​the previous frame. If so, the candidate text instance is inter-frame bound to the same candidate text instance of the previous frame and output it to the inter-frame binding output result set; if not, the candidate text instance is written to the buffer area corresponding to the current video frame and output it to the inter-frame binding output result set.

[0055] Optionally, the global processing module includes:

[0056] The global binding unit is used to compare all candidate text instances in pairs and bind the candidate text instances with the same comparison results.

[0057] Optionally, the global processing module includes:

[0058] A global screening unit is used to screen out candidate text instances whose occurrence frequency is greater than a preset threshold from all candidate text instances; wherein the occurrence frequency includes the ratio of the number of video frames in which the candidate text instance appears to the total number of frames in the video frame sequence.

[0059] Optionally, the global processing module includes:

[0060] A line division processing unit, configured to divide all candidate text instances into lines according to the ordinate values ​​of the candidate text instances to obtain at least one line of candidate text instances;

[0061] The row binding unit is used to sort and sequentially associate multiple candidate text instances in each row of candidate text instances according to the horizontal coordinate values ​​of the candidate text instances to obtain candidate text instances associated by row.

[0062] Optionally, the global processing module includes:

[0063] The row-by-row screening unit is configured to determine whether to retain or delete the candidate text instances row by row based on whether the number of candidate text instances in each row meets a preset threshold condition.

[0064] Another aspect of the present application provides an electronic device, comprising:

[0065] at least one processor; and

[0066] a memory communicatively connected to the at least one processor; wherein,

[0067] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method for extracting chapter identification text from a video as described above.

[0068] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the method for extracting chapter identification text in the video.

[0069] In the solution provided by the embodiment of the present application, frame extraction processing is performed on the video file to be processed to obtain a video frame sequence, and then chapter marker text recognition is performed on each video frame in the video frame sequence according to the preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame, and then the same candidate text instances between adjacent video frames are bound between frames through text tracking, and the candidate text instances are output to an inter-frame binding output result set according to the binding relationship, and then global processing is performed on all candidate text instances in the inter-frame binding output result set, wherein the global processing includes at least one of global binding, global filtering, row-by-row association and row-by-row filtering, and the chapter marker text can be determined based on the candidate text instance after global processing. The embodiment of the present application fully utilizes the structural features of the chapter marker text to identify the chapter marker text, and realizes the binding, filtering, association and other tasks of the chapter marker text through the combination of inter-frame binding and global processing, so that the chapter marker text in the video can be accurately extracted. At the same time, the pre-performed frame extraction processing can effectively reduce the workload of subsequent processing and improve the overall extraction efficiency of the chapter marker text. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction is given below to the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0071] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0072] Figure 1A schematic diagram of a video with chapter text identifiers provided in an embodiment of the present application;

[0073] Figure 2 A flowchart of a method for extracting chapter identifier text from a video provided in one embodiment of the present application;

[0074] Figure 3 A flowchart of a method for extracting chapter identifier text from a video provided in yet another embodiment of the present application;

[0075] Figure 4 A schematic diagram of the structure of a device for extracting chapter marker text from a video provided in one embodiment of the present application;

[0076] Figure 5 is a block diagram of an exemplary computer system / server 12 suitable for implementing embodiments of the present application;

[0077] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0078] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0079] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0080] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0081] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. The information can be computer program instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc-read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0082] An embodiment of the present application provides a method for extracting chapter marker text in a video. The method aims to extract frames from a video file to be processed to obtain a video frame sequence, and then perform chapter marker text recognition on each video frame in the video frame sequence based on preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame. Then, through text tracking, the same candidate text instances between adjacent video frames are bound between frames, and the candidate text instances are output to an inter-frame binding output result set according to the binding relationship. Then, global processing is performed on all candidate text instances in the inter-frame binding output result set, wherein the global processing includes at least one of global binding, global filtering, row-by-row association, and row-by-row filtering, so that the chapter marker text can be determined based on the candidate text instances after global processing. The embodiment of the present application fully utilizes the structural features of the chapter marker text to recognize the chapter marker text, and realizes the binding, filtering, association and other tasks of the chapter marker text through the combination of inter-frame binding and global processing, so that the chapter marker text in the video can be accurately extracted. At the same time, the pre-performed frame extraction processing can effectively reduce the workload of subsequent processing and improve the overall extraction efficiency of the chapter marker text.

[0083] In actual scenarios, the execution subject of this method can be a user device, or a device formed by integrating a user device and a network device through a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablets, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computers.

[0084] Figure 2 The following is a processing flow of a method for extracting chapter identifier text from a video provided by an embodiment of the present application. The method includes at least the following processing steps:

[0085] Step S101: extract frames from the video file to be processed to obtain a video frame sequence.

[0086] Among them, the video file to be processed can be any video file with a chapter identification progress bar, and the embodiment of the present application does not limit its format.

[0087] Taking into account that the chapter marker progress bar has a strong time persistence, that is, the chapter marker progress bar on each frame is consistent within a period of time, therefore, some video frames can be extracted by extracting frames to recognize and extract chapter marker text, thereby reducing the computational workload and improving the recognition and extraction efficiency of chapter marker text.

[0088] The frequency of extracting frames is not limited in this application, and can be implemented according to a frequency of one frame per second, one frame per thirty seconds, one frame per minute, etc., and those skilled in the art can flexibly set it according to actual needs. In specific implementation, it can be implemented using an open source video codec tool such as FFMPEG, and the embodiments of this application are also not limited thereto.

[0089] In addition, considering that some videos do not have chapter identification progress bars at the beginning and end of the video, it is also possible to block the beginning and / or end of the video based on experience or statistical data, and only extract frames from the middle part. For example, 10% of the front and back of the video are respectively blocked, and only the middle 80% is extracted. This can further reduce the computational workload and improve the recognition and extraction efficiency of chapter identification text.

[0090] The extracted video frames are sorted according to the playback time to obtain a video frame sequence.

[0091] Step S102 : performing chapter marker text recognition on each video frame in the video frame sequence according to preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame.

[0092] The above-mentioned chapter marker text structural features may include display position features and / or format features, wherein the display position features refer to the position features of the chapter marker text in the video frame. For example, most of the chapter marker text is set at the bottom of the video, its height is generally less than 10% of the video frame height, and its width generally spans the entire time axis, etc.; the format features may refer to the rotation angle, character string length, etc. of the chapter marker text. For example, the chapter marker text is generally set horizontally, its rotation angle is generally less than 5 degrees, and the character string length generally does not exceed 32, etc.; technical personnel in this field can pre-set appropriate chapter marker text structural features according to actual needs or according to statistical data, and then perform targeted recognition of the chapter marker text based on the chapter marker text structural features to improve recognition efficiency and accuracy.

[0093] When performing chapter marker text recognition, there are multiple implementation methods. For example, one feasible implementation method is to first identify all text in the video frame through OCR (Optical Character Recognition) technology, and then match candidate text instances based on preset chapter marker text structure features. In another feasible implementation method, the OCR recognition area in each video frame can be determined based on the display position features, and then text recognition is performed on the OCR recognition area, and then candidate text instances are matched based on format features. These will be further explained in the subsequent embodiment descriptions.

[0094] It should be noted that, among the candidate text instances obtained by the above identification, each candidate text instance may include at least one piece of information such as the text corresponding to a text box, location coordinates, format, text recognition confidence, video frame sequence number, etc., so as to facilitate subsequent in-depth processing to improve the accuracy of chapter identification text extraction.

[0095] In step S103 , the same candidate text instances between adjacent video frames are bound to each other through text tracking, and the candidate text instances are output to an inter-frame binding output result set according to the binding relationship.

[0096] Since the chapter identification progress bar has a strong time persistence, the chapter identification text between adjacent video frames has a certain consistency. For example, the same chapter identification text should be consistent in at least two adjacent video frames. If the same candidate text instances between adjacent video frames are bound, then the candidate text instances with a binding relationship are more likely to be chapter identification texts. Based on this, the embodiment of the present application uses text tracking to inter-frame bind the same candidate text instances between adjacent video frames and output them to the inter-frame binding output result set, so as to further screen and determine the chapter identification text based on the candidate text instances in the inter-frame binding output result set.

[0097] Step S104, performing global processing on all candidate text instances in the inter-frame binding output result set, and determining the chapter identification text based on the globally processed candidate text instances, wherein the global processing includes at least one of global binding, global filtering, row association, and row filtering.

[0098] In order to improve the accuracy of chapter marker text extraction, the embodiment of the present application adopts a method that combines inter-frame binding and global processing. After inter-frame binding, global processing is further performed to accurately screen and obtain the chapter marker text.

[0099] Among them, global binding can mean comparing all candidate text instances pairwise and binding the candidate text instances with the same comparison results, thereby further binding the candidate text instances after inter-frame binding to reduce misjudgment caused by inter-frame binding errors.

[0100] Global screening may refer to screening out candidate text instances whose occurrence frequency is greater than a preset threshold from all candidate text instances; wherein the occurrence frequency includes the ratio of the number of video frames in which the candidate text instance appears to the total number of frames in the video frame sequence. For example, since chapter identification text is generally displayed during most of the video playback time, so that users can understand the playback progress and content in real time, its occurrence frequency should not be lower than a certain preset threshold. The threshold can be set according to actual needs, such as 70%, 80%, etc., and is not limited in the embodiments of the present application. Through global screening, candidate text instances can be further screened from the dimensions of time and frequency of occurrence to further improve the accuracy of chapter identification text extraction.

[0101] Row-by-row association may involve dividing all candidate text instances into rows based on their ordinate values ​​to obtain at least one row of candidate text instances. Then, for each row of candidate text instances, multiple candidate text instances in that row are sorted and associated in sequence based on their abscissa values ​​to obtain row-by-row associated candidate text instances. Considering that chapter identifier text may contain multiple lines, row-by-row association allows candidate text instances to be associated by line, thereby obtaining chapter identifier text that fits the actual display situation.

[0102] Filtering by line can refer to determining whether to retain or delete the candidate text instances on a line-by-line basis based on whether the number of candidate text instances in each line meets a preset threshold condition. Since the chapter identification text on the chapter identification progress bar is generally distributed at intervals, it will be recognized as multiple text boxes distributed at intervals during text recognition, each text box corresponding to a candidate text instance. Based on this, the same line should have multiple candidate text instances to meet the distribution characteristics of the chapter identification text. Therefore, based on whether the number of candidate text instances in each line meets the preset threshold condition, the candidate text instances can be further filtered, and candidate text instances that do not meet the above distribution characteristics can be deleted, further improving the accuracy of chapter identification text extraction.

[0103] It should be noted that the above-mentioned global processing methods such as global binding, global filtering, row association and row filtering can be used one by one, or multiple combinations can be selected for use. When multiple combinations are selected for use, the embodiments of the present application do not limit the implementation order of the multiple global processing methods. Those skilled in the art can flexibly set the implementation order according to actual needs. Among them, the input data of the subsequent global processing method should be the output data of the previous global processing method. For example, all candidate text instances input by row association should be all candidate text instances after global binding output by the previous global binding. The like will not be repeated one by one.

[0104] The embodiment of the present application provides a method for extracting chapter marker text in a video. The method extracts frames from a video file to be processed to obtain a video frame sequence. Then, based on preset chapter marker text structural features, chapter marker text recognition is performed on each video frame in the video frame sequence to obtain a candidate text instance corresponding to each video frame. Then, the same candidate text instances between adjacent video frames are bound between frames through text tracking, and the candidate text instances are output to an inter-frame binding output result set according to the binding relationship. Then, global processing is performed on all candidate text instances in the inter-frame binding output result set, wherein the global processing includes at least one of global binding, global filtering, row-by-row association, and row-by-row filtering. The chapter marker text can be determined based on the candidate text instances after global processing. The embodiment of the present application fully utilizes the structural features of the chapter marker text to identify the chapter marker text, and realizes tasks such as binding, filtering, and association of the chapter marker text through the combination of inter-frame binding and global processing, thereby accurately extracting the chapter marker text in the video. At the same time, the pre-performed frame extraction processing can effectively reduce the workload of subsequent processing and improve the overall extraction efficiency of the chapter marker text.

[0105] Regarding various implementations of performing chapter marker text recognition in step S102, in some modified implementations, step S102 performs chapter marker text recognition on each video frame in the video frame sequence based on preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame, including:

[0106] Performing text recognition on each video frame in the video frame sequence to obtain a text instance corresponding to each video frame;

[0107] Candidate text instances that meet preset chapter identification text structure features are screened out from the text instances.

[0108] In this embodiment, the OCR text recognition algorithm can be used to directly perform text recognition on all the text in each video frame to obtain all the text boxes corresponding to each video frame. Then, based on the text structure features such as the position features and / or format features of each text box, feature matching is performed according to the preset chapter identification text structure features to screen out candidate text instances. Since OCR text recognition is a relatively mature technology, it can be directly applied to text recognition in video frames. This method of first recognizing the text and then screening the instances is less difficult to implement and can achieve better recognition accuracy and efficiency.

[0109] Based on the above embodiment, the above preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0110] The step of selecting candidate text instances that meet preset chapter identification text structure features from the text instances may include:

[0111] According to the display position information and format information of each text instance, candidate text instances that meet the display position feature and the format feature are matched from the text instances.

[0112] Among them, the display position feature refers to the position feature of the chapter marking text in the video frame. For example, most of the chapter marking text is set at the bottom of the video, its height is generally less than 10% of the video frame height, and its width generally spans the entire timeline; the format feature can refer to the rotation angle, character string length, etc. of the chapter marking text. For example, the chapter marking text is generally set horizontally, its rotation angle is generally less than 5 degrees, and the character string length generally does not exceed 32; by presetting the chapter marking text structural features and then comparing them with the display position features and format features of the text instance, the text instance that meets the chapter marking text structural features can be matched and recorded as a candidate text instance.

[0113] Through this embodiment, the text instances can be preliminarily screened from the two dimensions of display position and format to obtain candidate text instances with a high probability of containing chapter marking text, so as to further identify the chapter marking text based on the candidate text instances.

[0114] In some other embodiments of the chapter marker text recognition in step S102, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0115] The above step S102, performing chapter marker text recognition on each video frame in the video frame sequence according to the preset chapter marker text structure features to obtain a candidate text instance corresponding to each video frame, includes:

[0116] determining a chapter identification text recognition region of each video frame in the video frame sequence according to the display position feature;

[0117] Performing text recognition on the chapter identification text recognition area of ​​each video frame to obtain a regional text instance;

[0118] The regional text instances are screened according to the format features to obtain candidate text instances.

[0119] Through this implementation, text recognition can be first performed on a specific area of ​​each video frame based on the display position features, which can effectively reduce the workload of text recognition and avoid ineffective work of text recognition on other areas. After obtaining the regional text instance, the candidate text instance is screened out based on the format feature matching. This method of first demarcating the area for text recognition and then performing format matching is highly targeted in the text recognition stage, which can effectively improve the text recognition efficiency and thus improve the overall recognition efficiency of chapter identification text.

[0120] In some more specific implementations, the above step S103, binding the same candidate text instances between adjacent video frames through text tracking, and outputting the candidate text instances to an inter-frame binding output result set according to the binding relationship, may include:

[0121] Traverse the candidate text instances of each video frame in the video frame sequence, add the candidate text instance of the first video frame to the cache area corresponding to the video frame and output it to the inter-frame binding output result set, and for each subsequent candidate text instance of the video frame, determine whether the same candidate text instance exists in the cache area of ​​the previous frame. If so, inter-frame bind the candidate text instance with the same candidate text instance of the previous frame and output it to the inter-frame binding output result set; if not, write the candidate text instance into the cache area corresponding to the current video frame and output it to the inter-frame binding output result set.

[0122] Through this implementation, the same candidate text instances in adjacent video frames can be bound between frames, and all traversed candidate text instances can be output to the inter-frame binding output result set, reducing the probability of misjudgment and missed judgment.

[0123] Among them, for the comparison between candidate text instances, the text box-based target detection algorithm provided by the prior art can be used as the basic algorithm. In this embodiment of the application, the intersection over union (IOU) is selected. The intersection-over-union (IOU) and edit distance are used as indicators for determining whether candidate text instances are identical. The intersection-over-union (IOU) calculates the ratio of the intersection to the union of two text boxes, which can be used to measure the degree of matching of the position information of two candidate text instances. For example, if the text box positions of two candidate text instances completely overlap, the IOU is 1. The smaller the value, the less overlap. The edit distance calculates the minimum number of times a string is processed to convert one string into another string, which can be used to measure the similarity of the texts in two candidate text instances. In the implementation of the embodiment of the present application, a threshold for the IOU and a threshold for the edit distance can be set. For example, the threshold for the IOU is set to 0.5. If it is less than 0.5, the two text boxes are considered to be different. The threshold for the edit distance is set to 40% of the average length of the two compared texts. If it is greater than the threshold, the two text boxes are considered to be different. In specific implementation, the above two indicators can be used simultaneously. For example, the two candidate text instances are considered to be identical only when the IOU is greater than 0.5 and the edit distance is less than 40% of the average length of the two compared texts. Thus, the consistency of the candidate text instances is judged from the two dimensions of position matching and string matching, which can effectively improve the accuracy of the comparison of candidate text instances. It should be noted that the above thresholds can be flexibly set according to actual needs. The above is only an exemplary description and should not be understood as limiting the scope of protection of this application.

[0124] Regarding the above step S104, global processing is performed on all candidate text instances in the inter-frame binding output result set, and the chapter identification text is determined based on the globally processed candidate text instances. In some specific implementations, the global processing includes global binding;

[0125] The global binding includes:

[0126] All candidate text instances in the inter-frame binding output result set are compared pairwise, and the candidate text instances with the same comparison results are bound.

[0127] Through global binding, the candidate text instances after inter-frame binding can be further bound to reduce misjudgment caused by inter-frame binding errors.

[0128] Based on the above implementation, in some modified implementations, the global processing further includes global screening;

[0129] The global screening includes:

[0130] From the globally bound candidate text instances, candidate text instances having an appearance frequency greater than a preset threshold are screened out; wherein the appearance frequency includes a ratio of the number of video frames in which the candidate text instance appears to the total number of frames in the video frame sequence.

[0131] Through global filtering, candidate text instances can be further filtered from the dimensions of time and frequency of occurrence to further improve the accuracy of chapter marker text extraction.

[0132] Based on the above implementation, in some modified implementations, the global processing further includes row-by-row association;

[0133] The row-by-row association includes:

[0134] For all candidate text instances selected by global screening, dividing the candidate text instances into rows according to their ordinate values ​​to obtain at least one row of candidate text instances;

[0135] For each row of candidate text instances, multiple candidate text instances in the row are sorted according to the horizontal coordinate values ​​of the candidate text instances and sequentially associated to obtain candidate text instances associated by row.

[0136] By associating by line, the candidate text instances can be associated by line, so as to obtain chapter identification text that fits the actual display situation.

[0137] Based on the above implementation, in some modified implementations, the global processing further includes filtering by row;

[0138] The row filtering includes:

[0139] For the candidate text instances associated by row, whether to retain or delete the candidate text instances by row is determined based on whether the number of candidate text instances in each row meets a preset threshold condition.

[0140] Since the chapter marking text on the chapter marking progress bar is generally distributed at intervals, it will be recognized as multiple text boxes distributed at intervals during text recognition. Each text box corresponds to a candidate text instance. Based on this, the same line should have multiple candidate text instances to meet the distribution characteristics of the chapter marking text. Therefore, based on whether the number of candidate text instances in each line meets the preset threshold conditions, the candidate text instances can be further screened, and the candidate text instances that do not meet the above distribution characteristics can be deleted to further improve the accuracy of chapter marking text extraction.

[0141] For ease of understanding, the method for extracting chapter identification text in the above-mentioned video of this application is further explained as follows with reference to a specific example.

[0142] For a specific example, see Figure 3 It is understood that the method for extracting chapter identification text in the video includes the following processes:

[0143] First, the input video, i.e., the video file to be processed, is subjected to frame extraction processing (one frame per second by default) to obtain a video frame sequence. Since the chapter identifier text appears throughout the entire video by default, some frames can be sampled to improve the algorithm's computational efficiency. For example, the default 10% of the video duration is the prologue and ending, and in the middle 80% of the area, 10% of the video frames are sampled at one frame per second to obtain a video frame sequence. Then, text detection and text recognition are performed in sequence according to the frame order. The text recognition results are further filtered out of irrelevant text using a structured strategy (such as the preset chapter identifier text structural features), thereby obtaining candidate text instances on each frame. Before switching to the next frame calculation, the result of the current frame is stored in the cache and compared with the result of the next frame to track the text instance. Finally, the tracking result is post-processed to remove duplicates between text instances and output the chapter identifier text after binding.

[0144] Among them, the extraction of frames can be implemented using the solution provided by the open source computer program FFMPEG, the detection can be implemented using a DB (Differentiable Binarization, a differentiable binarization processing, which is an open source text detection method) detector trained based on business data, and the recognition can be implemented using a CRNN (Convolutional Recurrent Neural Network) recognizer obtained based on business data and synthetic data. These are all relatively mature technical solutions. Those skilled in the art can refer to the existing technology for flexible implementation, and will not be repeated here.

[0145] for Figure 3 The inter-frame strategy in [1] aims to bind identical text instances in adjacent frames, filter out irrelevant text, and locate chapter-related text information. Therefore, this strategy is mainly set based on aspects such as text box coordinates and recognition results, including the following strategies 1.1-1.5:

[0146] 1.1. Considering the y-axis position of the text coordinates, chapter marker text should be located at the bottom of the video. For example, the y-axis coordinate of the center point of the text box should be within the range of 95% to 100% of the image height.

[0147] 1.2. Considering the x-axis coordinate of the center point of the text box, there should be a certain margin between the center point of the chapter mark text and the edge of the video. For example, the x-axis coordinate of the center point of the text box should be within the range of 1% to 99% of the video width to avoid difficult cases in some scenarios (such as some buttons in games).

[0148] 1.3. The rotation angle of the chapter mark text is less than 5°.

[0149] 1.4. The length of the chapter identifier text string shall not exceed 32 characters.

[0150] 1.5. Keep the text boxes that meet the above conditions on the current frame, store them in the current frame buffer, and compare them with the cache results of the previous frame to perform inter-frame text instance binding (text tracking).

[0151] The specific methods of text tracking are as follows 2.1-2.6:

[0152] 2.1. Traverse the extraction results of the current frame in sequence and compare them with the cached results of the previous frame.

[0153] 2.2. To consider that the current frame text box A and the previous frame buffer result text box B are the same text instance, the following conditions must be met: a. The IOU value is greater than 0.5; b. The edit distance of the recognition result is less than 40% of the average length of the two.

[0154] 2.3. The text box of the current frame can find an instance that meets 2.2 from the cache results, and the two are bound.

[0155] 2.4. If there is no matching object in the cached result of the current frame, it is placed in the cached result of the current frame as a new instance and independently added to the final output result set.

[0156] 2.5. After processing the current frame, add the binding results in the current frame buffer to the final output result set, write the current frame buffer to the previous frame buffer, and clear the current frame buffer.

[0157] 2.6. Perform the calculations in 2.1-2.5 frame by frame and return the final output result set.

[0158] So far, we have obtained the results of text instance binding frame by frame for the sampled video frames. We will further use a global strategy to deduplicate instance information and associate information between instances. The specific steps of the global strategy are as follows 3.1-3.5:

[0159] 3.1. To prevent tracking disconnection caused by disturbances in detection or recognition results, or deviations in thresholds set by the strategy, compare the IOU and edit distance between instances, and further bind those that match.

[0160] 3.2. The frequency of each text instance is required to be greater than 70% of the total number of sampled frames, that is, it is considered to be a text instance throughout the entire video.

[0161] 3.3. Compare the y-axis coordinates of the center points of different instances one by one. If the difference between any two instances' y-axis coordinates is within 1% of the video height jitter, they are considered to be from the same industry and are linked. The average of the two y-axis coordinates is used as a reference value for linking the next instance. This step enables row-by-row binding (i.e., row-by-row association). Sort the bound text lines by their center point's x-axis coordinates from smallest to largest.

[0162] 3.4. If there are multiple bound rows, and at least one of them meets the requirement that the number of instances is greater than or equal to 3, then all the bound rows will be retained. If none of them meet the requirement, they will be skipped.

[0163] 3.5. From the binding results, select the recognition result with the highest confidence as the final output to obtain the chapter text set. The confidence level refers to the confidence level of the text recognition result during the text recognition phase and can be retained as information about the text instance. Selecting the text instance with the highest confidence level as the final output can further improve the accuracy of the recognized chapter text.

[0164] Based on the same inventive concept, an embodiment of the present application also provides a device for extracting chapter identification text in a video. The method corresponding to the device may be the method for extracting chapter identification text in a video in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The device for extracting chapter identification text in a video provided in the embodiment of the present application may implement the aforementioned method for extracting chapter identification text in a video, and the device for extracting chapter identification text in a video may be implemented by software, hardware, or a combination of software and hardware. For example, the device for extracting chapter identification text in a video may include integrated or separate functional modules or units to perform the corresponding steps in the aforementioned methods. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts may refer to the partial description of the method embodiment. The device embodiment described below is merely illustrative. Please refer to Figure 4 The device for extracting chapter identification text from the video may include:

[0165] The frame extraction module 101 is used to extract frames from the video file to be processed to obtain a video frame sequence;

[0166] The chapter text recognition module 102 is configured to perform chapter identification text recognition on each video frame in the video frame sequence according to preset chapter identification text structural features, and obtain a candidate text instance corresponding to each video frame;

[0167] An inter-frame binding module 103 is configured to inter-frame bind the same candidate text instances between adjacent video frames through text tracking, and output the candidate text instances to an inter-frame binding output result set according to the binding relationship;

[0168] The global processing module 104 is used to perform global processing on all candidate text instances in the inter-frame binding output result set and output the chapter identification text, wherein the global processing includes at least one of global deduplication, global screening, global binding and row-by-row association.

[0169] In some modified implementations, the chapter text recognition module 102 includes:

[0170] a text recognition unit, configured to perform text recognition on each video frame in the video frame sequence to obtain a text instance corresponding to each video frame;

[0171] The text instance screening unit is used to screen out candidate text instances that meet preset chapter identification text structure features from the text instances.

[0172] In some modified implementations, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0173] The text instance screening unit includes:

[0174] The text instance screening subunit is used to match the text instances according to the display position information and format information of each text instance to obtain candidate text instances that meet the display position characteristics and the format characteristics.

[0175] In some modified implementations, the preset chapter marker text structural features include display position features and format features of the chapter marker text in the video frame;

[0176] The chapter text recognition module 102 includes:

[0177] a recognition region determining unit, configured to determine a chapter identification text recognition region of each video frame in the video frame sequence according to the display position feature;

[0178] a regional text recognition unit, configured to perform text recognition on the chapter identification text recognition region of each video frame to obtain a regional text instance;

[0179] The format screening unit is used to screen the regional text instances according to the format features to obtain candidate text instances.

[0180] In some modified implementations, the inter-frame binding module 103 includes:

[0181] An inter-frame binding unit is used to traverse the candidate text instances of each video frame in the video frame sequence, add the candidate text instance of the first video frame to the buffer area corresponding to the video frame and output it to the inter-frame binding output result set, and for each subsequent candidate text instance of the video frame, determine whether the same candidate text instance exists in the buffer area of ​​the previous frame. If so, the candidate text instance is inter-frame bound to the same candidate text instance of the previous frame and output it to the inter-frame binding output result set; if not, the candidate text instance is written to the buffer area corresponding to the current video frame and output it to the inter-frame binding output result set.

[0182] In some modified implementations, the global processing module 104 includes:

[0183] The global binding unit is used to compare all candidate text instances in pairs and bind the candidate text instances with the same comparison results.

[0184] In some modified implementations, the global processing module 104 includes:

[0185] A global screening unit is used to screen out candidate text instances whose occurrence frequency is greater than a preset threshold from all candidate text instances; wherein the occurrence frequency includes the ratio of the number of video frames in which the candidate text instance appears to the total number of frames in the video frame sequence.

[0186] In some modified implementations, the global processing module 104 includes:

[0187] A line division processing unit, configured to divide all candidate text instances into lines according to the ordinate values ​​of the candidate text instances to obtain at least one line of candidate text instances;

[0188] The row binding unit is used to sort and sequentially associate multiple candidate text instances in each row of candidate text instances according to the horizontal coordinate values ​​of the candidate text instances to obtain candidate text instances associated by row.

[0189] In some modified implementations, the global processing module 104 includes:

[0190] The row-by-row screening unit is configured to determine whether to retain or delete the candidate text instances row by row based on whether the number of candidate text instances in each row meets a preset threshold condition.

[0191] The device for extracting chapter marking text in a video provided in an embodiment of the present application is based on the same inventive concept as the method for extracting chapter marking text in a video provided in the aforementioned embodiment of the present application and has the same beneficial effects.

[0192] Based on the same inventive concept, an embodiment of the present application further provides an electronic device. The method corresponding to the device may be the method for extracting chapter marker text from a video in the aforementioned embodiment, and the principle of solving the problem is similar to that of the aforementioned method. The device includes a memory for storing computer program instructions and a processor for executing the computer program instructions. When the computer program instructions are executed by the processor, the device is triggered to execute the aforementioned method for extracting chapter marker text from a video.

[0193] The device can be a user device, or a device formed by integrating a user device and a network device via a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablets, smart watches, and wristbands. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections, and can be used to implement some of the processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing). Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computers.

[0194] Figure 5 A block diagram of an exemplary computer system / server 12 suitable for implementing embodiments of the present application is shown. Figure 5 The computer system / server 12 shown is merely an example and should not limit the functionality and scope of use of the embodiments of the present application.

[0195] like Figure 5 As shown, computer system / server 12 is implemented as a general-purpose computing device. Components of computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, storage or system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing units 16).

[0196] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0197] Computer system / server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer system / server 12, including volatile and non-volatile media, removable and non-removable media.

[0198] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, often called a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present application.

[0199] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0200] The computer system / server 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer system / server 12, and / or any device that enables the computer system / server 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can occur via an input / output (I / O) interface 44. Furthermore, the computer system / server 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer system / server 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the computer system / server 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0201] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing Figure 2 A method for extracting chapter identification text from a video provided by any corresponding embodiment.

[0202] Another embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, Figure 2 A method for extracting chapter identification text from a video provided by any corresponding embodiment.

[0203] Specifically, any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0204] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0205] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0206] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0207] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0208] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0210] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0211] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0212] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0213] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

Claims

1. A method for extracting chapter identification text in a video, wherein: The method comprises: Extract frames from the video file to be processed to obtain a video frame sequence; Performing chapter marker text recognition on each video frame in the video frame sequence based on preset chapter marker text structural features to obtain candidate text instances corresponding to each video frame, wherein each candidate text instance includes at least one of information selected from the group consisting of text corresponding to a text box, location coordinates, format, text recognition confidence, and a video frame sequence number; Inter-frame binding is performed on the identical candidate text instances between adjacent video frames through text tracking, and the candidate text instances are output to an inter-frame binding output result set according to the binding relationship, wherein the inter-frame binding is used to determine the binding relationship of the identical candidate text instances in adjacent video frames; Global processing is performed on all candidate text instances in the inter-frame binding output result set, and chapter identification text is determined based on the globally processed candidate text instances, wherein the global processing includes at least one of global binding, global filtering, row-by-row association, and row-by-row filtering.

2. The method for extracting chapter identification text in a video according to claim 1, wherein: The step of performing chapter identification text recognition on each video frame in the video frame sequence according to preset chapter identification text structural features to obtain a candidate text instance corresponding to each video frame includes: Performing text recognition on each video frame in the video frame sequence to obtain a text instance corresponding to each video frame; Candidate text instances that meet preset chapter identification text structure features are screened out from the text instances.

3. The method for extracting chapter identification text in a video according to claim 2, wherein: The preset chapter mark text structural features include display position features and format features of the chapter mark text in the video frame; The step of selecting candidate text instances that meet preset chapter identification text structure features from the text instances includes: According to the display position information and format information of each text instance, candidate text instances that meet the display position feature and the format feature are matched from the text instances.

4. The method for extracting chapter identification text in a video according to claim 1, wherein: The preset chapter mark text structural features include display position features and format features of the chapter mark text in the video frame; The step of performing chapter identification text recognition on each video frame in the video frame sequence according to preset chapter identification text structural features to obtain a candidate text instance corresponding to each video frame includes: determining a chapter identification text recognition region of each video frame in the video frame sequence according to the display position feature; Performing text recognition on the chapter identification text recognition area of ​​each video frame to obtain a regional text instance; The regional text instances are screened according to the format features to obtain candidate text instances.

5. The method for extracting chapter identification text in a video according to claim 1, wherein: The step of inter-frame binding the same candidate text instances between adjacent video frames through text tracking, and outputting the candidate text instances to an inter-frame binding output result set according to the binding relationship, includes: Traverse the candidate text instances of each video frame in the video frame sequence, add the candidate text instance of the first video frame to the cache area corresponding to the video frame and output it to the inter-frame binding output result set, and for each subsequent candidate text instance of the video frame, determine whether the same candidate text instance exists in the cache area of ​​the previous frame. If so, inter-frame bind the candidate text instance with the same candidate text instance of the previous frame and output it to the inter-frame binding output result set; if not, write the candidate text instance into the cache area corresponding to the current video frame and output it to the inter-frame binding output result set.

6. The method for extracting chapter identification text in a video according to claim 1, wherein: The global processing includes global binding; The global binding includes: All candidate text instances are compared pairwise, and the candidate text instances with the same comparison results are bound.

7. The method for extracting chapter identification text in a video according to claim 1, wherein: The global processing also includes global screening; The global screening includes: From all candidate text instances, candidate text instances having an appearance frequency greater than a preset threshold are screened out; wherein the appearance frequency includes a ratio of the number of video frames in which the candidate text instance appears to the total number of frames in the video frame sequence.

8. The method for extracting chapter identification text in a video according to claim 1, wherein: The global processing further includes row-by-row association; the row-by-row association includes: For all candidate text instances, dividing the candidate text instances into rows according to their ordinate values ​​to obtain at least one row of candidate text instances; For each row of candidate text instances, multiple candidate text instances in the row are sorted according to the horizontal coordinate values ​​of the candidate text instances and sequentially associated to obtain candidate text instances associated by row.

9. The method for extracting chapter identification text in a video according to claim 8, wherein: The global processing also includes filtering by row; the filtering by row includes: According to whether the number of candidate text instances in each line meets a preset threshold condition, it is determined whether to retain or delete the candidate text instances on a line-by-line basis.

10. A device for extracting chapter identification text in a video, wherein: The device comprises: The frame extraction module is used to extract frames from the video file to be processed to obtain a video frame sequence; a chapter text recognition module, configured to perform chapter identifier text recognition on each video frame in the video frame sequence based on preset chapter identifier text structural features, and obtain candidate text instances corresponding to each video frame, wherein each candidate text instance includes at least one of information selected from the group consisting of text corresponding to a text box, location coordinates, format, text recognition confidence, and video frame sequence number; An inter-frame binding module is used to inter-frame bind the same candidate text instances between adjacent video frames through text tracking, and output the candidate text instances to an inter-frame binding output result set according to the binding relationship, wherein the inter-frame binding is used to determine the binding relationship of the same candidate text instances in adjacent video frames; A global processing module is used to perform global processing based on all candidate text instances in the inter-frame binding output result set, and determine the chapter identification text based on the globally processed candidate text instances, wherein the global processing includes at least one of global binding, global filtering, row association and row filtering.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

12. A computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 9.

Images