Text information extraction method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device and storage medium for extracting text information. The method comprises: obtaining N frame images of a video to be detected, determining a target frame image set according to the N frame images, each target frame image contained in the target frame image set contains a target text, extracting the target text according to the target frame image set, the start timestamp and the end timestamp of the target text are the timestamps corresponding to the target frame image at the start position in the target frame image set, and the end timestamp is the timestamp corresponding to the target frame image at the end position in the target frame image set. In the process of text extraction, the present invention can identify and completely extract the same target text, and extract the time information of the same target text according to the corresponding timestamps of all target frame images containing the same target file, without the need for manual time marking, thereby improving the accuracy and efficiency of text information extraction.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a text information extraction method, device, electronic device and storage medium. Background Art

[0002] At present, the video client on the terminal device has become an important tool for users to watch videos. Various videos can be translated or edited through subtitles, and video information from different countries, regions and languages ​​can be converted into video information that users can understand. If subtitles can be extracted from videos and converted into editable text files, it will greatly reduce the burden of subtitle translation and subtitle processing.

[0003] In actual applications, for some use needs, it is necessary to extract subtitles from videos. However, in the process of extracting subtitles from videos, existing subtitle extraction methods have the following disadvantages: low accuracy, inability to extract the same sentence completely, and the timestamps of texts are mostly manually annotated later, which is time-consuming and inefficient. Summary of the invention

[0004] In view of this, the present invention provides a text information extraction method, device, electronic device and storage medium to improve the accuracy and efficiency of text information extraction. In order to achieve the above beneficial effects, the technical solution of the present invention is as follows:

[0005] In a first aspect, the present invention provides a method for extracting text information, the method comprising: acquiring N frame images of a video to be detected, wherein N is greater than zero; determining a target frame image set based on the N frame images, wherein each target frame image contained in the target frame image set contains a target text, and the target frame image belongs to the N frame images; determining a target text, a start timestamp and an end timestamp of the target text based on the target frame image set, wherein the start timestamp is a timestamp corresponding to a target frame image at a start position in the target frame image set; and the end timestamp is a timestamp corresponding to a target frame image at an end position in the target frame image set.

[0006] Optionally, determining the target frame image set based on the N frame images includes: judging whether any adjacent frame images among the N frame images have the same target text; when the target text is present in the nth frame image and the target text is not present in the n-1th frame image, using the nth frame image as the target frame image at the starting position of the target frame image set; when the target text is not present in the n+mth frame image and the target text is present in the n+m-1th frame image, using the n+m-1th frame image as the target frame image at the ending position of the target frame image set.

[0007] Optionally, the determining whether any adjacent frame images among the N frame images have the same target text includes: when there is a first target text in the nth frame image and a second target text in the n-1 frame image, determining whether a repetition rate between the first target text and the second target text is greater than or equal to a repetition threshold; if the repetition rate is greater than or equal to the repetition threshold, determining that the nth frame image and the n-1 frame image have the same target text.

[0008] Optionally, the method of obtaining N frame images of the video to be detected includes: obtaining the starting text coordinates of the video to be detected; determining a text detection area based on the starting text coordinates; the text detection area is an area obtained by expanding the area where the starting text is located; obtaining the N frame images based on the text detection area; the N frame images are screenshots of each frame image of the video to be detected that includes the text detection area.

[0009] Optionally, before obtaining the starting text coordinates of the video to be detected, the method further includes: extracting all text information of the video to be detected; and the starting text coordinates are included in the all text information.

[0010] Optionally, after extracting all text information of the video to be detected, the method further includes: when there are at least two texts in any frame image, splicing all texts corresponding to the any frame image, and using the spliced ​​text as the text of the any frame image.

[0011] Optionally, the method also includes: obtaining a character set corresponding to each character position of the target text; the character set contains characters in at least one font format; for each character set corresponding to the character position, taking the character corresponding to the maximum probability value as the target character of the character position; the target text is a text composed of the target characters of each character position.

[0012] In a second aspect, the present invention provides a text information extraction device, comprising: an acquisition module, used to acquire N frame images of a video to be detected, wherein N is greater than or equal to zero; a determination module, used to determine a target frame image set based on the N frame images, wherein each target frame image contained in the target frame image set contains a target text, and the target frame image belongs to the N frame images; an extraction module, used to extract the target text, a start timestamp and an end timestamp of the target text based on the target frame image set, wherein the start timestamp is a timestamp corresponding to the target frame image at the start position in the target frame image set; and the end timestamp is a timestamp corresponding to the target frame image at the end position in the target frame image set.

[0013] In a third aspect, the present invention provides an electronic device, comprising a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the processor executes the machine-executable instructions, the electronic device implements the text information extraction method described in the first aspect.

[0014] In a fourth aspect, the present invention provides a storage medium storing machine executable instructions, which, when executed, implement the text information extraction method described in the first aspect.

[0015] The present invention provides a method, device, electronic device and storage medium for extracting text information, the method comprising: obtaining N frame images of a video to be detected, wherein N is greater than or equal to zero; determining a target frame image set according to the N frame images, wherein each target frame image contained in the target frame image set contains a target text, and the target frame image belongs to the N frame images; determining a target text, a start timestamp and an end timestamp of the target text according to the target frame image set, wherein the start timestamp is a timestamp corresponding to the target frame image at the start position of the target frame image set; and the end timestamp is a timestamp corresponding to the target frame image at the end position of the target frame image set. In the process of text extraction, the present invention can identify and completely extract the same target text, and extract the time information of the same target text according to the corresponding timestamps of all target frame images containing the same target file, without the need for manual time marking, thereby improving the accuracy and efficiency of text information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic flow chart of a text information extraction method provided by an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of the results of the text information extraction method provided by an embodiment of the present invention;

[0019] Figure 3 A schematic flow chart of an implementation method of step S101 provided by the present invention;

[0020] Figure 4 A schematic diagram of a text detection area is provided for an embodiment of the present invention;

[0021] Figure 5 A schematic flow chart of another implementation of step S101 provided in an embodiment of the present invention;

[0022] Figure 6 A schematic flow chart of another implementation of step S101 provided in an embodiment of the present invention;

[0023] Figure 7 A schematic diagram of a text splicing scenario is provided for an embodiment of the present invention;

[0024] Figure 8 A schematic flow chart of an implementation method of step S102 provided in an embodiment of the present invention;

[0025] Fig. 9 A text example diagram of a continuous frame image provided by an embodiment of the present invention;

[0026] Fig.10 A schematic flowchart of an implementation method of step S102-1 provided in an embodiment of the present invention;

[0027] Fig.11 A schematic flow chart of another text information extraction method provided by an embodiment of the present invention;

[0028] Fig.12 A functional module diagram of a text information extraction device provided by an embodiment of the present invention;

[0029] Fig.13 A structural block diagram of an electronic device is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0031] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0033] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0034] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0035] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0036] At present, with the development of network video technology, various videos, including dialogues and monologues in TV dramas, movies, musicals, and stage plays, can be converted into subtitles by adding and editing text at a later stage. This allows users to master and learn information from different countries, regions, and languages ​​without leaving home. Users can also extract subtitles from videos, convert them into editable text files, and add them to their favorite video works. Therefore, subtitle extraction will greatly reduce the burden of subtitle translation and subtitle processing.

[0037] In actual applications, related technologies can only extract subtitles from one frame of video images during video subtitle extraction, and cannot recognize the same sentence in the video. In addition, the time information corresponding to the extracted subtitles is often manually added later, which is time-consuming.

[0038] For example, two adjacent frame images contain the same sentence "Have you eaten today?", and the text extraction results [text][start time, end time] of the related technology may be: [Have you eaten today][500,1000], [Have you eaten today][1200,1700]. Obviously, in the process of text extraction, the text "Have you eaten today?" appears twice in the text file, which is obviously not in line with the user's reality. The same text in adjacent frames should eventually appear as a complete sentence, such as [Have you eaten today][500,1700], and the above time information is added manually in the later stage, which obviously reduces the accuracy and efficiency of text information extraction.

[0039] In order to solve the above technical problems, the inventors, after a series of studies, proposed a text information extraction method provided in an embodiment of the present invention.

[0040] On the one hand, in the process of extracting text from video images, the text information extraction method of the embodiment of the present invention can obtain all frame images containing the same sentence, and set the starting timestamp corresponding to the target frame image at the starting position of all target frame images; and set the starting timestamp corresponding to the target frame image at the ending position of all target frame images. In this way, it can not only complete the recognition of the same text, but also accurately extract the time information corresponding to the text.

[0041] On the other hand, the execution subject of the text information extraction method provided by the embodiment of the present invention can be the text extraction device provided by the embodiment of the present invention, or an electronic device integrated with the text extraction device, wherein the text extraction device can be implemented in hardware or software. Among them, the electronic device can be a smart phone, a tablet computer, a PDA, a laptop computer, or a desktop computer. In other words, the embodiment of the present invention can realize a fully automated text extraction process, thereby improving the efficiency of text information extraction.

[0042] The following will be combined Figures 1 to 11 The text information extraction method provided by the embodiment of the present invention is introduced.

[0043] See first Figure 1 , Figure 1 A schematic flow chart of a text information extraction method provided by an embodiment of the present invention includes the following steps:

[0044] S101. Obtain N frame images of a video to be detected, where N is greater than zero.

[0045] S102 . Determine, based on the N frame images, a target frame image set in which each target frame image contains a target text.

[0046] It can be understood that the target frame image belongs to the above-mentioned N frame images.

[0047] S103: Extracting target text and a start timestamp and an end timestamp of the target text according to the target frame image set.

[0048] It can be understood that the above-mentioned start timestamp is the timestamp corresponding to the target frame image at the start position in the target frame image set; the end timestamp is the timestamp corresponding to the target frame image at the end position in the target frame image set.

[0049] To show the extraction results of the above text information extraction method, see Figure 2 , Figure 2 A schematic diagram of the results of the text information extraction method provided in an embodiment of the present invention.

[0050] like Figure 2As shown, for a video to be detected, through the text information extraction method provided by the embodiment of the present invention, the extracted text information includes "text content (text): ..., start time (bg): ..., end time (ed): ...", that is:

[0051] "text: If I do it today for love", bg: 560, ed: 1880";

[0052] "text: chose you", bg: 1920, ed: 3240";

[0053] "text: Then one day", bg: 3360, ed: 3920";

[0054] "text: I will definitely", bg: 3960, ed: 4760";

[0055] "text: I abandon you for love", bg: 4800, ed: 5640".

[0056] A text file can be generated based on the text information obtained above, and can also be applied in scenarios such as video dubbing, subtitle translation or subtitle editing.

[0057] It can be seen that the present invention provides a text information extraction method, which obtains N frame images of a video to be detected and determines a target frame image set containing the same target text. Since each frame image corresponds to a timestamp, the time information of the same target text is extracted based on the corresponding timestamps of all target frame images containing the same target file. There is no need to manually mark the timestamp information, which improves the accuracy and efficiency of text information extraction.

[0058] Optionally, the above-mentioned method of obtaining N frame images can be a complete video image corresponding to each frame of the video to be detected. The overall image obtained in this way will inevitably increase the detection time in the text detection process and reduce the efficiency of subsequent text extraction. In order to solve this problem, an embodiment of the present invention provides a method for obtaining N frame images, that is, the above-mentioned N frame images can be screenshots of each frame of the complete image, and the captured area is the area where the text appears in each frame of the complete image. In order to facilitate understanding of this implementation method of obtaining N frame images, the following is Figure 1 Based on this, a possible implementation is given, see Figure 3 , Figure 3 A schematic flowchart of an implementation method of step S101 provided by the present invention.

[0059] S101-2. Obtain the starting text coordinates of the video to be detected.

[0060] In the embodiment of the present invention, the starting text here refers to the text that appears for the first time in the entire video to be detected.

[0061] S101 - 3. Determine a text detection area according to the starting text coordinates.

[0062] It can be understood that the above-mentioned starting text is the text that appears for the first time in the entire video to be detected. In actual scenarios, the height or width of the area of ​​the subsequently appearing text in the image is different from the length or height of the area where the starting text is located. Therefore, in order to obtain all the texts and reduce the risk of text omission, the embodiment of the present invention takes the area where the starting text is located as a reference and expands the area where the starting text is located, so that the area obtained after the expansion contains all the texts as much as possible.

[0063] Based on the above principles, in a possible implementation method, the embodiment of the present invention expands the height of the area of ​​the starting text by a preset multiple (for example, 1) toward the top and bottom of the image, respectively, based on the height of the area of ​​the starting text, and expands the width of the area to be consistent with the width of the entire image, thereby obtaining a text detection area, which can contain all texts and reduce the risk of text missing.

[0064] S101 - 4 . Obtain N frame images according to the text detection area. The N frame images are screenshots of each frame image of the video to be detected that includes the text detection area.

[0065] For ease of understanding, an embodiment of the present invention provides a schematic diagram of a text detection area. Figure 4 , Figure 4 A schematic diagram of a text detection area is provided for an embodiment of the present invention.

[0066] like Figure 4 As shown in the figure, the image is a complete video image, and "What day is today" is the text where the subtitle appears for the first time, that is, the first text of the entire video to be detected, wherein the black solid rectangular box is the area where the starting text is located. On the basis of the black solid rectangular box, the height of the rectangular box is extended by a preset multiple (for example, 1 times) toward the top and bottom of the image, respectively, to obtain the following Figure 4 The black dotted rectangular frame shown in the figure, the area where the black dotted rectangular frame is located is the above-mentioned text detection area, and then each frame of the video image is captured, and the captured area is the area where the rectangular frame is located.

[0067] The N frame images obtained in the above manner are used for text detection and text coordinate extraction. Since the detection area is the area where text may exist, the text extraction software does not need to perform text detection on the entire image, thereby improving the speed of text detection and extraction.

[0068] Optionally, the above method for obtaining the coordinates of the starting text can be: perform text detection from the starting frame image of the video to be detected, stop detecting the text detected for the first time, use the text as the starting text, and extract the coordinates of the text. However, this method has to start detection from the first frame of the video each time, which is slow and time-consuming. An embodiment of the present invention provides a method: input the video to be detected into a text recognition network, extract all the text and store it in advance as a text database, and then directly extract the starting text from the text database. Therefore, in the following Figure 3 Based on this, a possible implementation method is given, see Figure 5 , Figure 5 This is a schematic flow chart of another implementation of step S101 provided in an embodiment of the present invention. Step S101 may further include:

[0069] S101-0, extract all text information including the starting text coordinates from the video to be detected.

[0070] It is understandable that in the actual implementation process, the text in all frame images of the video to be detected can be extracted by a text extractor or text extraction software to form a text set. The text extraction software can be a connectionist text proposal network (CTPN) model. All the extracted text information can be used as a text database. When the starting text coordinates are needed, they can be directly read from the text database, which improves the efficiency of data acquisition.

[0071] Optionally, in one embodiment, when all text information is extracted through step S101-0, since the text in each frame image may be scattered and fragmented, two texts may be extracted from the same frame image during the extraction process. This may cause the text in the same frame image to be confused and inaccurate when stored. Therefore, in order to solve this problem, the embodiment of the present invention provides a solution, that is, in Figure 5 Based on Figure 6 , Figure 6 This is a schematic flow chart of another implementation of step S101 provided in an embodiment of the present invention. Step S101 may further include:

[0072] S101-1. When any frame of image contains at least two texts, all the texts corresponding to the frame of image are spliced ​​together, and the spliced ​​texts are used as the texts of the frame of image.

[0073] It can be understood that all the text information extracted in step S101-0 includes the coordinate information corresponding to each text. Therefore, when two text coordinates are detected in any frame image, the points at the outermost edges of the corner points of each text coordinate box can be spliced ​​and fused into a rectangle to improve recognition accuracy and sentence judgment.

[0074] For ease of understanding, here is an example to illustrate. First, see Figure 7 , Figure 7 A schematic diagram of a text splicing scenario is provided for an embodiment of the present invention.

[0075] like Figure 7 As shown in (a), “Zhang San and Li Si” appear in the same frame of the image and should be considered as one text. However, in the actual text extraction process, “Zhang San” and “Li Si” may be returned to the user as two texts (e.g. Figure 7 (a) shows the actual extracted text coordinates). At this time, it is necessary to splice the coordinates of the corner points of the coordinate boxes of "Zhang San" and "Li Si" to obtain a target text coordinate (such as Figure 7 (a) shown in the dashed box); for example, Figure 7 (b) shows that the same text is divided into two lines, one above the other. When extracting text, two texts, "Please stop talking" and "What day is today?" may be returned. At this time, it is necessary to splice the coordinates of the corner points of the coordinate boxes of "Please stop talking" and "What day is today?" to obtain a target text coordinate (such as Figure 7 (b) is shown in the dashed box.

[0076] Optionally, after determining the target text in each frame image, the inventor found in the research that the existing method of determining the frame image set containing the same sentence is often to manually mark the frame images containing the same sentence, which has low accuracy and efficiency. Therefore, the embodiment of the present invention will provide an implementation method for obtaining the target text time information of each target text set on the basis of obtaining accurate text information. Figure 1 Based on this, we give a possible approach, see Figure 8 , Figure 8 A schematic flowchart of an implementation of step S102 provided in an embodiment of the present invention, wherein step S102 may include the following steps:

[0077] S102-1. Determine whether any adjacent frame images among the N frame images have the same target text.

[0078] S102-2: When the nth frame image has the target text and the n-1th frame image does not have the target text, the nth frame image is used as the target frame image at the starting position of the target frame image set.

[0079] S102-3. When there is no target text in the n+mth frame image and there is target text in the n+m-1th frame image, use the n+m-1th frame image as the target frame image at the end position of the target frame image set.

[0080] The above sub-steps can be understood as follows: for the above N frame images, the text detection result of each frame is compared with the detection result of the previous frame. No text is detected in the previous frame (n-1 frame), and text is detected in the current frame (n frame), indicating that the current frame (n frame) is the starting frame of the text. At this time, the text is extracted as the target text, and the timestamp corresponding to the current frame (n frame) is used as the starting time of the text; continue to detect frame by frame, and detect that the previous frame (n+m-1 frame) has the above text, and the current frame (n+m frame) does not detect the above text, indicating that the previous frame (n+m-1 frame) is the ending frame of the text. At this time, the timestamp corresponding to the previous frame (n+m-1 frame) is used as the end time of the above text.

[0081] It is understood that the above-mentioned “without target text” can be combined with Fig. 9 For understanding, see Fig. 9 , Fig. 9 This is an example diagram of text of continuous frame images provided by an embodiment of the present invention.

[0082] like Fig. 9 In the first case shown in FIG. 1 , the same target text “Let me see where you are” exists in the nth frame image to the n+m-1th frame image, but no text exists in the n-1th frame image and the n+mth frame image. This case corresponds to the case where the text display in the corresponding video image is paused. Fig. 9 In the second case shown, the same target text "Let me see where you are" exists in the nth frame image to the n+m-1th frame image, but the text in the nth frame image is "OK" and the text in the n+mth frame image is "here". Obviously, the nth frame image and the n+mth frame image do not have the same text as in the nth frame image to the n+m-1th frame image. This situation is caused by non-pause text display.

[0083] It can be understood that, for the above two situations, the target frame image set is all frame images with the same text, that is, the nth frame image to the n+m-1th frame image in the figure, the nth frame image is the target frame image at the starting position of the target frame image set, and the n+m-1th frame image is the target frame image at the ending position of the target frame image set.

[0084] Optionally, for Fig. 9In the first case shown, since there is a situation where there is no text in two adjacent frames, the same text in each frame image and the time information of the start position image and the end position image can be directly extracted. However, for Fig. 9 In the second case shown, since there is text in both adjacent frames, in order to accurately extract the target text, the embodiments of the present invention provide an implementation method for determining whether it is the same text according to the repetition rate. Next, on the basis of Figure 8 , an implementation method is given. Refer to Fig.10 , Fig.10 FIG. is a schematic flowchart of the implementation method of step S102-1 provided by the embodiment of the present invention, where step S102-1 includes the following sub-steps:

[0085] S102-1-1. When there is a first target text in the nth frame image and a second target text in the (n-1)th frame image, determine whether the repetition rate of the first target text and the second target text is greater than or equal to the repetition threshold.

[0086] S102-1-2. If the repetition rate is greater than or equal to the repetition threshold, it is determined that the nth frame image and the (n-1)th frame image have the same target text.

[0087] In an actual implementation, the first target text in the nth frame image and the second target text in the (n-1)th frame image can be input into the text recognition network to recognize the meaning of the text, and then the repetition rate of the two texts is calculated based on the meaning of the text. When the repetition rate is greater than or equal to the repetition threshold (for example, the repetition threshold is 20%), it is considered that the nth frame image and the (n-1)th frame image have the same target text, that is, the first target text and the second target text are the same target text.

[0088] Through the above implementation method, the target text can be determined more accurately, and then the time information of the target text can be determined, improving the accuracy of text extraction.

[0089] Optionally, in an actual scenario, for the same sentence, the following situation may occur: characters with the same semantics but different font formats appear at the same character position. For example, please look back at Fig. 9 , the text in the nth frame image to the (n+m-2)th frame image is all "I'll see where you are", but the text in the (n+m-1)th frame image is "I'll see where 伱 are". Obviously, the text from the nth frame image to the (n+m-1)th frame image is essentially the same sentence, but at the fourth character position of this same sentence, characters with the same semantics but different font formats appear: "你" and "伱". For this situation, to determine the final target text, in order to improve the accuracy of the target text, the embodiments of the present invention give a possible implementation method on the basis of Figure 1 , refer to Fig.11 , Fig.11 is a schematic flowchart of another text information extraction method provided by an embodiment of the present invention, further including:

[0090] S104. Obtain the character set corresponding to each character position of the target text.

[0091] It can be understood that the above character set contains characters in at least one font format; for example, please continue to refer to Fig. 9 , if the target text has 6 character positions, then the character sets corresponding to these 6 character positions will be obtained. For example, the character set corresponding to the first character position is [我, 我…我], which only contains the character "我" in one font format. For each character set, there may be characters with the same semantics but different font formats. For example, in the fourth character position, its character set contains characters with the same semantics but different font formats: "你" and "伱".

[0092] S105. For the character set corresponding to each character position, use the character corresponding to the maximum probability value as the target character of this character position, and the target text is the text composed of the target characters of each character position.

[0093] It can continue to refer to in Fig. 9 , obviously, the character "你" appears with the highest probability, so the character "你" can be used as the target character at this character position, and the finally obtained target text is "我看看你在哪里". In this way, an accurate target text can be obtained, reducing the repetition rate of the same text.

[0094] To implement each step in the embodiment of the above text information extraction method to achieve the corresponding technical effects, the following gives an implementation manner of a text information extraction device. Refer to Fig.12 , Fig.12 is a functional module diagram of a text information extraction device provided by an embodiment of the present invention. Among them, the text information extraction device 12 includes: an acquisition module 121, a determination module 122, and an extraction module 123.

[0095] The acquisition module 121 is used to acquire N frame images of the video to be detected, where N is greater than zero;

[0096] The determination module 122 is used to determine a target frame image set according to the N frame images. Each target frame image included in the target frame image set contains the target text, and the target frame image belongs to the N frame images;

[0097] The extraction module 123 is used to extract the target text, the start timestamp and the end timestamp of the target text according to the target frame image set, wherein the start timestamp is the timestamp corresponding to the target frame image at the start position in the target frame image set; and the end timestamp is the timestamp corresponding to the target frame image at the end position in the target frame image set.

[0098] It is understandable that the acquisition module 121, the elimination module 122 and the extraction module 123 can also be used to collaboratively execute Figure 1 , 3 , 5, 6, 8, 10, and 11 to achieve the corresponding technical effects.

[0099] The embodiment of the present invention further provides an electronic device, such as Fig.13 , Fig.13 The electronic device structure block diagram of the embodiment of the present invention is provided. The electronic device 13 includes a communication interface 131, a processor 132 and a memory 133. The processor 132, the memory 133 and the communication interface 131 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 133 can be used to store software programs and modules, such as program instructions / modules corresponding to the text information extraction method provided in the embodiment of the present invention. The processor 132 executes various functional applications and data processing by executing the software programs and modules stored in the memory 133. The communication interface 131 can be used for signaling or data communication with other node devices. In the present invention, the electronic device 13 can have multiple communication interfaces 131.

[0100] Among them, the memory 133 can be but is not limited to, random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0101] The processor 132 may be an integrated circuit chip with signal processing capability. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0102] It is understandable that the various modules of the above-mentioned text information extraction device 12 can be stored in the memory 133 of the electronic device 13 in the form of software or firmware and executed by the processor 132. At the same time, the data, program code, etc. required to execute the above-mentioned modules can be stored in the memory 133.

[0103] An embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, a text information extraction method as described in any one of the above embodiments is implemented. The computer-readable storage medium can be, but is not limited to, a USB flash drive, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic disk, or an optical disk, etc., which can store program codes.

[0104] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Various changes or substitutions that can be easily thought of by technicians familiar with the technical field within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A text information extraction method, characterized in that: The method comprises: Obtain N frame images of the video to be detected, where N is greater than zero; Determine a target frame image set according to the N frame images, wherein each target frame image contained in the target frame image set contains the same target text, and the target frame image belongs to the N frame images; wherein the target frame image set includes a plurality of continuous target frame images; According to the target frame image set, determine the target text, the start timestamp and the end timestamp of the target text, wherein the start timestamp is the timestamp corresponding to the target frame image at the start position in the target frame image set; the end timestamp is the timestamp corresponding to the target frame image at the end position in the target frame image set; Obtain a character set corresponding to each character position of the target text; the character set includes characters in at least one font format; for each character set corresponding to the character position, use the character corresponding to the maximum probability value as the target character of the character position; the target text is a text composed of the target characters at each character position; Determining the target frame image set according to the N frame images includes: judging whether any adjacent frame images among the N frame images have the same target text; when the target text is present in the nth frame image and the target text is not present in the n-1th frame image, taking the nth frame image as the target frame image at the starting position of the target frame image set; when the target text is not present in the n+mth frame image and the target text is present in the n+m-1th frame image, taking the n+m-1th frame image as the target frame image at the ending position of the target frame image set; The determining whether any adjacent frame images among the N frame images have the same target text includes: when there is a first target text in the nth frame image and a second target text in the n-1 frame image, determining whether a repetition rate of the first target text and the second target text is greater than or equal to a repetition threshold; if the repetition rate is greater than or equal to the repetition threshold, determining that the nth frame image and the n-1 frame image have the same target text.

2. The text information extraction method according to claim 1, characterized in that: The step of obtaining N frames of images of the video to be detected includes: Obtaining the starting text coordinates of the video to be detected; Determine a text detection area according to the starting text coordinates; the text detection area is an area obtained by expanding the area where the starting text is located; The N frame images are obtained according to the text detection area; the N frame images are screenshots of each frame image of the video to be detected that includes the text detection area.

3. The text information extraction method according to claim 2, characterized in that: Before obtaining the starting text coordinates of the video to be detected, the method further includes: Extract all text information of the video to be detected; the all text information includes the starting text coordinates.

4. The text information extraction method according to claim 3, characterized in that: After extracting all text information of the video to be detected, the method further includes: When there are at least two texts in any frame of image, all the texts corresponding to the any frame of image are spliced, and the spliced ​​texts are used as the texts of the any frame of image.

5. A text information extraction device, characterized in that: include: An acquisition module, used to acquire N frame images of the video to be detected, where N is greater than zero; A determination module, configured to determine a target frame image set according to the N frame images, wherein each target frame image contained in the target frame image set contains the same target text, and the target frame image belongs to the N frame images; wherein the target frame image set includes a plurality of continuous target frame images; An extraction module, configured to extract a target text, a start timestamp and an end timestamp of the target text according to the target frame image set, wherein the start timestamp is a timestamp corresponding to the target frame image at the start position in the target frame image set; and the end timestamp is a timestamp corresponding to the target frame image at the end position in the target frame image set; The acquisition module is further used to acquire a character set corresponding to each character position of the target text; the character set includes characters in at least one font format; the determination module is further used to, for each character set corresponding to the character position, use the character corresponding to the maximum probability value as the target character of the character position; the target text is a text composed of the target characters at each character position; The determination module is specifically used to determine whether any adjacent frame images among the N frame images have the same target text; when the target text is present in the nth frame image and the target text is not present in the n-1th frame image, the nth frame image is used as the target frame image at the starting position of the target frame image set; when the target text is not present in the n+mth frame image and the target text is present in the n+m-1th frame image, the n+m-1th frame image is used as the target frame image at the ending position of the target frame image set; The determination module is also specifically used to determine whether the repetition rate of the first target text and the second target text is greater than or equal to a repetition threshold when there is a first target text in the nth frame image and a second target text in the n-1 frame image; if the repetition rate is greater than or equal to the repetition threshold, it is determined that the nth frame image and the n-1 frame image have the same target text.

6. An electronic device, characterized in that: The electronic device includes a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions. When the processor executes the machine-executable instructions, the electronic device implements the text information extraction method described in any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores machine executable instructions, and when the machine executable instructions are executed, the text information extraction method described in any one of claims 1 to 4 is implemented.

Images