Video-to-text conversion method, mobile terminal, and computer-readable storage medium

Abstract

This application discloses a method for converting video to text, comprising the following steps: identifying the content of a current keyframe of a video to be converted; determining that the current keyframe contains text, and using the current keyframe as a target keyframe; and converting the text content of the target keyframe into a text document. This application also discloses a terminal device and a computer-readable storage medium. This application can save document conversion time and thereby improve video-to-document conversion efficiency.

Description

Technical Field

[0001] The present application relates to the field of text recognition, and in particular to a video-to-text conversion method, a mobile terminal, and a computer-readable storage medium. Background Art

[0002] With the advancement of technology, people have more and more ways to obtain and record information, such as recording videos and taking photos. In some scenarios, people prefer to use video recording to obtain and record information. For example, in a PowerPoint presentation, users generally record the PowerPoint content in real time by recording a video for convenience. However, using video to record PowerPoint content can be inconvenient for subsequent review. For example, when users click on a video to play it, they need to constantly pause the video to retrieve the content of the current PowerPoint page. Therefore, users often want to be able to directly access the text document within the video.

[0003] Therefore, video-to-text conversion technology has emerged. However, in the exemplary video-to-text conversion technology, all video frames of the video are generally converted into text documents. After obtaining the text documents, some invalid documents are manually deleted by the user. This video-to-document conversion process requires text conversion processing on all video frames, which is time-consuming and inefficient.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a video-to-text conversion method, a mobile terminal and a computer-readable storage medium, aiming to solve the technical problems of the existing video-to-text conversion process being time-consuming and inefficient.

[0006] To achieve the above objectives, the present application provides a video-to-text conversion method, which includes the following steps:

[0007] Identify the content of the current key frame of the video to be converted;

[0008] Determining that the current key frame has text content, and using the current key frame as a target key frame;

[0009] Convert the text content of the target keyframe into a text document.

[0010] Optionally, the step of identifying the content of the current key frame of the video to be converted includes:

[0011] Identifying whether there is a text mark in the content of the current key frame of the video to be converted;

[0012] If so, it is determined that the current key frame has text content.

[0013] Optionally, the text identifier includes a text border.

[0014] Optionally, the step of determining that the current key frame has text content and using the current key frame as a target key frame includes:

[0015] Determining that the current key frame has text content, and identifying text information in the text content;

[0016] Determine the text information recognized in the text content and use the current key frame as a target key frame.

[0017] Optionally, the step of determining that the current key frame has text content and using the current key frame as a target key frame includes:

[0018] Determining that the current key frame has text content, and obtaining a text area;

[0019] cropping the current key frame according to the text area to form a cropped key frame with the text area as the picture;

[0020] The cropping key frame is used as the target key frame.

[0021] Optionally, the step of using the cropped key frame as the target key frame includes:

[0022] identifying a shape of the cropping keyframe;

[0023] If the shape of the cropping key frame does not match the preset shape, the cropping key frame is adjusted according to the preset shape, and the adjusted cropping key frame is used as the target key frame.

[0024] Optionally, the preset shape includes one of a rectangle and a square.

[0025] Optionally, the text area is an area surrounded by a text border.

[0026] Optionally, the step of obtaining the text area includes:

[0027] If the content of the current key frame contains at least two text frames, obtaining a text frame that meets a preset condition;

[0028] The area surrounded by the text frame that meets the preset conditions is used as the text area.

[0029] Optionally, the step of determining that the current key frame has text content and using the current key frame as a target key frame includes:

[0030] If the current key frame has text content, obtaining the distribution ratio of the three primary colors of the current key frame;

[0031] Obtaining an error between the three primary color distribution ratios of the current key frame and the three primary color distribution ratios of the previous key frame;

[0032] If the error is outside the preset error range, the current key frame is used as the target key frame.

[0033] Optionally, the video-to-text conversion method further comprises:

[0034] If the error is within a preset error range, the current key frame is deleted.

[0035] Optionally, while executing the step of saving the target key frame, the following steps are also executed:

[0036] The distribution ratio of the three primary colors of the target key frame is saved.

[0037] Optionally, the distribution ratios of the three primary colors corresponding to the same text content are the same.

[0038] Optionally, after the step of converting the text content of the target key frame into a text document, the method further includes:

[0039] Obtaining a degree of matching between the text content of the text document and the text content of the text document of the previous target key frame;

[0040] If the matching degree is greater than a preset threshold, obtaining a target text document with more text content;

[0041] The target text document is saved.

[0042] In order to achieve the above-mentioned purpose, the present application also provides a mobile terminal, which includes: a memory, a processor, and a video-to-text conversion program stored in the memory and runnable on the processor. When the video-to-text conversion program is executed by the processor, the various steps of the video-to-text conversion method described above are implemented.

[0043] In order to achieve the above objectives, the present application also provides a computer-readable storage medium, on which a video-to-text conversion program is stored. When the video-to-text conversion program is executed by a processor, the various steps of the video-to-text conversion method described above are implemented.

[0044] The embodiments of the present application propose a video-to-text conversion method, a mobile terminal, and a computer-readable storage medium. By identifying the content of the current key frame of the video to be converted and determining that the current key frame has text content, the current key frame is converted into a text document. For key frames that do not have text content, no text document conversion processing is performed, which can save document conversion time and thereby improve the conversion efficiency of video to document. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application;

[0046] Figure 2 This is a flowchart of the first embodiment of the video-to-text conversion method provided by this application;

[0047] Figure 3 This is a detailed flow chart of an embodiment of S20 in the second embodiment of the video-to-text conversion method provided by this application;

[0048] Figure 4 This is a schematic diagram of another embodiment of the detailed flow chart of S20 in the third embodiment of the video-to-text conversion method provided by this application;

[0049] Figure 5 yes Figure 4 Detailed process diagram of S25;

[0050] Figure 6 This is a detailed flowchart of S23 in the fourth embodiment of the video-to-text conversion method provided by this application;

[0051] Figure 7 This is a schematic diagram of a detailed flow chart of another embodiment of S20 in the fifth embodiment of the video-to-text conversion method provided by this application;

[0052] Figure 8 This is a flowchart of the sixth embodiment of the video-to-text conversion method provided by this application.

[0053] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0055] The main solution of the embodiment of the present application is: identifying the content of the current key frame of the video to be converted; determining that the current key frame has text content, and using the current key frame as the target key frame; converting the text content of the target key frame into a text document.

[0056] like Figure 1 As shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0057] The terminal in the embodiment of the present application can be a PC, or a mobile terminal device with photo taking and display functions, such as a smart phone and a tablet computer.

[0058] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0059] Optionally, the terminal may further include a camera, a sensor, an audio circuit, a WiFi module, etc.

[0060] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0061] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a video-to-text conversion program.

[0062] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the video-to-text conversion program stored in the memory 1005 and perform the following operations:

[0063] Identify the content of the current key frame of the video to be converted;

[0064] Determining that the current key frame has text content, and using the current key frame as a target key frame;

[0065] Convert the text content of the target keyframe into a text document.

[0066] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and perform the following operations:

[0067] Identifying whether there is a text mark in the content of the current key frame of the video to be converted;

[0068] If so, it is determined that the current key frame has text content.

[0069] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and perform the following operations:

[0070] Determining that the current key frame has text content, and identifying text information in the text content;

[0071] Determine the text information recognized in the text content and use the current key frame as a target key frame.

[0072] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and perform the following operations:

[0073] Determining that the current key frame has text content, and obtaining a text area;

[0074] cropping the current key frame according to the text area to form a cropped key frame with the text area as the picture;

[0075] The cropping key frame is used as the target key frame.

[0076] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and perform the following operations:

[0077] identifying a shape of the cropping keyframe;

[0078] If the shape of the cropping key frame does not match the preset shape, the cropping key frame is adjusted according to the preset shape, and the adjusted cropping key frame is used as the target key frame.

[0079] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and may also perform the following operations:

[0080] If the content of the current key frame includes at least two text frames, obtaining a text frame that meets a preset condition;

[0081] The area surrounded by the text frame that meets the preset conditions is used as the text area.

[0082] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and may also perform the following operations:

[0083] If the current key frame has text content, obtaining the distribution ratio of the three primary colors of the current key frame;

[0084] Obtaining an error between the three primary color distribution ratios of the current key frame and the three primary color distribution ratios of the previous key frame;

[0085] If the error is outside the preset error range, the current key frame is used as the target key frame.

[0086] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and may also perform the following operations:

[0087] If the error is within a preset error range, the current key frame is deleted.

[0088] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and may also perform the following operations:

[0089] The distribution ratio of the three primary colors of the target key frame is saved.

[0090] Furthermore, the processor 1001 may call the video-to-text conversion program stored in the memory 1005 and may also perform the following operations:

[0091] Obtaining a degree of matching between the text content of the text document and the text content of the text document of the previous target key frame;

[0092] If the matching degree is greater than a preset threshold, obtaining a target text document with more text content;

[0093] The target text document is saved.

[0094] Reference Figure 2 The present application provides a first embodiment of a video-to-text conversion method, the video-to-text conversion method comprising:

[0095] Step S10, identifying the content of the current key frame of the video to be converted;

[0096] Step S20, determining that the current key frame has text content, and using the current key frame as a target key frame;

[0097] Step S30: converting the text content of the target key frame into a text document.

[0098] The execution subject of this embodiment may be a mobile terminal device, such as a mobile phone, tablet computer, or camera, all of which have a camera function. Alternatively, the execution subject of this embodiment may be a terminal device, such as a server, which is connected to the mobile phone, tablet computer, or camera, all of which have a camera function.

[0099] If the execution subject is the aforementioned terminal device with a camera function, the video recorded in real time by the terminal device, or the video stored in the terminal device, is the video to be converted. After the video recording is completed, the video conversion instruction can be triggered to convert the video to text; or after the recording is completed, the user can manually trigger the video conversion instruction to convert the video to text. If the execution subject is the server connected to the aforementioned terminal device with a camera function, the video uploaded by the terminal device to the server is the video to be converted. After receiving the video uploaded by the terminal device, the video to be converted is converted into text.

[0100] The videos to be converted are mainly recorded videos of courses. For example, when a lecturer is playing a PowerPoint presentation, students can use their terminal devices to record the PowerPoint presentation. This eliminates the need to take notes while listening to the lecture, and the recorded video can be used to record the PowerPoint content later. This embodiment converts the videos to text documents, saving users the time of taking notes by playing the recorded video.

[0101] This embodiment uses a terminal device as the execution subject to illustrate the specific process of video-to-text conversion:

[0102] The terminal device of this embodiment has a text content recognition function. The terminal device starts from the initial key frame of the video to be converted, identifies the content of each key frame, and when a key frame is identified as having text content, converts the text content of the key frame into a text document. By identifying the content of the key frames of the video, only the key frames with text content are converted to text. The speed of text recognition is much faster than the speed of document extraction. Compared with the exemplary technology that first extracts the content of all key frames of the video to form a document and then discards the invalid parts of the document, this embodiment can reduce the time of video-to-text conversion and improve the efficiency of video-to-text conversion.

[0103] Specifically, this embodiment uses a preset text identifier to identify whether a key frame contains text content. The text identifier can be a text frame. The specific steps of identifying the content of the current key frame of the video to be converted include:

[0104] Identifying whether there is a text mark in the content of the current key frame of the video to be converted;

[0105] If yes, it is determined that the current key frame has text content. And / or,

[0106] If not, it is determined that the current key frame does not have text content, and the terminal device obtains the next key frame of the video to be converted and identifies the content of the next key frame.

[0107] After the terminal device obtains the current key frame of the video to be converted, it identifies whether there is a preset text border in the content of the current key frame. If so, it means that the current key frame has text content; if not, it means that the current key frame does not have text content. At this time, the document conversion processing is not performed on the current key frame to save document conversion time.

[0108] It should be noted that the content recognition technology in this embodiment may adopt image recognition technology to directly recognize whether the current key frame contains the text identifier.

[0109] It is determined that the current key frame has text content, and the current key frame is used as a target key frame; then the text content of the target key frame is converted into a text document, and the text document is saved, thereby achieving text extraction of the content of the current key frame.

[0110] After the terminal device identifies all key frames of the video to be converted and performs document conversion on the key frames with text content, a text document set is obtained, which contains all text content of the video to be converted.

[0111] In this embodiment, by identifying the content of the current key frame of the video to be converted and determining that the current key frame has text content, the current key frame is converted into a text document. For key frames that do not have text content, no text document conversion is performed, which can save document conversion time and thereby improve the conversion efficiency of video to document.

[0112] Furthermore, based on the first embodiment above, in order to further improve the conversion efficiency of text documents, please refer to Figure 3 In the second embodiment of the video-to-text conversion method provided by the present application, S20 includes:

[0113] S21, determining that the current key frame has text content, and identifying text information of the text content;

[0114] S22: Determine the text information recognized in the text content and use the current key frame as a target key frame.

[0115] When it is determined that the text content does not contain any text information, the current key frame is deleted.

[0116] After identifying the content of the current key frame of the video to be converted, when it is determined that the current key frame has text content, since the text content is determined by a text identifier, the current key frame with a text identifier may also have unclear text content. When converting a text document under the condition that the text content is unclear, the converted document may be garbled, resulting in poor text document transfer effect and increased text document conversion time.

[0117] Therefore, in this embodiment, when it is determined that the current key frame has text content, the text information of the text content is identified; when it is determined that the text information in the text content is identified, the current key frame is converted into a text document, and / or, if it is determined that no text information is identified in the text content, it means that the text content in the current key frame is blurred and unclear. If the current key frame is forcibly converted, garbled characters will appear. At this time, the current key frame is deleted, and the terminal device identifies the content of the next key frame to omit the text document conversion process of the current key frame, thereby improving the conversion efficiency of video to document.

[0118] Furthermore, based on the first or second embodiment above, in order to prevent the terminal device from extracting text other than the target text content and reducing the conversion efficiency of the text document, this embodiment proposes a third embodiment of the video text conversion method. For details, please refer to Figure 4 , the step S20 includes:

[0119] S23, determining that the current key frame has text content, and obtaining a text area;

[0120] S24, cropping the current key frame according to the text region to form a cropped key frame with the text region as the picture;

[0121] S25: Use the cropped key frame as the target key frame.

[0122] It is understandable that when the terminal device is recording a video, based on any possibility of the recording process, the video will not only include text content, but may also include other content. In order to reduce the amount of text content recognition and reduce the conversion time of text documents, this embodiment is set to only extract and convert the target text content, and other content outside the target text content is not recognized and extracted, which reduces the amount of text content recognition and reduces the conversion efficiency of text documents.

[0123] Specifically, the text area is the area surrounded by the text border. The text border is a preset text identifier. After the terminal device recognizes that the current key frame has the text identifier, that is, the current key frame has the text border, it obtains the text border, uses the text border as the boundary, and crops the current key frame along the boundary to form a cropped key frame with the text area as the picture. By cropping, other areas outside the text area are deleted, so that the current key frame is the cropped key frame. In this way, during the text document conversion process, only the text content within the text area is converted, eliminating the extraction and conversion of other text contents, thereby improving the conversion efficiency of the text document. Among them, the cropped key frame described in this embodiment is a key frame saved after the current key frame is cropped according to the text area.

[0124] In this embodiment, the text area is set as the area where the target text content is located. Other content outside the text area does not belong to the target text content. By cropping the text area, the content except the target text content can be deleted, and only the content of the text area is extracted and converted, which can save the number of extractions and conversions and reduce the conversion time of the text document.

[0125] Furthermore, in order to improve recognition accuracy and thus improve the conversion efficiency of text documents, please refer to Figure 5 , the S25 includes:

[0126] S251, identifying the shape of the cropping key frame;

[0127] S252, determining whether the shape of the cropped key frame matches a preset shape;

[0128] S253: If there is no match, adjusting the cropping key frame according to the preset shape, and using the adjusted cropping key frame as the target key frame.

[0129] And / or, if they match, using the cropped key frame as the target key frame.

[0130] The preset shape includes one of a rectangle and a square.

[0131] During video recording, the text area may be tilted due to the shaking of the terminal device or the manual movement of the terminal device. Therefore, in this embodiment, after capturing the cropping key frame, the shape of the cropping key frame is identified, and whether the cropping key frame is tilted is identified. If it is tilted, it is adjusted before document conversion to make text recognition more accurate.

[0132] Specifically, the shape of the target key frame is preset, such as a rectangle or a square, etc. After identifying the shape of the cropping key frame, the cropping key frame is matched with the preset shape. If the cropping key frame matches the preset shape, it is determined that the cropping key is not tilted, and / or, if the cropping key frame does not match the preset shape, it is determined that the cropping key frame is tilted.

[0133] When the shape of the cropping key frame does not match the preset shape, the cropping key frame is adjusted according to the preset shape, wherein the adjustment manner includes rotating and / or stretching, so that the cropping key frame matches the preset shape.

[0134] Based on the third embodiment above, the fourth embodiment of the video text conversion method provided by this application, please refer to Figure 6 , the step of obtaining the text area includes:

[0135] S231, if the content of the current key frame includes at least two text frames, obtaining a text frame that meets a preset condition;

[0136] S232: Taking the area surrounded by the text frame that meets the preset condition as the text area.

[0137] Wherein, when the text frame matches a preset text frame, it is determined that the text frame meets the preset condition;

[0138] Alternatively, a text frame with the largest size among all text frames is obtained, and it is determined that the text frame with the largest size meets a preset condition.

[0139] This embodiment is based on the third embodiment described above. In order to reduce the amount of text content to be recognized and the time it takes to convert text documents, this embodiment only extracts and converts the target text content, and does not recognize or extract other content outside the target text content, thereby reducing the amount of text content to be recognized and the efficiency of text document conversion. Specifically, the text area is obtained, and the target text content is determined based on the text area. However, in actual video recording scenarios, there may be multiple text areas (with multiple text borders) in the same video frame. However, among the multiple text areas, generally only the content of the text area corresponding to the PPT is the target text content.

[0140] In the third embodiment, the terminal device extracts and converts the contents of all text areas into documents. However, some of the contents in the text areas are not what the user wants. If all the contents of the text areas are converted into documents, not only will the user have to manually delete them, but the conversion time of the text documents will also be increased.

[0141] Based on this, this embodiment proposes that if the content of the current key frame includes at least two text borders, that is, when the terminal device recognizes that there are at least two text areas in the current key frame, a text border that meets the preset conditions is obtained according to the program setting rules, and the area surrounded by the text border that meets the preset conditions is used as the target text area, and then the current key frame is cropped according to the target text area, and only the target text area is retained.

[0142] The program setting rule may be a preset text frame, and multiple text frames are sequentially compared with the preset text frame to obtain a text frame that matches the preset text frame. The area enclosed by the text frame that matches the preset text frame is the text area, and then the text area is cropped, and only the cropped text area is subjected to text document recognition and conversion. Matching with the preset text frame may include shape matching and / or size matching. The preset text frame may be manually set by the user, or the text area frame in the first key frame may be used as the preset text frame, or the text frame in the key frame with only one text frame may be used as the preset text frame.

[0143] Alternatively, the program may set a rule to compare all the text frames one by one, obtain the text frame with the largest size, use the area enclosed by the text frame with the largest size as the text area, and then crop the text area, and only perform text document recognition and conversion on the cropped text area. Since the text frame presented by the PPT is mainly used as the main frame for recording during video recording, generally speaking, the size of the text frame presented by the PPT is generally larger. Therefore, by comparing the size of the text frame, the text frame of the PPT can be obtained to achieve the interception of the target text content.

[0144] In this embodiment, if the content of the current key frame includes at least two text frames, only the text frames that meet the preset conditions are cropped, and then only the content corresponding to the text frame is extracted and converted into the text document. This can save the need to identify and convert other content outside the text frame, and the user can avoid deleting the document, and can further improve the conversion efficiency of the text document.

[0145] Reference Figure 7 In the fifth embodiment of the video-to-text conversion method provided by the present application, based on all the above embodiments, the step of determining that the current key frame has text content and setting the current key frame as the target key frame includes:

[0146] S26, if the current key frame has text content, obtaining the distribution ratio of the three primary colors of the current key frame;

[0147] S27, obtaining an error between the three primary color distribution ratios of the current key frame and the three primary color distribution ratios of the previous key frame;

[0148] S28, if the error is outside the preset error range, the current key frame is used as the target key frame. And / or,

[0149] S29: If the error is within a preset error range, the current key frame is deleted.

[0150] After converting the text content of the target keyframe into a text document, the method further includes: saving the three primary color distribution ratios of the target keyframe. Therefore, after obtaining the three primary color distribution ratios of the current keyframe, the three primary color distribution ratios can be compared with the pre-stored three primary color distribution ratios of the previous keyframe.

[0151] Based on the fact that the distribution ratios of the three primary colors corresponding to the same text content are the same, if the ratios of the three primary colors of the current key frame and the previous key frame are the same or not much different, it means that the text content in the current key frame and the previous key frame are the same text content. In order to avoid repeated conversion of text documents, increase the conversion amount of text documents, and increase the conversion time, this embodiment, during the video text conversion process, compares the distribution ratios of the three primary colors with the previous key frame, deletes the key frames that are the same as the text documents, and avoids rereading and conversion.

[0152] It should be noted that the three primary colors refer to red, green and blue. The image is formed by combining the three primary colors to form different patterns or texts. When the terminal device is recording a video, the patterns and texts corresponding to the same text content are the same. Therefore, the distribution ratio of the three primary colors of the key frames corresponding to the same text content is the same, wherein the distribution ratio of the three primary colors is the total distribution ratio of the three primary colors of all fonts in the text content.

[0153] When determining that the current key frame has text content, this embodiment obtains the three primary color distribution ratio of the current key frame, compares the three primary color ratio of the current key frame with the three primary color ratio of the previous key frame, and if the error between the three primary color distribution ratio of the current key frame and the three primary color distribution ratio of the previous key frame is within a preset error range, it is determined that the current key frame and the previous key frame are key frames corresponding to the same text content. To avoid rereading and converting, the current key frame is deleted, and the terminal device executes S10 to identify the content of the next key frame. And / or, if the error is outside the preset error range, it means that the text content corresponding to the current key frame is different from the text content corresponding to the previous key frame and is the text content of the next page. At this time, the terminal device executes S30 to convert the text content of the current key frame into a text document.

[0154] It is understandable that the preset error range may be between 0% and 30%.

[0155] The video-to-text conversion process of this embodiment can effectively avoid repeated document conversion of the same text content, reduce the document conversion amount, and further improve the document conversion efficiency.

[0156] Further, refer to Figure 8 The sixth embodiment of the video-to-text conversion method provided by the present application, based on all the above embodiments, further includes, after the step of converting the text content of the target key frame into a text document:

[0157] S40, obtaining a matching degree between the text content of the text document and the text content of the text document of the previous target key frame;

[0158] S50, determining whether the matching degree is greater than or equal to a preset threshold;

[0159] S60, if yes, obtain the target text document with more text content;

[0160] S70, saving the target text document. And / or,

[0161] S80: If not, save the text document.

[0162] The target text document may be the text document corresponding to the previous key frame or the text document corresponding to the current key frame. If the target text document is the text document corresponding to the previous target key frame, the text document corresponding to the current key frame is deleted; and / or if the target text document is the text document corresponding to the current key frame, the text document corresponding to the previous target key frame is deleted.

[0163] This embodiment is based on the first to fifth embodiments. Since the same text content has multiple keyframes during video recording, the terminal device has a high duplication rate after converting the keyframe text documents corresponding to the same text content during the video-to-text conversion process. In order to avoid the need for the user to manually delete duplicate documents, after converting the text content of the target keyframe to a text document, the text content of the text document is obtained, and the content of the text document of the current keyframe is compared with the content of the text document of the previous keyframe to obtain a matching degree. When the matching degree is greater than or equal to a preset threshold, it indicates that the text document corresponding to the current keyframe is the same as the text document of the previous keyframe. Only one of the text documents can be saved, and the remaining text documents are deleted. Specifically, this embodiment determines the target text document to be saved based on the amount of text content. When part of the content in the keyframe is blocked and not recorded, the text document's text content is not fully extracted, but the most complete text document can still be saved. If the matching degree is less than the preset threshold, it indicates that the text document corresponding to the current keyframe is different from the text document of the previous keyframe, and the text document corresponding to the current keyframe needs to be saved.

[0164] This embodiment is based on the fourth embodiment described above. During the video-to-text conversion process, the error between the three primary color distribution ratios of the current key frame and the three primary color distribution ratios of the previous key frame is used to determine whether the current key frame and the previous key frame correspond to the same text content. If so, the current key frame is deleted without repeating the document conversion. Therefore, in the fourth embodiment, the repetition rate of the text document is low. However, during the video recording process, the text content may be obscured, such as when a lecturer obscures the PPT content when walking. At this time, the error between the three primary color distribution ratio of the current key frame and the three primary color distribution ratio of the previous key frame may be outside the preset error range. The terminal device identifies and extracts the content of the current key frame to obtain a text document. However, the text document of the current key frame and the text document of the previous key frame may still be converted from the text content of the same page. Therefore, in order to avoid duplication of the converted text documents in this scenario, this embodiment converts the text content of the target key frame into a text document, obtains the matching degree between the text content of the text document and the text content of the text document of the previous target key frame; determines whether the matching degree is greater than or equal to a preset threshold, and if so, obtains the target text document with more text content, saves the target text document, and deletes other text documents. And / or, if not, saves the text document. Among them, saving the target text document with more text content can ensure that the content of the converted text document is the most complete, thereby improving the text conversion effect.

[0165] The preset threshold may be 70% to 100%, and the matching degree is determined based on the number of words and / or the matching degree of text in the two text documents.

[0166] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code is run on a computer, the computer executes the methods described in the various possible implementation modes above.

[0167] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods described in the various possible implementation modes above.

[0168] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0169] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "if" as used herein may be interpreted as "at the time of," "when," or "in response to a determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, meaning any one or any combination. Thus, “A, B, or C” or “A, B, and / or C” means “any of: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition occurs only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0170] It should be noted that in this article, step codes such as S10 and S20 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the scope of protection of this application.

[0171] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0172] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0173] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video-to-text conversion method, characterized in that: The video-to-text conversion method comprises the following steps: Identify the content of the current key frame of the video to be converted; Determining that the current key frame has text content, and using the current key frame as a target key frame; Converting the text content of the target keyframe into a text document; The step of determining that the current key frame has text content and using the current key frame as a target key frame includes: If the current key frame has text content, obtaining the distribution ratio of the three primary colors of the current key frame; Obtaining an error between the three primary color distribution ratios of the current key frame and the three primary color distribution ratios of the previous key frame; If the error is outside the preset error range, the current key frame is used as the target key frame; If the error is within a preset error range, the current key frame is deleted.

2. The video-to-text conversion method according to claim 1, wherein: The step of identifying the content of the current key frame of the video to be converted includes: Identifying whether there is a text mark in the content of the current key frame of the video to be converted; If so, it is determined that the current key frame has text content.

3. The video-to-text conversion method according to claim 2, wherein: The text mark includes a text frame, and the text area is the area surrounded by the text frame.

4. The video-to-text conversion method according to claim 3, wherein: The step of determining that the current key frame has text content and using the current key frame as a target key frame includes: Determining that the current key frame has text content, and identifying text information in the text content; Determine the text information recognized in the text content and use the current key frame as a target key frame.

5. The video-to-text conversion method according to claim 1, wherein: The step of determining that the current key frame has text content and using the current key frame as a target key frame includes: Determining that the current key frame has text content, and obtaining a text area; cropping the current key frame according to the text area to form a cropped key frame with the text area as the picture; The cropping key frame is used as the target key frame.

6. The video-to-text conversion method according to claim 5, wherein: The step of using the cropped key frame as the target key frame comprises: identifying a shape of the cropping keyframe; If the shape of the cropping key frame does not match the preset shape, the cropping key frame is adjusted according to the preset shape, and the adjusted cropping key frame is used as the target key frame.

7. The video-to-text conversion method according to claim 3, wherein: The steps to get the text area include: If the content of the current key frame contains at least two text frames, obtaining a text frame that meets a preset condition; The area surrounded by the text frame that meets the preset conditions is used as the text area.

8. The video-to-text conversion method according to claim 1, wherein: After the step of converting the text content of the target key frame into a text document, the method further includes: The distribution ratio of the three primary colors of the target key frame is saved.

9. The video-to-text conversion method according to claim 1, wherein: After the step of converting the text content of the target key frame into a text document, the method further includes: Obtaining a degree of matching between the text content of the text document and the text content of the text document of the previous target key frame; If the matching degree is greater than a preset threshold, obtaining a target text document with more text content; The target text document is saved.

10. A mobile terminal, characterized in that: The mobile terminal includes: a memory, a processor, and a video-to-text conversion program stored in the memory and executable on the processor. When the video-to-text conversion program is executed by the processor, the steps of the video-to-text conversion method according to any one of claims 1 to 9 are implemented.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a video-to-text conversion program, which, when executed by a processor, implements the steps of the video-to-text conversion method according to any one of claims 1 to 9.

Images