Text elimination method, device, electronic device and storage medium

Abstract

The present invention provides a text elimination method, device, electronic device and storage medium. The text elimination method comprises: obtaining an image to be processed; the image to be processed has a characteristic region and the characteristic region is used to indicate that the characteristic region is to be eliminated; the characteristic region is matched with a region of text to be processed in the image to be processed; the image to be processed is input into a preset region elimination model so that the preset region elimination model eliminates the characteristic region and then outputs a target image; the target image does not contain the above-mentioned text to be processed. The present invention can eliminate the characteristic region in the image to be processed through the region elimination model, and can improve the accuracy and processing efficiency of text elimination.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a text elimination method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of electronic devices, the functions of electronic devices are gradually improved, and people often use electronic devices to obtain information, store information, and watch videos. When playing a video, an electronic device can display subtitle information while displaying the video image, including subtitle information related to the video content and video watermark information, so that users can grasp the relevant information of the video.

[0003] When watching or using some videos, a considerable number of viewers do not want to see subtitles or video watermarks in the video for various reasons. Therefore, there is a need to remove various text information in the video for this group of people. Traditional subtitle masking solutions are slow and have poor effects. Most watermark removal methods also require manual acquisition of watermark information and outlines in advance. Summary of the invention

[0004] In view of this, the present invention provides a text elimination method, device, electronic device and storage medium to improve the accuracy and efficiency of text elimination. In order to achieve the above beneficial effects, the technical solution of the present invention is as follows:

[0005] In a first aspect, the present invention provides a text elimination method, the method comprising: acquiring an image to be processed; the image to be processed has a feature area; the feature area is used to indicate that elimination processing is to be performed on the feature area; the feature area is matched with a region of text to be processed in the image to be processed; the image to be processed is input into a preset region elimination model, so that the preset region elimination model performs elimination processing on the feature area and then outputs a target image; the target image does not contain the text to be processed.

[0006] Optionally, obtaining the image to be processed includes: obtaining the target text coordinates of the video image of the target frame; the video image of the target frame has the text to be processed; adding marking information to the area in the video image of the target frame that matches the target text coordinates to generate the image to be processed; the marking information is used to indicate that the area is the feature area.

[0007] Optionally, the method of acquiring target text coordinates of the video image of the target frame includes: acquiring text coordinates extracted from the video image of the target frame and historical text coordinates of at least one historical frame video image adjacent to the video frame of the video image of the target frame; matching the historical text coordinates with an area of ​​the text to be processed; when an error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is within a threshold range, using the historical text coordinates as the target text coordinates; when an error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is not within a threshold range, using the text coordinates extracted from the video image of the target frame as the target text coordinates.

[0008] Optionally, before obtaining the target text coordinates of the video image of the target frame, the method further includes: obtaining a text coordinate set of the video to be processed through a text recognition network; the text coordinate set includes text coordinates extracted from the video image of the target frame and text coordinates of the video image of the historical frame.

[0009] Optionally, before acquiring the image to be processed, the method further includes: receiving a user operation instruction; the user operation instruction is used to instruct acquiring the image to be processed.

[0010] Optionally, before acquiring the image to be processed, the method further comprises: acquiring the preset region elimination model; the region elimination model is obtained by training a preset number of sample images; the sample image has a characteristic region; the characteristic region is used to instruct the region elimination model to eliminate the characteristic region; the characteristic region has the following characteristic information: color format, size and layout angle;

[0011] Optionally, the method also includes: inputting the image to be processed into a preset region elimination model so that the region elimination model replaces the pixel values ​​in the feature region according to the target pixel value to obtain the target image; the target pixel value is the pixel value corresponding to the pixel point in the area outside the feature region that is adjacent to the pixel point in the feature region.

[0012] Optionally, the method further includes: inputting the image to be processed into a preset region elimination model, so that the region elimination model performs a cutting process on the feature region to obtain the target image.

[0013] In a second aspect, the present invention provides a text elimination device, comprising: an acquisition module, used to acquire an image to be processed; the image to be processed has a feature area; the feature area is used to indicate that the feature area is to be eliminated; the feature area matches the area of ​​the text to be processed in the image to be processed; an elimination module, used to input the image to be processed into a preset area elimination model, so that the preset area elimination model performs elimination processing on the feature area and then outputs a target image; the target image does not contain the text to be processed.

[0014] In a third aspect, the present invention provides an electronic device, comprising a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the processor executes the machine-executable instructions, the electronic device implements the text elimination method described in the first aspect.

[0015] In a fourth aspect, the present invention provides a storage medium, wherein the storage medium stores machine executable instructions, and when the machine executable instructions are executed, the text elimination method described in the first aspect is implemented.

[0016] The present invention provides a text elimination method, device, electronic device and storage medium, and the text elimination method includes: obtaining an image to be processed; the image to be processed has a characteristic region; the characteristic region is used to indicate that the characteristic region is to be eliminated; the characteristic region matches the region of the text to be processed in the image to be processed; the image to be processed is input into a preset region elimination model, so that the preset region elimination model eliminates the characteristic region and then outputs a target image; the target image does not contain the text to be processed. The present invention eliminates text from the image to be processed by using a region elimination model. Since the region elimination model can identify the characteristic region during the training process and eliminate the content in the characteristic region based on the indication of the characteristic region, the region elimination model can be used to realize the function of text elimination. The entire process does not need to be determined as a text position or outline by manual means, which can not only improve the accuracy of the elimination process, but also improve the processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A region elimination model training method provided by an embodiment of the present invention;

[0019] Figure 2 It is a schematic diagram of the structure of a Unet network;

[0020] Figure 3 A text removal effect diagram provided by an embodiment of the present invention;

[0021] Figure 4 is a schematic flow chart of an implementation method of step S101 of an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of obtaining a training sample image provided by an embodiment of the present invention;

[0023] Figure 6 A schematic flow chart of a text elimination method provided by an embodiment of the present invention;

[0024] Figure 7 A schematic flow chart of an implementation of S603 provided in an embodiment of the present invention;

[0025] Figure 8 A schematic flow chart of an implementation method of step S603-1 provided in an embodiment of the present invention;

[0026] Fig. 9 A schematic flow chart of another text elimination method provided by an embodiment of the present invention;

[0027] Fig.10 A schematic flow chart of another text processing method provided by an embodiment of the present invention;

[0028] Fig.11 A schematic diagram of a function display interface of an electronic device provided by an embodiment of the present invention;

[0029] Fig.12 A schematic diagram of a function display interface of another electronic device provided by an embodiment of the present invention;

[0030] Fig.13 A functional module diagram of a text elimination device provided by an embodiment of the present invention;

[0031] Fig.14 A structural block diagram of an electronic device is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0035] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0036] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0037] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0038] At present, most online videos carry watermarks of video producers or advertisers to prevent "piracy". At the same time, in order to facilitate users to understand the content of the video, subtitle information will be added to the video. However, when watching or using some videos, a considerable number of viewers do not want to see subtitles or video watermarks in the video for different reasons. Therefore, for this group of people who want to remove subtitle information or watermark information from the video, relevant technologies will remove subtitles or watermarks from the video. However, the current subtitle or watermark removal solutions are slow, accurate and inefficient. The reason is that the existing solutions often mark the subtitle area or watermark outline by manual methods, and start the removal process after determining the location of the subtitle or watermark. Obviously, this manual method of marking the location of the subtitle or watermark often has errors, resulting in low accuracy of the determined removal area. At the same time, the manual processing method is slow, which in turn leads to slow removal speed and low efficiency.

[0039] Therefore, in order to solve the above technical problems, the inventors have conducted a series of studies and proposed a method of using a region elimination model to remove subtitles or watermarks in a video. That is, after the video image to be processed (including the text or watermark to be removed) is input into the region elimination model, the model can output an image with the text information removed.

[0040] In order to achieve the above-mentioned elimination effect, during the research process, the inventor proposed a text elimination method, which achieves the purpose of text elimination through a pre-trained regional elimination model. The regional elimination model can accurately identify the area to be eliminated in the image to be processed, and eliminate the area and the content in the area. The pre-trained regional elimination model can be used to eliminate subtitles or watermarks from each frame of the video to be processed. The entire process does not require manual determination of the location of the subtitles or watermarks, but instead completes the elimination process through the regional elimination model, which can not only improve the accuracy of the elimination process, but also improve the processing efficiency.

[0041] As can be seen from the above, the text removal method of the embodiment of the present invention is based on a pre-trained region removal model to achieve the effect of text removal. Figures 1 to 5 The implementation method of pre-obtaining the above-mentioned region elimination model in an embodiment of the present invention is described.

[0042] See first Figure 1 , Figure 1 A region elimination model training method is provided in an embodiment of the present invention. The region elimination model training method can be performed by a region elimination model training device. Accordingly, the region elimination model training device can be set in an electronic device. The region elimination model training method may include the following process:

[0043] S101: Obtain a preset number of training sample images.

[0044] In the embodiment of the present invention, each training sample image contains a feature region; the feature region can be, but is not limited to, a rectangular region of any color format and any size. The feature region is used to indicate that the feature region is to be eliminated. By using the training sample images in the embodiment of the present invention to perform model training, it can be foreseen that the model finally obtained can perform elimination processing on the image with the feature region, including eliminating the content in the feature region (which can be text content such as subtitles or watermarks).

[0045] S102: Input the training sample images into the recognition network for training.

[0046] In the actual implementation process, the recognition network can be a network model with feature segmentation function such as a fully convolutional network FCN (Fully Convolutional Network, FCN), a U-type network Unet (Unity net, Unet), etc. The embodiment of the present invention adopts a Unet network. In the process of constructing the Unet network, the inventor has found through research that by constructing a 7*7 Unet network and combining the training method of the embodiment of the present invention to train the 7-layer recognition network, the region elimination model can be finally obtained with an ideal elimination effect and a stronger topology of the network. The structural diagram of the Unet network can be seen in Figure 2 , Figure 2 A schematic diagram of the structure of a Unet network.

[0047] like Figure 2 As shown in the figure, each gray box corresponds to a multi-channel feature map, where the number of channels is marked on the top of the box, and the white box represents a duplicated feature map; arrows represent different operations. The U-net network consists of a contraction path (left) and an expansion path (right). The contraction path follows the typical convolutional network structure, which consists of two repeated 3*3 convolution kernels (unpadded convolution), and both use the rectified linear unit (ReLU) activation function and a 2*2 maximum pooling operation with a step size of 2 for downsampling, and in each downsampling step, the number of feature channels is doubled. In the expansion path, each step includes upsampling the feature map; then convolution operation (up-convolution) with a 2*2 convolution kernel is performed to reduce the number of feature channels by half; then the corresponding cropped feature map in the contraction path is cascaded; and convolution operation is performed with two 3*3 convolution kernels, and both use the ReLU activation function. Since there is a problem of missing boundary pixels in each convolution operation, it is necessary to crop the feature map. In the last layer, a 1*1 convolution kernel is used for convolution operation to map each 64-dimensional feature vector to the output layer of the network. In total, the network has 23 convolution layers.

[0048] S103: When the loss value of the loss function of the recognition network is within a threshold range, the recognition network is used as a region elimination model.

[0049] In the embodiment of the present invention, a large number of training steps are used to train the recognition network. After each round of training, the recognition network outputs a processed image corresponding to each training sample image. The loss value of the constructed loss function (such as the L1 loss function) is calculated based on the error between the output image and the training sample image. When the loss value is within the threshold range, for example, the threshold range can be 0.001-s<0.001<0.001+s, where s can be a preset error value, it indicates that the recognition network is maturely trained, and the recognition network at this time is used as a region elimination model. It can be foreseen that after the image to be processed containing text information is input into the region elimination model for processing, the output image will no longer contain the text information. At this time, the model has completed the text elimination processing work, with fast speed and high efficiency.

[0050] To demonstrate the text elimination function of the region elimination model in the embodiment of the present invention, see Figure 3 , Figure 3 A text removal effect diagram provided by an embodiment of the present invention.

[0051] like Figure 3 As shown, Figure 3 (a) is a video image with subtitles, and the region elimination model eliminates the content in the feature region. Therefore, before the video image is input into the region elimination model, the embodiment of the present invention adds a feature region to the video image. It can be imagined that the feature region matches the region where the subtitles are located. Figure 3 (b), and then Figure 3 The image (b) is input into the region elimination model, and the region elimination model will eliminate the subtitles in the feature area. The model output result is as follows: Figure 3 As shown in (c), it can be seen Figure 3 The subtitle does not exist in (c), which shows that the region elimination model can not only complete the subtitle elimination, but also improve the accuracy and efficiency of subtitle elimination.

[0052] It should be noted that the region elimination model training method provided in the embodiment of the present invention may be executed by the region elimination model training device provided in the embodiment of the present invention, or an electronic device integrating the region elimination model training device, wherein the region elimination model training device may be implemented in hardware or software. The electronic device may be a smart phone, tablet computer, PDA, laptop computer, desktop computer or other device.

[0053] The embodiment of the present invention provides a region elimination model obtained through the above-mentioned region elimination model training method. The region elimination model can perform elimination processing on the video image input into the model. The entire process does not require manual determination of the position of subtitles or watermarks, but the elimination processing is completed through the region elimination model, which can not only improve the accuracy of the elimination processing, but also improve the processing efficiency.

[0054] Optionally, before model training, in order to obtain the training image in the embodiment of the present invention, in order to enable the region recognition model obtained by using the training sample image to have the text removal function, the embodiment of the present invention also provides an implementation method for obtaining the training sample image. Figure 1 Based on this, a possible implementation method is given, see Figure 4 , Figure 4 This is a schematic flow chart of an implementation of step S101 of an embodiment of the present invention. Step S101 may also include the following sub-steps:

[0055] S101 - 1 . Obtain a preset number of images without text.

[0056] In an embodiment of the present invention, the above-mentioned text-free images may be images obtained by batch cropping video images without subtitles in various video files, such as cartoons, TV series, and the like.

[0057] S101-2. Add feature regions to the text-free image to obtain a training sample image.

[0058] It can be understood that since the region elimination model finally obtained in the embodiment of the present invention needs to have the function of region elimination, during the training process, the recognition network needs to identify various forms of feature regions existing in the image. The training sample image in the embodiment of the present invention may have various feature regions. For example, the feature region may be but is not limited to a rectangle, and the feature region may have any color format and size.

[0059] In one possible implementation, in order to expand the set of training sample images, feature regions can be added to text-free images by adding the feature regions to any position of the text-free image at any layout angle to generate training sample images, thereby obtaining training sample images with various feature regions.

[0060] To understand the above process of obtaining training sample images, please refer to Figure 5 , Figure 5 A schematic diagram of obtaining a training sample image provided by an embodiment of the present invention.

[0061] like Figure 5 As shown, Figure 5(a) You can batch-crop images without subtitles from video files such as animation and TV series. In order to expand the dataset, you can Figure 5 The image in (a) is cropped to any size to obtain a large number of Figure 5 (b) and then randomly generate rectangles of any size and color and add them to the image at any layout angle. Figure 5 (b) in the image, we can obtain Figure 5 (c) shows a large number of training sample images.

[0062] It can be understood that based on the above training process, there are characteristic areas in the training sample images, and the characteristic areas can indicate the elimination of the content in the area. Then the content in the area may be subtitles, or watermarks (such as various text logos, graphic logos, image logos, advertising logos, etc.). Therefore, the above area elimination model can be applied to video subtitle elimination or watermark elimination.

[0063] Based on the region elimination model obtained above, an implementation method of a text elimination method provided by an embodiment of the present invention is given below. Figure 6 , Figure 6 A schematic flow chart of a text elimination method provided by an embodiment of the present invention includes:

[0064] S603: Obtain an image to be processed.

[0065] In the embodiment of the present invention, the above-mentioned image to be processed has a characteristic region, and the characteristic region is used to indicate that the characteristic region is to be eliminated. The characteristic region can be, but is not limited to, a rectangular region of any color format and any size, and the elimination process in the characteristic region can be understood as: eliminating the watermark or subtitle contained in the characteristic region.

[0066] It can be understood that in order to improve the accuracy of the elimination process, the feature region is matched with the region of the text to be processed in the image to be processed, and the text to be processed may be a subtitle or a watermark.

[0067] S604: Input the image to be processed into a preset region elimination model, so that the preset region elimination model performs elimination processing on the feature region and then outputs a target image.

[0068] In the embodiment of the present invention, the preset model is a model pre-trained by the above-mentioned training method, and the output target image does not contain the text to be processed.

[0069] To demonstrate the beneficial effects of this text elimination method, please continue to refer to Figure 3 , Figure 3(b) can be understood as the image to be processed, in which the feature area and Figure 3 (a) matches the area where the subtitles are located. Figure 3 The image in (b) is input into the pre-trained region elimination model, and the processed image is as follows: Figure 3 As shown in (c), it can be seen Figure 3 The subtitle does not exist in (c), which shows that the region elimination model can not only complete the subtitle elimination, but also improve the accuracy and efficiency of subtitle elimination.

[0070] The embodiment of the present invention provides a method for removing text, which inputs the acquired image to be processed into a preset region removal model, and the preset region removal model performs removal processing on the feature region according to the indication of the feature region in the image to be processed, and then outputs the target image, at which time the target image no longer has text information. The embodiment of the present invention can accurately identify the information to be removed in the image through the pre-trained region removal model, and then perform text removal processing on the image, and no longer needs the traditional manual method to mark the region to be removed, so the whole process can not only improve the accuracy of the removal processing, but also improve the processing efficiency.

[0071] Optionally, in one embodiment, the above-mentioned preset region elimination model can eliminate the feature region in the image to be processed in a manner such that: the region elimination model can replace the pixel values ​​in the feature region according to the target pixel values ​​to obtain the target image; the target image does not contain the text to be processed. It can be understood that the target pixel value is the pixel value corresponding to the pixel points in the area outside the feature region that are adjacent to the pixel points in the feature region. For example, the pixel value corresponding to the pixel point in the area outside the feature region that is the smallest distance from the pixel point on the boundary of the feature region is taken as the target pixel value, and the pixel values ​​of all the pixels in the feature region are replaced by the target pixel value, so as to achieve the effect of eliminating the text to be processed, so that the obtained target image does not contain the text to be processed.

[0072] Optionally, in another embodiment, the above-mentioned preset region elimination model can also be used to eliminate the feature region in the image to be processed in the following manner: the image to be processed is input into the preset region elimination model, so that the region elimination model performs a cutting process on the feature region to obtain a target image, which does not contain the text to be processed. It can be understood that after the cutting process, the cut image area can also be filled with background pixels to make the text removal effect of the cut area more natural.

[0073] Optionally, in one embodiment, from Figure 3It can be seen from the image to be processed that in order to eliminate text in a video image, the embodiment of the present invention does not input the obtained original video image into the region elimination model, but first adds a feature region to the original video image, and the model performs elimination processing according to the indication of the feature region. Therefore, before obtaining the image to be processed, it is necessary to accurately identify the text coordinates in the image to be processed, and then add a feature region to the image according to the text coordinates. Therefore, in the following Figure 1 Based on this, a possible implementation method is given, see Figure 7 , Figure 7 A schematic flow chart of an implementation of S603 provided in an embodiment of the present invention includes:

[0074] S603-1. Obtain the target text coordinates of the video image of the target frame.

[0075] It can be understood that the video image of the target frame may be, but is not limited to, any frame of the video image of the video to be processed, and the video image of the target frame contains the text to be processed.

[0076] S603-2. Adding marking information to the area matching the target text coordinates in the video image of the target frame to generate an image to be processed; the marking information is used to indicate that the area is a feature area.

[0077] In the actual implementation process, after obtaining the target text coordinates, marking information can be added to the area matching the target text coordinates. The marking information can be but is not limited to color information. For example, the pixel color of the area is set to red, then the red color data can be regarded as marking information. The region elimination model identifies the area where the text to be processed is located based on the marking information, that is, the feature area, and then eliminates the area.

[0078] Optionally, in one embodiment, in order to obtain an ideal text removal effect, accurately remove the text in the area where the text is located, and reduce the risk of flickering instability during the text removal process, the embodiment of the present invention also provides an implementation method for obtaining a target text of the text to be processed, which can improve the accuracy of the target text and reduce the removal error. Figure 7 Based on this, a possible implementation is given, see Figure 8 , Figure 8 A schematic flowchart of an implementation of step S603-1 provided in an embodiment of the present invention, wherein S603-1 may include the following sub-steps:

[0079] S603-1-1. Acquire the coordinates of the text extracted from the video image of the target frame and the coordinates of the historical text in the video image of at least one historical frame adjacent to the target frame.

[0080] In an embodiment of the present invention, the above-mentioned historical frame may be a previous frame adjacent to the target frame, and the above-mentioned text elimination processing has not been performed in the video image of the historical frame. The above-mentioned "historical text coordinates" match the area where the text in the video image of the historical frame is located. It can be understood that the historical text coordinates can be used to represent the actual position of the text in the historical frame image in the image.

[0081] S603-1-2: When the error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is within a threshold range, the historical text coordinates are used as the target text coordinates.

[0082] S603-1-3: When the error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is not within a threshold range, the text coordinates extracted from the video image of the target frame are used as the target text coordinates.

[0083] In an embodiment of the present invention, when the error between the text coordinates in the video image of the target frame and the historical text coordinates is within a threshold range, it indicates that the text in the video image of the target frame and the text in the historical frame video image are the same sentence. At this time, considering that the text coordinates extracted from the video image of the target frame may fluctuate with the actual position of the text in the image, but the historical text coordinates represent the actual position of the text, therefore, the historical text coordinates can be used as the target text coordinates of the video image of the target frame. Conversely, when the error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is not within the threshold range, it indicates that the text in the video image of the target frame and the text in the historical frame video image are not the same sentence. At this time, the text coordinates extracted from the video image of the target frame can be directly used as the target text coordinates.

[0084] It can be understood that the above-mentioned target text coordinates can represent the actual location of the text in the video image of the target frame. By eliminating the text in the area represented by the target text coordinates, the accuracy can be improved and the risk of flickering instability during the text elimination process can be reduced.

[0085] Optionally, in one embodiment, in order to quickly obtain the target text coordinates of the video image of the target frame, the following Figure 7 Based on this, a possible implementation method is given, see Fig. 9 , Fig. 9 A schematic flow chart of another text elimination method provided by an embodiment of the present invention, the method further comprising:

[0086] S602: Obtain a text coordinate set of the video to be processed through a text recognition network.

[0087] In an embodiment of the present invention, the above-mentioned text coordinate set includes the text coordinates extracted from the video image of the target frame and the historical text coordinates of the video image of the historical frame. The above-mentioned text recognition network can be a connectionist text proposal network (CTPN). Before obtaining the target text coordinates, the text coordinates extracted from the video image of the target frame and the historical text coordinates of the video image of the historical frame can be quickly obtained from the text coordinate set to save processing time.

[0088] Optionally, in one embodiment, the execution subject of each of the above steps may be an electronic device. In order to meet the user's need to delete text, the interactive interface of the electronic device in the embodiment of the present invention may include a function area for receiving user operations. The user's operation on the function area may trigger the electronic device to execute each of the above steps to complete the text deletion process. Figure 1 Based on this, a possible implementation method is given, see Fig.10 , Fig.10 A schematic flow chart of another text processing method provided by an embodiment of the present invention, the method further comprising:

[0089] S601: Receive a user operation instruction, where the user operation instruction is used to instruct to obtain an image to be processed.

[0090] To facilitate the human-computer interaction process, see Fig.11 , see Fig.11 , Fig.11 A schematic diagram of a function display interface of an electronic device provided by an embodiment of the present invention, Fig.11 The function display interface shown may have, but is not limited to, two main function areas: a file upload area and a function selection area, wherein the file upload area is used to instruct the user to input the video file to be processed, and the function selection area is used to instruct the user to select a function, for example, "Function One" corresponds to the text elimination function provided in an embodiment of the present invention, and "Function Two" corresponds to other forms of text processing functions. When the user clicks or touches the area corresponding to "Function One", the electronic device receives the user's operation instruction, which triggers the electronic device to execute the text elimination method provided in an embodiment of the present invention, and starts to perform text elimination processing on each frame of the video image for the video file in the file upload area through the above-mentioned text elimination method.

[0091] Optionally, Figure 1 The training process of the text elimination area shown can also be carried out by using a preset training sample before obtaining the image to be processed. Therefore, in a possible implementation, before step S601, the text elimination method can also include:

[0092] S600: Obtain a preset region elimination model.

[0093] It can be understood that the above-mentioned region elimination model is obtained by training a preset number of sample images; the sample image has a feature region; the feature region is used to instruct the region elimination model to eliminate the feature region; the feature region has the following feature information: color format, size and layout angle.

[0094] It can be understood that in order to enable the above-mentioned preset text elimination model to have the function of identifying feature areas and eliminating feature areas, any position of the above-mentioned preset training sample image can have a feature area; the feature area can have the following feature information: color format, size and layout angle.

[0095] In one implementation, the electronic device in the embodiment of the present invention further provides another functional interface, see Fig.12 , Fig.12 A schematic diagram of another functional display interface provided in an embodiment of the present invention, wherein the functional interface is used to display the processing progress of each video to be processed, so that the user can directly grasp the processing information.

[0096] In order to implement the various steps in the above-mentioned text elimination method embodiment and achieve the corresponding technical effect, a text elimination device implementation method is given below. Fig.13 , Fig.13 This is a functional module diagram of a text elimination device provided by an embodiment of the present invention, wherein the text elimination device 13 includes: an acquisition module 131 and an elimination module 132.

[0097] The acquisition module 131 is used to acquire the image to be processed; the image to be processed has a characteristic region; the characteristic region is used to indicate that the characteristic region is to be eliminated; the characteristic region matches the region of the text to be processed in the image to be processed.

[0098] The elimination module 132 is used to input the image to be processed into the region elimination model so that the region elimination model eliminates the feature area and outputs the processed image to be processed; the processed image to be processed does not contain the text to be processed; the region elimination model is a model obtained when the loss value of the loss function of the recognition network training is within a threshold range.

[0099] It is understandable that the acquisition module 131 and the elimination module 132 can be used to perform Figure 6 , Figure 7 Figure 8 Each step in the process is performed to achieve the corresponding technical effect.

[0100] Optionally, the text elimination device 13 further includes an extraction module, which is used to extract a text coordinate set of the video to be processed.

[0101] It is understandable that the acquisition module 131, the elimination module 132 and the extraction module can also be used to cooperate to perform Fig. 9 Each step in the process is performed to achieve the corresponding technical effect.

[0102] Optionally, the text elimination device 13 further includes a receiving module, wherein the receiving module is used to receive a user operation instruction, wherein the user operation instruction is used to instruct to obtain an image to be processed.

[0103] It is understandable that the acquisition module 131, the elimination module 132 and the extraction module can also be used to cooperate to perform Fig.10 Each step in the process is performed to achieve the corresponding technical effect.

[0104] In one embodiment, in order to obtain a pre-trained region elimination model, the embodiment of the present invention may also provide an implementation method of a region elimination model training device, and the region elimination model training device includes: an acquisition module, a training module and a determination module.

[0105] The acquisition module is used to acquire a preset number of training sample images; each training sample image contains a feature area; the feature area is used to indicate the elimination process of the feature area.

[0106] The training module is used to input the training sample images into the recognition network for training.

[0107] The determination module is used to use the recognition network as a region elimination model when the loss value of the loss function of the recognition network is within a threshold range.

[0108] It is understood that the acquisition module, the training module and the determination module can be used to perform Figure 1 , Figure 4 Each step in the process is performed to achieve the corresponding technical effect.

[0109] The embodiment of the present invention further provides an electronic device, such as Fig.14 , Fig.14A block diagram of the structure of an electronic device is provided for an embodiment of the present invention. The electronic device 14 includes a communication interface 141, a processor 142 and a memory 143. The processor 142, the memory 143 and the communication interface 141 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 143 can be used to store software programs and modules, such as program instructions / modules corresponding to the text elimination method or the area elimination model training method provided in the embodiment of the present invention. The processor 142 executes various functional applications and data processing by executing the software programs and modules stored in the memory 143. The communication interface 141 can be used for signaling or data communication with other node devices. In the present invention, the electronic device 14 can have multiple communication interfaces 141.

[0110] Among them, the memory 143 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0111] The processor 142 may be an integrated circuit chip with signal processing capability. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0112] It can be understood that each module of the above-mentioned text elimination device 13 and the area elimination model training device can be stored in the memory 143 of the electronic device 14 in the form of software or firmware, and executed by the processor 142. At the same time, the data, program code, etc. required to execute the above-mentioned modules can be stored in the memory 143.

[0113] An embodiment of the present invention provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the text elimination method or the region elimination model training method as described in any of the above embodiments is implemented. The computer-readable storage medium may be, but is not limited to, a U disk, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic disk, or an optical disk, and other media that can store program codes.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Various changes or substitutions that can be easily thought of by technicians familiar with the technical field within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A text elimination method, characterized in that: The method comprises: Acquiring an image to be processed, including: acquiring target text coordinates of a video image of a target frame; the video image of the target frame has the text to be processed; the image to be processed has a feature area; the feature area is used to indicate that the feature area is to be eliminated; the feature area is matched with a region of the text to be processed in the image to be processed; Obtaining target text coordinates of a video image of a target frame, including: obtaining text coordinates extracted from the video image of the target frame and historical text coordinates of at least one historical frame video image adjacent to the target frame; matching the historical text coordinates with an area of ​​a text to be processed; when an error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is within a threshold range, using the historical text coordinates as the target text coordinates; when an error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is not within a threshold range, using the text coordinates extracted from the video image of the target frame as the target text coordinates; The image to be processed is input into a preset region elimination model, so that the preset region elimination model performs elimination processing on the feature region and then outputs a target image; the target image does not contain the text to be processed.

2. The text elimination method according to claim 1, characterized in that: The obtaining of the image to be processed also includes: adding marking information to the area matching the target text coordinates in the video image of the target frame to generate the image to be processed; the marking information is used to indicate that the area is the feature area.

3. The text elimination method according to claim 1, characterized in that: Before acquiring the target text coordinates of the video image of the target frame, the method further includes: A text coordinate set of the video to be processed is obtained through a text recognition network; the text coordinate set includes text coordinates extracted from the video image of the target frame and text coordinates of the video image of the historical frame.

4. The text elimination method according to claim 1, characterized in that: Before acquiring the image to be processed, the method further includes: Receive a user operation instruction; the user operation instruction is used to instruct to obtain the image to be processed.

5. The text elimination method according to claim 1, characterized in that: Before acquiring the image to be processed, the method further includes: Obtain the preset region elimination model; the region elimination model is obtained by training a preset number of sample images; the sample image has a feature region; the feature region is used to instruct the region elimination model to eliminate the feature region; the feature region has the following feature information: color format, size and layout angle.

6. The text elimination method according to claim 1, characterized in that: The step of inputting the image to be processed into a preset region elimination model so that the preset region elimination model performs elimination processing on the feature region and then outputs a target image comprises: The image to be processed is input into a preset region elimination model so that the region elimination model replaces the pixel values ​​in the feature region according to the target pixel values ​​and then outputs the target image; wherein the target pixel value is the pixel value corresponding to the pixel point in the area outside the feature region that is adjacent to the pixel point in the feature region.

7. The text elimination method according to claim 1, characterized in that: The step of inputting the image to be processed into a preset region elimination model so that the preset region elimination model performs elimination processing on the feature region and then outputs a target image comprises: The image to be processed is input into a preset region elimination model, so that the region elimination model performs a cutting process on the feature region and then outputs the target image.

8. A text elimination device, characterized in that: include: An acquisition module is used to acquire an image to be processed, including: acquiring target text coordinates of a video image of a target frame; the video image of the target frame has the text to be processed; the image to be processed has a feature area; the feature area is used to indicate that the feature area is to be eliminated; the feature area is matched with the area of ​​the text to be processed in the image to be processed; an elimination module, used for inputting the image to be processed into a preset region elimination model, so that the preset region elimination model performs elimination processing on the feature region and then outputs a target image; the target image does not contain the text to be processed; The acquisition module is also specifically used to: acquire the text coordinates extracted from the video image of the target frame and the historical text coordinates of at least one historical frame video image adjacent to the target frame; match the historical text coordinates with the area of ​​the text to be processed; when the error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is within a threshold range, use the historical text coordinates as the target text coordinates; when the error between the text coordinates extracted from the video image of the target frame and the historical text coordinates is not within a threshold range, use the text coordinates extracted from the video image of the target frame as the target text coordinates.

9. An electronic device, characterized in that: The electronic device includes a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions. When the processor executes the machine-executable instructions, the electronic device implements the text elimination method described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores machine executable instructions, and when the machine executable instructions are executed, the text elimination method described in any one of claims 1 to 7 is implemented.

Images