A method for processing text in an image and a related device

Abstract

The present application discloses a method for processing text in an image and a related device, and relates to the field of image processing technology. The method for processing text in an image includes: obtaining a target image containing text; performing text line attribute detection on the target image to obtain text line attributes of the text line in the target image; performing text attribute recognition on the text line in the target image to obtain text attributes of the text line in the target image; obtaining attributes selected by a user from text line attributes and text attributes as target text attributes; and erasing or editing the text line of the target text attribute in the target image. The method for processing text in an image disclosed in the present application can obtain the text attributes of the text line in the target image by performing text attribute detection on the target image. On this basis, the user can specify the text attributes, and then the text line of the text attribute specified by the user in the target image can be erased or edited, so as to obtain an image that meets the user's requirements.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for processing text in an image and a related device. Background Art

[0002] In some scenarios, it is necessary to process part or all of the text in an image containing text to meet the user's requirements. For example, in some scenarios, the user wants to hide part or all of the text in an image containing text. In other scenarios, the user wants to edit part or all of the text in an image containing text into other text (for example, edit the English text in the image into the corresponding Chinese text). How to process an image containing text into an image that meets the user's requirements is an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present application provides a method and a related device for processing text in an image, which are used to process an image containing text into an image that meets user requirements. The technical solution is as follows:

[0004] The first aspect of the present application provides a method for processing text in an image, which comprises:

[0005] Acquire a target image, wherein the target image contains text;

[0006] Performing text line attribute detection on the target image to obtain text line attributes of the text lines in the target image;

[0007] Performing text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image;

[0008] Obtaining the attribute selected by the user from the text line attributes obtained by detection and the character attributes obtained by recognition as the target text attribute;

[0009] The text line of the target text attribute in the target image is erased or edited.

[0010] In a possible implementation, the text line attributes include part or all of the following attributes: the position of the text line, the font of the text line, and the rotation angle of the text line;

[0011] The text attributes include part or all of the following attributes: text content, text position, text direction, and text language.

[0012] In a possible implementation, the performing text line attribute detection on the target image to obtain the text line attribute of the text line in the target image includes:

[0013] Using a pre-trained text line attribute detection model, the text line position and text line font attribute detection is performed on the target image to obtain the position and font of the text line in the target image, wherein the text line attribute detection model is trained using a training image in which the real text line position and the real text line font are marked;

[0014] According to the position of the text line in the target image, a rotation angle of the text line in the target image is determined.

[0015] In a possible implementation, the training process of the text line attribute detection model includes:

[0016] Extracting visual features of multiple scales from the training image using a text line attribute detection model, and fusing the visual features of multiple scales extracted from the training image to obtain multi-scale fused features of the training image;

[0017] Using a text line attribute detection model and taking the multi-scale fusion features of the training image as a basis, predicting the position and font of the text line in the training image to obtain a first prediction result;

[0018] Predicting the position and font of a text line in the training image according to the visual features of each scale extracted from the training image to obtain a plurality of second prediction results;

[0019] Acquire text semantic features of the training image, fuse the text semantic features of the training image with multi-scale fusion features of the training image, and predict the position and font of a text line in the training image according to the fused features to obtain a third prediction result;

[0020] With the goal of making each prediction result close to the annotation information of the training image, the parameters of the text line attribute detection model are updated.

[0021] In a possible implementation, the performing text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image includes:

[0022] Using a pre-trained text attribute recognition model, extract visual features from the area where the text line in the target image is located, and obtain text temporal features based on the visual features;

[0023] Determining the text direction of the text line in the target image by using the text attribute recognition model and based on the visual features;

[0024] The text attribute recognition model is used to obtain the text content and text position of the text line in the target image based on the text temporal features; wherein the text attribute recognition model is trained using training text line images with real text direction, real text content and real text position annotated;

[0025] The language of the text line in the target image is recognized by performing language recognition on the text content in the text line in the target image to obtain the language of the text in the text line in the target image.

[0026] In a possible implementation manner, erasing the text line of the target text attribute in the target image includes:

[0027] Generate an erased area mask according to the position of the text line of the target text attribute, wherein the erased area mask has the same size as the target image and can indicate an area in the target image that needs to be erased;

[0028] The text line of the target text attribute in the target image is erased using the erase area mask.

[0029] In a possible implementation manner, editing the text line of the target text attribute in the target image includes:

[0030] Erasing the text line of the target text attribute in the target image to obtain an erased image;

[0031] Obtain the edited text corresponding to the text line of the target text attribute;

[0032] According to the position of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

[0033] In a possible implementation, the step of pasting the edited text corresponding to the text line of the target text attribute into the erased image according to the position of the text line of the target text attribute includes:

[0034] According to the position of the text line of the target text attribute and in combination with the font and / or text direction of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

[0035] In a possible implementation, pasting the edited text corresponding to the text line of the target text attribute into the erased image includes:

[0036] Determine the text foreground color of the text line of the target text attribute;

[0037] According to the text foreground color of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

[0038] In a possible implementation, determining the text foreground color of the text line of the target text attribute includes:

[0039] Acquire a target text image, wherein the target text image is an image having the same size as the target image and only including the text strokes of the text line of the target text attribute in the target text image;

[0040] Determine the area where each character in the target text image is located according to the position of each character in the text line of the target text attribute;

[0041] Determine the foreground color of each character in the text line of the target text attribute by clustering the foreground color of the character strokes in the area where each character in the target text image is located;

[0042] According to the difference in foreground colors of adjacent characters of each character in the text line of the target text attribute, a final foreground color of each character in the text line of the target text attribute is determined.

[0043] In a possible implementation, erasing the text line of the target text attribute in the target image to obtain an erased image, and acquiring the target text image includes:

[0044] Generate an erased area mask according to the position of the text line of the target text attribute, wherein the erased area mask has the same size as the target image and can indicate an area in the target image that needs to be erased;

[0045] Inputting the target image and the erased area mask into a pre-trained text erasure model to obtain an erased image and a target text image output by the text erasure model;

[0046] Among them, the text erasure model is trained with training images and erased area masks corresponding to the training images as training samples, and with real erased images and erased text images corresponding to the training images as sample labels. The erased text image has the same size as the training image and only includes the strokes of the text lines to be erased in the training image.

[0047] A second aspect of the present application provides a device for processing text in an image, comprising: an image acquisition module, a text line attribute detection module, a text attribute recognition module, a target text attribute acquisition module, and a text processing module;

[0048] The image acquisition module is used to acquire a target image, wherein the target image contains text;

[0049] The text line attribute detection module is used to perform text line attribute detection on the target image to obtain text line attributes of the text lines in the target image;

[0050] The text attribute recognition module is used to perform text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image;

[0051] The target text attribute acquisition module is used to acquire the attribute selected by the user from the text line attributes detected and the character attributes recognized as the target text attribute;

[0052] The text processing module is used to erase or edit the text line of the target text attribute in the target image.

[0053] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0054] The memory is used to store computer programs;

[0055] The processor is used to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned methods for processing text in an image.

[0056] In a fourth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned methods for processing text in an image.

[0057] A fifth aspect of the present application provides a computer program product, comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement the steps of any one of the above-described methods for processing text in an image.

[0058] By means of the above technical scheme, the method for processing text in an image provided by the present application first obtains a target image containing text, then performs text line attribute detection on the target image to obtain the text line attributes of the text line in the target image, then performs text attribute recognition on the text line in the target image to obtain the text attributes of the text line in the target image, then obtains the attributes selected by the user from the detected text line attributes and the recognized text attributes as the target text attributes, and finally erases or edits the text line of the target text attribute in the target image. The method for processing text in an image provided by the present application can obtain the text line attributes and text attributes of the text line in the target image by performing text line attribute detection and text attribute recognition on the target image, on this basis, the user can specify the text attributes, and then erases or edits the text line of the text attribute specified by the user in the target image, so as to obtain an image that meets the user's requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0060] Figure 1 A schematic diagram of a system architecture involved in this application;

[0061] Figure 2 A schematic diagram of a hardware structure of a terminal provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of a hardware structure of a server provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of a flow chart of a method for processing text in an image provided in an embodiment of the present application;

[0064] Figure 5 Schematic diagram of several text directions provided in the embodiments of the present application;

[0065] Figure 6 A schematic diagram of a training method for a text line attribute detection model provided in an embodiment of the present application;

[0066] Figure 7 A schematic diagram of the structure of a text attribute recognition model provided in an embodiment of the present application;

[0067] Figure 8 A schematic diagram of text erasure based on a text erasure model provided in an embodiment of the present application;

[0068] Fig. 9 A schematic diagram of pasting edited text corresponding to a text line of a target text attribute in a target image according to a text foreground color of the text line of a target text attribute in an embodiment of the present application in an erased image;

[0069] Fig.10 A schematic diagram of the structure of a device for processing text in an image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0071] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0072] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0073] In one possible implementation, Figure 1 As shown, the system architecture involved in the present application may include a terminal 101 and a server 102, and the terminal 101 may interact with the server 102 via a network (wired network or wireless network). The server 102 may include one or more servers ( Figure 1 The example includes a server. The terminal 101 can obtain an image containing text (such as a natural scene image containing text), and transmit the image containing text to the server 102 through the network. The server 102 uses the text processing method in the image provided by the present application to process the image containing text, and feeds back the processing result to the terminal.

[0074] In another possible implementation, the system architecture involved in the present application may include a terminal. The terminal has strong data processing capabilities. The terminal can obtain an image containing text (such as a natural scene image containing text), and use the text processing method in the image provided by the present application to process the image containing text.

[0075] Next, the product form of the above-mentioned terminal is described.

[0076] The above-mentioned terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a robot, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiments of the present application do not impose any restrictions on this.

[0077] Figure 2 An optional hardware structure diagram of a terminal is shown.

[0078] refer to Figure 2 As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), an earphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290 and other components. Those skilled in the art will appreciate that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0079] The input unit 230 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 270, and can receive and execute the command sent by the processor 270; the touch signal at least includes the contact point coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch screen 231, the input unit 230 can also include other input devices. Specifically, other input devices 232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0080] The display unit 240 may be used to display information input by a user or information provided to a user, various menus of the terminal, an interactive interface, file display, and / or playback of any multimedia file.

[0081] The memory 220 can be used to store instructions and data. The memory 220 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; it provides the processor 270 with hardware, software, and data resources including management of computing and processing equipment, and supports control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0082] The processor 270 is the control center of the terminal. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing instructions stored in the memory 220 and calling data stored in the memory 220, it executes various functions of the terminal and processes data, thereby controlling the terminal as a whole. Optionally, the processor 270 may include one or more processing units; preferably, the processor 270 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 270. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 270 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 220, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0083] Among them, the memory 220 can be used to store software codes related to the processing method of text in the image, and the processor 270 can execute the software codes in the memory 220, and can also schedule other units (such as the above-mentioned input unit 230 and display unit 240) to realize corresponding functions.

[0084] The radio frequency unit 210 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 270 for processing; in addition, the designed uplink data is sent to the base station. Generally, the radio frequency unit 210 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LowNoiseAmplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 210 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General PacketRadio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0085] In the embodiment of the present application, the radio frequency unit 210 can send data to other devices, and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional, and can be replaced by other communication interfaces, such as a network port.

[0086] The terminal also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0087] The terminal also includes an external interface 280, which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal to communicate with other devices, or to connect a charger to charge the terminal.

[0088] Although not shown, the terminal may further include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here.

[0089] Next, the product form of the above server is described.

[0090] Figure 3 A structural diagram of the above server is provided, such as Figure 3As shown, the server may include a bus 301, a processor 302, a communication interface 303, and a memory 304. The processor 302, the memory 304, and the communication interface 303 communicate with each other via the bus 301.

[0091] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0092] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0093] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0094] The memory 304 may be used to store software codes related to the method for processing text in an image. The processor 302 may call the software codes stored in the memory 304 or schedule other units to implement corresponding functions.

[0095] The processors in the above-mentioned terminals and servers (for example, processor 270 and processor 302) may be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (DSP), microprocessors or microcontrollers, etc.), or combinations of these hardware circuits. For example, the processor may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0096] Next, the method for processing text in an image provided by the present application is introduced through the following embodiments.

[0097] See also Figure 4 , shows a schematic flow chart of a method for processing text in an image provided by an embodiment of the present application, the method may include:

[0098] Step S401: Acquire a target image.

[0099] Among them, the target image contains text.

[0100] The target image in this embodiment may be, but is not limited to, a natural scene image.

[0101] Step S402: Perform text line attribute detection on the target image to obtain text line attributes of the text lines in the target image.

[0102] The text line attributes in this embodiment may include, but are not limited to, some or all of the following attributes: the position of the text line, the font of the text line, and the rotation angle of the text line.

[0103] Among them, the position of a text line can be the regional envelope position of the text line, the font of a text line can be but is not limited to one of printed and handwritten fonts, and the rotation angle of a text line is the inclination angle of the text line (for example, the inclination angle relative to the horizontal direction).

[0104] In one possible implementation, text line position detection and text line font attribute detection may be performed on the target image simultaneously to obtain the position and font of the text line in the target image. After obtaining the position of the text line, the rotation angle of the text line may be determined based on the position of the text line.

[0105] Step S403: performing text attribute recognition on the text line in the target image to obtain the text attributes of the text line in the target image.

[0106] The text attributes in this embodiment may include, but are not limited to, some or all of the following attributes: text content, text position, text direction, and text language. Figure 4 , showing schematic diagrams of several text directions.

[0107] According to the position of the text line in the target image, the area where the text line is located can be extracted from the target image, and then the text attribute recognition can be performed on the area where the text line is located to obtain the text attribute of the text line in the target image.

[0108] In one possible implementation, text direction recognition, text content, and text position recognition can be performed simultaneously on the area where the text line is located to obtain the text direction, text content, and text position of the text line in the target image. After obtaining the text content of the text line in the target image, language recognition can be performed on the obtained text content to obtain the text language of the text line in the target image.

[0109] Step S404: obtaining a text attribute selected by the user from the text line attributes detected and the character attributes recognized as a target text attribute.

[0110] The user can select one text attribute from the detected text attributes (including text line attributes and character attributes), such as handwriting. The user can also select multiple text attributes from the detected text attributes, such as Chinese and printed text.

[0111] Step S405: erasing or editing the text line of the target text attribute in the target image.

[0112] Exemplarily, if the text attribute selected by the user, ie, the target text attribute, is Chinese printed text, then the text line of Chinese printed text in the target image is erased or edited.

[0113] The text line of the target text attribute in the target image is erased to achieve hiding of the specified text in the target image, while the text line of the target text attribute in the target image is edited to achieve changing of the text content in the target image.

[0114] The method for processing text in an image provided by an embodiment of the present application first obtains a target image containing text, then performs text line attribute detection on the target image to obtain the text line attributes of the text line in the target image, then performs text attribute recognition on the text line in the target image to obtain the text attributes of the text line in the target image, then obtains the text attributes selected by the user from the text line attributes obtained by detection and the text attributes obtained by recognition as the target text attributes, and finally erases or edits the text line of the target text attribute in the target image. The method for processing text in an image provided by an embodiment of the present application can obtain the text line attributes and text attributes of the text line in the target image by performing text line attribute detection and text attribute recognition on the target image, on this basis, the user can specify the text attributes, and then erases or edits the text line of the text attribute specified by the user in the target image, so as to obtain an image that meets the user's requirements.

[0115] In another embodiment of the present application, the specific implementation process of "step S402: performing text line attribute detection on the target image to obtain text line attributes of the text line in the target image" in the above embodiment is introduced.

[0116] In a possible implementation manner, the specific implementation process of performing text line attribute detection on the target image and obtaining the text line attributes of the text line in the target image may include:

[0117] Step a1: Use the pre-trained text line attribute detection model to detect the text line position and text line font attributes of the target image to obtain the position and font of the text line in the target image.

[0118] Among them, the text line attribute detection model is trained using training images that mark the real text line position and the real text line font.

[0119] Specifically, using the pre-trained text line attribute detection model to detect the text line position and text line font attribute of the target image, the process of obtaining the position and font of the text line in the target image may include:

[0120] Step a11: using the pre-trained text line attribute detection model, extract visual features of multiple scales from the target image, and fuse the visual features of multiple scales to obtain multi-scale fused features of the target image.

[0121] In a possible implementation, the text line attribute detection model may include a multi-scale visual feature extraction module and a multi-scale visual feature fusion module. The target image may be input into the multi-scale visual feature extraction module of the text line attribute detection model, the multi-scale visual feature extraction module extracts and outputs visual features of multiple scales from the input target image, the visual features of multiple scales output by the multi-scale visual feature extraction module are input into the multi-scale visual feature fusion module, the multi-scale visual feature fusion module fuses the visual features of multiple scales input (for example, splices the visual features of multiple scales), and outputs multi-scale fusion features.

[0122] Step a12: using the text line attribute detection model and taking the multi-scale fusion features of the target image as a basis, predict the position and font of the text line in the target image.

[0123] The text line attribute detection model may also include a target information prediction module. The multi-scale fusion features of the target image are input into the target information prediction module. The target information prediction module predicts the position and font of the text line in the target image based on the input multi-scale fusion features.

[0124] Step a2: Determine the rotation angle of the text line in the target image according to the position of the text line in the target image.

[0125] After the position of the text line is determined, the minimum bounding rectangle of the text line can be obtained, and then the rotation angle of the text line (such as the tilt angle relative to the horizontal direction) can be determined according to the minimum bounding rectangle of the text line.

[0126] As mentioned above, the text line attribute detection model is trained using training images that annotate the real text line positions and real text line fonts. Next, the training process of the text line attribute detection model is introduced.

[0127] In a possible implementation, the training process of the text line attribute detection model may include:

[0128] Step b1: using the text line attribute detection model to extract visual features of multiple scales from the training image, thereby obtaining visual features of multiple scales of the training image.

[0129] like Figure 6 As shown, the training image is input into the multi-scale visual feature extraction module of the text line attribute detection model, and the multi-scale visual feature extraction module extracts visual features of multiple scales from the input training image and outputs them.

[0130] Step b2: using the text line attribute detection model, the visual features of multiple scales extracted from the training image are fused to obtain the multi-scale fused features of the training image.

[0131] like Figure 6As shown, the visual features of multiple scales of the training image are input into the multi-scale visual feature fusion module of the text line attribute detection model. The multi-scale visual feature fusion module fuses the visual features of multiple scales of the training image (for example, splicing the visual features of multiple scales), and outputs the multi-scale fused features of the training image.

[0132] Step b3: using the text line attribute detection model and taking the multi-scale fusion features of the training image as a basis, predicting the position and font attributes of the text line in the training image to obtain a first prediction result.

[0133] like Figure 6 As shown, the multi-scale fusion features of the training image are input into the target information prediction module of the text line attribute detection model. The target information prediction module predicts the position and font of the text line in the training image according to the multi-scale fusion features of the training image to obtain a first prediction result.

[0134] Step b4: predict the position and font of the text line in the training image based on the features of each scale extracted from the training image to obtain multiple second prediction results.

[0135] like Figure 6 As shown, the features of each scale extracted from the training image can be input into the first prediction module of the auxiliary training, and the first prediction module of the auxiliary training predicts the position and font of the text line in the training image according to the input features to obtain multiple second prediction results.

[0136] This embodiment does not limit the execution order of step b4, as long as step b4 is executed after step b1.

[0137] Step b5, obtaining text semantic features of the training image, fusing the text semantic features of the training image with the multi-scale fusion features of the training image, predicting the position and font of the text line in the training image based on the fused features, and obtaining a third prediction result.

[0138] This embodiment does not limit the execution order of step b5, as long as step b5 is executed after step b2.

[0139] like Figure 6 As shown, the text semantic features of the training image can be obtained based on the text semantic feature acquisition module, and the text semantic features of the training image can be combined with the multi-scale fusion features of the training image using the feature fusion module. The fused features output by the feature fusion module are input into the second prediction module of the auxiliary training. The second prediction module of the auxiliary training predicts the position and font of the text line in the training image according to the input fused features to obtain a third prediction result.

[0140] Step b6: update the parameters of the text line attribute detection model with the goal of making each prediction result close to the annotation information of the training image.

[0141] Specifically, the prediction loss can be determined according to each prediction result and the annotation information of the training image to obtain several prediction losses, and the several prediction losses are fused, and the parameters of the text line attribute detection model are updated according to the fused losses.

[0142] The training images in the training data set of the text line attribute detection model can be used to perform multiple iterative training on the text line attribute detection model according to the training method of steps b1 to b6 above until the training end conditions are met (such as model convergence, or reaching a preset number of training iterations, etc.).

[0143] It should be noted that this embodiment is not limited to the training method of steps b1 to b6 for training the text line attribute detection model. For example, the text line attribute detection model can also be trained according to the training method of steps b1 to b4 and step b6 (only the first prediction result and the second prediction result are obtained), and the text line attribute detection model can also be trained according to the training method of steps b1 to b3 and steps b5 and step b6 (only the first prediction result and the third prediction result are obtained).

[0144] In another embodiment of the present application, the specific implementation process of "step S403: performing text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image" in the above embodiment is introduced.

[0145] In a possible implementation, the specific implementation process of performing text attribute recognition on the text line in the target image and obtaining the text attribute of the text line in the target image may include:

[0146] Step c1: using the pre-trained text attribute recognition model, extract visual features from the area where the text line is located in the target image, and obtain text temporal features based on the visual features.

[0147] When using the pre-trained text attribute recognition model to extract visual features from the region where the text line is located in the target image, the region where the text line is located may be preprocessed first, and then the preprocessed image may be input into the text attribute recognition model.

[0148] The preprocessing of the area where the text line is located may include, but is not limited to, text line angle correction and / or text line curvature correction. The text line angle correction means that if the text line is tilted, the text line is rotated to make it horizontal, and the text line curvature correction means correcting the curved text line (equidistant sampling can be performed according to the text line contour, and then curvature correction can be performed through TPS (Thin-Plate-Spline) transformation).

[0149] In one possible implementation, Figure 7 As shown, the text attribute recognition model may include an encoder, and the encoder may include a backbone network (such as a residual network Resnet, a densely connected convolutional network Desnet, etc.) and a bidirectional long short-term memory network BiLSTM. The area where the text line in the target image is located is input into the backbone network of the encoder for feature extraction, and the backbone network outputs visual features. The visual features output by the backbone network are further input into the BiLSTM, and the BiLSTM extracts text temporal features based on the input visual features.

[0150] Step c2-a: using the text attribute recognition model and based on visual features, determine the text direction of the text line in the target image.

[0151] like Figure 7 As shown, the text attribute recognition model may also include a text direction attribute prediction module. The visual features output by the backbone network of the encoder are input into the text direction attribute prediction module, and the text direction attribute prediction module predicts the text direction according to the input visual features.

[0152] Step c2-b: using the text attribute recognition model and taking the text temporal features as the basis, obtain the text content and text position of the text line in the target image.

[0153] like Figure 7 As shown, the text attribute recognition model can also be a decoder. The text temporal features output by the BiLSTM of the encoder are input into the decoder. The decoder uses a decoding method based on the attention mechanism to decode the input text temporal features and output the text content and text position of the text line in the target image.

[0154] The text attribute recognition model in this embodiment is trained using training text line images with the real text direction, real text content and real text position marked.

[0155] When training the text attribute recognition model, the training text line image is first input into the text attribute recognition model to obtain the text direction, text content and text position predicted by the text attribute recognition model for the training text line image, and then the prediction loss is determined based on the predicted text direction, text content and text position and the information annotated on the training text line image (real text direction, real text content and real text position), and finally the parameters of the text attribute recognition model are updated based on the prediction loss. The text attribute recognition model can be iteratively trained multiple times in the above manner using the training text line images in the training data set of the text attribute recognition model until the training end condition is met (such as model convergence, or the preset number of training iterations is reached, etc.).

[0156] Step c3: perform language recognition on the text content of the text line in the target image to obtain the text language of the text line in the target image.

[0157] After obtaining the text content of the text line in the target image, the language of the text content in the text line in the target image may be recognized to obtain the language of the text line in the target image.

[0158] In another embodiment of the present application, the specific implementation process of "step S404: erasing or editing the text line of the target text attribute in the target image" in the above embodiment is introduced.

[0159] Firstly, the specific implementation process of erasing the text line of the target text attribute in the target image is introduced.

[0160] In a possible implementation manner, the specific implementation process of erasing the text line of the target text attribute in the target image may include:

[0161] Step d1: Generate an erased area mask according to the position of the text line of the target text attribute.

[0162] The erased area mask has the same size as the target image, and the erased area mask can indicate the area in the target image that needs to be erased.

[0163] Step d2: using the erased area mask, erase the text line of the target text attribute in the target image to obtain an erased image.

[0164] In a possible implementation, the erased area mask and the target image may be input into a pre-trained first text erasure model to obtain an erased image output by the first text erasure model. The first text erasure model is trained using a training image and an erased area mask corresponding to the training image as training samples, and a real erased image corresponding to the training image as a sample label.

[0165] Next, the specific implementation process of editing the text line of the target text attribute in the target image is introduced.

[0166] In a possible implementation manner, the specific implementation process of editing the text line of the target text attribute in the target image may include:

[0167] Step e1, erasing the text lines of the target text attributes in the target image to obtain an erased image.

[0168] The erased area mask can be generated based on the position of the text line of the target text attribute, such as Figure 8As shown, the target image and the erased area mask are input into the pre-trained second text erasure model to obtain the erased image and the target text image output by the second text erasure model. The target text image is an image with the same size as the target image and only includes the text strokes of the text line of the target text attribute in the target text image.

[0169] The second text erasure model in this embodiment is trained with the training image and the erased area mask corresponding to the training image as training samples, and with the real erased image and erased text image corresponding to the training image as sample labels. The erased text image has the same size as the training image and only includes the strokes of the text line to be erased in the training image.

[0170] Step e2: Obtain the edited text corresponding to the text line of the target text attribute.

[0171] The edited text corresponding to the text line of the target text attribute can be the translation result of the text content of the text line of the target text attribute, or can be the text after the text content of the text line of the target text attribute is modified, or can be other text unrelated to the text content of the text line of the target text attribute. The specific text of the edited text corresponding to the text line of the target text attribute depends on the actual application requirements.

[0172] Step e3: Paste the edited text corresponding to the text line of the target text attribute into the erased image according to the position of the text line of the target text attribute.

[0173] According to the position of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image so that the position of the edited text corresponding to the text line of the target text attribute is consistent with the position of the text line of the target text attribute.

[0174] Preferably, when the edited text corresponding to the text line of the target text attribute is pasted in the erased image according to the position of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute can also be pasted in the erased image according to the position of the text line of the target text attribute and in combination with the font and / or text direction of the text line of the target text attribute, so that the position of the edited text corresponding to the text line of the target text attribute is consistent with the position of the text line of the target text attribute, and the font and / or text direction of the edited text corresponding to the text line of the target text attribute is consistent with the font and / or text direction of the text line of the target text attribute.

[0175] Preferably, when pasting the edited text corresponding to the text line of the target text attribute into the erased image, the text foreground color of the text line of the target text attribute can also be determined, and then, according to the text foreground color of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted into the erased image.

[0176] For example, the text line of the target text attribute in the target image is "wash your hand, please!", where "hand" is gray and the rest is black. The edited text corresponding to the text line of the target text attribute is the translation result of "wash your hand, please!", which is "Please wash your hands!". When "Please wash your hands!" is pasted into the erased image, "hand" can be gray and the rest can be black. Fig. 9 shown.

[0177] Exemplarily, the text lines of the target text attribute in the target image are two lines of English, the first line of English is red, and the second line of English is black, and the edited text corresponding to the text lines of the target text attribute is two lines of Chinese. Then, when the edited text corresponding to the text lines of the target text attribute is pasted into the erased image, the first line of Chinese can be red and the second line of Chinese can be black.

[0178] Among them, the process of determining the foreground color of the text line of the target text attribute may include: obtaining the target text image through the above-mentioned second text erasure model; determining the area where each character is located in the target text image according to the position of each character in the text line of the target text attribute; determining the foreground color of each character in the text line of the target text attribute by clustering the foreground colors of the text strokes in the area where each character is located in the target text image; determining the final foreground color of each character in the text line of the target text attribute according to the difference in foreground colors of adjacent characters of each character in the text line of the target text attribute (the foreground colors of adjacent characters of each character may be class-center smoothed; if the difference in class centers of adjacent characters is large, no smoothing is performed, that is, the personalized color of the character is maintained; if the difference in class centers of adjacent characters is not large, class-center smoothing is performed).

[0179] The above describes the method for processing text in an image provided by an embodiment of the present application. The following describes a device corresponding to the method for processing text in an image.

[0180] See also Fig.10 , Fig.10 A structural schematic diagram of a device for processing text in an image provided in an embodiment of the present application, the device for processing text in an image may include: an image acquisition module 1001, a text line attribute detection module 1002, a text attribute recognition model 1003, a target text attribute acquisition module 1004 and a text processing module 1005.

[0181] The image acquisition module 1001 is used to acquire a target image, wherein the target image contains text.

[0182] The text line attribute detection module 1002 is used to perform text line attribute detection on the target image to obtain text line attributes of the text lines in the target image.

[0183] The text attribute recognition model 1003 is used to perform text attribute recognition on the text line in the target image to obtain the text attributes of the text line in the target image.

[0184] The target text attribute acquisition module 1004 is used to acquire the text attribute selected by the user from the text line attributes obtained by detection and the character attributes obtained by recognition as the target text attribute.

[0185] The text processing module 1005 is used to erase or edit the text line of the target text attribute in the target image.

[0186] In one possible implementation, text line attributes include some or all of the following attributes: text line position, text line font, and text line rotation angle; text attributes include some or all of the following attributes: text content, text position, text direction, and text language.

[0187] In a possible implementation, when the text line attribute detection module 1002 performs text line attribute detection on the target image to obtain the text line attribute of the text line in the target image, it is specifically used to:

[0188] Using a pre-trained text line attribute detection model, the text line position and text line font attribute detection are performed on the target image to obtain the position and font of the text line in the target image, wherein the text line attribute detection model is trained using a training image in which the real text line position and the real text line font are annotated;

[0189] According to the position of the text line in the target image, the rotation angle of the text line in the target image is determined.

[0190] In a possible implementation, the apparatus for processing text in an image may further include: a model training module for training a text line attribute detection model.

[0191] When training the text line attribute detection model, the model training module is specifically used to:

[0192] Using the text line attribute detection model, visual features of multiple scales are extracted from the training image, and the visual features of multiple scales extracted from the training image are fused to obtain multi-scale fused features of the training image;

[0193] Using the text line attribute detection model and taking the multi-scale fusion features of the training image as a basis, the position and font of the text line in the training image are predicted to obtain a first prediction result;

[0194] Predicting the position and font of a text line in the training image based on the visual features of each scale extracted from the training image to obtain a plurality of second prediction results;

[0195] Acquire text semantic features of the training image, fuse the text semantic features of the training image with multi-scale fusion features of the training image, and predict the position and font of a text line in the training image according to the fused features to obtain a third prediction result;

[0196] With the goal of making each prediction result close to the annotation information of the training image, the parameters of the text line attribute detection model are updated.

[0197] In a possible implementation, when the text attribute recognition model 1003 performs text attribute recognition on a text line in a target image to obtain the text attribute of the text line in the target image, it is specifically used to:

[0198] Using the pre-trained text attribute recognition model, the visual features of the area where the text line is located are extracted, and the text temporal features are obtained based on the visual features;

[0199] Using the text attribute recognition model, based on visual features, determine the text direction of the text line in the target image;

[0200] The text attribute recognition model is used to determine the text content and text position of the text line in the target image based on the text temporal features; wherein the text attribute recognition model is trained using training text line images with real text direction, real text content and real text position annotated;

[0201] The language of the text line in the target image is recognized by performing language recognition on the text content in the text line in the target image to obtain the language of the text in the text line in the target image.

[0202] In a possible implementation, when the text processing module 1005 performs the erasing process on the text line of the target text attribute in the target image, it is specifically used to:

[0203] Generate an erased area mask according to the position of the text line of the target text attribute, wherein the erased area mask has the same size as the target image, and the erased area mask can indicate the area to be erased in the target image;

[0204] Using the erased region mask, the text lines of the target text attributes in the target image are erased.

[0205] In a possible implementation, when the text processing module 1005 performs editing processing on the text line of the target text attribute in the target image, it is specifically used to:

[0206] Erasing the text line of the target text attribute in the target image to obtain an erased image;

[0207] Get the edited text corresponding to the text line of the target text attribute;

[0208] According to the position of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

[0209] In a possible implementation, when the text processing module 1005 pastes the edited text corresponding to the text line of the target text attribute into the erased image according to the position of the text line of the target text attribute, it is specifically configured to:

[0210] According to the position of the text line of the target text attribute and in combination with the font and / or text direction of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

[0211] In a possible implementation, when pasting the edited text corresponding to the text line of the target text attribute into the erased image, the text processing module 1005 is specifically configured to:

[0212] Determine the foreground color of the text line of the target text attribute;

[0213] According to the text foreground color of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted into the erased image.

[0214] In a possible implementation, when determining the text foreground color of the text line of the target text attribute, the text processing module 1005 is specifically configured to:

[0215] Acquire a target text image, wherein the target text image is an image having the same size as the target image and only including the text strokes of a text line with target text attributes in the target text image;

[0216] According to the position of each character in the text line of the target text attribute, determine the area where each character in the target text image is located;

[0217] Determine the foreground color of each character in the text line of the target text attribute by clustering the foreground color of the character strokes in the area where each character in the target text image is located;

[0218] According to the difference in foreground colors of adjacent characters of each character in the text line of the target text attribute, the final foreground color of each character in the text line of the target text attribute is determined.

[0219] In a possible implementation, the text processing module 1005 erases the text line of the target text attribute in the target image to obtain the erased image, and obtains the target text image, specifically for:

[0220] Generate an erased area mask according to the position of the text line of the target text attribute, wherein the erased area mask has the same size as the target image, and the erased area mask can indicate the area to be erased in the target image;

[0221] Input the target image and the erased area mask into the pre-trained text erasure model to obtain the erased image and the target text image output by the text erasure model;

[0222] Among them, the text erasure model is trained with the training image and the erased area mask corresponding to the training image as training samples, and the real erased image and erased text image corresponding to the training image as sample labels. The erased text image has the same size as the training image and only includes the strokes of the text line to be erased in the training image.

[0223] The apparatus for processing text in an image provided by an embodiment of the present application first acquires a target image containing text, then performs text line attribute detection on the target image to obtain text line attributes of the text line in the target image, then performs text attribute recognition on the text line in the target image to obtain text attributes of the text line in the target image, then acquires the attributes selected by the user from the detected text line attributes and the recognized text attributes as the target text attributes, and finally erases or edits the text line of the target text attribute in the target image. The apparatus for processing text in an image provided by an embodiment of the present application can obtain text line attributes and text attributes of the text line in the target image by performing text line attribute detection and text attribute recognition on the target image, on this basis, the user can specify the text attributes, and then erases or edits the text line of the text attribute specified by the user in the target image, thereby obtaining an image that meets the user's requirements.

[0224] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0225] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0226] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0227] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0228] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to implement the steps of the method for processing text in an image provided in the above embodiment.

[0229] An embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the method for processing text in an image provided in the above embodiment.

[0230] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the method for processing text in an image provided in the above embodiment.

[0231] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0232] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0233] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0234] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for processing text in an image, characterized in that: include: Acquire a target image, wherein the target image contains text; Performing text line attribute detection on the target image to obtain text line attributes of the text lines in the target image; Performing text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image; Obtaining the attribute selected by the user from the text line attributes obtained by detection and the character attributes obtained by recognition as the target text attribute; The text line of the target text attribute in the target image is erased or edited.

2. The method for processing text in an image according to claim 1, characterized in that: The text line attributes include some or all of the following attributes: the position of the text line, the font of the text line, and the rotation angle of the text line; The text attributes include part or all of the following attributes: text content, text position, text direction, and text language.

3. The method for processing text in an image according to claim 1, characterized in that: The performing text line attribute detection on the target image to obtain text line attributes of the text lines in the target image includes: Using a pre-trained text line attribute detection model, the text line position and text line font attribute detection is performed on the target image to obtain the position and font of the text line in the target image, wherein the text line attribute detection model is trained using a training image in which the real text line position and the real text line font are marked; According to the position of the text line in the target image, a rotation angle of the text line in the target image is determined.

4. The method for processing text in an image according to claim 3, characterized in that: The training process of the text line attribute detection model includes: Extracting visual features of multiple scales from the training image using a text line attribute detection model, and fusing the visual features of multiple scales extracted from the training image to obtain multi-scale fused features of the training image; Using a text line attribute detection model and taking the multi-scale fusion features of the training image as a basis, predicting the position and font of the text line in the training image to obtain a first prediction result; Predicting the position and font of a text line in the training image according to the visual features of each scale extracted from the training image to obtain a plurality of second prediction results; Acquire text semantic features of the training image, fuse the text semantic features of the training image with multi-scale fusion features of the training image, and predict the position and font of a text line in the training image according to the fused features to obtain a third prediction result; With the goal of making each prediction result close to the annotation information of the training image, the parameters of the text line attribute detection model are updated.

5. The method for processing text in an image according to claim 1, characterized in that: The step of performing text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image includes: Using a pre-trained text attribute recognition model, extract visual features from the area where the text line in the target image is located, and obtain text temporal features based on the visual features; Determining the text direction of the text line in the target image by using the text attribute recognition model and based on the visual features; The text attribute recognition model is used to determine the text content and text position of the text line in the target image based on the text temporal features; wherein the text attribute recognition model is trained using training text line images with real text direction, real text content and real text position annotated; The language of the text line in the target image is recognized by language recognition to obtain the language of the text line in the target image.

6. The method for processing text in an image according to claim 1, characterized in that: The step of erasing the text line of the target text attribute in the target image includes: Generate an erased area mask according to the position of the text line of the target text attribute, wherein the erased area mask has the same size as the target image and can indicate an area in the target image that needs to be erased; The text line of the target text attribute in the target image is erased using the erase area mask.

7. The method for processing text in an image according to claim 1, characterized in that: Editing a text line of the target text attribute in the target image includes: Erasing the text line of the target text attribute in the target image to obtain an erased image; Obtain the edited text corresponding to the text line of the target text attribute; According to the position of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

8. The method for processing text in an image according to claim 7, characterized in that: The step of pasting the edited text corresponding to the text line of the target text attribute into the erased image according to the position of the text line of the target text attribute comprises: According to the position of the text line of the target text attribute and in combination with the font and / or text direction of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

9. The method for processing text in an image according to claim 7 or 8, characterized in that: Pasting the edited text corresponding to the text line of the target text attribute in the erased image includes: Determine the text foreground color of the text line of the target text attribute; According to the text foreground color of the text line of the target text attribute, the edited text corresponding to the text line of the target text attribute is pasted in the erased image.

10. The method for processing text in an image according to claim 9, characterized in that: The step of determining the foreground color of the text line of the target text attribute includes: Acquire a target text image, wherein the target text image is an image having the same size as the target image and only including the text strokes of the text line of the target text attribute in the target text image; Determine the area where each character in the target text image is located according to the position of each character in the text line of the target text attribute; Determine the foreground color of each character in the text line of the target text attribute by clustering the foreground color of the character strokes in the area where each character in the target text image is located; According to the difference in foreground colors of adjacent characters of each character in the text line of the target text attribute, a final foreground color of each character in the text line of the target text attribute is determined.

11. The method for processing text in an image according to claim 10, characterized in that: Erasing the text line of the target text attribute in the target image to obtain an erased image, and acquiring the target text image, including: Generate an erased area mask according to the position of the text line of the target text attribute, wherein the erased area mask has the same size as the target image and can indicate an area in the target image that needs to be erased; Inputting the target image and the erased area mask into a pre-trained text erasure model to obtain an erased image and a target text image output by the text erasure model; Among them, the text erasure model is trained with training images and erased area masks corresponding to the training images as training samples, and with real erased images and erased text images corresponding to the training images as sample labels. The erased text image has the same size as the training image and only includes the strokes of the text lines to be erased in the training image.

12. A device for processing text in an image, characterized in that: include: Image acquisition module, text line attribute detection module, text attribute recognition module, target text attribute acquisition module and text processing module; The image acquisition module is used to acquire a target image, wherein the target image contains text; The text line attribute detection module is used to perform text line attribute detection on the target image to obtain text line attributes of the text lines in the target image; The text attribute recognition module is used to perform text attribute recognition on the text line in the target image to obtain the text attribute of the text line in the target image; The target text attribute acquisition module is used to acquire the attribute selected by the user from the text line attributes detected and the character attributes recognized as the target text attribute; The text processing module is used to erase or edit the text line of the target text attribute in the target image.

13. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the method for processing text in an image as described in any one of claims 1 to 11.

14. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the method for processing text in an image as described in any one of claims 1 to 11.

15. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the steps of the method for processing text in an image as described in any one of claims 1 to 11.

Images