Image processing method and device and electronic equipment

Abstract

The embodiment of the invention provides an image processing method. The method comprises the following steps: receiving a natural language for editing an image; converting the natural language into an editing instruction for the image; and editing the image according to the editing instruction. According to the image processing method provided by the embodiment of the invention, the editing processing of the image can be realized through the natural language, so that the editing operation of the image is simplified. The embodiment of the invention further provides an image processing device and electronic equipment.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and more specifically, to an image processing method, device, and electronic device. Background Art

[0002] As the visual basis for human beings to perceive the world, images are an important means for humans to obtain, express and transmit information. Digital image processing is to process images with computers.

[0003] When processing an image, it is usually necessary to create different layers, masks, etc., and the local editing and processing methods of the image can be divided into two categories: active methods and passive methods. The existing methods have their own characteristics and are suitable for different application scenarios and technical requirements, but the operation is cumbersome and inconvenient for users. Summary of the invention

[0004] In response to the problems existing in the above-mentioned prior art, the embodiments of the present application provide an image processing method, device, and electronic device, which can implement image editing processing through natural language, thereby simplifying image editing operations.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising the following steps:

[0006] receiving natural language for editing the image;

[0007] converting the natural language into editing instructions for the image; and

[0008] The image is edited according to the editing instruction.

[0009] Furthermore, converting the natural language into an editing instruction for the image includes:

[0010] The natural language is analyzed by natural language processing technology and / or a large language model, and the natural language is converted into editing instructions for the image.

[0011] Further, editing the image according to the editing instruction includes:

[0012] identifying elements in the image;

[0013] Locating a region to be edited according to the identified elements of the image and the editing instruction; and

[0014] The area to be edited is edited.

[0015] Furthermore, before editing the area to be edited, the method further includes:

[0016] Create a mask corresponding to the area to be edited.

[0017] Further, the identifying the elements in the image includes:

[0018] The elements in the image are identified by using image semantic recognition technology.

[0019] Furthermore, locating the area to be edited according to the identified elements of the image includes:

[0020] Segmenting the image into multiple regions using image segmentation techniques; and

[0021] A region to be edited is located from among the multiple regions according to the identified elements of the image and the editing instruction.

[0022] Further, editing the image according to the editing instruction includes:

[0023] The image is edited according to the editing instruction by using a generative adversarial network technology.

[0024] In a second aspect, an embodiment of the present application further provides an image processing device, including:

[0025] A language receiving module, used for receiving a natural language for editing the image;

[0026] an instruction conversion module, used for converting the natural language into an editing instruction for the image; and

[0027] The image editing module is used to edit the image according to the editing instruction.

[0028] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the image processing method according to the first aspect when executing the program.

[0029] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, wherein the computer program is used to implement the image processing method according to the first aspect above.

[0030] The embodiments of the present application bring the following beneficial effects:

[0031] In the image processing method provided in the embodiment of the present application, after receiving the natural language for editing the image, the natural language can be converted into an editing instruction for the image, and the image can be edited according to the editing instruction. Therefore, the image processing method provided in the embodiment of the present application can edit the image according to the natural language, thereby simplifying the image editing operation, improving the image processing efficiency, and further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0033] Figure 1 A schematic diagram of the process of an image processing method provided in an embodiment of the present application;

[0034] Figure 2 A structural block diagram of an image processing device provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0036] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments described in the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0038] In the specification and claims of this application and the above-mentioned drawings, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "multiple" means two or more. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0039] Figure 1FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application. Figure 1 As shown, the image processing method of the embodiment of the present application includes the following steps:

[0040] S101: receiving a natural language for editing the image;

[0041] Specifically, natural language is the language form used by humans to communicate and express ideas in daily life, including oral language and written language. It is a tool used by humans to communicate, transmit information and express meaning, and has characteristics in terms of vocabulary, grammar and semantics. Natural language can be in the form of text or voice, and can be in various languages, such as Chinese and English. The natural language of editing an image is the natural language used to process and edit the image, such as changing the color, moving, deleting, etc. of a specific area of ​​the image.

[0042] S102: Converting the natural language into editing instructions for the image; and

[0043] That is, after receiving the natural language for editing the image, the natural language needs to be converted into image editing instructions so as to perform editing processing on the image.

[0044] S103: Edit the image according to the editing instruction.

[0045] As described above, the image is edited according to the image editing instructions converted from natural language, such as changing the color, moving, deleting, etc. of a specific area of ​​the image.

[0046] Therefore, in the image processing method provided in the embodiment of the present application, after receiving the natural language for editing the image, the natural language can be converted into an editing instruction for the image, and the image can be edited according to the editing instruction. Therefore, the image processing method provided in the embodiment of the present application can edit the image according to the natural language, thereby simplifying the image editing operation, improving the image processing efficiency, and further improving the user experience.

[0047] Furthermore, in some embodiments of the present application, converting the natural language into an editing instruction for the image includes:

[0048] The natural language is analyzed by natural language processing technology and / or a large language model, and the natural language is converted into editing instructions for the image.

[0049] Specifically, Natural Language Processing (NLP) is a branch of computer science and artificial intelligence that aims to enable computers to understand, interpret and generate human language. The goal of NLP is to enable machines to process language like humans to perform various tasks, such as text analysis, language translation, sentiment analysis, automatic summarization, chatbots, etc. Large Language Models (LLM) are machine learning models trained with large amounts of data, with billions or even hundreds of billions of parameters. They are able to understand, generate, translate and answer questions, covering a wide range of language tasks. LLMs are usually based on deep learning technology, especially models based on the Transformer architecture, such as GPT (Generative Pretrained Transformer).

[0050] Therefore, converting the natural language into editing instructions for the image includes: analyzing the natural language through natural language processing technology and / or a large language model, and converting the natural language into editing instructions for the image, so as to accurately analyze the natural language and ensure accurate generation of editing instructions for the image.

[0051] Further, in some embodiments of the present application, editing the image according to the editing instruction includes:

[0052] identifying elements in the image;

[0053] Locating a region to be edited according to the identified elements of the image and the editing instruction; and

[0054] The area to be edited is edited.

[0055] The image processing method provided in the embodiment of the present application can accurately identify the area to be edited through the above steps, and then edit the area to be edited according to the editing instructions, thereby simplifying the image processing operation and improving the efficiency of image processing.

[0056] Furthermore, in some embodiments of the present application, before editing the area to be edited, the method further includes:

[0057] Create a mask corresponding to the area to be edited.

[0058] The mask here refers to the layer mask, which is a glass sheet covering the current layer. This glass sheet can be transparent, semi-transparent, or completely opaque. Before editing the area to be edited, you need to create a mask in the area to be edited, so that you can only edit the area to be edited, while the area outside the area to be edited will be protected, so that you can edit the image in a targeted manner, further simplify the image processing operation, and improve the efficiency of image processing.

[0059] Further, in some embodiments of the present application, the identifying the element in the image includes:

[0060] The elements in the image are identified by using image semantic recognition technology.

[0061] Specifically, Image Semantic Recognition is a computer vision technology used to extract and understand high-level semantic information from images. Unlike simple image classification or object detection, Image Semantic Recognition focuses not only on identifying objects present in an image, but also on the relationships between objects, the overall semantics of the scene, and a deeper understanding of the image content. Therefore, by identifying the elements in the image through Image Semantic Recognition technology, the image can be accurately analyzed and divided, thereby ensuring the accuracy of the image editing process.

[0062] Further, in some embodiments of the present application, locating the area to be edited according to the identified elements of the image includes:

[0063] Segmenting the image into multiple regions using image segmentation techniques; and

[0064] A region to be edited is located from among the multiple regions according to the identified elements of the image and the editing instruction.

[0065] Specifically, image segmentation technology is to divide an image into multiple regions, each of which corresponds to a specific semantic label, such as "Sky", "Building", "Road", etc. The segmentation result is usually represented in the form of a pixel-level label map. The image is divided into multiple regions by image segmentation technology, and the region to be edited is located from the multiple regions according to the identified elements of the image and the editing instructions, thereby further ensuring the accuracy of the image editing process.

[0066] In addition, the image processing method provided in the embodiment of the present application can also perform operations such as recognition, division and classification of images through scene understanding technology, convolutional neural networks and other AI models, thereby further improving the efficiency and accuracy of the image.

[0067] Further, in some embodiments of the present application, editing the image according to the editing instruction includes:

[0068] The image is edited according to the editing instruction by using a generative adversarial network technology.

[0069] Specifically, GAN (Generative Adversarial Networks) refers to the generation of realistic images through adversarial training of two neural networks (generator and discriminator), and is widely used in tasks such as image generation and style transfer. According to the editing instruction, the image is edited through the generative adversarial network technology, so as to improve the quality of image processing and further meet user needs.

[0070] Figure 2 2 is a block diagram of the structure of the image processing device 200 provided in the embodiment of the present application. Figure 2 As shown, the image processing device 200 of the embodiment of the present application includes: a language receiving module 210, an instruction conversion module 220 and an image editing module 230, wherein:

[0071] A language receiving module 210, used for receiving a natural language for editing the image;

[0072] An instruction conversion module 220, configured to convert the natural language into an editing instruction for the image; and

[0073] The image editing module 230 is used to edit the image according to the editing instruction.

[0074] In the image processing device provided in the embodiment of the present application, after receiving the natural language for editing the image, the natural language can be converted into an editing instruction for the image, and the image can be edited according to the editing instruction. Therefore, the image processing method provided in the embodiment of the present application can edit the image according to the natural language, thereby simplifying the image editing operation, improving the image processing efficiency, and further improving the user experience.

[0075] It should be noted that the specific implementation method of the image processing device of the embodiment of the present application is similar to the specific implementation method of the image processing method of the embodiment of the present application. Please refer to the description of the method part for details, and no further details will be given here.

[0076] Figure 3 Schematic diagram of the structure of an electronic device 300 according to an embodiment of the present application.

[0077] like Figure 3As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from the storage part 302 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0078] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

[0079] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, the above-mentioned functions defined in the electronic device of the present application are executed.

[0080] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electronic device, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0081] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction-executing electronic device, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction-executing electronic device, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0082] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the processing receiving device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based electronic device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The units or modules involved in the embodiments of the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor, and the processor is used to implement the image processing method when executing the program:

[0084] receiving natural language for editing the image;

[0085] converting the natural language into editing instructions for the image; and

[0086] The image is edited according to the editing instruction.

[0087] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently and not be assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the image processing method described in the present application:

[0088] receiving natural language for editing the image;

[0089] converting the natural language into editing instructions for the image; and

[0090] The image is edited according to the editing instruction.

[0091] As another aspect, the present application further provides a computer program product, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer program product stores one or more programs, and when the above programs are used by one or more processors to execute the image processing method described in the present application:

[0092] receiving natural language for editing the image;

[0093] converting the natural language into editing instructions for the image; and

[0094] The image is edited according to the editing instruction.

[0095] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. All equivalent structural changes made by using the contents of the present application specification and drawings under the application concept of the present application, or directly / indirectly used in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: The following steps are involved: receiving natural language for editing the image; converting the natural language into editing instructions for the image; and The image is edited according to the editing instruction.

2. The image processing method according to claim 1, characterized in that: The converting the natural language into an editing instruction for the image comprises: The natural language is analyzed by natural language processing technology and / or a large language model, and the natural language is converted into editing instructions for the image.

3. The image processing method according to claim 2, characterized in that: The step of editing the image according to the editing instruction comprises: identifying elements in the image; Locating a region to be edited according to the identified elements of the image and the editing instruction; and The area to be edited is edited.

4. The image processing method according to claim 3, characterized in that: Before editing the area to be edited, the method further includes: Create a mask corresponding to the area to be edited.

5. The image processing method according to claim 3, characterized in that: The identifying the element in the image comprises: The elements in the image are identified by using image semantic recognition technology.

6. The image processing method according to claim 3, characterized in that: The locating the area to be edited according to the identified elements of the image and the editing instruction comprises: Segmenting the image into multiple regions using image segmentation techniques; and A region to be edited is located from among the multiple regions according to the identified elements of the image and the editing instruction.

7. The image processing method according to claim 1, characterized in that: The step of editing the image according to the editing instruction comprises: The image is edited according to the editing instruction by using a generative adversarial network technology.

8. An image processing device, characterized in that: include: A language receiving module, used for receiving a natural language for editing the image; An instruction conversion module, used for converting the natural language into an editing instruction for the image; and The image editing module is used to edit the image according to the editing instruction.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the image processing method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the image processing method according to any one of claims 1 to 7.

Images