Image processing method, apparatus, device, storage medium, program product

Abstract

This paper provides an image processing method, apparatus, device, storage medium, and program product. The method includes: displaying a first mark on a first image, the first mark indicating a first element of the first image, and the first mark representing a first editing operation on the first element; displaying a second image obtained through an intelligent system, the second image being matched with the first element and the first editing operation. This process improves the efficiency of image editing.

Description

Technical Field

[0001] This article relates to the field of computer technology, and in particular to an image processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Intelligent systems, such as AI (Artificial Intelligence) systems, possess image generation and editing capabilities, and an increasing number of users are utilizing these systems for image editing. Therefore, improving the image editing efficiency of intelligent systems is one of the problems that needs to be addressed. Summary of the Invention

[0003] In one instance, an image processing method, apparatus, device, storage medium, or program product is provided that can improve image editing efficiency.

[0004] In a first aspect, an image processing method is provided, comprising: displaying a first mark in a first image, the first mark indicating a first element of the first image, and the first mark representing a first editing operation on the first element; and displaying a second image obtained by an intelligent system, the second image being matched with the first element and the first editing operation.

[0005] In a second aspect, an image processing apparatus is provided, comprising: a first display unit for displaying a first mark on a first image, the first mark indicating a first element of the first image, and the first mark representing a first editing operation on the first element; and a second display unit for displaying a second image obtained by an intelligent system, the second image being matched with the first element and the first editing operation.

[0006] Thirdly, an electronic device is provided, comprising: a processor; and a memory for storing computer-executable instructions for causing the processor to perform the method described in the first aspect.

[0007] Fourthly, a computer-readable storage medium is provided for storing computer-executable instructions, wherein a processor executes the computer-executable instructions to implement the method described in the first aspect.

[0008] Fifthly, a computer program product is provided, the computer program product comprising a computer program, wherein a processor executes the computer program to implement the method described in the first aspect above.

[0009] In one scenario, a first marker is displayed on a first image to indicate a first element of the first image and to represent a first editing operation on the first element. A second image is then displayed, obtained through an intelligent system, and matched with the first element and the first editing operation. In this technical solution, by marking the first image, the intelligent system can process the first image based on the first editing operation corresponding to the first marker to obtain the second image. This enriches the user's image editing methods using the intelligent system, and the first marker more intuitively reflects the editing intent of the first image, thereby improving image editing efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solution, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description only record some implementation situations. For those skilled in the art, other drawings can be obtained from these drawings without creative effort. Figure 1 This is a schematic diagram illustrating an application scenario of an image processing method provided in one particular situation. Figure 2 This is a flowchart illustrating an image processing method provided in one scenario. Figure 3a This is a schematic diagram of the first label in one scenario; Figure 3b This is a schematic diagram of the first label in yet another scenario; Figure 3c This is a schematic diagram of the first label in yet another scenario; Figure 4 This is a schematic diagram of an editing scenario for the first image in one particular situation. Figure 5a This is a schematic diagram showing the distribution of the first and second regions in one scenario. Figure 5b This is a schematic diagram showing the distribution of the first and second regions in another scenario. Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in one scenario; Figure 7 This is a schematic diagram of the structure of an electronic device provided in one scenario. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in one or more of the scenarios described below, the technical solutions in one or more of the scenarios described below will be clearly and completely described below in conjunction with one or more accompanying drawings. Obviously, the scenarios described are only some of the scenarios, not all of them. Based on one or more of the scenarios described below, all other implementations obtained by those skilled in the art without inventive effort should fall within the protection scope of this document.

[0012] It is understood that before using any of the technical solutions described below, relevant parties should be informed of the type, scope of use, and usage scenarios of the information involved and their authorization obtained in an appropriate manner in accordance with relevant laws and regulations.

[0013] For example, upon receiving a user's proactive request, a prompt message can be sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the technical solution, based on the prompt message.

[0014] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0015] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of the various scenarios described below. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the various scenarios described below.

[0016] In one scenario, an image processing method, apparatus, device, storage medium, or program product is provided, capable of improving the image editing efficiency of an intelligent system. This method can be implemented on and executed by a terminal device, which includes, but is not limited to, laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smartphones, smart speakers, smartwatches, smart TVs, in-vehicle terminals, and various other types of terminal devices.

[0017] In this paper, an intelligent system refers to a system capable of autonomous control based on machine learning models. Intelligent systems can be configured in the front end or back end. For example, an intelligent system is a virtual object or physical entity capable of making decisions and autonomously executing actions based on machine learning models to achieve preset goals or complete preset tasks. An intelligent system can be an automated program that understands user intent and can utilize models or invoke tools to complete various types of tasks. In some cases, examples of intelligent systems may include, but are not limited to: agents, bots, chatbots, digital avatars, intelligent customer service, digital assistants, etc. Alternatively, an intelligent system can also be an intelligent role implemented based on machine learning models. Intelligent systems can process user requests based on generative models (e.g., language models, multimodal models) to perform specified types of tasks. In some cases, intelligent systems may also be associated with virtual accounts, which may have corresponding avatars or nicknames.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of the image processing method provided in one particular situation, such as... Figure 1 As shown, the scenario includes terminal device 101. Figure 1 In this system, terminal device 101 can be any user's terminal device. Terminal device 101 can run an intelligent system and display an interactive interface 1001. Users can interact with the intelligent system through the interactive interface, sending various forms of dialogue messages such as text, voice, images, and video. The intelligent system processes the dialogue messages sent by the user and displays the results on the interactive interface 1001. The processed results can be various forms of dialogue messages such as text, voice, images, and video. The intelligent system fulfills user needs through dialogue, for example, fulfilling various needs such as image generation, image editing, and video generation. Figure 1 The example shown is a user requesting an intelligent system to generate an image.

[0019] Figure 2 This is a flowchart illustrating an image processing method provided in one scenario, which can be executed by the aforementioned terminal device. For example... Figure 2 As shown, the method includes: Step 202: Display a first mark in the first image, the first mark being used to indicate a first element of the first image, and the first mark being used to indicate a first editing operation on the first element; In this scenario, the first image may be generated by the intelligent system through a dialogue between the user and the intelligent system, such as a reference. Figure 1The first image can be generated by the intelligent system based on the user's input, "Please generate a picture that shows the scenery of spring"; or, the first image can be uploaded by the user. The first image uploaded by the user can come from the user's local photo album, etc., and there are no restrictions here.

[0020] Step 204: Display the second image, which is obtained through the intelligent system and is matched with the first element and the first editing operation.

[0021] In one scenario, a first mark is displayed in the first image. The first mark may include at least one of the following: lines, patterns, etc. The first image contains at least one element, each element corresponding to a portion of the image contained in the first image. This portion may be a part of a display object or an entire display object. For example, if the display object of the first image includes a "kettle," the element may include image portions such as the "spoof" of the "kettle." Or, for example, if the display object of the first image includes multiple "apples," then each "apple" may be considered as an element.

[0022] In this scenario, the first mark is used to indicate a first element of the first image, and also to represent a first editing operation on the first element. The first mark is drawn by the user according to their editing intent. The first mark itself can indicate the editing intent through its appearance (e.g., shape, color, size, etc.). This editing intent is represented in this scenario by the first editing operation, which may include at least one of the following: deletion, moving, copying, zooming in, zooming out, color filling, etc. Therefore, the second image obtained by the intelligent system matches the first element and the first editing operation. For example, if the user's editing intent includes deleting a certain image element of the first image, they might draw an "×" on that image element as the first mark to indicate the intention to delete the image element. In this scenario, the first editing operation could be a "delete" operation.

[0023] In some possible scenarios, the first mark can be implemented based on the brush function, which can provide a virtual brush that can be manually and freely painted, drawn, and doodled. The appearance of the brush can be customized, that is, the user can draw on the first image using the brush provided by the brush function, and the drawing result is presented as the first mark.

[0024] In this scenario, in response to a triggering operation, including a triggering operation for a pen function associated with an intelligent system (i.e., the intelligent system has an interface for calling the pen), a first mark is displayed in the first image.

[0025] In some possible scenarios, the first label can be obtained based on the input semantic information. In this case, the intelligent system obtains the voice information input by the user for the first image and performs semantic recognition on the voice information to obtain the first label. For example, if the input voice information includes: "Draw a cross on the spout of the sports water bottle", then the intelligent system draws the corresponding cross at the "spout of the sports water bottle" position in the first image as the first label.

[0026] Through the above process, a first mark is displayed on the first image. The intelligent system can process the first image based on the first mark to obtain a second image. The first editing operation on the first image can be realized based on the first mark. Moreover, the first mark can more intuitively reflect the first editing operation on the first image than text expression. Compared with text input and other methods of image editing, the first editing operation on the first image can be accurately obtained without multiple rounds of dialogue guidance. This can improve the image editing efficiency of the intelligent system, enrich the image editing operation methods, and enhance the image editing experience.

[0027] In summary, the image processing method provided in this scenario displays a first mark on a first image, the first mark indicating a first element of the first image, and the first mark representing a first editing operation on the first element. A second image is then displayed, obtained through an intelligent system, and matched with the first element and the first editing operation. In this technical solution, by marking the first image, the intelligent system can process the first image based on the first editing operation corresponding to the first mark to obtain the second image. This enriches the user's operation methods when editing images using the intelligent system, and the first mark more intuitively reflects the editing intent of the first image, thereby improving image editing efficiency.

[0028] As mentioned above, the appearance of the first marker can intuitively indicate the corresponding editing intent. That is, the first editing operation indicating the corresponding editing intent can be identified based on the first marker. The following are some examples of possible first editing operations: In some possible cases, there may be only one first marker, which is used to indicate the first element and to represent the first editing operation; A first marker can be considered as consisting of all the drawn lines that form a complete marker semantics. For example, when the first marker includes an "arrow trajectory," then since the "arrow trajectory" corresponds to a complete marker semantics, the "arrow trajectory" is a first marker. When drawing the corresponding first marker with a pen, the trajectory drawn from the start of drawing to the end of drawing (calling the pen is the start of drawing, and stopping the pen is the end of drawing) belongs to a first marker.

[0029] In this case, there is one first marker, which is used to indicate both the first element and the first editing operation. The first marker can indicate the first element based on appearance parameters and the first editing operation based on the trajectory semantics of the marker trajectory.

[0030] In this scenario, there is one first marker. The first marker indicates the first element. It uses appearance parameters (e.g., shape, color, size, position) to indicate the first element. When the first marker is a "circle" shape, the element contained within the "circle" is the first element. In some possible cases, the position of the first marker is associated with the position of the first element (e.g., the positions of the first marker and the first element overlap; for example, when the first marker has positional indicative semantics, the position indicated by the first marker is the first element). That is, the first marker is visually associated with the position of the first element, thus obtaining the first element corresponding to the first marker. For example, the position of the first marker coincides with the position of the first element (e.g., when the first marker is a "circle" shape, the element contained within the "circle" is the first element). The first marker represents a first editing operation, which can be indicated by the trajectory semantics of the first editing operation. For example, when the first marker operation is "×", the corresponding trajectory semantics is "delete", and the corresponding first editing operation is a deletion operation.

[0031] For example, Figure 3a This is a schematic diagram of the first mark in one scenario. The first mark is an arrow-shaped trajectory. The position of the "trajectory starting point" of the trajectory is visually associated with the "hanging rope". That is, the position of the "trajectory starting point" is located at the position of the "hanging rope". Therefore, the first element is determined to be the "hanging rope". Since the trajectory semantics corresponding to the "arrow trajectory" is "move", the first editing operation corresponding to the first mark is "move".

[0032] In this scenario, the first element can be indicated by a first marker, which enhances the intuitiveness of locking the first element. Furthermore, the first editing operation on the first element can be presented through the first marker, enriching the diversity of input methods for image editing operations.

[0033] In some possible cases, there may be multiple first markers, some of which are used to indicate the first element and some of which are used to indicate the first editing operation; In this scenario, there are multiple first markers, and their appearances can differ. For example, the first image may contain two first markers, one red and the other green, which are visually distinguishable. Some first markers indicate the first element, while others indicate the first editing operation. For instance, some first markers may indicate the first element, while others may indicate the first editing operation. Alternatively, some first markers may indicate both the first element and the first editing operation, while others may only indicate the first element or only the first editing operation. In this case, the multiple first markers systematically indicate the first marker operation and the first element, further ensuring the accuracy of image editing intent recognition and thus improving image editing efficiency.

[0034] In this scenario, a portion of the first marker is used to select the first element. The selection method can be determined by the position of the first marker. For example, if the position of the first marker overlaps with the position of the first element, it is determined that the first marker selects the first element. A portion of the first marker is used to represent the first editing operation, which can be represented by the trajectory semantics of the first marker.

[0035] For example, Figure 3b This is a schematic diagram of the first marker in another scenario, where the first image contains two first markers: one first marker C is an "×" trajectory, and the other first marker D is a circle trajectory. C indicates the first editing operation as "delete," and D indicates the first element as "lantern." In this scenario, the "lantern" will be deleted in the resulting second image.

[0036] In some possible cases, there may be multiple first markers. Some first markers are used to indicate the first element corresponding to the result of the editing operation, to indicate the first element. The result of the editing operation includes the result of the first editing operation. Some first markers are used to indicate the first editing operation.

[0037] In this scenario, the editing operation results include: moving position, moving effect, etc. The editing operation results include the result of the first editing operation. For example, if the first editing operation includes "moving," then the corresponding editing operation result includes moving position. In this scenario, among multiple first markers, some first markers are used to represent the first element corresponding to the editing operation result, that is, to represent the first element after the editing operation, such as the position of the first element after the move operation, etc., while other first markers are used to represent the first editing operation. Alternatively, among multiple first markers, some first markers are used to represent the first element and the first editing operation corresponding to the editing operation result, while other first markers are used to represent the first editing operation or the first element. Thus, multiple first markers collaboratively indicate the first element and the first editing operation, which can improve the recognition accuracy of the first element and the first editing operation, thereby improving the image editing efficiency.

[0038] For example, Figure 3c This is a schematic diagram of the first mark in another scenario, where one first mark A is an arrow-shaped trajectory and the other first mark B is a circle-shaped trajectory. A indicates that the first element is a "lanyard" and the first editing operation is "move". B indicates that the result of the first editing operation, i.e., the first element after the movement, is located in the "middle of the kettle handle".

[0039] In this scenario, the second image obtained shows the "lanyard" moving to the middle of the handle of the "sports water bottle".

[0040] Based on the above, when the second image is generated, the first image and the first mark can be input into the intelligent system. The intelligent system generates the second image based on the first image and the first mark. The intelligent system can identify the first element and the first editing operation corresponding to the first mark, and obtain the second image by performing the first editing operation on the first element.

[0041] In some possible scenarios, the intelligent system can generate a second image for each set of first elements and first editing operations based on the recognition of the first marker, and display multiple second images.

[0042] In some possible scenarios, for a first image, the location information of a first marker is determined, including the coordinates of the marker point. The first image and the location information are input into an intelligent system. Based on the first image, the intelligent system obtains the first marker and a first element from the location information. Then, based on the first marker and the first element, the intelligent system generates a second image. Specifically, the intelligent system can generate a bounding box region including the location information, identify the first marker and the first element within the bounding box region, and generate the second image based on the first marker and the first element. Therefore, the intelligent system does not need to recognize the entire first image, which helps improve image processing efficiency.

[0043] In some possible scenarios, the intelligent system determines the movement direction of the first element based on the trajectory direction of the first marker. In this scenario, the movement direction of the first element is determined based on the trajectory direction of the first marker. The intelligent system then moves the first element in the first image according to the movement direction to obtain the second image. Where the movement distance is not indicated, it can be generated by the intelligent system. In this scenario, there can be one first marker, which serves to indicate both the first element and the first editing operation. The intelligent system can identify the first marker to obtain the first element and the first editing operation, and then process the second image based on the first element and the first editing operation.

[0044] In this scenario, an intelligent system can determine the movement direction of the first element based on the trajectory direction of the first marker. The intelligent system then moves the first element in the first image according to the movement direction to obtain the second image. Since the first marker does not indicate the movement distance, this distance can be generated by the intelligent system. Alternatively, the movement distance can be determined based on the image semantics of the first image identified by the intelligent system. The movable position of the first element can be determined based on the image semantics. A target movement position can be randomly determined from the movable position, and the movement distance between the current position and the target movement position can be determined. For example, if the first element is "an apple on a tree," the movable position of the "apple" can be determined based on the image semantics of the first image. Several positions within the area directly below the apple can be identified. A target movement position corresponding to the direction indicated by the first trajectory can be randomly determined from these positions. Furthermore, the movement distance of the "apple" can be calculated based on the target movement position.

[0045] For example, if the first marker is determined to be an "arrow" trajectory based on the location information, then the first editing operation is determined to be "move", the movement direction is the trajectory direction of the "arrow", and the first element is a "lantern". The intelligent system can then move the "lantern" of the first image according to the trajectory direction indicated by the arrow to obtain the second image. The movement distance can be determined by the intelligent system.

[0046] In this scenario, there can be multiple first markers. Some of these first markers indicate the first element, while others indicate the first editing operation. Since the first markers do not indicate the movement distance, etc., this distance can be determined by the intelligent system, as described above.

[0047] For example, the first image contains two first markers, one of which is a "circle" to indicate that the first element is a "rope" and the other is an "arrow" to indicate that the first editing operation is "move" and the direction of movement is the trajectory direction of the "arrow". The intelligent system moves the "rope" in the first image according to the trajectory direction. The movement distance is determined by the intelligent system. Alternatively, the intelligent system can randomly generate a position in the trajectory direction as the target movement position and calculate the distance between the target movement position and the current position as the movement distance.

[0048] In some possible scenarios, the intelligent system determines the position to which the first element should be moved based on the position of the first marker, and then moves the first element in the first image according to the position to which the first element should be moved, thereby obtaining the second image.

[0049] In this scenario, there can be one first marker. The first marker indicates the first element and the first marker operation of the first element. For example, if the first marker is an "arrow", then the "head" of the "arrow" trajectory indicates the position to be moved to, and the "tail" indicates the first element to be moved. For example, if the first image contains an "apple", then the "tail" of the first marker "arrow" is on the "apple", indicating that the "apple" is the first element, and the "head" of the "arrow" is located on the "plate". This is equivalent to the "arrow" also indicating that the position to be moved to is the "plate". Then, the "apple" is moved to the "plate" by the intelligent system to obtain the second image.

[0050] In this scenario, there may be multiple first markers, some of which indicate the position to which the first element is to be moved, and some of which indicate that the first editing operation is a move operation.

[0051] For example, in this case, the first marker is determined to be an "arrow" trajectory and a "circle" trajectory based on the location information. The first element corresponding to the "arrow" is the "lanyard", the first editing operation is "move", and the position to which the "circle" is moved is the "middle position of the handle". In the second image obtained by the intelligent system, the "lanyard" is in the "middle position of the handle".

[0052] In summary, the image processing method in this scenario can flexibly indicate the first element and the first editing intent through the cooperation of multiple first markers, ensuring the accuracy of image editing and improving the editing efficiency of the first image.

[0053] In some possible scenarios, in order to further improve the editing efficiency of the first image, first information may also be received, wherein the first information corresponds to the first element and the first editing operation. The first information can be regarded as a textual supplementary description of the first element and the first editing operation. The first information may be input by the user or generated by the intelligent system. The intelligent system recognizes the first mark to obtain the first information describing the first element and the first editing operation.

[0054] For example, regarding Figure 3b In the scenario shown, the first piece of information may include "delete the lanyard".

[0055] In some possible scenarios, the first page includes a first area, and a first image is displayed in the first area. The first page is an interaction page between the intelligent system and the user, receiving first information, including: receiving first information through a first dialog box; the first dialog box is displayed in the first area, wherein the first dialog box may be displayed on the first image, or the first dialog box and the first image may be displayed in two separate areas of the first area, i.e., the first dialog box is not displayed on the first image. The first dialog box is movable, and the corresponding first dialog box can be moved when the user triggers it.

[0056] In some possible scenarios, the first page includes a first area and a second area, a first image is displayed in the first area, and receiving first information includes: receiving the first information through a second dialog box; the second dialog box is displayed in the second area; the first area and the second area are distributed horizontally, or the first area and the second area are distributed vertically.

[0057] In some possible scenarios, the first information is visualized in the second area, and the first information is editable. Thus, the second dialog box contains a message sending component, and the user can further edit and modify the first information to more accurately reflect the editing intent of the first image. That is, in this scenario, the user can know whether the intelligent system can perform editing operations on the first image that conform to the editing intent through the visualized first information. If it does not conform to the editing intent, the user can change the first information to make the edited second image conform to the editing intent without having to perform multiple rounds of operation marking, thus improving the editing efficiency of the first image.

[0058] Therefore, to further facilitate the marking of the first image, the intelligent system provides a first area, in which the first area displays the first image. That is, the first area is dedicated to displaying the first image, making it easy for the user to intuitively see the details of the first image. In some possible cases, in response to the confirmation operation for the first mark, the second image is displayed in the first area. Thus, both the first image before editing and the second image after editing are displayed in the first area. In addition to making it easy for the user to intuitively see the details of the first image to facilitate the execution of the first mark, it also makes it easy for the user to see the details of the edited second image, thus improving the user's image editing experience.

[0059] In some cases, the confirmation operation is implemented in the second area of ​​the interactive page, meaning the confirmation operation can be implemented independently in the second area, which may be a dialogue input box with the intelligent system. The confirmation operation may include at least one of the following: voice confirmation, trajectory confirmation, triggering operation of related components, etc. In some possible cases, the second area may also include a message sending component, and the confirmation operation includes a first triggering operation on the message sending component, the first triggering operation being associated with first information.

[0060] In some cases, when the first information is displayed in the first area, and a confirmation operation is performed in the first area, both the first information and the first image with the first mark are visually presented in the second area. Furthermore, the first information is editable. Thus, before triggering the message sending component, the user can further edit and modify the first information to more accurately reflect the editing intent of the first image. In this case, the user can determine whether the intelligent system can perform editing operations on the first image that conform to the editing intent through the visual first information. If the editing intent is not met, the user can change the first information to make the edited second image conform to the editing intent without having to perform multiple rounds of first marking, thereby improving the editing efficiency of the first image.

[0061] In some scenarios, after a user triggers the message sending component, the corresponding initial information (which can be the initial information or the modified information) and initial image are presented in the form of a dialogue on the interaction page with the intelligent system. Figure 4 This is a schematic diagram of an editing scene for the first image in one scenario. Figure 4In the scenario shown, after the user triggers the message sending component, the first message "Move the rope to the top of the sports water bottle" and the first image are presented as the message sent by the user. The user can determine whether the intelligent system's first editing operation on the first image satisfies their editing intent based on the first message. If the editing intent is not satisfied, further editing operations on the second image can be performed through a further dialogue. Further editing of the second image can include inputting new first information in the second area, or drawing new first marks on the second image, etc. When further editing the second image, it can be considered a new first image. Specific editing methods are described in the above scenario and will not be elaborated upon here.

[0062] As mentioned above, the purpose of displaying the first and second images in the first area is to make it easier for users to see the details of the relevant images. Therefore, in some possible cases, the area of ​​the first area is larger than that of the second area, that is, the first area is divided into larger areas as much as possible.

[0063] Because the display sizes and shapes of terminal devices used in intelligent systems vary, the first and second areas may differ in location to maximize the area allocated to the first area. For example, in some cases where the terminal device is a landscape-oriented device such as a computer, the first and second areas may be distributed horizontally. Figure 5a This is a schematic diagram showing the distribution of the first and second regions in one scenario. The second region can be the area where the input box for dialogue with the intelligent system is located. The first region is located to the right of the second region. The display area of ​​the first region is larger, allowing users to see the image details of the relevant images intuitively and clearly, and facilitating the drawing of the first mark on the first image.

[0064] For example, in some possible scenarios, when the terminal device is a vertically oriented screen device such as a mobile phone, the first and second areas can be distributed vertically. Figure 5b This is a schematic diagram of the distribution of the first and second regions in another scenario. In the diagram, the second region can be the area where the input box for dialogue with the intelligent system is located. The first region is located above the second region. The display area of ​​the first region is larger, so that the user can intuitively and clearly see the image details of the relevant image, and it is convenient to draw the first mark on the first image.

[0065] Therefore, by separating the input of the display of the relevant image and the confirmation operation of the first mark through the first area and the second area, the drawing efficiency of the first mark of the first image is improved, making it easier for users to see the image details of the relevant image, and making it easier for users to take care of the drawing and confirmation operation of the first mark, avoiding the problem of operation confusion, and improving the editing efficiency of the relevant image.

[0066] In summary, the image processing method in this case can also receive first information, which corresponds to the first element and the first editing operation. Thus, based on the first information, it can be known in advance whether the first editing operation on the first image conforms to the user's editing intention for the first image, thereby improving the intuitiveness of editing the first image.

[0067] In practical applications, after drawing the first mark on the first image, the initial first mark may not fully meet the user's first editing operation on the first image. Therefore, the first mark can be further edited to ensure that the processed second image matches the user's first editing operation.

[0068] In some possible scenarios, the first image may have multiple first marks, each corresponding to a variety of line appearances. In this scenario, multiple operation result identifiers may be displayed, each corresponding to a line appearance. The line appearance may include at least one of the following: color, thickness, and style (e.g., whether it is a dashed line or a solid line). The operation result identifier may be in at least one of the following forms: text, image, animation, etc. One operation result identifier may correspond to one or more first marks of the first image.

[0069] In this scenario, the operation result indicator can be displayed in any of the following ways: floating window display, control display, etc., and the display position of the operation result indicator is not limited.

[0070] In this scenario, multiple operation result identifiers corresponding to the multiple first marks contained in the first image can be provided, thus providing functional support to meet the user's editing intent for the first marks of the first image.

[0071] In some possible scenarios, in response to a triggering of a first operation result identifier, the corresponding first mark is displayed as editable. This can be indicated by displaying the first mark in a preset selection style (e.g., highlighting the first mark, or displaying the first mark as a preset dotted line). Each operation result identifier includes a first operation result identifier, and each first mark includes a first mark. The first mark corresponds to the appearance of a first line, and the appearance of the first line corresponds to the first operation result identifier. In this scenario, when the first mark is in an editable state, various editing operations can be performed on it. These editing operations include, but are not limited to, at least one of the following: changing the appearance of the line, deleting, moving, copying, drawing other first marks based on the appearance of the line corresponding to the first mark, etc. For example, the first operation result identifier "pen 1" can be selected, and then the first mark corresponding to "pen 1" is displayed as editable, at which point related editing operations can be performed on the first mark.

[0072] When there are multiple first markers, and each operation result identifier corresponds to multiple first markers, multiple first markers can be switched to editable state at once by triggering the operation result identifier, which further improves the editing efficiency of the first markers.

[0073] Therefore, by triggering the operation result identifier, the corresponding first mark can be switched to the editable state with one click, which can realize the editing of the first mark more quickly and efficiently, thus providing support for improving the editing efficiency of the first image.

[0074] In some possible scenarios, at least one prompt word corresponding to the first mark is displayed. The prompt word can be triggered by triggering the operation result identifier mentioned above, or by directly triggering the corresponding first mark. The prompt word is used to indicate the first editing operation related to the first mark. For example, the prompt word may include at least one of the following: delete, move, change line appearance, etc.

[0075] In this scenario, in response to a triggering of a first prompt word, a second image is displayed. Each prompt word includes the first prompt word, which is the prompt word triggered by the user among multiple prompt words. The second image is matched with the first prompt word; that is, the second image is obtained by updating the first image's first marker based on the first prompt word, identifying a first editing operation based on the updated first image's first marker, and processing the first image according to the first editing operation. For example, if the first prompt word includes "move," the corresponding first marker is switched to a movable state. Based on the user's movement operation on the first marker, the relevant first marker is moved. The intelligent system processes the first image based on the moved first marker to obtain the second image. The intelligent system determines a first element and a first editing operation based on the first marker, and performs the first editing operation on the first element to obtain the second image.

[0076] Therefore, the first marker is not fixed once it is drawn. At least one prompt word can be displayed for the first marker. The prompt word is used to indicate the first editing operation related to the first marker. Based on the trigger operation of the prompt word, the corresponding editing operation can be performed on the first marker, which improves the flexibility of drawing the first marker and ensures the matching between the first marker and the user's first editing operation on the first image.

[0077] In summary, the image processing method in this case can further instruct the first editing operation on the relevant object presented in the first image by triggering the first prompt word of the first marker, thereby ensuring the accuracy of the first editing operation and improving the editing efficiency of the first image.

[0078] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in one scenario, such as... Figure 6 As shown, the device includes: a first display unit 62 and a second display unit 64, wherein, The first display unit 62 is configured to display a first mark in the first image, the first mark being used to indicate a first element of the first image, and the first mark being used to indicate a first editing operation on the first element; The second display unit 64 is used to display a second image, which is obtained through an intelligent system and is matched with the first element and the first editing operation.

[0079] Optionally, the first marker is used to indicate the first element and represent the first editing operation; or, there are multiple first markers, some of which are used to indicate the first element and some of which are used to represent the first editing operation.

[0080] Optionally, the number of the first markers is one, and the position of the first marker matches the position of the first element to indicate the first element. The first marker is used to represent the first editing operation.

[0081] Optionally, there may be multiple first tags, some of which are used to select the first element to indicate the first element, and some of which are used to indicate the first editing operation.

[0082] Optionally, there may be multiple first markers, some of which are used to indicate the first element corresponding to the editing operation result, to indicate the first element, the editing operation result including the operation result of the first editing operation, and some of the first markers are used to indicate the first editing operation.

[0083] Optionally, it also includes: a receiving unit, used for: Receive first information; the first information corresponds to the first element and the first editing operation.

[0084] Optionally, the first page includes a first area, and the first image is displayed in the first area; the receiving unit is specifically used for: The first information is received through a first dialog box; the first dialog box is displayed in the first area.

[0085] Optionally, the first page includes a first area and a second area, and the first image is displayed in the first area; the receiving unit is specifically used for: The first information is received through a second dialog box; the second dialog box is displayed in the second area; the first area and the second area are distributed horizontally, or the first area and the second area are distributed vertically.

[0086] Optionally, the first display unit 62 is specifically used for: In response to a triggering operation, the first marker is displayed in the first image, the triggering operation including a triggering operation for a brush function associated with the intelligent system.

[0087] Optionally, the second display unit 64 is specifically used for: Input the first image and the first marker into the intelligent system; The intelligent system generates the second image based on the first image and the first marker.

[0088] Optionally, the second display unit 64 is specifically used for: For the first image, determine the location information of the first marker; The first image and the location information are input into the intelligent system; The intelligent system obtains the first marker and the first element based on the first image and the location information. The intelligent system generates the second image based on the first marker and the first element.

[0089] Optionally, the second display unit 64 is specifically used for: The intelligent system determines the movement direction of the first element based on the trajectory direction of the first marker. Using the intelligent system, the first element in the first image is moved according to the moving direction to obtain the second image.

[0090] Optionally, the second display unit 64 is specifically used for: The intelligent system determines the location to which the first element should be moved based on the position of the first marker. Using the intelligent system, the first element is moved in the first image according to the position to which the first element is to be moved, to obtain the second image.

[0091] In summary, the image processing apparatus provided in this embodiment displays a first mark on a first image, the first mark indicating a first element of the first image, and the first mark representing a first editing operation on the first element; it also displays a second image obtained through an intelligent system, the second image being matched with the first element and the first editing operation. In this technical solution, by marking the first image, the intelligent system can process the first image based on the first editing operation corresponding to the first mark to obtain the second image, thereby enriching the user's operation methods when using the intelligent system for image editing. Furthermore, the first mark can more intuitively reflect the editing intent of the first image, thus improving image editing efficiency.

[0092] The image processing apparatus described above can implement each process of the image processing method described above and achieve the same effect and function, which will not be repeated here.

[0093] In one instance, an electronic device was also provided. Figure 7 This is a schematic diagram of the structure of an electronic device provided in one scenario, such as... Figure 7 As shown, electronic devices can vary considerably due to differences in configuration or performance. They may include one or more processors 701 and memories 702, with the memory 702 storing one or more application programs or data. The memory 702 can be temporary or persistent storage. The application programs stored in the memory 702 may include one or more modules (not shown), each module including a series of computer-executable instructions from the electronic device. Furthermore, the processor 701 may be configured to communicate with the memory 702, executing the series of computer-executable instructions stored in the memory 702 on the electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input or output interfaces 705, one or more keyboards 706, etc.

[0094] In one specific scenario, the electronic device includes a processor; and a memory for storing computer-executable instructions for causing the processor to perform the following processes: A first mark is displayed in the first image, the first mark being used to indicate a first element of the first image, and the first mark being used to indicate a first editing operation on the first element; Display a second image, which is obtained through an intelligent system, and match the second image with the first element and the first editing operation.

[0095] The electronic devices described above can implement each process of the information recommendation method and achieve the same effect and function, so they will not be repeated here.

[0096] In one embodiment, a computer-readable storage medium is also provided for storing computer-executable instructions, which, when executed by a processor, perform the following process: A first mark is displayed in the first image, the first mark being used to indicate a first element of the first image, and the first mark being used to indicate a first editing operation on the first element; Display a second image, which is obtained through an intelligent system, and match the second image with the first element and the first editing operation.

[0097] The computer-readable storage media described above can implement each process of the information recommendation method described above and achieve the same effect and function, so they will not be repeated here.

[0098] In one scenario, a computer program product is also provided, the computer program product comprising a computer program, wherein when a processor executes the computer program, it implements the following process: A first mark is displayed in the first image, the first mark being used to indicate a first element of the first image, and the first mark being used to indicate a first editing operation on the first element; Display a second image, which is obtained through an intelligent system, and match the second image with the first element and the first editing operation.

[0099] The computer program products described above can implement each process of the information recommendation method and achieve the same effect and function, so they will not be repeated here.

[0100] In all the above scenarios, the computer-readable storage medium includes read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0101] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0102] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. Memory controllers can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0103] The systems, devices, modules, or units described above can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0104] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing the above scenarios, the functions of each unit can be implemented in one or more software and / or hardware components.

[0105] Those skilled in the art will understand that one or more of the above-described embodiments can be provided as methods, systems, or computer program products. Therefore, one or more of the above-described embodiments can take the form of entirely hardware, entirely software, or a combination of software and hardware aspects. Furthermore, one or more of the above-described embodiments can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] The above description refers to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the above descriptions. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0109] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] One or more of the above scenarios can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more of the above scenarios can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0111] The above scenarios are described in a progressive manner. Similar or identical parts between scenarios can be referred to interchangeably. Each scenario focuses on its differences from the others. In particular, the system is described more simply because it is fundamentally similar to the method; relevant parts can be found in the method section.

[0112] The above description is only a partial description and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims herein.

Claims

1. An image processing method, comprising: A first mark is displayed in the first image, the first mark being used to indicate a first element of the first image, and the first mark being used to indicate a first editing operation on the first element; Display a second image, which is obtained through an intelligent system, and match the second image with the first element and the first editing operation.

2. The method according to claim 1, wherein the number of the first markers is one, and the first markers are used to indicate the first element and represent the first editing operation; or, the number of the first markers is multiple, some of the first markers are used to indicate the first element, and some of the first markers are used to represent the first editing operation.

3. The method according to claim 2, wherein the number of the first markers is one, the position of the first markers matches the position of the first element to indicate the first element, and the first markers are used to indicate the first editing operation.

4. The method according to claim 2, wherein the number of the first tags is multiple, some of the first tags are used to select the first element to indicate the first element, and some of the first tags are used to indicate the first editing operation.

5. The method according to claim 2, wherein the number of the first markers is multiple, some of the first markers are used to indicate the first element corresponding to the editing operation result, to indicate the first element, the editing operation result includes the operation result of the first editing operation, and some of the first markers are used to indicate the first editing operation.

6. The method according to claim 1, further comprising: Receive the first message; The first information corresponds to the first element and the first editing operation.

7. The method according to claim 6, wherein the first page includes a first region, and the first image is displayed in the first region; The receiving of the first information includes: Receive the first information through the first dialog box; The first dialog box is displayed in the first area.

8. The method according to claim 6, wherein the first page includes a first region and a second region, and the first image is displayed in the first region; The receiving of the first information includes: The first information is received through a second dialog box; the second dialog box is displayed in the second area; the first area and the second area are distributed horizontally, or the first area and the second area are distributed vertically.

9. The method according to claim 1, wherein displaying the first mark in the first image comprises: In response to a triggering operation, the first marker is displayed in the first image, the triggering operation including a triggering operation for a brush function associated with the intelligent system.

10. The method according to claim 1, wherein the second image is obtained by: The first image and the first marker are input into the intelligent system; The intelligent system generates the second image based on the first image and the first marker.

11. The method according to claim 1, wherein the second image is obtained by: For the first image, determine the location information of the first marker; The first image and the location information are input into the intelligent system; The intelligent system obtains the first marker and the first element based on the first image and the location information. The intelligent system generates the second image based on the first marker and the first element.

12. The method according to claim 11, wherein generating the second image by the intelligent system based on the first marker and the first element comprises: The intelligent system determines the movement direction of the first element based on the trajectory direction of the first marker. Using the intelligent system, the first element in the first image is moved according to the moving direction to obtain the second image.

13. The method according to claim 11, wherein generating the second image based on the first marker and the first element through the intelligent system comprises: The intelligent system determines the location to which the first element should be moved based on the position of the first marker. Using the intelligent system, the first element is moved in the first image according to the position to which the first element is to be moved, to obtain the second image.

14. An image processing apparatus, comprising: A first display unit is configured to display a first mark in a first image, the first mark indicating a first element of the first image, and the first mark indicating a first editing operation on the first element; The second display unit is used to display a second image, which is obtained through an intelligent system and is matched with the first element and the first editing operation.

15. An electronic device comprising: processor; as well as, A memory for storing computer-executable instructions for causing the processor to implement the method described in any one of claims 1-13.

16. A computer-readable storage medium for storing computer-executable instructions, wherein a processor, when executing the computer-executable instructions, implements the method according to any one of claims 1-13.

17. A computer program product comprising a computer program, wherein a processor executes the computer program to implement the method described in any one of claims 1-13.

Images