Visual prompt method and electronic equipment

Abstract

The embodiment of the invention provides a visual prompt method and electronic equipment, and the method comprises the steps: determining a target processing object and a target processing operation in response to a processing instruction for a to-be-processed image; taking the target processing object and the target processing operation as input information, and inputting the input information into an image reasoning model to obtain output information output by the image model; in the process of executing the target processing operation on the target processing object through the image reasoning model, a visual display effect corresponding to the target processing operation is presented in a position area matched with the target processing object, and the visual display effect is used for indicating the state of the target processing operation.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and relate to but are not limited to a visual prompting method and an electronic device. BACKGROUND

[0002] With the continuous development of artificial intelligence (AI) technology, image processing functions are increasingly rich, and user demand for visual prompting of the operation process is also increasing. In particular, in the AI-assisted image processing process, users expect to be able to intuitively understand the processing region, processing state, and completion status, in order to improve the interactive experience and use efficiency.

[0003] In related technologies, when processing a picture through AI, there is a lack of prompting of the image processing process, and users cannot determine the specific content and state changes of the current AI processing. Therefore, there is an urgent need for a method that can generate corresponding prompting effects based on AI processing content and state. SUMMARY

[0004] Based on the problems in related technologies, embodiments of the present application provide a visual prompting method and an electronic device.

[0005] The technical solution of the embodiments of the present application is as follows:

[0006] In a first aspect, the embodiments of the present application provide a visual prompting method, which includes:

[0007] In response to a processing instruction for a to-be-processed image, determining a target processing object and a target processing operation;

[0008] inputting the target processing object and the target processing operation as input information to an image inference model to obtain output information output by the image model;

[0009] In the process of executing the target processing operation on the target processing object through the image inference model, a visual display effect corresponding to the target processing operation is presented at a position region matched with the target processing object, and the visual display effect is used to indicate the state of the target processing operation.

[0010] In some embodiments, the visual prompting method further includes: for different target processing operations, setting a visual display effect corresponding to each target processing operation based on the image inference model; the visual display effects corresponding to different target processing operations are the same or different.

[0011] In some embodiments, the process of presenting the visual display effect corresponding to the target processing operation in the position area matching the target processing object during the execution of the target processing operation on the target processing object by the image inference model comprises: in response to the image inference model starting to execute the target processing operation on the target processing object, determining a first display effect representing that the target processing object is processing; and presenting the first display effect at the target processing object.

[0012] In some embodiments, the process of presenting the visual display effect corresponding to the target processing operation in the position area matching the target processing object during the execution of the target processing operation on the target processing object by the image inference model comprises: in response to the image inference model completing processing of the target processing object, determining a second display effect representing that the target processing object has completed processing; and presenting the second display effect at the target processing object or the region of the target processing object that has changed.

[0013] In some embodiments, the second display effect is different from the first display effect; and the visual prompting method further comprises: canceling the presentation of the first display effect in the case of presenting the second display effect.

[0014] In some embodiments, the process of presenting the visual display effect corresponding to the target processing operation in the position area matching the target processing object during the execution of the target processing operation on the target processing object by the image inference model comprises: in response to the image inference model starting to execute the target processing operation on the target processing object, presenting a target light band gradually surrounding the target processing object from a target position of an edge of the target processing object; the length of the target light band is directly proportional to the progress of the target processing operation; and in response to the image inference model completing processing of the target processing object, the target light band surrounds the target processing object.

[0015] In some embodiments, the visual prompting method further comprises: in response to the processing instruction, determining a target processing algorithm for processing the image to be processed; and in the process of executing the target processing operation on the target processing object based on the target processing algorithm, presenting a target display effect corresponding to the target processing operation in the position area matching the target processing object; the target display effect is different from the visual display effect.

[0016] In some embodiments, the determining the target processing object and the target processing operation in response to the processing instruction for the to-be-processed image comprises: comparing an image size of the to-be-processed image with a specified size of an image display pop-up window of the image inference model in response to the processing instruction; performing scaling processing on the to-be-processed image to obtain a display image in response to the image size of the to-be-processed image being greater than the specified size, and displaying the display image based on the image display pop-up window; and determining the target processing object on the display image and the target processing operation in response to a selection operation for the image display pop-up window displaying the display image.

[0017] In some embodiments, the visual prompt method further comprises: determining a first processing object corresponding to the target processing object on the display image on the to-be-processed image; and correspondingly, the inputting the target processing object and the target processing operation as input information to the image inference model comprises: inputting the first processing object and the target processing operation as input information to the image inference model.

[0018] In a second aspect, the embodiments of the present application provide an electronic device, which comprises:

[0019] a memory configured to store a computer program executable on the processor and an image inference model;

[0020] a processor configured to load the image inference model and implement the following steps when the computer program is executed: determining a target processing object and a target processing operation in response to a processing instruction for a to-be-processed image; and inputting the target processing object and the target processing operation as input information to the image inference model to obtain output information output by the image model;

[0021] a display module configured to present a visual display effect corresponding to the target processing operation in a position region matched with the target processing object during execution of the target processing operation on the target processing object by the image inference model, and the visual display effect is used to indicate a state of the target processing operation. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a structural schematic diagram of an electronic device provided by the embodiments of the present application;

[0023] Figure 2 is an optional flowchart of a visual prompt method provided by the embodiments of the present application;

[0024] Figure 3 is an image display pop-up window diagram of an image inference model provided by the embodiments of the present application. DETAILED DESCRIPTION

[0025] In order to illustrate the purpose, technical solutions and advantages of the embodiments of the present application more clearly, the embodiments of the present application will be described in detail below with reference to the drawings. It should be understood that the following description of the embodiments is intended to explain and illustrate the general inventive concept of the embodiments of the present application, and should not be construed as limiting the embodiments of the present application. In the specification and drawings, the same or similar reference signs refer to the same or similar components or members. For the sake of clarity, the drawings are not necessarily drawn to scale, and some well-known components and structures can be omitted from the drawings.

[0026] In some embodiments, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood as the usual meaning understood by a person with ordinary skills in the art to which the embodiments of the present application belong. The terms "first", "second", and the like used in the embodiments of the present application do not represent any order, number, or importance, but are only used to distinguish different components. The word "one" or "a" does not exclude a plurality. The words "including" or "containing" and the like mean that the elements or objects before the word encompass the elements or objects listed after the word and their equivalents, without excluding other elements or objects. The words "connected" or "connected" and the like are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The words "up", "down", "left", "right", "top" or "bottom" are only used to represent relative positional relationships, which may change accordingly when the absolute position of the described object changes. When an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element can be "directly" on or under another element, or there can be an intermediate element.

[0027] Currently, when a user processes an image through an AI tool (such as PenGen), there is a lack of unified and intuitive dynamic effect prompts to reflect the processing area, processing progress, and processing result boundary. This makes it difficult for the user to determine whether the AI is processing the image, to which step the processing has progressed, and the range of the final processing result. Especially in the case of multi-task concurrency or long processing time, the user is prone to confusion and even misoperation.

[0028] Based on the problems in the related art, the embodiment of the present application provides a visual prompt method, in response to a processing instruction for a to-be-processed image, determining a target processing object and a target processing operation; inputting the target processing object and the target processing operation as input information to an image inference model to obtain output information output by the image model; in the process of executing the target processing operation on the target processing object through the image inference model, a visual display effect corresponding to the target processing operation is presented at a position region matched with the target processing object, and the visual display effect is used to indicate the state of the target processing operation.

[0029] In this way, after receiving a user processing instruction, the target processing object and the target processing operation can be accurately identified, and the target processing object and the target processing operation are input to the image inference model for processing. During the processing, a visual display effect corresponding to the processing state can be presented in real time at the position matched with the target processing object, such as a color flowing light edge, a light scanning effect, etc. In this way, the user can be helped to understand the area and progress of image processing in real time, and the boundary of the processing result can be clearly displayed after the processing is completed, reducing the misoperation caused by lack of feedback, thereby significantly improving the user experience. In addition, through the unified motion effect design language, the visual feedback of different functions has consistency, improving the overall interaction fluency of the product.

[0030] The visual prompt method provided by the embodiments of the present application can be executed by an electronic device, Figure 1 is a structural schematic diagram of an electronic device provided by the embodiments of the present application, Figure 1 The electronic device 10 shown includes at least one processor 110, a memory 150, at least one network interface 120 and a user interface 130. The various components in the electronic device are coupled together through a bus system 140. It can be understood that the bus system 140 is used to realize the connection communication between the components. In addition to including a data bus, the bus system 140 also includes a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 140 in Figure 1 .

[0031] The processor 110 can be an integrated circuit chip having a processing capability of signals, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 110 can be used to load the image inference model and implement the following steps when executing the computer program: in response to a processing instruction for a to-be-processed image, determining a target processing object and a target processing operation; inputting the target processing object and the target processing operation as input information to the image inference model to obtain output information output by the image model.

[0032] The user interface 130 includes one or more output devices 131, such as a display module, enabling presentation of media content, and one or more input devices 132. Here, the input device 132 can be an input device such as a mouse and a keyboard, and the output device 131 can be a display module of an electronic device, such as a display screen

[0033] The memory 150 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 150 optionally includes one or more storage devices physically located in proximity to the processor 110. The memory 150 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. Non-volatile memory can be read only memory (ROM), and volatile memory can be random access memory (RAM). The memory 150 described in the embodiments of the present application is intended to include any suitable type of memory. In some embodiments, the memory 150 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are described below.

[0034] The operating system 151 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0035] The network communication module 152 is used to reach other computing devices via one or more (wired or wireless) network interfaces 120, and exemplary network interfaces 120 include Bluetooth, wireless compatibility certification (WiFi), and universal serial bus (USB), etc.

[0036] The input processing module 153 is configured to detect one or more inputs or interactions from one or more input devices 132.

[0037] In some embodiments, the visual cue device provided by the embodiments of the present application can be implemented in a software manner. Figure 1 A visual cue device 154 stored in the memory 150 is shown. The visual cue device 154 can be a visual cue device in the visual cue method, which can be software in the form of programs and plug-ins, etc., including the following software modules: a determination module 1541, an input module 1542, and a presentation module 1543. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of the modules will be described below.

[0038] In some embodiments, the visual cue device provided by the embodiments of the present application can be implemented in a software manner.

[0039] Figure 1 An image inference model 155 stored in the memory 150 is also shown. The image inference model 155 is configured to perform a target processing operation on a target processing object in a to-be-processed image to obtain output information.

[0040] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings.

[0041] Figure 2 is an optional flowchart of the visual cue method provided by the embodiments of the present application. As shown in Figure 2 The visual cue method provided by the embodiments of the present application can be implemented through steps S201 to S203.

[0042] Step S201, in response to a processing instruction for a to-be-processed image, determining a target processing object and a target processing operation.

[0043] Here, the processing instruction can be an operation command initiated by the user to indicate the image processing behavior, and the processing instruction can contain the specific processing type (such as cropping, enhancement, subject extraction, etc.) that the user wants to perform on the image. For example, it can be that the user right-clicks on the image, clicks the super-resolution button in the menu that appears, or selects a part of the image area using a smudge tool and selects style migration.

[0044] After obtaining the processing instruction, the user's operation can be identified to obtain the image area on the image to be processed (i.e., the target processing object) that the user wants to process and the specific processing task type (i.e., the target processing operation) that the user wants to perform.

[0045] In some embodiments, the target processing object can refer to a specific area selected by the user on the image to be processed or the entire image that needs to be processed. The target processing object can be the entire image or a partial area on the image, such as a specific area selected by a smudge tool.

[0046] In some embodiments, the target processing operation can refer to the image processing task type that the user wants to perform on the target processing object through the image inference model, such as image enhancement, image segmentation, image completion, color correction, style migration, and denoising.

[0047] Step S202, input the target processing object and the target processing operation as input information to the image inference model to obtain output information output by the image model.

[0048] In some embodiments, the image inference model can refer to a pre-trained machine learning model for performing image processing tasks, which can process image content according to the input target processing object and target processing operation and output the processing result, i.e., the output information. For example, in the image enhancement scenario, the image inference model optimizes the contrast, brightness, and clarity of the image to be processed; in the image segmentation scenario, the image inference model identifies the boundaries of different objects in the image to be processed and labels them separately.

[0049] In some embodiments, the image inference model can be a model based on a deep learning architecture, such as a convolutional neural network (CNN) or a Transformer structure, which can understand image semantics and process the image to be processed according to the user's intention to generate the corresponding output result.

[0050] Here, the output information of the image inference model can refer to the result data generated by the image inference model after receiving the input information, which can be image data processed by the model or intermediate processing results such as image feature vectors, processing progress information, etc.

[0051] In step S203, during the execution of the target processing operation on the target processing object by the image inference model, a visual display effect corresponding to the target processing operation is presented at a position region matching the target processing object, and the visual display effect is used to indicate the state of the target processing operation.

[0052] In some embodiments, when the image inference model starts to perform a processing operation on a target processing object, a dynamic visual display effect can be presented at the corresponding position on the target processing object to reflect the current processing state. For example, when the image inference model processes a picture, a color light border with flowing colors will first appear around the picture to show that the image inference model has started processing; when the processing is completed, a blue-purple light will sweep across the region where the target processing object is located, prompting the user that the processing of the image inference model on this region has been completed.

[0053] In some embodiments, the visual display effect can reflect the range, process and completion of image processing through color flow, light and shadow change, boundary highlighting, etc.

[0054] Here, different target processing operations can correspond to different visual display effects. For example, a red color display effect is presented when performing image enhancement operation; a green visual display effect is presented when performing blur processing operation.

[0055] In some embodiments, the position region matching the target processing object can refer to the region in the image corresponding to the target processing object, which can be the region being processed or the region where changes have occurred after image processing. For example, if the user selects a corner of the image, then the corner of the image and the nearby region can be considered as the position region matching the target processing object.

[0056] In some embodiments, the visual display effect refers to a dynamic graphical effect or animation effect presented on the interface during image processing to reflect the state of the target processing operation. For example, a flowing light strip can be observed around the image and a flickering light and shadow appears at the edge. These visual display effects help the user to clearly understand which region is being processed, how far the processing has gone, and clearly show the range of the processing result after the processing is completed, thereby improving the user's interactive experience and understanding efficiency.

[0057] The embodiments of the present application can accurately identify the target processing object and the target processing operation after receiving the user processing instruction, and input the target processing object and the target processing operation into the image inference model for processing. During the processing, the visual display effect corresponding to the processing state can be presented in real time at the position matched by the target processing object, such as color flowing light edges, light scanning effects, etc. In this way, the user can understand the area and progress of image processing, and the boundary of the processing result can be clearly displayed after processing, reducing the misoperation caused by lack of feedback, thereby significantly improving the user experience. In addition, through the unified motion effect design language, the visual feedback of different functions is consistent, improving the overall interaction fluency of the product.

[0058] In some embodiments, for different target processing operations, the visual display effect corresponding to each target processing operation is set based on the image inference model; the visual display effects corresponding to different target processing operations are the same or different.

[0059] In the embodiments of the present application, when the user uses the image AI processing tool to process the image to be processed, the visual display effect matched with different target processing operations can be set in advance. The visual display effect can include color light edges, color light scanning, boundary hatching, etc., and the visual display effect can vary according to the operation type and the image processing area. For example, when processing the entire picture, the blue-violet light scanning covers the entire image area; and when processing locally, only the image processing area and the surrounding range of the area will appear light scanning prompts.

[0060] In some embodiments, the visual display effects corresponding to different processing operations can be different. For example, all processing operations can use the same light scanning color, but the image area and path of light scanning are different; or different types of operations can use different colors or animation styles.

[0061] The embodiments of the present application can clearly display the image area, processing progress and result boundary during image processing by setting the visual display effects corresponding to different target processing operations, thereby improving the user experience.

[0062] In some embodiments, for different stages of processing the image by the image inference model, the corresponding animation effect is set to prompt the user of the progress of image processing. For example, when the image starts processing, step S203 can be implemented through step S2031 and step S2032:

[0063] Step S2031, in response to the image inference model starting to execute the target processing operation on the target processing object, determining a first display effect representing that the target processing object is being processed.

[0064] In some embodiments, the first display effect can be visual feedback displayed on the target processing object, and the first display effect is used to indicate that the image inference model starts processing the target processing object. For example, the user can be prompted that the current target processing object has started processing through dynamic light edges or light scanning. The first display effect enhances the user's perception of the processing process.

[0065] In some embodiments, the first display effect can be a light edge flowing around the edge of the target processing object. When the image inference model starts real-time processing of the target processing object, the light edge can slowly flow along the edge of the target processing object. The first display effect can also be a light edge flashing on the edge of the target processing object. When the image inference model starts real-time processing of the target processing object, a light edge is displayed on the edge of the target processing object, and the light edge can flash at a certain frequency. The first display effect can also be a light strip moving on the surface of the target processing object. When the image inference model starts real-time processing of the target processing object, the light strip moves horizontally or vertically on the surface of the target processing object to send a prompt signal to the user.

[0066] Step S2032, presenting the first display effect at the target processing object.

[0067] The embodiments of the present application determine and present the first display effect when the image inference model starts processing the target processing object. In this way, the processing state of the image inference model can be effectively conveyed, clear visual feedback is provided for the user, the user can be helped to clearly understand the processing area and process, and the user can be prevented from making mistakes due to lack of information, thereby improving the user's sense of control over the processing process.

[0068] In some embodiments, when the image processing is completed, step S203 can also be implemented through steps S2033 and S2034.

[0069] Step S2033, in response to the image inference model completing processing of the target processing object, determining a second display effect representing completion of processing of the target processing object.

[0070] In the embodiments of the present application, the second display effect can be one of highlighting, target color covering, or light scanning with a target color sliding in a target direction. The selection is based on maintaining a balance between visual uniformity and clarity of expression.

[0071] Here, highlighting can be achieved by increasing brightness or contrast; target color covering can mean completely replacing the color of the original area with a specified color to form a visual impact; and light scanning can be a colored light strip slowly moving in a certain direction to simulate the effect of light sweeping, with stronger dynamic sense and guidance.

[0072] For example, when a user selects an image for intelligent editing (such as adding a filter, removing the background, etc.), the image inference model processes in the background. Once the processing is completed, the second display effect is triggered to be displayed on the image to ensure that the user is promptly informed of the processing status.

[0073] Step S2034, presenting the second display effect at the target processing object or the region in the target processing object that has changed.

[0074] In the process of processing the target processing object, the embodiments of the present application can accurately locate the specific position affected by the image inference model processing on the image and apply the second display effect at the target processing object or the region in the target processing object that has changed. For example, in the picture local smearing operation, the blue-violet light only covers the modified regions and the surrounding regions of these regions, rather than covering the entire picture. The localized feedback helps the user quickly identify which parts have been modified by the image inference model, thereby helping the user avoid misoperation, which not only maintains the neatness of the user interface but also highlights the changed content in the image.

[0075] The instant feedback mechanism provided by the embodiments of the present application can effectively reduce the uncertainty of the user during the waiting process and improve the overall operation fluency.

[0076] In some embodiments, the second display effect is different from the first display effect. The first display effect refers to the visual prompt method used when the AI starts processing the image, and the second display effect refers to the visual prompt method used to prompt the user that the operation has been completed when the image processing is completed.

[0077] Correspondingly, the visual prompt method provided by the embodiments of the present application can also include step S1:

[0078] Step S1, cancel the presentation of the first display effect in the case of presenting the second display effect.

[0079] In some embodiments, during the image processing process, the display effects of different processing stages are different, and the corresponding display effects are switched when the processing stage changes. For example, when the AI completes the operation on the target processing object, the display effect is switched, the first display effect originally used to represent the processing process is closed, and the second display effect used to represent the completion of the processing is opened. For example, if the target processing object is the subject in the image to be processed, such as a vehicle, the user extracts the vehicle, and the scanning light only appears in the subject area during the processing process, and a persistent outline effect is left on the edge of the subject after the scanning is completed, to further highlight the boundary of the image inference model processing.

[0080] In practical applications, when a user uses an AI image processing application such as PenGen to doodle or crop a picture, a color flowing light edge (first display effect) can first be played on the edge of the image, indicating that the user input is being recognized and processed. Once the processing is completed, the light edge controlling the image edge disappears, and a blue-purple sweeping light (second display effect) is played on the image to clearly inform the user that the task has been successfully completed.

[0081] In the embodiments of the present application, by using different display effects in different processing processes and timely switching the display state after processing is completed, the working stage of artificial intelligence can be more clearly conveyed to the user, and the interaction efficiency with the artificial intelligence system and the user's satisfaction can be improved.

[0082] In some embodiments, step S203 can also be implemented by steps S2035 and S2036:

[0083] Step S2035, in response to the image inference model starting to perform the target processing operation on the target processing object, a target light strip gradually surrounding the target processing object is presented from a target position on the edge of the target processing object; the length of the target light strip is directly proportional to the progress of the target processing operation.

[0084] In some embodiments, the target light strip can refer to a dynamic expanding light strip around the edge of the target processing object, which is used to intuitively represent the progress of the image inference model processing the target processing object. The target light strip can have visual features such as color gradient, flow, or flicker to enhance user perception.

[0085] Here, the length of the target light strip can be directly proportional to the progress of the target processing operation, meaning that as the target processing operation progresses, the target light strip will continuously lengthen until it completely surrounds the entire target processing object at the completion of processing. This allows the user to clearly see the progress of the target processing operation without relying on traditional forms such as text or progress bars.

[0086] In practical applications, when a user selects a part of a picture and triggers AI processing, an initial light point or a small section of the target light strip can be generated around the selected area, and this light point or light strip will gradually expand as the target processing operation progresses. For example, in the image repair process, the target light strip may start from the starting point of the user's smearing and gradually expand along the repair path, eventually surrounding the entire repair area.

[0087] Step S2036, in response to the image inference model completing processing of the target processing object, the target light strip surrounds the target processing object.

[0088] In the embodiments of the present application, when the image inference model completes the operation on the target processing object, the target light band forms a closed shape and no longer continues to expand, and the light band stably surrounds the target processing object, indicating that the processing has been completed. At this time, the user can confirm that the target processing operation has been completed, without waiting or guessing whether the image inference is still running.

[0089] In actual application, when the target processing operation such as image inpainting or style conversion is completed, the originally dynamically expanding target light band can be locked as a complete annular or contour line, and at this time, the closed target light band can be accompanied by color change (for example, from color to blue-violet) or other animation effects (for example, slight shaking), which clearly indicates the processing state of the target processing operation through these changes.

[0090] In some embodiments, the shape of the target light band is the same as or different from the shape of the edge of the target processing object. For example, the edge of the target processing object can be an irregular shape selected by the user through a brush, and the shape of the target light band can be the same as the shape or a closed shape different from the irregular shape such as a circle.

[0091] The target light band in a positive proportional relationship with the progress of the target processing operation in the embodiments of the present application not only represents the processing progress, but also clearly provides a target processing operation completion signal, avoiding the user's misjudgment of the current processing state due to no changes in the interface. This feedback mechanism helps to improve the overall experience smoothness of the user using the image inference tool and the satisfaction of the image inference application.

[0092] In some embodiments, the target processing algorithm can be determined according to the processing instruction, and the corresponding processing is performed on the to-be-processed image, and at this time, the display effect when the image is processed is different from the display effect when the image is processed through the image inference model, so that the user can intuitively determine which way is used to process the image.

[0093] Correspondingly, the visual prompting method provided by the embodiments of the present application can further include steps S11 and S12:

[0094] Step S11, in response to the processing instruction, determining a target processing algorithm for processing the to-be-processed image.

[0095] The target processing algorithm can be an image enhancement algorithm, an image segmentation algorithm, an edge detection algorithm, an image restoration algorithm, an image recognition algorithm, or the like.

[0096] The processing instruction can be an operation command initiated by the user to indicate an image processing behavior. The processing instruction can include a specific processing type (such as cropping, enhancement, subject extraction, etc.) that the user wants to perform on the image. Based on the processing instruction, the corresponding target processing algorithm can be determined. For example, the user can right-click on the image with the mouse and click the super-resolution button in the pop-up menu. At this time, the target processing algorithm is the super-resolution algorithm.

[0097] In step S12, during the process of performing the target processing operation on the target processing object based on the target processing algorithm, a target display effect corresponding to the target processing operation is presented in a position area matched with the target processing object. The target display effect is different from the visual display effect.

[0098] Here, during the process of performing the target processing operation on the target processing object by the target processing algorithm, the target display effect corresponding to the target processing operation is presented in the position area matched with the target processing object, to convey to the user that the current image processing is not performed by the model, but by the target processing algorithm.

[0099] Here, the target display effect and the visual display effect can be significantly different in color, shape, duration, etc., thereby forming a clear visual distinction.

[0100] In the embodiments of the present application, different display effects are adopted for different image processing methods, which can enhance the user's perception of image processing, more accurately judge the processing method, progress and result, and improve the user experience and work efficiency.

[0101] In some embodiments, the image inference model has an image display pop-up window, Figure 3 is a schematic diagram of the image display pop-up window of the image inference model provided by the embodiments of the present application, as Figure 3 As shown in the image display pop-up window 301, a plurality of image processing function keys such as cropping, super-resolution and erasing are included, and the user's image processing intention can be determined through the interaction between the user and the image processing function keys. When processing the image to be processed, a visual display effect can be presented based on the image display pop-up window to realize visual prompting of the processing method, progress and result.

[0102] Correspondingly, step S201 can be implemented by steps S2011 to S2013:

[0103] In step S2011, the image size of the image to be processed is compared with the specified size of the image display pop-up window of the image inference model in response to the processing instruction.

[0104] In the embodiments of the present application, the processing instruction can be an operation request for triggering an image processing function issued by a user through a graphical interface interaction mode. For example, when a user is browsing a file and a webpage, the image on the webpage has low definition, and the user wants to improve the definition. At this time, the user right-clicks on the background webpage or the image to be processed in the file, and selects the super-resolution option to obtain a target processing operation required by the user.

[0105] Here, the image display pop-up window refers to a window component for displaying the image to be processed and its processing result on the user interface after receiving the processing instruction. The image display pop-up window has a preset size specification. The image display pop-up window can enable the image to be processed to be presented in a suitable scale within the image display pop-up window, and will not affect the layout and usability of other interface elements.

[0106] The specified size can refer to the maximum image input size allowed by the image display pop-up window. The maximum image input size can be uniformly defined by the front-end interface design specification to ensure good consistent display effect of the image display pop-up window under different devices and different resolutions. For example, under a screen resolution of 1920x1080, the maximum image input size allowed by the image display pop-up window can be set to 1200x800 pixels.

[0107] Here, by comparing the image size of the image to be processed with the specified size of the image display pop-up window, it can be determined whether scaling processing is required for the image, thereby avoiding the problem of incomplete display or layout disorder caused by the image being too large in size, and improving the visual experience and operation efficiency of the user.

[0108] Step S2012, in response to the image size of the image to be processed being greater than the specified size, performing scaling processing on the image to be processed to obtain a display image, and displaying the display image based on the image display pop-up window.

[0109] When the actual size of the image to be processed exceeds the specified size of the image display pop-up window, the image can be scaled down to a size that fits the size of the image display pop-up window by an image scaling algorithm. The image scaling processing process can use a bilinear interpolation algorithm to maintain the definition and detail information of the image.

[0110] The display image refers to an image for showing progress, status or preview after scaling. The display image will be presented to the user in the image display pop-up window for further operation, such as selecting super-resolution or cropping.

[0111] In the embodiments of the present application, the display image is only presented on the pop-up window or other visualization interface and does not participate in image processing calculation. The role of the display image is mainly to assist the user in confirming the current operation state or viewing the contrast effect before and after processing. For example, during the image processing process, the original image can be displayed in the pop-up window, and after the processing is completed, the processing result can be superimposed with the original image to confirm whether the processed image meets the expectation of the user.

[0112] Step S2013, in response to a selection operation for displaying the image display pop-up window of the display image, determining a target processing object on the display image and the target processing operation.

[0113] In the embodiments of the present application, the selection operation refers to the user's interactive selection behavior on the image in the image display pop-up window, which can be that the user selects a specific area on the display image as a target processing object using a mouse or a stylus (which can be by clicking or framing a local area or the entire image); it can also be that the user clicks a function button (such as smearing, repairing, obtaining a subject, etc.) in the image display pop-up window to clearly indicate the target processing operation that the user wants to perform.

[0114] In the embodiments of the present application, by comparing the size of the image to be processed and the image display pop-up window and performing adaptive scaling processing, and supporting interactive selection operation in the image display pop-up window to determine the processing object and operation type, the stability and visualization of the image processing process can be improved, and the operation efficiency and satisfaction of the user in the AI-assisted image editing scene can be improved.

[0115] In some embodiments, the image display pop-up window is based on the display image to present the visual display effect, but the image inference model actually processes the image to be processed, so it is necessary to determine the actual processing area on the image to be processed based on the target processing object determined by the user based on the display image. Therefore, the visual prompting method can further include step S21:

[0116] Step S21, determining a first processing object on the image to be processed corresponding to the target processing object on the display image.

[0117] Here, there is a clear mapping relationship between the first processing object and the target processing object. The target processing object is the processing object selected by the user on the display image in the image display pop-up window, and the first processing object is the specific form of the target processing object in the real image (i.e., the image to be processed). The mapping mechanism can accurately convert the user's intention into an operation on the target processing object in the actual image.

[0118] In some embodiments, the first processing object refers to the object for image processing located at the corresponding position in the actual image to be processed, based on the target processing object (such as a car) selected by the user in the displayed image. For example, if the user selects a car in the displayed image in the pop-up window, then the corresponding car is found in the image to be processed as the processing object for subsequent image processing.

[0119] This user-interaction-based approach transforms abstract user choices into concrete operations on actual images, thereby improving the accuracy of AI image processing.

[0120] Correspondingly, the input of the target processing object and the target processing operation as input information to the image inference model in step S202 can be achieved through step S2021:

[0121] Step S2021: Input the first processing object and the target processing operation as input information into the image reasoning model.

[0122] In this embodiment, the identified first processing object and the user-specified target processing operation (such as brushstroke style transfer, color enhancement, etc.) can be used as input and fed into an image inference model for calculation. The image inference model generates the final image processing result based on these inputs, ensuring that the AI ​​model can understand the user's true intention and perform image processing in the expected manner.

[0123] Here, the user-selected primary processing object and target processing operation can be passed to the AI ​​model in a structured manner to meet diverse image editing needs.

[0124] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0125] To address the problems existing in related technologies, this application proposes an artificial intelligence image processing prompting method based on visual feedback.

[0126] In some embodiments, during the processing of image objects in PenGen (an image processing application), a gradually appearing colored edge can first be displayed around the image to visually demonstrate the AI's performance in image processing. This gradually appearing colored edge not only provides a sensory signal that the AI ​​is working, but also enhances the user's interactive experience with the AI.

[0127] After the image or part of an image has been processed, a blue-purple sweeping light effect can be used to cover the corresponding area, indicating to the user that the AI ​​has completed processing of that area. At the same time, the colored light edges around the image will disappear, thus completing a full processing status feedback process.

[0128] In some embodiments, different processing methods have different specific manifestations of blue-violet light scanning, thereby achieving more accurate user guidance and feedback. For example, when the image processing model performs image enhancement operations on the entire picture, a blue-violet light scanning effect can be added to the entire picture; when performing smearing processing on a local area of the image, the blue-violet light scanning can be used to cover the smeared area and the adjacent pixels around the area; for obtaining the main content of the picture, the blue-violet light scanning will be limited within the main content area of the picture, and after the scanning is completed, a blue-violet outline can be retained to clearly indicate the boundary of the result after the image is processed by the artificial intelligence.

[0129] The method provided in the embodiments of the present application defines the AI processing area, so that the user can clearly identify which parts are being processed; the AI processing process is clear, which helps the user to judge the current processing stage; the application scope of the artificial intelligence processing result is defined, which helps the user to better understand the content presentation method after processing; a unified animation design language is used, which improves the animation consistency of different functions in use, and the unified animation design language enhances the user experience of the overall product.

[0130] Further, the scheme provided in the embodiments of the present application can be extended to other AI image processing tools or interactive interfaces, to improve the perceptibility and visual feedback level of user operation, and to ensure the consistency and professionalism of the product style.

[0131] To sum up, the embodiments of the present application introduce a unified and hierarchical animation prompt mechanism, effectively solving the problem of lack of clear feedback in AI image processing in the prior art, improving the operation efficiency and satisfaction of the user, and also providing a good reference for the design and development of subsequent related functions.

[0132] In the embodiments of the present application, please continue to refer to Figure 1 The visual prompt device can include a determination module 1541, an input module 1542, and a presentation module 1543. The determination module 1541 is configured to determine a target processing object and a target processing operation in response to a processing instruction for a to-be-processed image. The input module 1542 is configured to input the target processing object and the target processing operation as input information to an image inference model to obtain output information output by the image model. The presentation module 1543 is configured to present a visual display effect corresponding to the target processing operation in a position area matched with the target processing object during execution of the target processing operation on the target processing object by the image inference model, and the visual display effect is used to indicate a state of the target processing operation.

[0133] In some embodiments, the visual prompt device further includes a setting module configured to set, for different target processing operations, a visual display effect corresponding to each target processing operation based on the image inference model; the visual display effects corresponding to different target processing operations are the same or different.

[0134] In some embodiments, the presentation module 1543 is further configured to, in response to the image inference model starting to perform the target processing operation on the target processing object, determine a first display effect representing that the target processing object is being processed; and present the first display effect at the target processing object.

[0135] In some embodiments, the presentation module 1543 is further configured to, in response to the image inference model completing processing of the target processing object, determine a second display effect representing that the target processing object has completed processing; and present the second display effect at the target processing object or a region of the target processing object that has changed.

[0136] In some embodiments, the second display effect is different from the first display effect; and the visual prompt device further includes a canceling module configured to, in a case where the second display effect is presented, cancel presentation of the first display effect.

[0137] In some embodiments, the presentation module 1543 is further configured to, in response to the image inference model starting to perform the target processing operation on the target processing object, present a target light band that gradually encircles the target processing object from a target position at an edge of the target processing object; a length of the target light band is directly proportional to a progress of the target processing operation; and in response to the image inference model completing processing of the target processing object, the target light band encircles the target processing object.

[0138] In some embodiments, the visual prompt device further includes a first determination module configured to, in response to the processing instruction, determine a target processing algorithm for processing the image to be processed; and a first presentation module configured to, in a process of performing the target processing operation on the target processing object based on the target processing algorithm, present a target display effect corresponding to the target processing operation at a position region matching the target processing object; the target display effect is different from the visual display effect.

[0139] In some embodiments, the determining module 1541 is further configured to, in response to the processing instruction, compare an image size of the image to be processed with a specified size of an image display pop-up window of the image inference model; in response to the image size of the image to be processed being greater than the specified size, perform scaling processing on the image to be processed to obtain a display image, and display the display image based on the image display pop-up window; and in response to a selection operation on the image display pop-up window in which the display image is displayed, determine a target processing object on the display image and the target processing operation.

[0140] In some embodiments, the visual prompt device further includes a second determining module configured to determine, on the image to be processed, a first processing object corresponding to the target processing object on the display image; and the input module 1542 is further configured to input, as input information, the first processing object and the target processing operation to the image inference model.

[0141] It should be noted that the description of the device embodiments of the present application is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments, and thus is not described again. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0142] It should be noted that, in the embodiments of the present application, if the above-mentioned visual prompting method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a plurality of instructions for causing a terminal to execute all or part of the methods described in the embodiments of the present application. The storage medium mentioned above includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a magnetic disk or an optical disk, and various storage media that can store program codes. Thus, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0143] The embodiments of the present application provide a storage medium storing executable instructions, wherein the executable instructions are stored, and when the executable instructions are executed by a processor, the processor will execute the visual prompting method provided by the embodiments of the present application.

[0144] In some embodiments, the storage medium can be a computer-readable storage medium, such as a ferroelectric memory (FRAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disc, or a compact disc read only memory (CD-ROM), and the like. It can also be various devices including one or any combination of the above memories.

[0145] In some embodiments, the executable instructions can be in the form of programs, software, modules, scripts, or code, written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0146] By way of example, the executable instructions can or can not correspond to a file in a file system, can be stored in a part of a file that holds other programs or data, can be stored as one or more scripts stored in a markup language such as Hyper Text Markup Language (HTML), and other various forms of computer program code or web resources, can be stored as a subset of a single file, or can be stored in multiple coordinated files. As an example, the executable instructions can be deployed to be executed on one computing device or on multiple computing devices that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0147] The above merely provides an example of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application shall be included in the protection scope of the present application. It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that a particular feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that the size of the serial number of the above processes in various embodiments of the present application does not mean the execution order, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The serial number of the above embodiments of the present application is only for description, and does not represent the advantages or disadvantages of the embodiments.

[0148] It should be noted that the terms "comprising", "including", or any other variant thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, the element defined by the sentence "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus including the element. In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The above described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division, for example, a plurality of units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0149] The above merely provides an example of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement made within the technical range disclosed by the present application, which can be easily thought by those skilled in the art, shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A visual cueing method, the visual cueing method comprising: In response to processing instructions for the image to be processed, determine the target processing object and the target processing operation; The target processing object and the target processing operation are used as input information and input into the image reasoning model to obtain the output information output by the image model. During the execution of the target processing operation on the target processing object through the image reasoning model, a visual display effect corresponding to the target processing operation is presented in the position area that matches the target processing object, and the visual display effect is used to indicate the state of the target processing operation.

2. The visual cues method according to claim 1, further comprising: For different target processing operations, the visual display effect corresponding to each target processing operation is set based on the image reasoning model; The visual display effects corresponding to different target processing operations may be the same or different.

3. The visual cues method according to claim 1, wherein during the execution of the target processing operation on the target processing object through the image inference model, presenting a visual display effect corresponding to the target processing operation in a position area matching the target processing object includes: In response to the image inference model starting to perform the target processing operation on the target processing object, a first display effect characterizing the target processing object being processed is determined; The first display effect is presented at the target processing object.

4. The visual cues method according to claim 3, wherein during the execution of the target processing operation on the target processing object through the image reasoning model, presenting a visual display effect corresponding to the target processing operation in a position area matching the target processing object includes: In response to the completion of processing of the target processing object by the image reasoning model, a second display effect representing the completion of processing of the target processing object is determined; The second display effect is presented at the target processing object or the area in the target processing object where changes occur.

5. The visual cues method according to claim 4, wherein the second display effect is different from the first display effect; the visual cues method further includes: When the second display effect is displayed, the first display effect is canceled.

6. The visual cueing method according to any one of claims 1 to 5, wherein during the execution of the target processing operation on the target processing object through the image inference model, presenting a visual display effect corresponding to the target processing operation in a position area matching the target processing object includes: In response to the image inference model starting to perform the target processing operation on the target processing object, a target light band gradually surrounds the target processing object, starting from the target position at the edge of the target processing object; the length of the target light band is proportional to the progress of the target processing operation. In response to the completion of processing of the target processing object by the image inference model, the target light band surrounds the target processing object.

7. The visual cueing method according to any one of claims 1 to 5, further comprising: In response to the processing instruction, a target processing algorithm for processing the image to be processed is determined; During the execution of the target processing operation on the target processing object based on the target processing algorithm, a target display effect corresponding to the target processing operation is presented in the position area matching the target processing object; the target display effect is different from the visual display effect.

8. The visual cues method according to any one of claims 1 to 5, wherein determining the target processing object and the target processing operation in response to a processing instruction for the image to be processed includes: In response to the processing instruction, the image size of the image to be processed is compared with the specified size of the image display pop-up window of the image inference model; In response to the fact that the image size of the image to be processed is larger than the specified size, the image to be processed is scaled to obtain a display image, and the display image is displayed in a pop-up window based on the image. In response to a selection operation on an image display pop-up window that displays the displayed image, a target processing object and the target processing operation on the displayed image are determined.

9. The visual cues method according to claim 8, further comprising: A first processing object is determined on the image to be processed, corresponding to the target processing object on the displayed image; Correspondingly, the step of inputting the target processing object and the target processing operation as input information into the image inference model includes: The first processing object and the target processing operation are used as input information and input into the image reasoning model.

10. An electronic device, the electronic device comprising: The memory is configured to store computer programs and image reasoning models that can run on the processor; When a processor is configured to load the image inference model and execute the computer program, it performs the following steps: in response to a processing instruction for an image to be processed, it determines a target processing object and a target processing operation; and inputs the target processing object and the target processing operation as input information into the image inference model to obtain output information output by the image model. The display module is configured to present a visual display effect corresponding to the target processing operation in a position area matching the target processing object during the execution of the target processing operation on the target processing object through the image inference model, and the visual display effect is used to indicate the state of the target processing operation.

Images