Interaction processing method and electronic equipment

Abstract

The invention discloses an interaction processing method and electronic equipment, and belongs to the technical field of man-machine interaction, and the method comprises the steps that input information is obtained, and the input information can form a track; determining a target object based on the position information of the input information; determining a target processing mode based on a track formed by the input information; and processing the target object in the target processing mode.

Description

Technical Field

[0001] The present application belongs to the field of human-computer interaction technology, and in particular relates to an interaction processing method and electronic equipment. Background Art

[0002] Currently, when using electronic devices (such as mobile phones and tablets), users often need to click various buttons and other controls to switch between multiple screens or states to complete their tasks. However, this application approach is overly engineered, requiring users to adapt to the instructions of engineering software, making it difficult to intuitively operate or manage objects, resulting in low operational convenience and efficiency. Summary of the Invention

[0003] To this end, this application discloses the following technical solutions:

[0004] An interactive processing method, comprising:

[0005] obtaining input information, wherein the input information is capable of forming a trajectory;

[0006] determining a target object based on the position information of the input information;

[0007] determining a target processing method based on a trajectory formed by the input information;

[0008] The target object is processed in the target processing manner.

[0009] Optionally, obtain input information, including:

[0010] Input information generated by the interactive device performing writing and drawing on the device interface of the electronic device based on writing and drawing gestures is obtained; the writing and drawing gestures represent the movement trajectory and logic of the pen tip that conforms to the writing and drawing behavior of a person.

[0011] Optionally, determining a target processing method based on a trajectory formed by the input information includes:

[0012] Determining model input data; the model input data includes the input information, or includes the input information and the target object;

[0013] inputting the model input data into a processing model so that the processing model performs processing based on at least the trajectory formed by the input information;

[0014] A processing result of the processing model based on the model input data is obtained, wherein the processing result includes a target processing method for the target object.

[0015] Optionally, processing the target object in the target processing manner includes:

[0016] delivering the processing result to the first application, so that the first application performs a target operation corresponding to the target processing method in the processing result on the target object;

[0017] The target object is presented based on the first application.

[0018] Optionally, determining a target processing method based on a trajectory formed by the input information includes:

[0019] determining, in a first application, a target processing method based on a trajectory formed by the input information; and presenting the target object based on the first application;

[0020] Processing the target object in the target processing manner includes:

[0021] The first application performs a target operation corresponding to the target processing method on the target object.

[0022] Optionally, after obtaining the input information, the method further includes:

[0023] In response to the electronic device being in the first mode, triggering the processing model to determine the target object and the target processing mode based on the input information;

[0024] In response to the electronic device being in the second mode, triggering the first application to determine the target object and the target processing manner based on the input information;

[0025] The target object is presented based on the first application; and mode switching can be performed between the first mode and the second mode.

[0026] Optionally, the input information is obtained by the electronic device based on an operation of an interactive device, and the interactive device includes a first input terminal for writing to generate the input information and a second input terminal for mode control;

[0027] The method further comprises:

[0028] In response to the target control signal generated by the second input terminal, the electronic device is controlled to enter the first mode.

[0029] Optionally, the interaction processing method further includes:

[0030] displaying a trajectory corresponding to the input information;

[0031] In the first mode, the trajectory is displayed with a first visual effect, and in the second mode, the trajectory is displayed with a second visual effect, and the first visual effect and the second visual effect are different.

[0032] Optionally, the interaction processing method further includes:

[0033] In the first mode, the target object is marked and a corresponding marking effect is displayed; the marking effect of the target object and the first visual effect satisfy a consistency relationship.

[0034] An interactive processing device, comprising:

[0035] An acquisition module, configured to obtain input information, wherein the input information can form a trajectory;

[0036] A first determining module, configured to determine a target object based on the position information of the input information;

[0037] a second determination module, configured to determine a target processing method based on a trajectory formed by the input information;

[0038] The processing module is used to process the target object in the target processing manner.

[0039] An electronic device, comprising:

[0040] a memory for storing at least one set of computer instructions;

[0041] The processor is configured to implement the following processing by executing the instruction set stored in the memory:

[0042] obtaining input information, wherein the input information is capable of forming a trajectory;

[0043] determining a target object based on the position information of the input information;

[0044] determining a target processing method based on a trajectory formed by the input information;

[0045] The target object is processed in the target processing manner.

[0046] A storage medium carries one or more computer instruction sets, which, when executed by an electronic device, enable the electronic device to implement an interactive processing method as provided in any of the above method embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0048] Figure 1 This is a flowchart of the interactive processing method provided by this application;

[0049] Figure 2 This is another flowchart of the interactive processing method provided by this application;

[0050] Figure 3 This is another flowchart of the interactive processing method provided by this application;

[0051] Figure 4 This is another flowchart of the interactive processing method provided by this application;

[0052] Figure 5 This is another flowchart of the interactive processing method provided by this application;

[0053] Figure 6 This is another flowchart of the interactive processing method provided by this application;

[0054] FIG7( a ) and FIG7 ( b ) are different examples of quickly deleting an image using the symbol “×” provided by the present application;

[0055] Figure 8 This is an example of the latest image processing method based on the "+" symbol multiplexing provided by this application;

[0056] Figure 9 This is an example of quickly sharing pictures based on the symbol "@" provided by this application;

[0057] Figure 10 This is an example of a quick operation image based on the circle symbol "○" provided by this application;

[0058] Figure 11 This is an example of a quick operation image based on the check mark "√" provided by this application;

[0059] Figure 12 This is a structural diagram of the interactive processing device provided by this application;

[0060] Figure 13 This is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0062] The embodiments of the present application provide an interactive processing method and electronic device, which support users to intuitively operate device objects with writing and drawing gestures, making users' operation / management of device objects more free and quick, thereby improving the convenience and efficiency of operating / managing device objects.

[0063] The device objects may include but are not limited to various programs and applications installed on the electronic device, or corresponding objects within the programs and applications, such as various components / controls within the programs and applications that are used to provide corresponding functions, or data objects such as photos and documents.

[0064] The interactive processing method provided in the embodiments of the present application can be applied to, but not limited to, electronic devices with touch screens, such as mobile phones and tablet computers.

[0065] See also Figure 1 As shown in the flowchart of the interaction processing method, the interaction processing method provided in the embodiment of the present application may include the following steps 101 to 104, and these steps are described in detail below.

[0066] Step 101: Obtain input information, where the input information can form a trajectory.

[0067] The input information is obtained by the electronic device based on the operation of the interactive device.

[0068] The present application proposes a technical concept for conveniently operating / managing device objects based on an intuitive writing and drawing method. In this technical concept, an electronic device supports users using an interactive device to intuitively write and draw on the electronic device interface using writing and drawing gestures. Based on the intuitive writing and drawing information, the user determines the target object to be managed and the target processing method for the target object, thereby enabling operation and management of the target object.

[0069] The writing and drawing gesture representation conforms to the pen tip movement trajectory and logic of human writing and drawing behavior.

[0070] Optionally, the interactive device includes an active interactive device and a passive interactive device. The active interactive device may include but is not limited to an active stylus, and the passive interactive device may include but is not limited to a user's finger and a passive stylus.

[0071] The input information generated by the user using an interactive device to intuitively write and draw on the electronic device interface with writing and drawing gestures may include but is not limited to any one or more of graphics, symbols, characters or text, such as "×", "+", "○", "@", "delete", "scrapbook", "travel", etc., which is not limited to this and depends on the user's actual operation and management needs of the device object.

[0072] Different input information forms different trajectories, which are used to intuitively represent the processing methods for the specified device object, thereby supporting users to conveniently manage device objects based on intuitive writing and drawing methods. For example, "×" and "delete" indicate deleting the specified device object from the object set to which the specified device object belongs, such as deleting a picture from an album; "+" indicates repeating / reusing the most recent processing method for the device object. For example, if the most recent processing method for the device object is to delete picture X in the album, then "+" indicates deleting picture Y currently targeted by the "+" symbol; "@" indicates sharing the specified object; "○" indicates circling the required area from the object and performing the corresponding processing, such as circling a portrait area such as a face on a picture, and searching for a collection of pictures with the same portrait in the album based on the portrait information in the circled area; scrapbook / travel indicates moving the specified object to the scrapbook folder / travel folder, etc.

[0073] Accordingly, in this step, input information generated by the interactive device writing and drawing on the device interface of the electronic device based on the writing and drawing gestures can be obtained, such as the input information of various symbols or texts provided as examples above.

[0074] Step 102: Determine the target object based on the position information of the input information.

[0075] Optionally, when the user needs to operate / manage the target object on the device interface, the user can use the interactive device to intuitively draw corresponding graphics, symbols, characters and / or text and other writing and drawing information as needed in the area corresponding to the target object on the device interface, so as to specify the required target object and indicate the target processing method for the target object based on the generated writing and drawing information, that is, the input information.

[0076] The area corresponding to the target object can be, but is not limited to, any of the following:

[0077] a. The display area of the target object or its icon / thumbnail and other object information on the display interface;

[0078] b. Part of the display area of the target object or its icon / thumbnail and other object information, such as the label area on an image thumbnail;

[0079] c. An adjacent area corresponding to the display area of the target object or its icon / thumbnail and other object information, such as a blank area whose distance from the display area is less than a threshold.

[0080] Taking a photo album as an example, when a user needs to manage a picture in the album, they can intuitively write and draw on the picture's thumbnail area as needed to specify the picture and, based on the generated writing and drawing information (i.e., the input information), instruct the user on the desired processing method for the picture. For example, by drawing an "×" or "delete" on the picture's thumbnail, the user can indicate that the picture should be deleted from the album; by drawing the folder name "travel" on the picture's thumbnail, the user can indicate that the picture should be moved from the current folder in the album to the "travel" folder.

[0081] During implementation, when obtaining the input information, the location information of the input information may also be simultaneously detected. Specifically, but not limited to, while the user is writing or drawing on the interactive device, the location of the information input on the interactive device may be located based on capacitive positioning technology or electromagnetic induction positioning technology, thereby obtaining the location information of the input information. The obtained location information may include, but is not limited to, the location coordinates of the input information and / or the area where the input information is located (e.g., the thumbnail area of a certain image).

[0082] On this basis, the device object corresponding to the location information of the input information can be determined as the target object to be processed. The number of target objects can be one or more, depending on the actual situation. For example, the image corresponding to the thumbnail area where the symbol or text such as "×" or "delete" is located can be determined as the target image to be processed; or multiple images represented by multiple thumbnails with the same line drawn through them (for batch deletion of the multiple images) can be determined as individual target images to be processed, etc.

[0083] Step 103: Determine a target processing method based on the trajectory formed by the input information.

[0084] The target processing method is a processing method for the target object to be processed, including but not limited to deleting the target object from the object set where the target object is currently located, relocating the target object, sharing it, searching for related objects, adding tags, etc.

[0085] Different input information forms different trajectories, which can be used to intuitively characterize the processing method of the specified device object. Based on this, after obtaining the input information, in addition to determining the target object to be processed based on the position information of the input information, the trajectory formed by the input information can also be identified, and based on the trajectory formed by the input information, the target processing method for the target object can be determined.

[0086] For example, the trajectory of a symbol or text such as "×" or "delete" drawn by a user with a stylus on a thumbnail of a picture in an album is identified, and based on the identified "×" or "delete" trajectory, the target processing method for the current picture is determined to be deleting the picture, etc.

[0087] Step 104: Process the target object in the target processing manner.

[0088] On the basis of step 103 , a target operation corresponding to the target processing method may be further performed on the target object, thereby achieving intuitive operation management of the target object.

[0089] The target processing method corresponds to at least one operation instruction. For example, the image deletion processing method corresponds to a delete instruction for deleting the image from the collection to which the image currently belongs; the image sharing processing method corresponds to a sharing interface call instruction for calling a sharing interface, and a display instruction for displaying the called sharing interface; the image selection processing method corresponds to a portrait recognition instruction for identifying the portrait, and a portrait search instruction for searching for images in the album based on the identified portrait (to identify a collection of images of the same portrait), etc.

[0090] Based on this, after the target processing method for the target object is determined, a corresponding target operation can be performed on the target object based on the operation instruction corresponding to the target processing method to achieve processing of the target object.

[0091] The number of the target operations performed on the target object may be one or more, depending on the number of operation instructions corresponding to the target processing method.

[0092] Among them, when the target processing method corresponds to multiple operation instructions, the multiple operations corresponding to the multiple operation instructions can be further executed in serial or in parallel based on the characteristics of whether there is a dependency relationship between the multiple operations corresponding to the multiple operation instructions.

[0093] Taking the circle selection processing method for portraits in pictures as an example, since the portrait recognition and image search operations corresponding to the circle selection processing method are dependent on each other, these two operations need to be executed in serial manner, that is, the portrait recognition operation corresponding to the portrait recognition instruction is executed first, and then the album is searched for a set of pictures with the same portrait based on the portrait recognition results.

[0094] In summary, the embodiment of the present application proposes a technical concept for operating and managing device objects based on an intuitive writing and drawing method, and obtains input information generated by the user based on the intuitive writing and drawing method, determines the target object based on the position information of the input information, and determines the target processing method based on the trajectory formed by the input information, and then processes the target object in the target processing method, thereby realizing the above-mentioned technical concept and achieving the technical effect corresponding to the above-mentioned technical concept, and can support users to perform convenient and fast operation and management of device objects based on the intuitive writing and drawing method.

[0095] Based on the above technical concept, the user only needs to perform a writing and drawing operation on the target object based on the intuitive writing and drawing method, and can simultaneously specify the target object to be processed and the target processing method for the target object, without having to adapt to the predefined operation method of the engineering software, and gradually select the required target object and give the processing method for the target object through a series of tedious operations. For example, the user can directly draw the symbol "×" on the thumbnail of a picture in the album to select the picture and indicate to delete the picture from the album, without having to switch the album to editing mode in the conventional way, and then select the picture in the editing mode, and then continue to click the "Delete" button to indicate the deletion of the selected picture. Therefore, this application improves the convenience and efficiency of users' operations on device objects, and supports users to perform more flexible, free and convenient intuitive operations on device objects based on writing and drawing gestures.

[0096] In an alternative embodiment, see Figure 2 As shown in the flowchart of the interactive processing method, step 103 of the interactive processing method, i.e., determining a target processing method based on the trajectory formed by the input information, can be implemented as including the following steps 201-203:

[0097] Step 201: Determine model input data.

[0098] The model input data includes the input information, or includes the input information and the target object. That is, the model input data at least includes the input information, and may also include or exclude the target object. There is no limitation on this, and it depends on the actual situation.

[0099] Specifically, in some application scenarios, the processing model can determine the target processing method for the target object based solely on the trajectory formed by the input information. For example, if a user draws an "X" symbol or the word "delete" on a picture thumbnail, the processing model can directly identify the corresponding picture to be deleted based on this drawing information. Therefore, for such application scenarios, the input information can be determined as the model input data.

[0100] In other application scenarios, the processing model needs to combine the trajectory formed by the input information and the target object to determine the target processing method for the target object. For example, if a user circles an area on a thumbnail of an image in an album, the processing model needs to combine the trajectory formed by the circle symbol "○" and the image information of the image to determine the target processing method for the image. For example, based on the image information and the trajectory formed by the circle symbol "○", the processing model determines the content of the area circled by the user on the image, and then further determines the target processing method for the image based on the content of the circled area.

[0101] Optionally, if the user circles a portrait area, the target processing method for the image can be determined to be portrait search (searching the album based on the portrait to obtain a collection of images of the same portrait), portrait enhancement, portrait cutout, portrait beautification, and other optional processing methods; if the user circles a background area, the target processing method for the image can be determined to be background subtraction, background blur, and other optional processing methods. Therefore, for such application scenarios, the input information and the target object can be determined as model input data, that is, in this application scenario, the model input data includes the input information and the target object.

[0102] Step 202: Input the model input data into a processing model, so that the processing model performs processing based on at least the trajectory formed by the input information.

[0103] After the model input data is determined, the processing model may be called and the model input data may be input into the processing model, so as to determine a target processing manner for the target object through the processing model.

[0104] Optionally, the processing model learns the characteristics and patterns of the sample data by training the model based on a large amount of diverse sample data. The processing model can have model parameters ranging from hundreds of millions to hundreds of billions (model parameters are variables that control the model's behavior) to capture complex relationships and patterns in the model's input data.

[0105] The processing model can be, but is not limited to, various large models such as large language models (LLMs), large visual models, and large multimodal models, and can be a generative model or a non-generative model. There is no restriction on this. During implementation, the corresponding processing model can be called according to actual application requirements to achieve the required processing adapted to the current application scenario.

[0106] In addition, the processing model can be deployed locally on the electronic device, or can also be deployed in the cloud to support users to call the corresponding processing model through local calling or network calling to perform the required processing.

[0107] After the model input data is input into the processing model, the processing model can perform semantic recognition or symbolic recognition on the trajectory formed by the input information, such as semantic recognition of the trajectory formed by the input information "delete" or symbolic recognition of the symbol "×", etc., and based on the characteristics and rules of a large amount of diverse sample data learned by the model, capture the complex relationships and patterns in the model input data to ultimately determine the target processing method for the target object.

[0108] If the model input data contains the target object, it can also be combined with the processing of the target object, such as image type recognition, image area (such as circled area) content recognition, etc., based on the characteristics and rules of a large amount of diverse sample data learned by the model, to determine the target processing method for the target object.

[0109] Step 203: Obtain a processing result of the processing model based on the model input data, wherein the processing result includes a target processing method for the target object.

[0110] On the basis of step 202, this step further obtains the processing result of the processing model based on the model input data, so as to perform corresponding processing on the target object based on the target processing method in the processing result.

[0111] The processing result at least includes a target processing method for the target object, and optionally, may also include a confidence level of the target processing method.

[0112] The target processing mode of the processing model output can be one or more, which is also not limited and depends on the actual application situation.

[0113] If there are multiple target processing methods, in an optional embodiment, the electronic device can select a target processing method that meets the confidence conditions based on the confidence levels corresponding to each target processing method, and use the selected method as the final target processing method for the target object.

[0114] The confidence condition can be, but is not limited to, set to: the corresponding confidence is the highest, or the corresponding confidence is the top_k in the confidence descending sequence, where 1≤k<n, n is the number of target processing methods output by the processing model, and k and n are both integers.

[0115] The confidence descending sequence includes a sequence formed by arranging the confidences corresponding to the target processing methods in descending order.

[0116] In other optional implementations, the electronic device may further output various target processing methods on the current interface for the user to select a processing method as needed, and ultimately perform operations on the target object according to the target processing method selected by the user. For example, after the user circles a portrait area (such as a face) in a picture thumbnail, icons of various optional processing methods such as "Portrait Search," "Portrait Beautification," "Portrait Cutout," and "Portrait Enhancement" are displayed on the current interface for the user to select, and then the target object is processed according to the target processing method selected by the user.

[0117] In this embodiment, step 104 of the interaction processing method, i.e., processing the target object in the target processing manner, can be further implemented as the following step 204:

[0118] Step 204: delivering the processing result to the first application, so that the first application performs a target operation corresponding to the target processing method in the processing result on the target object.

[0119] The target object is presented based on the first application. For example, the first application is the application to which the target object currently belongs (such as the album application to which the photo belongs).

[0120] After obtaining the processing result of the processing model, by passing the processing result to the first application, the first application can further determine the target operation instruction for the target object, and perform the target operation corresponding to the target processing method on the target object based on the target operation instruction, thereby realizing the operation processing of the target object.

[0121] Among them, when there are multiple target processing methods, before the processing results of the processing model are passed to the first application, the final target processing method can be selected from them based on an automatic selection method (such as the selection method based on the confidence condition above) or a manual selection method, and the selected target processing method can be passed to the first application, or multiple target processing methods can be passed to the first application, and the first application can select the final target processing method from them through an automatic selection method or a manual selection method.

[0122] The target operation instruction is an operation instruction corresponding to the target processing method. When there are multiple target processing methods, it is specifically an operation instruction corresponding to the target processing method finally selected. The number of target operation instructions can be one or more, such as a deletion instruction corresponding to the deletion processing method, or a circle selection instruction corresponding to the circle selection processing method and a search instruction based on the circled area, etc.

[0123] In the case where there are multiple target operation instructions, the multiple operations corresponding to the multiple target operation instructions can be executed serially or in parallel based on whether there is a dependency relationship between the multiple operations corresponding to the multiple target operation instructions.

[0124] This embodiment uses processing models such as AI big models to identify the target processing method corresponding to the input information generated by the user based on the handwriting drawing gestures, and transmits the model processing results to the first application for corresponding processing of the target object. This allows the first application to support users to perform intuitive operations on the required objects based on handwriting drawing gestures without having to carry out complex functional improvements and secondary development of the first application, without being restricted by the predefined operation methods of the engineering software (such as predefined gesture operations, button operations, etc.), thereby improving the flexibility and convenience of user operations and at the same time improving the operation processing efficiency of device objects.

[0125] In an alternative embodiment, see Figure 3 In the interactive processing method flow chart shown in FIG. 1 , step 103 of the interactive processing method, i.e., determining a target processing method based on the trajectory formed by the input information, can be further implemented as step 301 including the following:

[0126] Step 301: Determine a target processing method based on a trajectory formed by the input information in a first application.

[0127] The target object is presented based on the first application. For example, the first application is the application to which the target object currently belongs (such as the album application to which the photo belongs).

[0128] Different from the previous embodiment, this embodiment determines the target processing method for the target object through the first application.

[0129] The first application may, but is not limited to, determine a target processing method for the target object by performing semantic or symbolic recognition on the trajectory formed by the input information and matching the recognition result with a pre-defined processing method table. Optionally, the processing method table includes different semantics / symbols and their corresponding processing methods, for example, "×" and "delete" indicate deleting the specified device object from the object set to which the specified device object belongs; "+" indicates repeating / reusing the most recent processing method for the device object; folder names (such as scrapbook, travel) indicate moving the specified object to the folder corresponding to the written folder name, etc.

[0130] During implementation, the first application can also determine the target processing method for the target object based on actual needs, combined with the trajectory and target object formed by the input information. For example, if a user circles an area on a thumbnail of a picture in the album, the content of the circled area can be first identified based on the image information of the picture and the circled trajectory, and then the processing method for the picture can be determined based on the content of the circled area. Among them, if the user circles a portrait, the target processing method for the picture can be determined to be portrait search (searching the album based on the portrait to obtain a set of pictures of the same portrait), portrait enhancement, portrait cutout, portrait beauty, and other optional processing methods; if the user circles the background, the target processing method for the picture can be determined to be background subtraction, background blur, and other optional processing methods.

[0131] Similar to the processing methods based on the processing model, the number of target processing methods determined by the first application can also be one or more. If there are multiple target processing methods, the first application can select one of them as the final processing method for the target object based on a selection strategy. The selection strategy can be, but is not limited to, a selection strategy based on the principle of recent use or the principle of most frequent use. Under this selection strategy, the target processing method that the user has used most recently or the most frequently / frequently among the multiple target processing methods can be determined as the final processing method for the target object.

[0132] Of course, various target processing methods can also be output on the current interface so that the user can select a processing method as needed, and finally the target object can be processed according to the target processing method selected by the user.

[0133] In this embodiment, step 104 of the interaction processing method, i.e., processing the target object in the target processing manner, can be further implemented as the following step 302:

[0134] Step 302: The first application executes a target operation corresponding to the target processing method on the target object.

[0135] The first application may first determine a target operation instruction corresponding to the target processing method, and then perform a target operation corresponding to the target processing method on the target object based on the target operation instruction to implement operation processing on the target object.

[0136] The number of target operation instructions can also be one or more, such as a delete instruction corresponding to the delete processing method, a circle selection instruction corresponding to the circle selection processing method, and a search instruction based on the circled area.

[0137] In the case where there are multiple target operation instructions, the multiple operations corresponding to the multiple target operation instructions can be executed serially or in parallel based on whether there is a dependency relationship between the multiple operations corresponding to the multiple target operation instructions.

[0138] Based on this embodiment, by improving the functions of applications on electronic devices (such as photo album applications), users can be supported in device applications to perform intuitive operations on required objects based on handwriting drawing gestures, overcoming the problems of cumbersome operations and low efficiency in conventional object processing methods, improving the flexibility and convenience of user operations, and at the same time improving the efficiency of operations on device objects.

[0139] In an optional embodiment, the interaction processing method may further include the following processing after obtaining the input information:

[0140] 11) In response to the electronic device being in the first mode, triggering the processing model to determine the target object and the target processing mode based on the input information;

[0141] 12) in response to the electronic device being in the second mode, triggering the first application to determine the target object and the target processing method based on the input information;

[0142] The target object is presented based on the first application; and mode switching can be performed between the first mode and the second mode.

[0143] In this embodiment, the electronic device has two device modes: a first mode and a second mode. The first mode is a mode in which the electronic device can call a processing model to perform intelligent processing related to artificial intelligence, and the second mode is a normal use mode in which the processing model is not called.

[0144] After obtaining the input information, this embodiment first determines the current device mode of the electronic device.

[0145] If the electronic device is in the first mode, the processing model is called and triggered to determine the target object to be processed and the target processing method for the target object based on the input information. In this case, the processing model can specifically determine the target object based on the location information of the input information, and determine the target processing method for the target object based on the trajectory formed by the input information, or in combination with the trajectory formed by the input information and the target object.

[0146] If the electronic device is in the second mode, the first application is triggered to determine the target object and the target processing method based on the input information. For example, the first application is triggered to determine the target object to be processed based on the location information of the input information, and the target processing method for the target object based on the trajectory formed by the input information.

[0147] For the detailed implementation process of executing the corresponding processing based on the processing model or the first application, please refer to the relevant description of the above embodiment, which will not be repeated here.

[0148] In actual applications, users can perform mode switching operations on electronic devices according to their needs to put the electronic devices in the desired mode, and then according to the processing logic or functions corresponding to the desired mode, support users to perform convenient operations and management of device objects based on intuitive writing and drawing methods.

[0149] The mode switching operation may include but is not limited to an operation performed by a user based on voice, button and / or gesture operation to switch the mode of the electronic device.

[0150] This embodiment designs two switchable device modes in the electronic device, which makes it convenient for users to adopt the corresponding mode as needed to operate and manage device objects in an intuitive writing and drawing manner. While improving the flexibility and convenience of user operations and the efficiency of operating and processing device objects, it also provides users with a variety of functional modes, thereby further improving the user experience.

[0151] In an optional embodiment, when the interactive device is an active interactive device such as an active pen, the interactive device includes a first input end for writing to generate the input information and a second input end for mode control.

[0152] Taking the active interactive device as an active pen as an example, the first input end may be the tip of the active pen, and the second input end may be the tail of the active pen.

[0153] The tip and tail of the active pen output different signals based on user operations, so that different interactive information is generated in the electronic device. For example, through the writing and drawing operation on the tip of the active pen, writing and drawing information (i.e., the input information) for determining the target object and the target processing method for the target object is generated in the electronic device, or through the operation on the tail of the active pen, a control signal for mode control is generated to the electronic device.

[0154] The signals generated by the first input end and the second input end are different, so that the electronic device can distinguish and identify them.

[0155] The signals generated by the first input end and the second input end are different, which may include but are not limited to different contents represented by the signals, or different signal characteristics such as signal strength and stability.

[0156] In this embodiment, see Figure 4 The interactive processing method shown in the flowchart may further include the following processing:

[0157] Step 401: In response to a target control signal generated by a second input terminal of an interactive device, control the electronic device to enter a first mode.

[0158] The target control signal is a first control signal for controlling the electronic device to enter the first mode, which is triggered by the user operating the second input end of the interactive device, such as by operating the end of the active pen.

[0159] The second input terminal of the interactive device, such as an active pen, can also provide a second control signal for controlling the electronic device to enter the second mode.

[0160] The first control signal and the second control signal may be the same or different. If the first control signal and the second control signal are different, the electronic device may directly perform a mode switching control that matches the received control signal.

[0161] If the first control signal is the same as the second control signal, the electronic device can switch modes based on the received control signal and in combination with the current device mode. For example, when receiving a control signal for mode control, if the electronic device is currently in the second mode, the electronic device can be switched to the first mode based on the control signal. Conversely, if the electronic device is currently in the first mode, the electronic device can be switched to the second mode based on the control signal.

[0162] The execution order of step 401 and other steps (such as steps 101 - 104 ) included in the interactive processing method is not limited and depends on the actual time when the second input terminal generates the target control signal.

[0163] This embodiment designs a first input end for writing to generate the input information and a second input end for mode control on the interactive device, so that the user can quickly and efficiently perform various types of operations or controls on the electronic device based on the interactive device, thereby further enriching the quick interaction function between the electronic device and improving the user experience.

[0164] In an alternative embodiment, see Figure 5 The interactive processing method flow chart shown in FIG. 1 further includes the following processing after step 101:

[0165] Step 501: Display a trajectory corresponding to the input information.

[0166] In the first mode, the trajectory is displayed with a first visual effect, and in the second mode, the trajectory is displayed with a second visual effect, and the first visual effect and the second visual effect are different.

[0167] In actual applications, as the user writes and draws the input information, the written and drawn information can be displayed synchronously according to the visual effects of the corresponding mode.

[0168] For example, when a user draws various symbols or characters such as "×", "+", "○", "@", and "delete", if the electronic device is in the second mode, the trajectory of the input information is displayed using conventional lines such as black, white, and gray. If the electronic device is in the first mode, the trajectory lines of the input information are displayed using a blue-pink gradient visual effect. When displaying the trajectory lines of the input information using the blue-pink gradient visual effect, the blue-pink gradient process of the trajectory lines of the input information can optionally be continuously cycled based on a cyclic gradient method, so that the electronic device can display information with a more advanced or cool display effect in the smart mode represented by the first mode.

[0169] This embodiment can realize the visualization of the user's writing and drawing information by displaying the trajectory corresponding to the input information, so that the user can timely understand the actual writing and drawing effects and make corrections when necessary, thereby further ensuring the correctness and efficiency of the user's intuitive operation and management of device objects with writing and drawing gestures; in addition, this embodiment can also enable the user to intuitively understand the current device mode of the electronic device based on the visual effect of the trajectory of the input information based on the visual effect of the trajectory of the input information, and can feel the matching visual effect in the device mode, thereby further improving the user experience.

[0170] In an alternative embodiment, see Figure 6 The interactive processing method shown in the flowchart may further include the following processing:

[0171] Step 601: In a first mode, mark the target object and display a corresponding marking effect; the marking effect of the target object and the first visual effect satisfy a consistency relationship.

[0172] Marking the target object may include but is not limited to framing / circling the target object or its thumbnail, icon, etc., or adding marking information / marking symbols on the target object or its thumbnail, icon, etc. For example, when a user writes information such as "×", "+", "○", "@", etc. on the thumbnail of a picture in the album, the thumbnail of the picture is framed and the formed framing lines are displayed.

[0173] Furthermore, the marking effect of the target object satisfies a consistency relationship with the first visual effect. The consistency relationship may include, but is not limited to, the target object's marking color being consistent with the color of the trajectory presented by the first visual effect, the target object's marking line type (such as solid line / dashed line, single line / double line, etc.) being consistent with the input information trajectory line type presented by the first visual effect, the target object's marking line parameters such as thickness being consistent with the trajectory line parameters such as thickness presented by the first visual effect, the target object's marking effect's dynamic and static features being consistent with the dynamic and static features presented by the first visual effect, and the like.

[0174] The term "consistent" may refer to being the same or close to each other. For example, the thickness of the target object's marking line is the same as the thickness of the trajectory line presented by the first visual effect, or the thicknesses of the two are different but the difference is less than a threshold.

[0175] For example, with respect to the above implementation method of displaying the trajectory lines of the input information with a blue-pink gradient visual effect, the frame selection mark lines of the image (target object) are also displayed with a blue-pink gradient visual effect. Furthermore, if the blue-pink gradient process of the trajectory lines of the input information is displayed in a cyclic gradient manner, the same method can be used to cycle the blue-pink gradient process of the frame selection mark lines of the image.

[0176] Similar to the display method in the first mode, in actual applications, in the second mode, the target object can also be marked and the corresponding marking effect can be displayed. Optionally, the marking effect of the target object in the second mode and the second visual effect satisfy a consistency relationship.

[0177] This embodiment marks the target object and displays the corresponding marking effect in the first mode and other device modes, and at the same time makes the marking effect of the target object and the visual effect of the trajectory of the input information in the mode satisfy a consistency relationship (such as the first visual effect). On the one hand, it realizes the visualization of the user's writing and drawing information and the target object targeted by the writing and drawing information, which makes it convenient for the user to visualize the actual writing and drawing effect and the target object, and make corrections when necessary, ensuring the correctness and efficiency of the user's intuitive operation and management of the device object with writing and drawing gestures; on the other hand, it also enables the user to feel the matching visual effect in the device mode, or intuitively understand the current device mode based on the visual effect, thereby ensuring the user experience of the user in the process of intuitive operation and processing with writing and drawing gestures in many aspects.

[0178] An application example of the method disclosed in this application is provided below.

[0179] This example uses the operation, management, or use of pictures in a photo album application as an example to illustrate the process of conveniently manipulating device objects using writing and drawing gestures.

[0180] See Figures 7(a) and 7(b), which illustrate examples of quickly deleting images using the "×" symbol. In Figure 7(a), the user quickly deletes an image from the album by drawing the "×" symbol on the image's thumbnail. In Figure 7(b), the user quickly deletes the image from the subfolder "scrapbpok" where it is currently located by drawing the "×" symbol on the folder name displayed on the image's thumbnail in the album. This quickly deletes the image and stores it in the entire folder corresponding to the album's first-level directory.

[0181] Typically, an album structure can correspond to a multi-level directory, wherein the first-level directory in the multi-level directory (referred to as the first-level directory) represents the overall album folder, and the non-first-level directories in the multi-level directory (referred to as the multi-level directory) represent subfolders belonging to the overall album folder. Each image, video, animated image, and other objects in the album can be located in a folder / subfolder represented by the corresponding directory based on demand. In the example provided in Figure 7(b), after deleting the corresponding image from the subfolder "scrapbpok" where it is located based on the symbol "×", it is added to the overall album folder corresponding to the first-level directory to avoid completely deleting the image from the album, making it easier for subsequent users to use the image. Of course, in actual applications, the image can also be completely deleted from the album.

[0182] Figure 8 An example of the latest processing method of the picture based on the repetition / multiplexing of the symbol "+" is shown. Figure 8 As shown, the most recent processing method for the picture is to quickly transfer the picture to the subfolder "scrapbpok" of the album by writing "scrapbpok" in the thumbnail area of the picture, and then the above operation can be repeated by drawing the symbol "+" on the thumbnail of the next picture to be processed, and the next picture to be processed corresponding to the symbol can be quickly transferred to the subfolder "scrapbpok" of the album.

[0183] In actual applications, when users write and draw symbols or text such as "×", "+", "@", "scrapbpok" on the corresponding areas of objects such as pictures, they can write part or all of the required information. If they only write part of the required information, the electronic device can complete all the information that the user needs to draw based on automatic matching and completion. For example, if the user only draws "scrap" on the thumbnail of a picture in the album, the electronic device recognizes that the user intends to draw "scrapbpok" based on the partial information drawn, and then automatically completes the information that the user needs to draw, so as to further improve the convenience and efficiency of object operations using writing and drawing gestures.

[0184] Figure 9 An example of quickly sharing a picture based on the symbol "@" is shown. In this example, the user draws the symbol "@" on the thumbnail of a picture in the album. The electronic device determines that the user wants to share the picture based on the symbol, and automatically determines the communication object that matches the portrait in the picture from the communication software that supports content sharing, such as WeChat, as the target recipient. For example, it determines the contact whose WeChat avatar contains the portrait as the target recipient, and sends the picture to the determined target recipient to realize the picture sharing with the target recipient. In actual applications, a variety of optional sharing interfaces can also be provided, such as interfaces of different communication software such as WeChat, Feishu, and Weibo, and the user can independently select the required communication object in the required communication software to share content.

[0185] Figure 10 An example of quick image manipulation based on the circle symbol "○" is shown. In this example, the user circles the portrait area (face area) in a picture to trigger portrait recognition. Based on this, the album is searched for a collection of pictures with the same portrait based on the user's selected search processing method. The searched picture collection can then be further processed as needed, such as moving the searched picture collection to a subfolder in the album by dragging it.

[0186] Figure 11 An example of quickly operating a picture based on a check mark “√” is shown. In this example, a cross-application operation is performed on a picture based on a check mark “√”, where Figure 11 As shown, the user first draws a travel route on the handbook interface and can set corresponding nodes on the travel route, such as setting travel process nodes corresponding to different travel destinations, etc., and based on the actual photo / video recording during the trip, the user can select the required pictures / videos by drawing the symbol "√" in the picture / video thumbnail area in the album (different from the related art of selecting pictures / videos after entering the editing mode), and can batch insert the selected pictures / videos into the required nodes in the travel route of the handbook interface by dragging and dropping, so as to realize convenient cross-application operations on pictures / videos in the album.

[0187] In actual applications, the trajectory formed by the user's drawing information and the marking information of the corresponding target object can be displayed with matching visual effects based on the mode of the electronic device.

[0188] It is easy to understand that the implementation is not limited to the "×", "+", "○", "@", "delete" and other gesture types provided in the embodiments of the present application. It is also possible to draw heart-shaped, star-shaped or batch gestures that can cover multiple objects as needed (such as drawing a long line on multiple picture thumbnails in the same row, column or inclination to represent the batch deletion of multiple pictures in the same row, column or inclination), etc.

[0189] This example uses the user's writing and drawing gestures to intuitively manage the pictures in the album, making the user's operation and management of the album more flexible and free, and improving the convenience and efficiency of album management.

[0190] Corresponding to the above-mentioned interactive processing method, the embodiment of the present application further provides an interactive processing device, the composition structure of which is as follows: Figure 12 Shown, including:

[0191] An acquisition module 1201 is configured to obtain input information, where the input information can form a trajectory;

[0192] A first determining module 1202 is configured to determine a target object based on the position information of the input information;

[0193] A second determination module 1203 is configured to determine a target processing method based on a trajectory formed by the input information;

[0194] The processing module 1204 is configured to process the target object in the target processing manner.

[0195] In an optional embodiment, the acquisition module 1201 is specifically used to: obtain input information generated by the interactive device writing and drawing on the device interface of the electronic device based on the writing and drawing gestures; the writing and drawing gestures represent the pen tip movement trajectory and logic that conforms to human writing and drawing behavior.

[0196] In an optional implementation manner, the second determining module 1203 is specifically configured to:

[0197] Determining model input data; the model input data includes the input information, or includes the input information and the target object;

[0198] inputting the model input data into a processing model so that the processing model performs processing based on at least the trajectory formed by the input information;

[0199] A processing result of the processing model based on the model input data is obtained, wherein the processing result includes a target processing method for the target object.

[0200] The processing module 1204 is specifically configured to:

[0201] delivering the processing result to the first application, so that the first application performs a target operation corresponding to the target processing method in the processing result on the target object;

[0202] The target object is presented based on the first application.

[0203] In an optional implementation manner, the second determining module 1203 is specifically configured to:

[0204] determining, in a first application, a target processing method based on a trajectory formed by the input information; and presenting the target object based on the first application;

[0205] In an optional implementation, the processing module 1204 is specifically configured to: execute, in the first application, a target operation corresponding to the target processing method on the target object.

[0206] In an optional embodiment, the above device further includes a control module, which is used to:

[0207] After obtaining the input information, in response to the electronic device being in the first mode, the trigger processing model determines the target object and the target processing method based on the input information; in response to the electronic device being in the second mode, the trigger first application determines the target object and the target processing method based on the input information.

[0208] The target object is presented based on the first application; and mode switching can be performed between the first mode and the second mode.

[0209] In an optional implementation, the input information is obtained by the electronic device based on an operation of an interactive device, and the interactive device includes a first input terminal for writing to generate the input information and a second input terminal for mode control.

[0210] The control module is further configured to: generate a target control signal in response to the second input terminal, and control the electronic device to enter the first mode.

[0211] In an optional implementation manner, the control module is further configured to:

[0212] Controlling the display of a track corresponding to the input information.

[0213] In the first mode, the trajectory is displayed with a first visual effect, and in the second mode, the trajectory is displayed with a second visual effect, and the first visual effect and the second visual effect are different.

[0214] In an optional implementation manner, the control module is further configured to:

[0215] In the first mode, the target object is marked and the corresponding marking effect is controlled to be displayed; the marking effect of the target object and the first visual effect satisfy a consistency relationship.

[0216] The present application also discloses an electronic device, the composition structure of the electronic device is as follows: Figure 13 As shown, including at least:

[0217] Memory 10, for storing a computer instruction set;

[0218] A set of computer instructions can be implemented in the form of a computer program.

[0219] The processor 20 is configured to implement the interactive processing method provided in any of the above method embodiments by executing the computer instruction set in the memory.

[0220] The processor 20 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural network processor (NPU), a deep learning processor (DPU) or other programmable logic devices, etc.

[0221] Optionally, the electronic device also includes storage resources such as memory and cache.

[0222] Optionally, the electronic device further includes a camera assembly, and / or is connected to an external camera assembly.

[0223] In addition, the electronic device may also include components such as a communication interface and a communication bus. The memory, processor, and communication interface communicate with each other via the communication bus.

[0224] Communication interfaces are used for communication between electronic devices and other devices. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, and a control bus.

[0225] An embodiment of the present application further discloses a storage medium, which carries one or more computer instruction sets. When the one or more computer instruction sets are executed by an electronic device, the electronic device can implement the interactive processing method provided in any of the above method embodiments.

[0226] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0227] For the convenience of description, the above systems or devices are described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0228] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application can be essentially or the part that makes a creative contribution in the form of a software product. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present application.

[0229] Finally, it should be noted that, in this document, relational terms such as first, second, third, and fourth are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0230] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An interactive processing method, comprising: obtaining input information, wherein the input information is capable of forming a trajectory; determining a target object based on the position information of the input information; determining a target processing method based on a trajectory formed by the input information; The target object is processed in the target processing manner.

2. The interactive processing method according to claim 1, wherein obtaining input information comprises: Obtaining input information generated by the interactive device performing writing and drawing on the device interface of the electronic device based on the writing and drawing gestures; The writing and drawing gesture representation conforms to the pen tip movement trajectory and logic of human writing and drawing behavior.

3. The interactive processing method according to claim 1, wherein determining a target processing method based on a trajectory formed by the input information comprises: Determine model input data; The model input data includes the input information, or includes the input information and the target object; inputting the model input data into a processing model so that the processing model performs processing based on at least the trajectory formed by the input information; A processing result of the processing model based on the model input data is obtained, wherein the processing result includes a target processing method for the target object.

4. The interactive processing method according to claim 3, processing the target object in the target processing manner, comprising: delivering the processing result to the first application, so that the first application performs a target operation corresponding to the target processing method in the processing result on the target object; The target object is presented based on the first application.

5. The interactive processing method according to claim 1, wherein determining a target processing method based on a trajectory formed by the input information comprises: determining, in the first application, a target processing method based on a trajectory formed by the input information; The target object is presented based on the first application; Processing the target object in the target processing manner includes: The first application performs a target operation corresponding to the target processing method on the target object.

6. The interactive processing method according to claim 1, after obtaining the input information, further comprising: In response to the electronic device being in the first mode, triggering the processing model to determine the target object and the target processing mode based on the input information; In response to the electronic device being in the second mode, triggering the first application to determine the target object and the target processing manner based on the input information; The target object is presented based on the first application; and mode switching can be performed between the first mode and the second mode.

7. The interactive processing method according to claim 6, wherein the input information is obtained by the electronic device based on an operation of an interactive device, the interactive device comprising a first input terminal for writing to generate the input information and a second input terminal for mode control; The method further comprises: In response to the target control signal generated by the second input terminal, the electronic device is controlled to enter the first mode.

8. The interactive processing method according to claim 6, further comprising: displaying a trajectory corresponding to the input information; In the first mode, the trajectory is displayed with a first visual effect, and in the second mode, the trajectory is displayed with a second visual effect, and the first visual effect and the second visual effect are different.

9. The interactive processing method according to claim 8, further comprising: In the first mode, the target object is marked and a corresponding marking effect is displayed; The marking effect of the target object and the first visual effect satisfy a consistency relationship.

10. An electronic device comprising: a memory for storing at least one set of computer instructions; The processor is configured to implement the following processing by executing the instruction set stored in the memory: obtaining input information, wherein the input information is capable of forming a trajectory; determining a target object based on the position information of the input information; determining a target processing method based on a trajectory formed by the input information; The target object is processed in the target processing manner.

Images