Processing method, device and equipment

Abstract

The invention discloses a processing method, device and equipment. The method comprises the following steps: obtaining a first operation; determining a first object based on the first operation; the first object is located at a first position; obtaining a second operation for the first object, wherein the second operation is used for indicating to change the position of the first object; in response to the second operation, the first object is displayed along with the input position of the second operation; obtaining a third operation for the first object; in response to the third operation, the first object is located at a second position, and the second position is different from the first position; before a third operation for the first object is obtained, if the parameter information meets a first condition, displaying a target identifier for a target function of the first object; the parameter information is related to the first object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a processing method, device and equipment. BACKGROUND

[0002] With the rapid development of artificial intelligence, in intelligent applications, if target content needs to be processed, a target model with intelligent function is generally called, and the calling process of the target model often depends on an additional operation program, leading to complex operation, increased delay and reduced processing efficiency. SUMMARY

[0003] Therefore, the present application provides a processing method, device and equipment.

[0004] The technical scheme of the present application is implemented as follows:

[0005] The present application provides a processing method, which comprises the following steps:

[0006] obtaining a first operation;

[0007] determining a first object based on the first operation; the first object is located at a first position;

[0008] obtaining a second operation for the first object, the second operation being used to indicate a change in the position of the first object;

[0009] in response to the second operation, the first object is displayed along with the input position of the second operation;

[0010] obtaining a third operation for the first object;

[0011] in response to the third operation, the first object is located at a second position, the second position being different from the first position;

[0012] if parameter information meets a first condition before the third operation for the first object is obtained, a target identifier of a target function for the first object is displayed; the parameter information is related to the first object.

[0013] In the above processing method, the target function can call an intelligent model to process first content corresponding to the first object.

[0014] In the method, the displaying the target identifier of the target function for the first object comprises one of the following: displaying a first display region, the first display region comprising the target identifier of the target function for the first object, the first display region comprising a second position; the target identifier is displayed at the second position; or displaying the first display region, the first display region comprising a plurality of indication identifiers of functions, the target identifier of the target function belonging to the plurality of indication identifiers of functions; a display effect of the target identifier is different from display effects of other indication identifiers, the first display region comprising a second position; the target identifier is displayed at the second position; or displaying the target identifier of the target function for the first object at a second position, the second position being an input position of the second operation.

[0015] In the method, the parameter information satisfies the first condition, and the first condition comprises one of the following: the first object moves to within a preset distance of the first display region along with the second operation corresponding to the input position; or a distance that the first object moves along with the second operation corresponding to the input position exceeds a preset distance threshold.

[0016] In the method, the parameter information satisfies the first condition, and the first condition comprises one of the following: when the first operation is performed on the first object, a window corresponding to the first object belongs to a preset window; the preset window indicates that the first object can obtain a storage path of the corresponding first content; or when the first operation is performed on the first object, a type corresponding to the first object belongs to a preset type.

[0017] In the method, the window corresponding to the first object belongs to the preset window, and the method comprises: a window process of the window corresponding to the first object is a target process, and a window scenario to which the window corresponding to the first object belongs is a desktop scenario or an explorer scenario.

[0018] In the method, the method further comprises: obtaining the window corresponding to the first object; determining a path obtaining manner of the first content based on a window scenario to which the window corresponding to the first object belongs; determining the storage path of the first object based on the window by using the path obtaining manner; and the storage path can obtain the corresponding first content.

[0019] In the method, before the displaying the target identifier of the target function for the first object, the method further comprises: identifying a category of the first content or the first object, and determining the target function corresponding to the first object based on an identification result.

[0020] Embodiments of the present application provide a processing device, comprising:

[0021] The obtaining module is configured to obtain the first operation.

[0022] The determining module is configured to determine the first object based on the first operation; the first object is located at the first position.

[0023] The obtaining module is further configured to obtain a second operation for the first object, the second operation being used to indicate a change in position of the first object.

[0024] The response module is configured to display the first object in response to the second operation, the first object being displayed in an input position of the second operation.

[0025] The obtaining module is further configured to obtain a third operation for the first object.

[0026] The response module is configured to display the first object in response to the third operation, the first object being displayed in a second position, the second position being different from the first position.

[0027] The display module is configured to display a target identifier of a target function for the first object before the third operation for the first object is obtained, if parameter information related to the first object meets a first condition.

[0028] The application embodiment provides a processing device, comprising a processor and a display.

[0029] The processor is configured to obtain a first operation, determine a first object based on the first operation, the first object being displayed in a first position, obtain a second operation for the first object, the second operation being used to indicate a change in position of the first object.

[0030] The display is configured to display the first object in response to the second operation, the first object being displayed in an input position of the second operation.

[0031] The processor is further configured to obtain a third operation for the first object.

[0032] The display is further configured to display the first object in response to the third operation, the first object being displayed in a second position, the second position being different from the first position, and display a target identifier of a target function for the first object before the third operation for the first object is obtained, if parameter information related to the first object meets a first condition.

[0033] The application embodiment provides a computer readable storage medium, the computer readable storage medium storing one or more computer programs, the one or more computer programs being executable by one or more processors to implement the processing method.

[0034] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions provided by the application embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on these drawings without creative labor.

[0036] Figure 1 A flowchart of a processing method provided by the embodiments of the present application is shown in FIG. 1;

[0037] Figure 2 A flowchart of an exemplary display of a target identifier provided by the embodiments of the present application is shown in FIG. 2;

[0038] Figure 3 An exemplary display of a target identifier provided by the embodiments of the present application is shown in FIG. 3; Figure 1

[0039] Figure 4 An exemplary display of a target identifier provided by the embodiments of the present application is shown in FIG. 4; Figure 2

[0040] Figure 5 An exemplary display of a target identifier provided by the embodiments of the present application is shown in FIG. 5; Figure 3

[0041] Figure 6 A flowchart of an exemplary acquisition of a first object storage path provided by the embodiments of the present application is shown in FIG. 6;

[0042] Figure 7 A flowchart of an exemplary determination of a valid drag provided by the embodiments of the present application is shown in FIG. 7;

[0043] Figure 8 A flowchart of an exemplary processing method provided by the embodiments of the present application is shown in FIG. 8;

[0044] Figure 9 A structural diagram of a processing device provided by the embodiments of the present application is shown in FIG. 9;

[0045] Figure 10 A structural diagram of a processing device provided by the embodiments of the present application is shown in FIG. 10. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that, in order to facilitate the description, only the parts related to the application are shown in the drawings.

[0047] ​​​In the related art, if an intelligent program needs to be called in the process of processing a file, the file is generally dragged to a processing position first, and then intelligent processing is performed on the file. Generally, additional triggering operations are required to start intelligent processing. For example, the triggering operations to start intelligent processing can include: 1. manual triggering, for example, Microsoft Office right-click menu selection of operations to be performed. This requires manual selection of operations to be performed after moving, which interrupts the workflow; 2. rule-based automatic triggering, for example, using an AutoHotkey script. This only supports preset rules and cannot dynamically adapt to the content of the file; 3. independent artificial intelligence (AI) application, for example, Grammarly / DeepL, which has nothing to do with the action of dragging the file and has a long operation path; and 4. cloud disk intelligent synchronization, for example, Dropbox SmartSync, which is limited to cloud files and has low real-time processing capability on the local. That is, in the related art, the movement of the file is the basis for the transmission function to be realized, and there is no corresponding relationship between the triggering operation of the AI and the movement of the file. The two are completely separated, and the efficiency of data processing is low.

[0048] To solve the above problems, an embodiment of the present application provides a processing method, which is realized by a processing device.

[0049] As shown in the following steps S101 to S107: Figure 1

[0050] Step S101, obtaining a first operation.

[0051] In an embodiment of the present application, the first operation can refer to an interaction behavior with an image interface completed by a user through a mouse, a touchpad or other input devices. For example, the first operation can be a mouse left button pressing or clicking operation, or a gesture operation of touching a display screen.

[0052] For example, in the case of a mouse event, the processing device can capture the user's original mouse event (click, left button press) through RAW Input, and use UI Automation to locate the click position.

[0053] In an embodiment of the present application, RAW Input is used to capture RAW input events, without waiting for a system message queue, which can reduce the delay by more than 50% compared with using a message hook in the related art.

[0054] ​In the embodiments of the present application, the processing device can be an electronic device with data processing function, which can be a tablet computer, a notebook computer, a smart phone, a palm computer, a personal digital assistant (PDA), a desktop computer, etc., and the specific processing device is not limited herein.

[0055] In step S102, a first object is determined based on the first operation; the first object is located at a first position.

[0056] In the embodiments of the present application, when the processing device obtains the first operation, the position where the first operation occurs is found to obtain the corresponding interface element, i.e., the first object. For example, when the first operation is a left mouse button click, the first object can be obtained in the following manner: when the left mouse button is clicked, the interface element at the position where the first operation occurs is obtained based on UI Automation.ElementFromPoint, i.e., the control (the first object) is directly located by the screen click position.

[0057] In the embodiments of the present application, the first object can refer to a visual element in the user interface, such as a file icon, a window title bar, a button, a shortcut, etc. The first object can include multiple attributes, such as position, size, class name, etc., which can be used to determine whether the object is a target that can be dragged. For example, the processing device can obtain the control class name of the first object based on UIAutomation.ElementFromPoint, and determine whether the click object corresponding to the first operation is a target that can be dragged based on the control class name. For example, if the control class name is blank, the object cannot be dragged and processed, i.e., the first object cannot be determined; if the control class name is a file icon, the object can be dragged and processed, i.e., the first object is determined to be the file icon.

[0058] In the embodiments of the present application, the first position is the coordinate value of the first object in the screen coordinate system, which is usually represented in pixels, and the first position is the position of the first object. For example, in a desktop environment, if a user clicks a shortcut icon, the shortcut icon is defined as the first object, and the display position of the shortcut icon is the first position; if the user clicks a file icon in a resource manager, the file icon is the first object, and the display position of the file icon is the first position.

[0059] Exemplarily, the determining the first object based on the first operation can include two steps: in a first step, the processing device calls GetClassName to obtain the control class name at the first operation position, and judges the control class name by the processing device to determine whether the control class name is an element (such as SysListView32) supporting dragging; in a second step, the processing device calls the UIAutomation.ElementFromPoint method to obtain the control type and context information at the first operation position, and further judges the first operation position by the processing device to determine whether the first operation position exists a draggable object. In this way, by combining the control class name with the context information of the UI Automation, the recognition module can accurately recognize the object of dragging, and this recognition mechanism effectively avoids misjudging the normal click operation as the dragging operation, thereby significantly enhancing the understanding and response ability of the system for the real intention of the user.

[0060] In step S103, a second operation for the first object is obtained, and the second operation is used to indicate changing the position of the first object.

[0061] In the embodiment of the present application, the second operation can refer to the interaction behavior of the user to the first object. Exemplarily, the second operation can be the behavior of moving the mouse while keeping the left mouse button pressed, that is, the first object is in the selected state. The second operation can change the display position of the first object.

[0062] In the embodiment of the present application, the processing device can use RAW Input to continuously track the displacement of the mouse. If the coordinate value of the mouse in the screen coordinate system changes and the left button is kept pressed, the second operation is successful.

[0063] In step S104, the first object is displayed along with the input position of the second operation in response to the second operation.

[0064] In the embodiment of the present application, the first object is displayed along with the change of the input position of the second operation, which is used to indicate the real-time position of the first object moving in the case of the second operation.

[0065] In the embodiment of the present application, the first object displayed along with the input position of the second operation can be other forms of the first object. For example, the first object displayed along with the input position of the second operation can be a semi-transparent first object, so that the moving path can be displayed while ensuring the visibility of the input position of the second operation. Alternatively, the first object displayed along with the input position of the second operation can be a reduced version of the first object, so that the visibility of the input position of the second operation can be increased as much as possible while ensuring the display of the moving path.

[0066] Step S105, obtaining a third operation for the first object.

[0067] In embodiments of the present application, the third operation can be an action of releasing the mouse button by the user, marking the end of the moving behavior. Illustratively, the third operation can also be based on the RAW Input interface to ensure that the release operation of the user can be captured in time, and trigger the subsequent logic processing according to the release operation of the user.

[0068] Step S106, in response to the third operation, the first object is located at a second position, the second position being different from the first position.

[0069] In embodiments of the present application, in the case that the third operation represents the release of the first object, the first object is located at the second position, and since the second operation is used to change the position of the first object, the third operation executed after the second operation makes the second position where the first object is located different from the first position where the first object is originally located. If the first position and the second position are the same, it means that the execution of the second operation does not change the position of the first object, which conflicts with the second operation, so the first position and the second position must have a positional deviation.

[0070] Illustratively, the positional deviation can be further limited here to avoid false triggering.

[0071] Of course, the processing device can further combine the window context information where the second position is located to ensure that the target position supports the file placement operation, for example, if the user tries to drag and drop the file to a window that does not support file placement, the processing device can pop up a prompt or ignore this operation, so as to avoid misoperation.

[0072] Step S107, before the third operation for the first object is obtained, if the parameter information meets a first condition, displaying a target identifier of a target function for the first object; the parameter information being related to the first object.

[0073] In embodiments of the present application, the target function is a function that the first object can perform, for example, if the first object is an image, the function that it can perform can include PS, image processing, outputting image features, displaying images, sending to a target position, etc.; if the first object is a file, the function that it can perform can be summarizing the content of the file, translating the file, converting the file format, etc.; the target identifier of the target function is the display identifier corresponding to the target function, which can also be displayed in the form of an icon. In this way, the target function that the first object can perform can be recommended in real time during the execution of the operation of the first object.

[0074] In the embodiments of the present application, if the parameter information satisfies the first condition, the first condition is related to the first object, for example, the first object performs the second operation so that the position of the first object satisfies the preset position condition, or the window in which the first object is located, the type of the first object or other conditions satisfy the preset condition, the parameter information corresponding to the first object and the corresponding condition can be set based on actual needs and application scenarios, and the present application is not limited thereto.

[0075] In this way, in the embodiments of the present application, the operation performed on the first object is linked to the target function that the first object can perform, and the target function that the first object can perform can be recommended in real time in the process of performing the operation on the first object, and then the corresponding target function is executed, so as to reduce the steps of triggering and improve the processing efficiency.

[0076] In some embodiments, the target function can call the intelligent model to process the first content corresponding to the first object.

[0077] In the embodiments of the present application, the intelligent model refers to a machine learning model or algorithm called to realize a specific intelligent function. The intelligent model can be a natural language processing model (such as BERT, GPT), an image recognition model (such as ResNet, YOLO), or a structured data analysis model (such as random forest, XGBoost), etc.

[0078] The intelligent model described in the embodiments of the present application is a machine learning model that can recognize natural language and / or other inputs (such as audio, video, images, tables, etc.) input to the intelligent model, and perform comprehensive language processing tasks such as semantic analysis and question answering, and then generate an output related to the input and / or respond to the input.

[0079] The intelligent model learns the characteristics and rules of natural language by training a large amount of diversified data, so as to be able to understand and generate natural language. Generally, it has hundreds of millions to tens of billions of model parameters (model parameters are variables that control the behavior of the intelligent model), which can capture the complex relationships and patterns in natural language.

[0080] The intelligent model can be a generative model (generative models), a generative language model (generative language models, GLMs). For example, it can specifically include a large language model (large language models, LLMs), a GPT (Generative Pre-trained Transformer), etc. The model involved in the embodiments of the present application can be a general large model, and can also be a specialist large model obtained by fine-tuning based on needs, and the embodiments of the present application are not limited thereto.

[0081] The intelligent model described in the embodiments of the present application is an intent recognition model. The target function corresponding to the intelligent model is determined by recognizing at least one of a target object (for example, an icon), a dragging track of the target object, a file path corresponding to the target object, and the like.

[0082] In the embodiments of the present application, there is a dependency relationship between the intelligent model and the target function. The target function implements the automatic processing of the first content corresponding to the first object by calling the intelligent model. For example, if the target function is image description, the intelligent model is an image recognition model related to the image, and the high image recognition model can perform image recognition on the key elements of the first content of the first object and generate descriptive text.

[0083] In the embodiments of the present application, the processing device displays the function identifier of the target function of the first object, and the second operation can move the first object to the function identifier of the target function to trigger the calling of the intelligent model to process the first content corresponding to the first object. In this way, there is no need to additionally set the step of calling the intelligent model, and the efficiency of processing the first content can be improved by combining the operation performed on the first object.

[0084] In some embodiments, when the processing device performs the step of displaying the target identifier of the target function of the first object in step S107, the following step S201 can be performed, as shown in the following table: Figure 2

[0085] Step S201, display a first display area, the first display area includes a target identifier of a target function of the first object, the first display area includes a first display position; the target identifier is displayed at the first display position.

[0086] In the embodiments of the present application, the processing device can display a first display area. The first display area can be an area defined on the display screen by the processing device for displaying the target identifier. The area is used to display the target identifier of the target function of the first object. Of course, considering that the first object can perform more than one target function, the first display area can include a first display position, which is the position of the target identifier display.

[0087] In the embodiments of the present application, the first display area can also be an area displayed in the process of performing the second operation on the first object, for example, a floating window displayed on the side of the display screen, which can disappear after the execution is completed, or an area displayed in the taskbar management window, or other areas capable of displaying the target identifier display.

[0088] For example, as shown in the following table: Figure 3 ​As shown, in the case where the first object is the image icon 31, the target identifier displayed by the first display position 33 in the first display area 32 can be a text generation identifier, an image recognition identifier, an image de-noising identifier, or other specific image processing identifiers, or can be directly an image identifier, and the target function corresponding to the image identifier is to intelligently process based on the image itself (first content) to obtain an image processing result. Here, the first display position 33 displays a text generation identifier 331, an image recognition identifier 332, and an image de-noising identifier 333.

[0089] When the processing device performs the step S107 of displaying the target identifier of the target function of the first object, as shown in Figure 2 As shown, the following step S202 can also be performed:

[0090] The step S202 displays a first display area, the first display area includes a plurality of function indication identifiers, and the target identifier of the target function belongs to the plurality of function indication identifiers. The display effect of the target identifier is different from the display effect of other indication identifiers. The first display area includes a first display position, and the target identifier is displayed in the first display position.

[0091] In the embodiments of the present application, the first display area not only displays the target identifier of the target function, but also displays the indication identifiers of other functions in the plurality of functions except the target function. The first display area not only includes the first display position, but also includes a second display position. The first display position is used to display the target identifier of the target function, and the second display position is used to display the indication identifiers of other functions in the plurality of functions except the target function.

[0092] In the embodiments of the present application, when the processing device performs the second operation on the first object, the display effect of the target identifier displayed in the first display position in the first display area is different from the display effect of other indication identifiers in the second display position. For example, the differential display can be realized by using a high-light color block, a dynamic flashing, a shadow effect, and other visual enhancement methods. These visual enhancement methods can make the target identifier corresponding to the target function more prominent among the many function options.

[0093] As shown, in the case where the first object is the image icon 31, the first display position 33 in the first display area 32 processes and displays a text generation identifier 331, an image recognition identifier 332, and an image de-noising identifier 333. The second display position 34 of the first display area 32 can also display text translation 341, music playing 342, video editing 343, and other function indication identifiers. Figure 4

[0094] ​In the embodiments of the present application, the first display area can also be in the form of a floating window, and can be displayed when the second operation is performed on the first object, or can be displayed in a blank area of the display screen, or can be fixedly displayed in a display area of the display screen. The present application is not limited in this regard, and the actual requirements and application scenarios can be set.

[0095] When the processing device displays the target identifier of the target function for the first object in step S107, the processing device can also perform the following step S203. Figure 2

[0096] In step S203, the target identifier of the target function for the first object is displayed at the second position, and the second position is the input position of the second operation.

[0097] In the embodiments of the present application, the processing device can also not set a fixed first display area on the display screen to display the target identifier of the target function, and can display the target identifier of the target function for the first object in real time at the input position of the second operation.

[0098] For example, as shown in FIG. 3, when the first object is an image icon 31, an image generation identifier 331, an image recognition identifier 332, and an image de-noising identifier 333 are displayed at the input position of the second operation. Figure 5

[0099] In this way, the target identifier of the target function is displayed at the second position, and when the user performs an operation at the second position, the function control corresponding to the user's intention is automatically displayed, without the need for the user to perform additional searching or interface switching. In this way, the target identifier of the target function is displayed at the second position, which can save the user's operation time, achieve faster processing, and improve processing efficiency.

[0100] In some embodiments, the parameter information satisfies the first condition, and can include one of the following:

[0101] The first object moves to within a preset distance of the first display area along the input position corresponding to the second operation, or the distance that the first object moves along the input position corresponding to the second operation exceeds a preset distance threshold.

[0102] ​​In the embodiments of the present application, before the third operation for the first object is obtained, if the first object moves to within the preset distance of the first display area along with the input position corresponding to the second operation, the target identifier of the target function for the first object is displayed. For example, the preset distance can be 100 pixel points. If the first object moves to a position 100 pixel points away from the first display area along with the input position corresponding to the second operation, the target identifier of the target function of the first object is displayed in the first display area. Of course, the preset distance can be set to other pixel points, and the size of the preset distance can be set based on actual needs and application scenarios, which is not limited in the present application.

[0103] In the embodiments of the present application, before the third operation for the first object is obtained, the distance that the first object moves along with the input position corresponding to the second operation exceeds the preset distance threshold, and the target identifier of the target function for the first object is displayed. For example, the preset distance threshold can be 40 pixel points. If the distance that the first object moves along with the input position corresponding to the second operation exceeds 40 pixel points, it can be determined as a valid second operation, and the target identifier of the target function of the first object is displayed. In this way, the preset distance threshold helps to further improve the accuracy of drag detection and prevent false judgments caused by rapid release in a short time.

[0104] In some other embodiments, the parameter information satisfies the first condition, which can include one of the following:

[0105] When the first operation is performed on the first object, the window corresponding to the first object belongs to a preset window. The preset window indicates that the first object can obtain a storage path corresponding to the first content. Alternatively, when the first operation is performed on the first object, the type corresponding to the first object belongs to a preset type.

[0106] In the embodiments of the present application, if the first operation is performed on the first object, the window corresponding to the first object belongs to a preset window, and the preset window is a window through which the first object can obtain a storage path corresponding to the first content. If the window corresponding to the first object cannot obtain the storage path of the first content, the first content cannot be obtained, and thus the first content cannot be processed.

[0107] In the embodiments of the present application, to avoid that the first content corresponding to the first object cannot be obtained, the preset window can be a window through which the first object can obtain a storage path corresponding to the first content, which is pre-set. For example, the preset window can be a file management window, a resource browsing window, or other folder view windows. In this way, the first content corresponding to the first object can be obtained, and the target function corresponding to the first object is determined based on the first content, so that the target identifier of the target function is displayed.

[0108] In the embodiments of the present application, the processing device can also determine the target function that can be executed by the first object based on the category corresponding to the first object, and determine that the parameter information satisfies the first condition if the type corresponding to the first object belongs to a preset type, determine the corresponding target function based on the preset type, and then display the target identifier of the target function of the first object.

[0109] In some embodiments, the window corresponding to the first object belongs to a preset window, including: the window process of the window corresponding to the first object is a target process, and the window scene to which the window corresponding to the first object belongs is a desktop scene or a resource manager scene.

[0110] In the embodiments of the present application, the window process of the window corresponding to the first object is a target process, and exemplarily, the target process can be explorer.exe or other processes that can obtain the storage path of the first content.

[0111] In the embodiments of the present application, the processing device can determine the target process through GetWindowThreadProcessId, if the target process is a windows process (including a desktop), it means that the processing device can obtain the storage path of the corresponding first content, and if it is a third party or other system process, the processing device can not be able to obtain the storage path of the corresponding first content. Exemplarily, the GetWindowThreadProcessId function is used to obtain the process ID associated with the specified window handle, and then match to the preset target process list, which can effectively distinguish between ordinary application windows and file management class windows, and avoid misjudgment.

[0112] In the embodiments of the present application, in the case that the first operation is mouse pressing, the window class name and the control type of the clicked window can be obtained based on GetClassName and UI Automation.ElementFromPoint, so that the window scene to which the window corresponding to the first object belongs can be distinguished based on the window class name and the control type, and exemplarily, the window scene can be a desktop scene or a file manager scene. Here, it is considered whether the first object is a draggable object and whether it can be processed (whether the corresponding content can be obtained). In this way, by judging the window scene to which the window corresponding to the first object belongs, it can be further confirmed whether the window corresponding to the first object belongs to the context related to file management, thereby helping to exclude the interference of irrelevant windows (such as browser windows, document editors, etc.).

[0113] In some embodiments, as shown in FIG. 6, Figure 6 The processing device can further perform steps S601 to S603 as follows:

[0114] Step S601, a window corresponding to the first object is acquired.

[0115] In the embodiments of the present application, considering that the window in which the first object is located can be one of the superimposed multiple windows, at this time, the processing device can enumerate all windows through IShellWindows, and the window information of each window can also be known, for example, the window information can include the window class name, and then the window corresponding to the first object is matched from all the enumerated windows based on the window class name and the control type of the window in which the first object is located. In this way, the desktop and the explorer can be distinguished according to the process window information, and the file scene of the current page drag is detected.

[0116] Step S602, determining a path acquisition manner of the first content based on a window scene to which the window corresponding to the first object belongs.

[0117] In the embodiments of the present application, the window scene refers to the context environment in which the window is located, for example, whether a certain window is an explorer window, a desktop window or a browser window, and different types of interface environments. Under different types of window scenes, the system will adopt a corresponding path acquisition mechanism to process the data access request, that is, the path acquisition manner of the first content is different under different window scenes.

[0118] Exemplarily, in the explorer window, the AI application in the processing device can acquire the file path through the COM interface (such as IShellView, IDataObject); and in the desktop scene, the application can use the clipboard or directly access the file system path, by identifying the window scene, the system can select the most suitable path acquisition manner, and the system improves the accuracy and security of path resolution.

[0119] Step S603, determining a storage path of the first object based on the window by using the path acquisition manner; the storage path can acquire the corresponding first content.

[0120] In the embodiments of the present application, after the path acquisition manner of the first content corresponding to the first object is acquired, the storage path of the first object can be determined from the window by using the path acquisition manner. The storage path refers to a string path used to identify and access the target content (such as a file or a directory), for example, `C:\Users\Example\Desktop\test.txt`, wherein the storage path is a result generated by dynamically analyzing the window scene and the path acquisition manner.

[0121] In the embodiments of the present application, compared with the related art that the path of the file is acquired only when the drag release occurs, the embodiments of the present application can acquire the storage path based on the first operation after the first operation is performed, then acquire the first content based on the storage path, and further process the first content, thereby improving the efficiency of file processing.

[0122] Exemplarily, when the user drags a file in the resource manager, the system acquires the CF_HDROP format data of the first object through the IShellView interface, extracts the file path from the format data, and then returns the file path to the AI application. The way of acquiring the CF_HDROP format data and extracting the file path through the IShellView interface avoids the dependence on the code of the target window, thereby improving the compatibility and security of the system.

[0123] In the embodiments of the present application, the processing device acquires the window corresponding to the first object, determines the path acquisition manner based on the window scenario, and parses the storage path by using the path acquisition manner. In this way, the path parsing logic in different window environments can be distinguished, thereby adapting to various application scenarios, and further improving the precision and compatibility of cross-scene file drag behavior detection.

[0124] In the embodiments of the present application, after the storage path is acquired, the processing device asynchronously delivers the storage path to the application layer through the ThreadSafeQueue, so that the AI application can acquire the first content based on the storage path, and further implement the processing of the first content. This can avoid blocking the event processing thread of the system. Here, the storage path of the acquired first object is given to the AI program, an execution queue of the AI application is created, system resources can be released, and the processing efficiency is improved.

[0125] In some embodiments, before the processing device performs the step of “displaying the target identifier of the target function for the first object” in the above step S107, the processing device can further perform the following step: identifying the category of the first content or the first object, and determining the target function corresponding to the first object based on the identification result.

[0126] In the embodiments of the present application, after the first content is acquired based on the storage path, the processing device can identify the first content to obtain the file category, and further determine the target function based on the file category.

[0127] In the embodiments of the present application, a corresponding relationship can be preset between different types. Thus, after the category of the first content or the category corresponding to the first object is known, the corresponding target function can be matched based on the corresponding relationship, and the target identifier of the target function is displayed. Exemplarily, the corresponding relationship is shown in Table 1.

[0128] Table 1

[0129] File Type Target Function Output Form .docx Abstract, Translation Floating Text .xlsx Chart, Summary Thumbnail .png OCR, Image Text Description Editable Text

[0130] As shown in Table 1, the corresponding target function can be matched based on the file type.

[0131] In some embodiments, after the first object corresponding to the first content selected target function, the AI application can perform processing operation of the corresponding function, and then output the processing result. The output form is also related to the corresponding file type, which is also embodied in Table 1.

[0132] In the embodiments of the present application, the processing result or processing process of the first object corresponding to the first content can also be displayed on the display screen of the processing device. The display method can be to use the DirectUI technology to draw a semi-transparent panel beside the drag cursor (the input position of the second operation) for display, or to dynamically update: the asynchronous thread receives the processing result of the AI program and refreshes the UI.

[0133] Exemplarily, as shown in Table 1, the processing result of the first object corresponding to the first content can be displayed on the display screen of the processing device. Figure 7 An exemplary implementation for determining valid dragging is provided, including a user, an AI application in a processing device: a RAW input (RAWInput), an analyzer (Analyzer), and an application layer (APP); wherein determining valid dragging includes the following steps S701 to S706:

[0134] Step S701, press the left key to drag the file.

[0135] Here, the user clicks the left mouse button, or other input devices select the file, which can be dragged.

[0136] Step S702, send the original event.

[0137] Here, the AI application obtains the original event (click, left button press, release, etc.) through RAWInput, and obtains the class name and control type (such as SysListView32 indicating the file list of the resource manager) of the clicked window through GetClassName and UI Automation.ElementFromPoint when the left mouse button is pressed.

[0138] Step S703, three-step verification.

[0139] Here, the AI application program verifies through the Analyzer in three steps, 1. Verify whether the control class name belongs to the invalid area, if so, the verification fails, if it belongs to the desktop, folder, resource manager and other valid areas, then this step verification is successful; 2. Whether it belongs to the windows process (explorer), if so, this step verification is successful, if not, the verification fails; 3. Whether the distance of dragging and moving is greater than the preset distance threshold, if so, this step verification is successful, if not, the verification fails; all three steps are verified successfully, which means that the verification is passed. If the verification fails, directly execute step S706. Among them, the information required for verification is RAWInput continuous tracking of mouse displacement, and the obtained raw event is sent to the Analyzer.

[0140] Step S704, cross-process path resolution.

[0141] Here, the AI application program enumerates all windows through the Analyzer's IShellWindows, locates the top-level window (such as the resource manager) of the drag starting point, obtains the IShellView interface through IServiceProvider, and calls GetItemObject to obtain the file path (CF_HDROP format) in IDataObject.

[0142] Step S705, asynchronous callback file path.

[0143] Here, the file path is asynchronously passed to the application layer through ThreadSafeQueue to avoid blocking the event processing thread.

[0144] Step S706, discard time.

[0145] Here, if the verification fails in step S703, it means that there is no effective drag behavior, and the event can be discarded directly.

[0146] In this way, based on three verifications, the accuracy of the drag behavior is greatly improved, and the possibility of false touch is avoided.

[0147] Exemplarily, as Figure 8 An exemplary intelligent processing implementation is provided, including a user, an AI application program in a processing device: raw input (RAWInput), a context analyzer (ContextAnalyzer), an artificial intelligence gateway (AIGateway), a translator (Translator), a summarizer (Summarizer), a renderer (Renderer); wherein the intelligent processing method includes the following steps S801 to S810:

[0148] Step S801, press the left button to drag the file.

[0149] Here, the user clicks the left mouse button, or other input device selected file, can be dragged.

[0150] Step S802, send coordinates and button state.

[0151] Here, the AI application program through RAWInput get the original event (click, left button down, release, etc.), in the mouse left button down, through the UI Automation.ElementFromPoint get the position of the click, and the state of the button, is pressed or released or click, and then sent to the ContextAnalyzer, exemplary, get information can refer to the content discussed in step S702 above, this will not be elaborated.

[0152] Step S803, multi-source verification.

[0153] Here, the AI application program through ContextAnalyzer based on the information obtained from RAWInput multi-source verification, exemplary verification method refer to the content discussed in step S703 above, this will not be elaborated, after verification, to AIGateway pass valid file information.

[0154] Step S804, pass valid file information.

[0155] Here, the effective file information can be passed in step S802 in the control type, control (corresponding to the first object discussed above) corresponding to the window, window process, window scene, etc.

[0156] Step S805, content extraction.

[0157] Here, the AI application program through AIGateway using COM interface (IShellView / IDataObject) from the file path of the resource manager security extraction, and then get the corresponding file (the first content) on the file path.

[0158] Step S806, based on file type processing file, get the processing result.

[0159] Here, exemplary examples: if the file type is a document, then execute step S8061 and step S8062.

[0160] Among them, step S8061, request summary generation.

[0161] Here, the AI application program through AIGateway will be sent to the file Summarizer, Summarizer on the file summary generation, get the abstract text.

[0162] Step S8062, return the summary text.

[0163] Here, the AI application returns the generated summary text to the AIGateway through the Summarizer.

[0164] Illustratively, if the file type is a table type, then steps S8063 and S8064 are performed.

[0165] In step S8063, a table translation is requested.

[0166] Here, the AI application sends the file to the Translator through the AIGateway, and the Translator translates the file (table) to obtain a translation result.

[0167] Step S8062, return the summary text.

[0168] Here, the AI application returns the translation result to the AIGateway through the Translator.

[0169] Step S807, send the processing result.

[0170] Here, the AI application sends the result to the Renderer through the AIGateway.

[0171] Step S808, render the floating panel.

[0172] Here, the AI application renders the floating panel through the Renderer.

[0173] Step S809, display the real-time result.

[0174] Here, the AI application renders the floating panel through the Renderer, and can ask the user whether to display in real time. If a user confirmation operation is obtained, the real-time result is displayed. This is also in the process of dragging, corresponding to the second operation discussed above.

[0175] Step S8010, release the mouse.

[0176] Here, the user releases the mouse, and the AI application sends the processing result to the AIGateway through the Renderer.

[0177] Step S8011, save the final result.

[0178] Here, the AI application saves the processing result obtained by the Renderer through the AIGateway.

[0179] Therefore, through the multi-level fusion judgment and the cross-process secure access technology, high-precision, low-delay and cross-scene detection and path analysis of the file dragging behavior are realized, the file processing is combined with the dragging scene, the AI function is triggered in the process of dragging without releasing the mouse, and the processing efficiency is improved.

[0180] The embodiment of the application provides a processing method, which comprises the following steps: obtaining a first operation; determining a first object based on the first operation; the first object is located at a first position; obtaining a second operation for the first object, the second operation being used for indicating a change of the position of the first object; in response to the second operation, the first object is displayed along with the input position of the second operation; obtaining a third operation for the first object; in response to the third operation, the first object is located at a second position, the second position being different from the first position; obtaining the third operation for the first object, if parameter information meets a first condition, displaying a target identifier of a target function of the first object; the parameter information is related to the first object. The method provided by the application links the operation performed on the first object with the target function that the first object can perform, can recommend the target function that can be performed by the first object in real time in the process of meeting the operation performed by the first object, and then performs the corresponding target function, so that the steps of triggering are reduced, and the processing efficiency is improved.

[0181] The embodiment of the application provides a processing device 9, as shown in the figure, comprising: Figure 9

[0182] The acquisition module 91 is used for obtaining a first operation;

[0183] The determination module 92 is used for determining a first object based on the first operation; the first object is located at a first position;

[0184] The acquisition module 91 is further used for obtaining a second operation for the first object, the second operation being used for indicating a change of the position of the first object;

[0185] The response module 93 is used for displaying the first object along with the input position of the second operation in response to the second operation;

[0186] The acquisition module 91 is further used for obtaining a third operation for the first object;

[0187] The response module 93 is used for locating the first object at a second position in response to the third operation, the second position being different from the first position;

[0188] The display module 94 is used for displaying a target identifier of a target function of the first object before the third operation for the first object is obtained, if parameter information meets a first condition; the parameter information is related to the first object.

[0189] ​In an embodiment of the present application, the target function can call the intelligent model to process the first content corresponding to the first object.

[0190] In an embodiment of the present application, the display module 94 is further configured to display a first display area, the first display area including a target identifier of the target function for the first object, the first display area including a first display position; the target identifier is displayed at the first display position; or the first display area is displayed, the first display area including a plurality of function indicators, the target identifier of the target function belonging to the plurality of function indicators; a display effect of the target identifier is different from a display effect of other function indicators, the first display area including a first display position; the target identifier is displayed at the first display position; or the target identifier of the target function for the first object is displayed at a second position, the second position being an input position of the second operation.

[0191] In an embodiment of the present application, the parameter information satisfies the first condition, including one of the following: the first object moves to within a preset distance of the first display area along with the input position corresponding to the second operation; or a distance moved by the first object along with the input position corresponding to the second operation exceeds a preset distance threshold.

[0192] In an embodiment of the present application, the parameter information satisfies the first condition, including one of the following: when the first operation is performed on the first object, a window corresponding to the first object belongs to a preset window; the preset window indicates that the first object can obtain a storage path of the first content; or when the first operation is performed on the first object, a type corresponding to the first object belongs to a preset type.

[0193] In an embodiment of the present application, a window process of the window corresponding to the first object is the target process, and a window scenario to which the window corresponding to the first object belongs is a desktop scenario or a resource manager scenario.

[0194] In an embodiment of the present application, the determination module 92 is further configured to obtain a window corresponding to the first object; determine a path obtaining manner of the first content based on a window scenario to which the window corresponding to the first object belongs; determine a storage path of the first object based on the window by using the path obtaining manner; and the storage path can obtain the first content.

[0195] In an embodiment of the present application, the determination module 92 is further configured to identify a category of the first content or the first object, and determine the target function corresponding to the first object based on the identification result.

[0196] An embodiment of the present application provides a processing device 10, as shown in the figure, the processing device 10 includes a processor 101 and a display 102. Figure 10

[0197] ​The processor 101 is configured to obtain a first operation; determine a first object based on the first operation; the first object is located at a first position; obtain a second operation for the first object, the second operation being used to indicate a change in the position of the first object;

[0198] The display 102 is configured to display the second object following the input position of the second operation in response to the second operation;

[0199] The processor 101 is further configured to obtain a third operation for the first object;

[0200] The display 102 is further configured to display the first object at a second position in response to the third operation, the second position being different from the first position; and display a target identifier of a target function for the first object before the third operation for the first object is obtained, if parameter information meets a first condition, the parameter information being related to the first object.

[0201] The embodiments of the present application provide a processing device, which obtains a first operation; determines a first object based on the first operation; the first object is located at a first position; obtains a second operation for the first object, the second operation being used to indicate a change in the position of the first object; displays the first object following the input position of the second operation in response to the second operation; obtains a third operation for the first object; displays the first object at a second position in response to the third operation, the second position being different from the first position; and displays a target identifier of a target function for the first object before the third operation for the first object is obtained, if parameter information meets a first condition, the parameter information being related to the first object. The processing device provided by the present application links the operation performed on the first object with the target function that the first object can perform, and can recommend the target function that the first object can perform in real time in the process of meeting the operation performed by the first object, and then performs the corresponding target function, so as to reduce the steps of triggering and improve the processing efficiency.

[0202] The embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores one or more computer programs, the one or more computer programs can be executed by one or more processors to implement the above processing method. The computer readable storage medium can be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (flash memory), a hard disk (Hard Disk Drive, HDD) or a solid state disk (Solid-State Drive, SSD); it can also be a respective device including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0203] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0204] The present application is described with reference to flowcharts and / or block diagrams according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0205] These computer program instructions can also be stored in a computer readable memory that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0206] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide the function of implementing the processes specified in the flowchart Figure 1 flowchart or multiple flowcharts and / or blocks Figure 1 flowchart or multiple flowcharts and / or blocks

[0207] The above description is merely illustrative of the application, and the scope of the application is not limited to the specific embodiments described herein. Any changes or modifications that can be made to the application by any person skilled in the art within the technical scope of the application should be encompassed within the scope of the application. Therefore, the scope of the application should be determined by the scope of the claims.

Claims

1. A processing method comprising: Get the first operation; determining a first object based on the first operation; The first object is located at a first position; Obtaining a second operation for the first object, where the second operation is used to instruct to change a position of the first object; In response to the second operation, the first object is displayed along with the input position of the second operation; Obtaining a third operation for the first object; In response to the third operation, the first object is located at a second position, the second position being different from the first position; Before obtaining the third operation for the first object, if the parameter information satisfies a first condition, displaying a target identifier for a target function for the first object; The parameter information is related to the first object.

2. According to the processing method according to claim 1, the target function can call an intelligent model to process the first content corresponding to the first object.

3. The processing method according to claim 1, wherein the target identifier for displaying the target function for the first object comprises one of the following: Displaying a first display area, the first display area including a target identifier for a target function of the first object, the first display area including a first display position; the target identifier being displayed at the first display position; or; Displaying a first display area, the first display area including indicator marks of multiple functions, the target mark of the target function being one of the indicator marks of the multiple functions; a display effect of the target mark being different from display effects of other indicator marks, the first display area including the first display position; the target mark being displayed at the first display position; or; A target identifier for a target function of the first object is displayed at the second position, where the second position is an input position for the second operation.

4. The processing method according to claim 1 or 3, wherein the parameter information satisfies the first condition, including one of the following: The first object moves to a preset distance within the first display area according to the input position corresponding to the second operation; or; The distance that the first object moves along with the input position corresponding to the second operation exceeds a preset distance threshold.

5. The processing method according to claim 1 or 3, wherein the parameter information satisfies the first condition, including one of the following: When the first operation is performed on the first object, the window corresponding to the first object is a preset window; the preset window indicates that the first object can obtain a storage path corresponding to the first content; or; When the first operation is performed on the first object, the type corresponding to the first object belongs to a preset type.

6. The processing method according to claim 5, wherein the window corresponding to the first object belongs to a preset window, comprising: The window process of the window corresponding to the first object is a target process, and the window scene to which the window corresponding to the first object belongs is a desktop scene or a resource manager scene.

7. The processing method according to claim 5, further comprising: Obtaining the window corresponding to the first object; Determining a path acquisition method for the first content based on a window scene to which the window corresponding to the first object belongs; The path acquisition method is used to determine a storage path of the first object based on the window; the storage path can acquire the corresponding first content.

8. The processing method according to claim 5, before displaying the target identifier of the target function for the first object, the method further comprises: Identify the category of the first content or the first object, and determine a target function corresponding to the first object based on the identification result.

9. A processing device comprising: An acquisition module, configured to obtain a first operation; a determining module, configured to determine a first object based on a first operation; The first object is located at a first position; The acquisition module is further configured to obtain a second operation for the first object, where the second operation is configured to instruct a change in the position of the first object; a response module, configured to, in response to the second operation, display the second object along an input position of the second operation; The acquisition module is further configured to obtain a third operation on the first object; The response module is configured to respond to the third operation, wherein the first object is located at a second position, the second position being different from the first position; a display module, configured to display a target identifier of a target function for the first object if parameter information satisfies a first condition before obtaining the third operation for the first object; The parameter information is related to the first object.

10. A processing device comprising: processor and display; The processor is configured to obtain a first operation; determining a first object based on the first operation; The first object is located at a first position; Obtaining a second operation for the first object, where the second operation is used to instruct to change a position of the first object; The display is configured to display the second object along with an input position of the second operation in response to the second operation; The processor is further configured to obtain a third operation for the first object; The display is further configured to respond to the third operation, wherein the first object is located at a second position, the second position being different from the first position; Before obtaining the third operation for the first object, if the parameter information satisfies a first condition, displaying a target identifier for a target function for the first object; The parameter information is related to the first object.

Images