Method, apparatus, device, medium and program product for providing reply

Abstract

The embodiment of the invention provides a method and device for providing reply, equipment, a storage medium and a computer program product. The method includes displaying a marker associated with a target object in a user interface above a target file, the target object being determined by a target model according to content of the target file. The method further includes displaying a description and an input component for the target object in a preset display area in response to receiving the user interaction for the marker, the description being determined by the target model according to the content and the target object. The method further includes, in response to receiving the user input based on the input component, displaying a reply to the user input, the reply determined by the target model from the content and the user input. According to the method disclosed by the embodiment of the invention, the user can be assisted in understanding the content in the target document when the user reads the target document, and the user can communicate with the model on the basis of displaying the document, so that the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of computers, and more particularly to a method, apparatus, device, computer-readable storage medium, and computer program product for providing a reply. BACKGROUND

[0002] At present, machine learning models are developing rapidly. After training on a large amount of text data, these models have formed strong language understanding and generation capabilities, and can perform multiple tasks such as translation, writing, and question answering, and build a cognitive system across multiple dimensions such as language, vision, and logic. For example, at the semantic analysis level, large models can accurately disassemble the contextual meaning of words, not only providing standard dictionary-level definitions, but also inferring emotional colors and industry-specific semantics in combination with the context.

[0003] For another example, at the text understanding level, large models have deep semantic association capabilities and can perform overall analysis on long documents and multi-modal content, not only extracting the core event context of news reports and the research framework and conclusions of academic papers, but also extracting operation steps from technical manuals to form a clear and logical guidance plan according to user needs. Such mature cognitive and processing capabilities make large models an "intelligent extension" of human cognition of the world. This human-computer interaction mode changes the information processing and knowledge output mode. The combination of machine learning models and prompt words has an impact on multiple industries such as education and science, and promotes the development of related fields. SUMMARY

[0004] According to an example embodiment of the present disclosure, a method, apparatus, device, computer storage medium, and computer program product for providing a reply are provided.

[0005] In a first aspect of the present disclosure, a method for providing a reply is provided, the method comprising displaying, in a user interface over a target file, a marker associated with a target object, the target object being determined by a target model according to content of the target file. The method further comprises, in response to receiving a user interaction for the marker, displaying, in a preset display area, a description for the target object and an input component, the description being determined by the target model according to the content and the target object. The method further comprises, in response to receiving a user input based on the input component, displaying a reply for the user input, the reply being determined by the target model according to the content and the user input.

[0006] In a second aspect of the disclosure, there is provided an apparatus for providing a reply, the apparatus comprising a marker display module configured to display a marker associated with a target object in a user interface above a target file, the target object being determined by a target model according to content of the target file. The apparatus further comprises a description and component display module configured to display a description for the target object and an input component in a preset display area in response to receiving a user interaction for the marker, the description being determined by the target model according to the content and the target object. The apparatus further comprises a reply display module configured to display a reply for a user input in response to receiving the user input based on the input component, the reply being determined by the target model according to the content and the user input.

[0007] In a third aspect of the disclosure, there is provided an electronic device comprising: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method described according to the first aspect of the disclosure.

[0008] In a fourth aspect of the disclosure, there is provided a computer-readable storage medium having stored thereon machine executable instructions which, when executed by a device, cause the device to perform the method described according to the first aspect of the disclosure.

[0009] In a fifth aspect of the disclosure, there is provided a computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method described according to the first aspect of the disclosure.

[0010] The summary is provided to introduce a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the disclosure will be apparent from review of the disclosure below. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figures 1A-1C A schematic diagram illustrating an example environment in which embodiments of the disclosure can be implemented is shown;

[0012] Figure 2 A flowchart of a method for providing a reply according to embodiments of the disclosure is shown;

[0013] Figure 3 A schematic diagram illustrating providing a description according to displayed content according to embodiments of the disclosure is shown;

[0014] Figure 4A schematic diagram showing a display description according to an embodiment of the disclosure is shown;

[0015] Figure 5 A schematic diagram showing a recommended term according to an embodiment of the disclosure is shown;

[0016] Figure 6 A schematic diagram showing a collaboration interface according to an embodiment of the disclosure is shown;

[0017] Figure 7 A schematic block diagram of an example apparatus according to some embodiments of the disclosure is shown;

[0018] Figure 8 A block diagram of an example device that can be used to implement embodiments of the disclosure is shown.

[0019] In all the drawings, like or similar reference numerals refer to like or similar elements. DETAILED DESCRIPTION

[0020] The names of the messages or information exchanged between the plurality of apparatuses in the embodiments of the disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information. It can be understood that, before using the technical solutions disclosed in the embodiments of the disclosure, the type, use range, use scenario, etc. of the personal information involved in the disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0021] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide personal information to the electronic device, application program, server or storage medium, etc. software or hardware performing the operation of the technical solutions of the disclosure according to the prompt information. As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending a prompt information to the user may, for example, be a pop-up window manner, and the prompt information may, for example, be presented in the form of text in the pop-up window. In addition, the pop-up window may, for example, carry a selection control for the user to select “agree” or “disagree” to provide personal information to the electronic device.

[0022] It can be understood that the above notification and obtaining of user authorization process is only illustrative, and does not limit the implementation manner of the disclosure, and other manners meeting relevant laws and regulations can also be applied to the implementation manner of the disclosure.

[0023] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein, but rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of the present disclosure.

[0024] In the description of embodiments of the present disclosure, the term "includes" and its derivatives, such as "including," should be understood in an open-ended way, that is, "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," and the like can refer to different or identical objects unless otherwise specified. Other explicit and implicit definitions can also be included below.

[0025] When a user is reading a document, he / she often encounters some difficult-to-understand content, such as rare words, ancient words, complex images, etc. When the user uses a machine learning model to understand, he / she needs to input these difficult-to-understand content into an application that specifically calls the machine learning model to reply, and then generate an answer from the other application. It is not possible to understand the content on the basis of the original document and communicate with the machine learning model, which reduces work efficiency and user experience.

[0026] To this end, the present disclosure proposes a method for providing a reply. The method actively analyzes the content of a target file to identify target objects that a user can have difficulty understanding, and generates a description for the target objects, and displays a prominent marker in the user interface on the target file, so that the user can immediately display the description of the object when he / she wants to understand the information of the target object without switching files. This can assist the user in understanding the content in the target document during the user's reading of the target document, and allow the user to communicate with the model during the presentation of the file content, which improves the user's reading efficiency and enhances the user experience.

[0027] Embodiments of the present disclosure will be described in further detail below with reference to the accompanying drawings, in which Figures 1A-1C A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown. In the example environment, an application program 110 and a large model application program 120 are included. The application program 110 can be an application program specifically used for reading a target file, which has basic file browsing, page turning, zooming, etc. The large model application program 120 can be an application program deployed on a local device that can call a large model. In this embodiment, the method of the embodiments of the present disclosure is performed by the large model application program 120.

[0028] AsFigure 1A As shown, a user can open a target file in the application 110, which has content 112. It can be seen that in this embodiment, the target file is an article, and the content 112 has many medical, health, and other field jargons that an ordinary user can not understand each word in the article. In some embodiments, the large model application 120 can listen to the user's operations in real time, such as through an event listening interface provided by the operating system, to detect whether the user opens a specific type of file, such as a txt, pdf, doc, and other document formats, and specific image formats, which are files that the large model application 120 can provide auxiliary reading support for. In some embodiments, if the large model application 120 detects that the user opens the target file, the large model application 120 can pop up an information box to prompt the user whether auxiliary reading is needed. Although Figure 1A "Let me help you explain the difficult words?" is shown in the figure, but the method of this embodiment is not limited to providing a description for the text content, and in fact, the method of this embodiment is widely applicable, not only limited to explaining difficult words in the text content, but also can provide targeted descriptions for other modalities of content such as difficult images and complex formulas. In some embodiments, the modality of the target object includes at least one of a text modality and an image modality.

[0029] In some embodiments, the large model application 120 can determine the target object in the target file and the description for the target object, which are determined by the target model according to the content of the target file. In some embodiments, the description for the target object is determined by a background device deployed with the target model according to the content of the target file and the target object. In some embodiments, after detecting that the target file is opened, the large model application 120 can transmit the content of the target file to the background device deployed with the target model through a secure and stable network connection, such as a network channel based on the HTTPS protocol. The target model, for example, is a language model trained on a large amount of data, which has strong semantic understanding and object recognition capabilities. After the background device receives the content of the target file, the target model starts to analyze the content in depth, and identifies target objects that can be difficult for the user to understand from text and image information through natural language processing technology and image recognition technology, such as professional words such as "anemia", "osteoporosis", "vagus nerve", and "cortisol", which are typical target objects. Then, the target model generates detailed and accurate description information according to the knowledge system obtained through training and the understanding of the content of the target file in combination with the target object. After obtaining the target object and the corresponding description, the background device encapsulates these data and sends them back to the large model application 120 through the same network connection, so that the large model application 120 performs subsequent display operations.

[0030] like Figure 1B As shown, the large model application 120 can generate a user interface for carrying a mark associated with the target object. In some embodiments, the user interface is designed with a transparent interface 121 of the same size as the target file. This design is based on graphics rendering technology. By setting the transparency attribute of the interface to be completely transparent, the user can hardly feel the existence of the interface visually, thereby not affecting the normal reading of the target file. In other embodiments, the user interface can be a transparent interface 121 with other transparencies. In some embodiments, the user interface can also be a non-transparent interface 121. For example, the large model application 120 can generate a non-transparent interface 121 at the location of the target object to carry a mark associated with the target object. For the convenience of description, the transparent interface 121 is used as an example for description below.

[0031] In some embodiments, the large model application 120 can display a marker associated with the target object in a transparent interface 121 above the target file. Based on the identified objects "anemia," "osteoporosis," "vagus nerve," and "cortisol," the large model application 120 can generate markers 122, 124, 126, and 128 in the transparent interface 121. In some embodiments, the marker is an underline. After the transparent interface 121 is superimposed on the interface of the target file, marker 122 is the underline of the object "anemia," marker 124 is the underline of the object "osteoporosis," marker 126 is the underline of the object "vagus nerve," and marker 128 is the underline of the object "cortisol." In some embodiments, the marker is a rectangular box covering the target object. The line below the rectangular box is displayed, while the other three lines of the rectangular box are not displayed, resulting in the rectangular box appearing as an underline. This rectangular box design has a stronger interactive advantage. Because the rectangular frame covers a certain area of ​​the target object, when the user's mouse pointer hovers over the rectangular frame for a time exceeding a preset threshold, the large model application 120 can determine that the user desires to display a description of the object through the mouse event monitoring interface provided by the operating system. Compared with the traditional underline format, the user does not need to accurately align the underline position, which greatly improves user reading efficiency, reduces the difficulty of interactive operations, and enhances the user experience. In some embodiments, all four sides of the rectangular frame can be displayed, and when the user performs a user interaction operation on an object in the rectangular frame, the rectangular frame can be highlighted.

[0032] like Figure 1CThe description can be information explaining the meaning of the target object (e.g., “osteoporosis”), which can include, for example, the definition of “osteoporosis”, the main causes of “osteoporosis”, the common symptoms of “osteoporosis”, and the like. As shown, if the user does not understand the meaning, main causes, and common symptoms of “osteoporosis” and expects to display the description for “osteoporosis”, the user can issue a user interaction for “osteoporosis”, so that the meaning, main causes, and common symptoms of “osteoporosis” can be easily understood. In embodiments in which the user interface is a transparent interface 121, the user cannot see the transparent interface 121 at all, as if the mark is in the target file, which can greatly facilitate the user's reading and does not block the content in the target file. Figure 1C

[0033] In some embodiments, the input component 132 integrates intelligent input method auxiliary functions such as auto-completion, spelling check, and grammar tips. The auto-completion function extracts high-frequency words and professional terms by analyzing the text content of the target file, and when the user inputs part of the characters, a drop-down list is displayed below the input box to show the relevant candidate words. In some embodiments, the target object and the description for the target object are determined based on the level of the user, and the level of the user is determined based on the history of the user's questions. The history of the user's questions can be temporarily stored on the server, and the level of the user can be classified as, for example, a layman, an experienced person, an expert, and the like. The target object thus determined can more efficiently improve the user's reading efficiency and enhance the user experience.

[0034] In some embodiments, if an input question 134 (i.e., user input) is received based on the input component 132, a target answer 136 is obtained and displayed, where the target answer 136 is determined by a language model based on the input question 134 and the content of the target file. For example, a launcher can be set in the large model application 120, which is used to call the language model to process the user input and the output of the language model. In some embodiments, when the user inputs a question, the question can be cleaned to remove noise data such as special characters and redundant spaces, and then the key words and key semantic information in the question can be extracted using natural language processing techniques such as word segmentation, part-of-speech tagging, and named entity recognition. In some embodiments, a connection is established with a language model server deployed in the cloud or locally through a network communication protocol, and the preprocessed question and the relevant content of the target file (context fragments extracted from the target file according to the key words of the question) are sent to the language model together.​

[0035] Generating the target reply 136 with reference to the content of the target file can make the generated reply more accurate. For example, a certain word has different meanings in different contexts, and for a question containing the word, the meaning of the word should be taken as the meaning consistent with the context in the reading of the target file, so that the generated target reply is more accurate. In some embodiments, an attention mechanism can be used in the language model to focus on the context paragraphs in the target file related to the question, calculate the semantic correlation between words, and ensure that the target reply is closely related to the question and consistent with the context of the target file.

[0036] In some embodiments, the target reply 134 is displayed in the pop-up window 130. In some embodiments, when the reply content is long, a virtual scrolling technique is used to render only the text content in the currently visible area, reducing memory usage and rendering pressure, while displaying a scroll bar on the right side, which can be scrolled by the user through the mouse wheel or touch sliding. In addition, in some embodiments, in the reply content, for content related to the specific location of the target file (such as "see page 3, paragraph 2"), a hyperlink is set, which can automatically jump to the corresponding location of the target file when clicked by the user, facilitating the user to further review the relevant information.

[0037] The technical solution improves the user experience when reading complex content by automatically scanning and analyzing the content of the target file, accurately identifying text words, complex images, complex charts and other target objects that the user may have difficulty understanding using natural language processing and other technologies. Then, detailed explanations of the corresponding target objects are generated. At the same time, in the user interface of the target file, the target objects are marked in a conspicuous form such as underlining and highlighting. When the user is interested in the marked content, only simple operations such as hovering and clicking are required, without the need to switch files or jump to applications, the user can instantly view the explanation of the target object, and communicate based on the displayed content of the file, thereby seamlessly obtaining auxiliary information during reading, significantly improving reading fluency and user experience.

[0038] It should be understood that the architecture and functionality in the example environment are described for illustrative purposes only and are not meant to limit the scope of the present disclosure. Embodiments of the present disclosure can also be applied to other environments with different structures and / or functionality.

[0039] The processes according to embodiments of the present disclosure will be described in detail below with reference to other drawings. For ease of understanding, the specific data mentioned in the following description are all exemplary and do not serve to limit the protection scope of the present disclosure. It can be understood that the embodiments described below can also include additional actions not shown or can omit the actions shown, and the scope of the present disclosure is not limited in this respect.

[0040] Figure 2 A flowchart of a method 200 for providing an answer according to certain embodiments of the present disclosure is shown. In this embodiment, the present method can be performed by the large model application 120. At block 202, a mark associated with a target object is displayed in a user interface over a target file, the target object being determined by a target model according to the content of the target file. The target file refers to a document that the user is preparing to read or is currently reading, which can be in the form of an e-book, an academic paper, a work report, etc. various text files, and can also contain images, formulas, and other non-textual content. The target object can be content in the target file that the user may have difficulty understanding, such as obscure vocabulary in a specialized field, ancient Chinese words, complex images, formulas, charts, etc. The target model may, for example, be a language model based on a transformer architecture. These models have powerful natural language understanding and generation capabilities, and can handle complex semantics and contextual information, showing unique advantages in analyzing the content of the target file. The mark can distinguish the target object from other content, for example, the mark can be an underline, a dotted line, etc.

[0041] For example, the user is reading a specific doc document, and with the user's consent, the target model can be called to determine the obscure vocabulary in the document. For example, when processing a medical paper, the target model can accurately identify "osteoporosis", "cortisol" and other professional terms as target objects through contextual understanding. For ancient Chinese words, the target model can identify difficult-to-understand words such as "taotie" and "aerosol" based on the learning of a large amount of ancient Chinese corpus. In addition, the target model can also be used to analyze the logical relationships and thematic context in the content of the target file in this step, and identify complex concept expressions in the text as target objects.

[0042] At block 204, in response to receiving the user interaction on the marker, display the description for the target object and an input component in a preset display area, the description being determined by the target model according to the content and the target object. The marker is a visual element on the user interface for identifying the target object, reminding the user that there is available auxiliary information here through different styles (such as underlining, highlighting, specific color, etc.). As described above, the user interface can be one of a non-transparent interface, a semi-transparent interface, and a fully transparent interface. In some embodiments, a prompt word for the description can be preset by the large model application, where the prompt word includes the content and the target object, and one or more of a basic definition, extended information, or a related link for the description. In some embodiments, the prompt word further includes format information of the basic definition, the extended information, and the related link. The basic definition for the target object is used to explain the meaning of the target object in the target file. The extended information for the target object can be information on the cause, effect, and function, such as the cause, symptoms (such as “pain, spinal deformity”), and prevention measures (such as “supplement calcium, proper exercise”). The description can also include a related link for the target object, which can jump to an authoritative medical website, a full text of a literature, or a related video explanation. The preset display area can be, for example, an area for displaying a pop-up window.

[0043] In some embodiments, the user interaction can be identified by a hover trigger, such as a mouse pointer staying at the target object for more than 300 ms (the threshold value can be configured), and the description is displayed. In some embodiments, the user interaction can be identified by a click trigger, such as a single click on the target object to pop up a pop-up window for the description. The input component is a component for facilitating user input of content, such as a text input box, a voice input box, an image embedding component, and the like.

[0044] At block 206, in response to receiving the user input based on the input component, display a reply for the user input, the reply being determined by the target model according to the content and the user input. After the user inputs the content through the input component, the user input can be sent to the target model (such as a language model), a corresponding reply is generated by the target model, and the reply is displayed in the preset display area, so that the user does not need to switch to another application when reading the file.

[0045] According to the method of embodiments of the present disclosure, by actively analyzing the content of the target file to identify target objects that the user can have difficulty understanding, and generating a description for the target object, and displaying a prominent marker in the user interface on the target file, the user can immediately display the description of the object when he or she wants to understand the information of the target object without switching files. This can assist the user in understanding the content in the target document during the user reading the target document, and allow the user to communicate with the model during the presentation of the file content, which improves the user's reading efficiency and enhances the user experience.

[0046] Figure 3 A schematic diagram of providing a description according to the displayed content is shown according to an embodiment of the present disclosure. At 302, the words and the description of the target file are determined by using a language model. The language model adopted is a deep learning model trained by a large amount of text data, such as a language model based on a transformer architecture. In some embodiments, the text content of the target file is taken as input, and the multi-layer attention mechanism inside the language model can perform deep analysis on the text, identifying professional terms, rare words, and other words that may be difficult to understand as target words. For each target word, the model generates a corresponding explanatory description according to its pre-trained knowledge and understanding of the current text.

[0047] In some embodiments, it is detected whether the target file is opened by a text processing application as the timing for performing the operation. In some embodiments, it is determined whether the target file is opened by a text processing application by listening to the file opening event of the text processing application. In some systems, a specific message is generated when the text processing application triggers a file opening operation by using the message mechanism provided by the operating system. After capturing the message, it is used as the timing for performing the operation of determining the words and the description by using the language model, of course, subject to the permission of the user. In some embodiments, the words and the description are determined if the target file is opened by a text processing application.

[0048] At 304, at least one relative position of the words is obtained, wherein the at least one relative position is determined by matching the words and the content. Taking “osteoporosis” as an example, assuming that the total length of the target file is N characters, when the algorithm searches for “osteoporosis” in the text, the position index of the starting and ending characters can be recorded. If “osteoporosis” appears twice in the text, at the 100th character to the 103rd character and the 200th character to the 203rd character respectively, in some embodiments, the at least one relative position is represented by at least one range object. In order to facilitate program processing and storage, the range object adopts an array form, such as “[100, 103]” to represent the position range of the first occurrence of “osteoporosis”, and “[200, 203]” to represent the position range of the second occurrence.

[0049] At 306, the displayed content of the target file is acquired. In this operation, the text processing application can be communicated with through a component of the text processing application to acquire the displayed content of the target file. In some embodiments, the text processing application can be communicated with through a component provided by the text processing application to acquire the displayed content of the target file. For example, for a target file in a doc format, the COM (Component Object Model) component technology is used to acquire the content currently displayed on the screen by calling the COM interface of the text processing application. In some other operating systems, an auxiliary function component is provided. The displayed content of the target file can be acquired through the auxiliary function component. In some embodiments, the displayed content is represented by a set of ranges (i.e., a first range). If the content from the 90th character to the 150th character is currently displayed on the screen, the first range is represented as “[90, 150]”.

[0050] At 308, it is detected whether the word is displayed, i.e., whether the displayed content includes the word. In some embodiments, it is detected whether the first range includes at least one range object. In some embodiments, this is achieved by specifically detecting whether the first range includes at least one range object. For example, for the target word “osteoporosis”, the location range objects are “[100, 103]” and “[200, 203]”, and the first range of the currently displayed content is “[90, 150]”. By comparison, it can be found that “[100, 103]” is completely contained in “[90, 150]”, indicating that the word is in the currently displayed content. If none of the range objects is contained in the first range, it is considered that the displayed content does not include the word, and the flow ends at 310.

[0051] If the displayed content includes the word, at 312, a transparent interface is superimposed on the target file, and the position and size thereof are set to completely coincide with the display area of the target file. Further, if the displayed content includes the word, at 314, a mark is displayed at the position of the user interface corresponding to the target object. In some embodiments, the mark is a colored mark to enhance the visual prompting effect. When the mark is displayed, according to the previously determined location range of the target word, the mark is drawn at the corresponding position of the transparent interface using a graphics drawing function. In some embodiments, if the first range includes a target range object in the at least one range object, the display coordinates of the target range object are determined using a target component, and the display coordinates are pixel coordinates of the target object on the screen. The target component can be a COM component or an auxiliary function component. In some embodiments, the mark is displayed on the user interface according to the display coordinates, and the mark is a colored mark. Thus, one auxiliary reading is completed.

[0052] Figure 4 A schematic diagram showing displaying the description is shown. At 400, a mark is displayed on the transparent interface. This operation is consistent with the operation at 314 of FIG. 3, and thus is not described again. At 402, it is detected whether a user interaction is received. If no user interaction is received, the user interaction is continuously detected. In some implementations, to avoid misjudgment due to mouse jitter, a filtering algorithm can be used to process the coordinate information of the mouse event. For example, a sliding average filtering algorithm is used to average the coordinate data in the continuous multiple messages, and only when the filtered coordinates are continuously and stably in the region where the mark is located for multiple times, it is determined as an effective hover interaction. In some implementations, for a multi-screen display environment, the screen where the mark is located can be automatically identified, and only the mouse event on the screen is effectively monitored to prevent the operation of other screens from interfering with the interaction detection. Figure 3 If the user interaction is received, at 406, a pop-up window is displayed on the preset display area, and the pop-up window displays the description and an input box. In some embodiments, when generating the pop-up window, a transition animation such as fade-in and fade-out, sliding, etc. can be added to the pop-up and disappearance of the pop-up window using an animation library to enhance the user experience. In some embodiments, for the description content in the pop-up window, the text can be segmented and typeset according to the semantic structure of the text, and different font styles and paragraph spacing are used to display the text formats such as title, body, list, etc. For example, the title uses bold and large font, the body uses regular font, and the list item uses indented format, so that the user can quickly and clearly read the description information and understand the meaning of the word and other content. The input box can be a text input box, for example, through which the user can communicate with the target model.

[0053] At 408, it is detected whether the displayed content is changed. For example, the COM component can be continuously used to obtain the displayed content in the target file, and it is compared whether the displayed content is the same as the last displayed content. If not, it means that the displayed content is changed. If the displayed content has been changed, the previous mark is no longer accurate, and thus at 412, the displayed content of the target file is obtained. Subsequently, it is continuously judged whether the word is in the displayed content. If the word is in the displayed content, the mark is displayed on the transparent interface accordingly. If the displayed content is not changed, the flow ends at 410.

[0054]

[0055] Figure 5 ​A schematic diagram of recommending a term according to an embodiment of the present disclosure is shown. A recommendation button 516 is also provided in the pop-up window displayed on the target file 510. If the recommendation button 516 is triggered, a first recommended object for replacing the target object is obtained, wherein the first recommended object is determined by the target model based on the content and the target object. For example, after the user clicks the recommendation button 516, a request 512 “recommendation” can be automatically sent. The target model can recommend a term to replace the currently interacted object “osteoporosis”. For example, the model’s reply 514 can be “the context of this file mainly talks about x problem, therefore it is recommended to use the term “osteosclerosis” to replace “osteoporosis”. A confirmation button 518 is displayed below the reply 514.

[0056] If the user confirmation is received, for example, the user clicks the confirmation button 518, the first recommended object is used to replace the target object in the target file 510. For example, “osteosclerosis” can be used instead of “osteoporosis” throughout the file, thereby achieving one-click replacement. This can help the user optimize the file content and improve the user experience. In some embodiments, if the recommendation button 516 is triggered again, a second recommended object for replacing the target object is obtained, wherein the second recommended object is determined by the target model based on the content, the target object and the first recommended object. If the user is not satisfied with the first recommended “osteosclerosis”, the recommendation button 516 can be clicked again to obtain a second recommended object. If the user confirmation is received, the second recommended object is used to replace the target object in the target file. For example, if the user is satisfied with the second recommended object, the confirmation button 518 can be clicked to achieve one-click replacement.

[0057] In some embodiments, if the target object has a misspelling, a modification suggestion and an accept button are displayed in a preset display area, wherein the modification suggestion is generated by the target model. For example, the target object can also be a word with a misspelling in the file, which can be highlighted by a marker. When the user interacts with the marker, the modification suggestion for the object and the accept button can be displayed in a pop-up window (i.e. the preset display area). In some embodiments, if the accept button is triggered, the misspelling is modified according to the modification suggestion. The application can replace the misspelling according to the content of the modification suggestion, thereby correcting the error. In this embodiment, the model can be used to check the errors in the file content, thereby improving the processing efficiency and user experience.

[0058] In some embodiments, if the user enters the keyword "continue" in the input component, the launcher can call the language model to perform the continuation task based on the content of the target file to enrich the content of the target file. In some implementations, the continued content is displayed in a pop-up window, or is displayed on the user interface, or is input into the target file. In other implementations, after the user opens the target file, a separate pop-up window can be displayed to prompt whether the document needs to be continued. If the user confirms that the document needs to be continued, the launcher can call the language model to perform the continuation task based on the content of the target file, otherwise no continuation will be performed.

[0059] Figure 6 Schematic diagram of a collaboration interface according to an embodiment of the present disclosure is shown. Figure 6 As shown, the large model application 120 may include an interactive interface 602 for setting which applications (such as application A and application B) the method of the embodiment of the present disclosure can be implemented on. The interactive interface 602 can list all applicable applications and further distinguish between the "line display entries in the application" function column 606 and the "display AI continuation document prompt" function column 608. A selection box may be provided on the right side of the function column. If it is selected, it means that the user agrees to execute the function, and if it is not selected, it means that the user does not agree to execute the function. In some embodiments, the collaborative interface can be used with Figure 1A If the user selects "OK" in the information box, that is, agrees to execute the non-disclosed embodiment for the target file, the function column 606 is automatically set to the selected state. If the user selects "Ignore" in the information box, that is, disagrees to execute the non-disclosed embodiment for the target file, the function column 606 is automatically set to the unselected state. In this embodiment, the collaboration interface is consistent with the initial pop-up window ( Figure 1A The user's immediate selection of "Assisted Reading" ("OK" or "Ignore") will be automatically synchronized to the selection status of the function column (606 selected / unselected), reducing repeated operations and improving configuration efficiency.

[0060] Figure 7 FIG2 shows a schematic block diagram of an example apparatus 700 according to some embodiments of the present disclosure. The apparatus 700 may be implemented in software, hardware, or a combination of both. Figure 7 As shown, the apparatus 700 includes a tag display module 710 , a description and component display module 720 , and a reply display module 730 .

[0061] In some embodiments, the mark display module 710 can be configured to display the mark associated with the target object in the user interface over the target file, the target object being determined by the target model according to the content of the target file. The description and component display module 720 can be configured to display the description for the target object and the input component in a preset display area in response to receiving the user interaction for the mark, the description being determined by the target model according to the content and the target object. The reply display module 730 can be configured to display the reply for the user input in response to receiving the user input based on the input component, the reply being determined by the target model according to the content and the user input.

[0062] In some embodiments, the mark display module 710 includes a first obtaining module configured to obtain at least one relative position of the target object, the at least one relative position being determined by matching the target object and the content. The mark display module 710 further includes a first acquiring module configured to acquire the displayed content of the target file. The mark display module 710 further includes a second mark display module configured to display the mark at a position of the user interface corresponding to the target object in response to the displayed content including the target object.

[0063] In some embodiments, the displayed content is represented by a first range, the at least one relative position is represented by at least one range object, and it is detected whether the at least one range object is included in the first range.

[0064] In some embodiments, the second mark display module includes a second determining module configured to determine display coordinates of a target range object of the at least one range object using the target component in response to the first range including the target range object. The second mark display module further includes a third mark display module configured to display the mark in the user interface according to the display coordinates, the mark being a colored mark.

[0065] In some embodiments, the mark is a rectangular frame covering the target object, and a lower line of the rectangular frame is displayed.

[0066] In some embodiments, the apparatus 700 further includes a third determining module configured to determine that the user interaction is received in response to the position indicating that the mark hovers over the rectangular frame for more than a threshold value.

[0067] In some embodiments, the target object is determined by the target model according to the content of the target file, and the description is determined by the target model according to the content of the target file and the target object.

[0068] In some embodiments, the target object and the description for the target object are determined based on a level of the user, the level of the user being determined based on historical questions raised by the user.

[0069] In some embodiments, the description and component display module 720 includes a popup module configured to display a popup on a preset display area in response to receiving a user interaction on the tag, wherein the popup displays the description and an input box.

[0070] In some embodiments, the popup includes a recommendation button, and the apparatus 700 further includes a second acquisition module configured to acquire a first recommended object for replacing the target object in response to the recommendation button being triggered, wherein the first recommended object is determined by the target model based on the content and the target object. The apparatus 700 further includes a first replacement module configured to replace the target object with the first recommended object in the target file in response to receiving a user confirmation.

[0071] In some embodiments, the apparatus 700 further includes a third acquisition module configured to acquire a second recommended object for replacing the target object in response to the recommendation button being triggered again, wherein the second recommended object is determined by the target model based on the content, the target object and the first recommended object. The apparatus 700 further includes a second replacement module configured to replace the target object with the second recommended object in the target file in response to receiving a user confirmation.

[0072] In some embodiments, the apparatus 700 further includes a second display module configured to display a modification suggestion and an accept button in a preset display area in response to the target object having a wrong character, wherein the modification suggestion is generated by the target model. The apparatus 700 further includes a modification module configured to modify the wrong character according to the modification suggestion in response to the accept button being triggered.

[0073] In some embodiments, the modality of the target object includes at least one of a text modality and an image modality.

[0074] In some embodiments, the apparatus 700 further includes a detection module configured to detect whether the target file is opened by a text processing application. The apparatus 700 further includes a fourth determination module configured to determine the target object and the description in response to the target file being opened by the text processing application.

[0075] In some embodiments, the apparatus 700 further includes a setting module configured to set a prompt word for the description, wherein the prompt word includes one or more of a basic definition for the description, extended information, or a related link.

[0076] The division of modules or units in the embodiments of the present disclosure is illustrative, and is only a logical function division. When actually implemented, another division manner can be used. In addition, each functional unit in the disclosed embodiments can be integrated into one unit, or can be physically separated, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0077] Figure 8 A block diagram of an example device 800 that can be used to implement embodiments of the present disclosure is shown. It should be understood that Figure 8 The device 800 shown is merely an example and should not be construed as limiting the functionality and scope of the implementations described herein.

[0078] As Figure 8 The device 800 is in the form of a general-purpose computing device, as shown. Components of the device 800 can include, but are not limited to, one or more processors or processing units 810, a memory 820, a storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 can be a real or virtual processor and can execute various processing in accordance with programs stored in the memory 820. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the device 800.

[0079] The device 800 typically includes a number of computer storage media. Such media can be any available media that is accessible by the device 800 and includes both volatile and non-volatile media, removable and non-removable media. The memory 820 can be a volatile memory (such as a register, cache, random access memory (RAM)), a non-volatile memory (such as read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable medium and can include machine-readable media such as a flash drive, a disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and that can be accessed by the device 800.

[0080] The device 800 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown, the device 800 can further include a network interface device that can be used to communicate with other devices over a network. Figure 8As shown in FIG. 8, a disk drive 822 or other computer readable media drive can be provided for reading from or writing to a removable, non- volatile, magnetically encoded disk (e.g., a "floppy disk"), and an optical disk drive 824 can be provided for reading from or writing to a removable, non- volatile, magnetically encoded disk (e.g., a "floppy disk"). In such cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 820 can include a computer program product 825 having one or more program modules configured to carry out the various methods or actions of the various implementations of the present disclosure.

[0081] The communication unit 840 enables communications with other computing devices over a communication medium. Additionally, the functionality of the components of the device 800 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating over a communication connection. Thus, the device 800 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0082] The input device 850 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 860 can be one or more output devices, such as a display, a speaker, a printer, etc. The device 800 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc., through the communication unit 840, as needed, with one or more devices that enable a user to interact with the device 800, or with any devices (e.g., a network card, a modem, etc.) that enable the device 800 to communicate with one or more other computing devices. Such communication can be carried out via an Input / Output (I / O) interface (not shown).

[0083] According to example implementations of the present disclosure, a computer readable storage medium is provided having computer executable instructions stored thereon, where the computer executable instructions are executed by a processor to implement the methods described above. According to example implementations of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the methods described above. According to example implementations of the present disclosure, a computer program product is provided having a computer program stored thereon, which program is executed by a processor to implement the methods described above.

[0084] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0085] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0086] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0087] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0088] While various implementations of the present disclosure have been described above, the foregoing descriptions are illustrative and non-exhaustive, and are not intended to limit the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for providing a response, comprising: Displaying a markup associated with a target object in a user interface above the target document, the target object being determined by the target model based on the content of the target document; In response to receiving a user interaction with the tag, displaying a description and an input component for the target object in a preset display area, the description being determined by the target model based on the content and the target object; as well as In response to receiving a user input based on the input component, a reply to the user input is displayed, the reply being determined by the target model based on the content and the user input.

2. The method of claim 1 , wherein displaying a tag associated with the target object in a user interface above the target file comprises: obtaining at least one relative position of the target object, wherein the at least one relative position is determined by matching the target object with the content; Obtaining the displayed content of the target file; as well as In response to the displayed content including the target object, the mark is displayed at a position of the user interface corresponding to the target object. 3 . The method according to claim 2 , wherein the displayed content is represented by a first range, the at least one relative position is represented by at least one range object, and the at least one range object is detected as being included in the first range.

4. The method according to claim 3, wherein in response to the displayed content including the target object, displaying the mark at a position corresponding to the target object on the user interface comprises: In response to the first range including a target range object in the at least one range object, determining display coordinates of the target range object using a target component; as well as The mark is displayed on the user interface according to the display coordinates, wherein the mark is a colored mark. The method according to claim 1 , wherein the mark is a rectangular frame covering the target object, and a line below the rectangular frame is displayed.

6. The method according to claim 5, further comprising: In response to a time period during which the position indication mark hovers over the rectangular frame exceeding a threshold, it is determined that the user interaction is received. 7 . The method according to claim 1 , wherein the target object is determined by the target model according to the content of the target file, and the description is determined by the target model according to the content of the target file and the target object. 8 . The method according to claim 1 , wherein the target object and the description for the target object are determined based on a user's level, and the user's level is determined based on historical questions asked by the user.

9. The method according to claim 1 , wherein in response to receiving a user interaction with the tag, displaying a description and an input component for the target object in a preset display area comprises: In response to receiving a user interaction with the tag, a pop-up window is displayed on the preset display area, wherein the pop-up window displays the description and the input box.

10. The method according to claim 9, wherein the pop-up window includes a recommendation button, and the method further comprises: In response to the recommendation button being triggered, obtaining a first recommended object for replacing the target object, wherein the first recommended object is determined by the target model based on the content and the target object; as well as In response to receiving user confirmation, the target object is replaced in the target file with the first recommended object.

11. The method according to claim 10, further comprising: In response to the recommendation button being triggered again, obtaining a second recommended object for replacing the target object, wherein the second recommended object is determined by the target model based on the content, the target object, and the first recommended object; and In response to receiving user confirmation, the target object is replaced in the target file with the second recommended object.

12. The method according to claim 1, further comprising: In response to a typo in the target object, displaying a modification suggestion and an accept button in the preset display area, wherein the modification suggestion is generated by the target model; as well as In response to the accept button being triggered, the typo is modified according to the modification suggestion. 13 . The method according to claim 1 , wherein the modality of the target object comprises at least one of a text modality and an image modality.

14. The method according to claim 1, further comprising: detecting whether the target file is opened by a text processing application; as well as In response to the target document being opened by the text processing application, the target object and the description are determined.

15. The method according to claim 1, further comprising: A prompt word is set for the description, wherein the prompt word includes one or more of a basic definition, extended information, or an associated link for the description.

16. An apparatus for providing a response, comprising: a tag display module configured to display a tag associated with a target object in a user interface above the target file, the target object being determined by the target model based on the content of the target file; a description and component display module configured to display a description and input components for the target object in a preset display area in response to receiving a user interaction with the tag, the description being determined by the target model based on the content and the target object; and The reply display module is configured to, in response to receiving a user input based on the input component, display a reply to the user input, the reply being determined by the target model according to the content and the user input.

17. An electronic device comprising: at least one processing unit; At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 15.

18. A computer program product having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 15 when executed by a processor.

Images