Information interaction method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an information interaction method and device, equipment and a storage medium. The method comprises the following steps: in response to a consultation initiation operation, providing an interactive interface between a user and a digital assistant for health consultation of the user; receiving a target file input by a user, wherein the target file is in a non-text message format; determining whether the target file meets a health consultation requirement or not based on the interaction context of the user and the digital assistant; and if it is determined that the target file meets the health consultation requirement, determining an analysis result of the digital assistant for the target file based on the information included in the target file. Therefore, it can be effectively avoided that when the analysis result of the health file is provided under the condition that the file does not meet the requirement, it may be caused that the feedback result is inaccurate, misleading occurs, and the reliability and user friendliness of health consultation are affected.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for information interaction. Background Art

[0002] With the development of Internet technology, people are expected to enjoy remote and online services in health consultation. Online medical care can improve the supply of high-quality medical resources efficiently and at low cost, help medical institutions expand the scope of medical services, and allow consulting objects to obtain authoritative and professional medical guidance at any time. At present, some websites, terminal applications, etc. try to provide online health consultation services. However, in online health consultation, the interactivity and professionalism need to be improved. Summary of the invention

[0003] In a first aspect of the present disclosure, an information interaction method is provided. The method includes: in response to a consultation initiation operation, providing an interactive interface between a user and a digital assistant for health consultation of the user; receiving a target file input by the user, the target file being in a non-text message format; determining whether the target file meets the health consultation requirements based on the interaction context between the user and the digital assistant; and if it is determined that the target file meets the health consultation requirements, determining the analysis result of the digital assistant for the target file based on the information included in the target file.

[0004] In a second aspect of the present disclosure, a device for information interaction is provided. The device includes: an interface providing module, configured to provide an interactive interface between a user and a digital assistant in response to a consultation initiation operation, for health consultation of the user; a file receiving module, configured to receive a target file input by the user, the target file being in the format of a non-text message; a requirement judgment module, configured to determine whether the target file meets the health consultation requirements based on the interaction context between the user and the digital assistant; and an analysis determination module, configured to determine the analysis result of the digital assistant for the target file based on the information included in the target file if it is determined that the target file meets the health consultation requirements.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0007] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0010] FIG. 2A to FIG. 2D A schematic diagram showing an example of an interactive interface according to some embodiments of the present disclosure;

[0011] FIG. 3A to FIG. 3D A schematic diagram showing examples of interaction interfaces according to other embodiments of the present disclosure;

[0012] FIG. 4A to FIG. 4D A schematic diagram showing examples of interaction interfaces according to still other embodiments of the present disclosure;

[0013] Figure 5 A flowchart of a process for information interaction according to some embodiments of the present disclosure is shown;

[0014] Figure 6 A block diagram showing an apparatus for information interaction according to some embodiments of the present disclosure; and

[0015] Figure 7 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0018] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0019] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0020] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0021] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0022] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0023] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0024] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0025] During online health consultations, users may have some examination reports, medical records, and other materials that they hope to upload directly for consultation. However, most current model-based digital assistants do not support processing information in non-text message formats. In addition, in actual applications, users may upload non-medical health-related pictures / files, or the photos uploaded by users are not taken in a standardized manner, which may cause the model to be unable to accurately identify and provide useful feedback.

[0026] According to an embodiment of the present disclosure, an improved scheme for information interaction is proposed. In this scheme, during the interaction between a user and a digital assistant for health consultation, if the user inputs a file in a non-text message format, based on the interaction context between the user and the digital assistant, it is determined whether the file meets the health consultation requirements. In the case where it is determined that the file meets the health consultation requirements, the analysis results of the digital assistant for the file are determined based on the information extracted from the file. In an embodiment of the present disclosure, for files in non-text message formats, the context is combined to determine whether the file meets the requirements, and the files that meet the requirements are analyzed. This can effectively avoid providing analysis results of health files when the files do not meet the requirements, which may lead to inaccurate and misleading feedback results, affecting the reliability and user-friendliness of health consultation.

[0027] Some example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the example environment 100, an application 120 is installed in a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or an attached device of the terminal device 110.

[0029] In some embodiments, the application 120 can be downloaded and installed on the terminal device 110. In some embodiments, the application 120 can also be accessed through other means, such as through a web page. Figure 1In the environment 100, if the application 120 is started, the terminal device 110 can present the interface 150 of the application 120. The interface 150 may include various user interfaces that the application 120 can provide. The interface 150 is sometimes also referred to as a client interface.

[0030] In an embodiment of the present disclosure, the application 120 includes any appropriate application capable of providing health consultation services. In some embodiments, the application 120 can implement a dialogue interaction with the user 140 based on the digital assistant 125 to provide health consultation services to the user 140. In some embodiments, in addition to the interaction with the digital assistant 125, the application 120 can also provide other services, and the embodiments of the present disclosure are not limited in this respect.

[0031] In some embodiments, the terminal device 110 can communicate with the server 130 to provide services for the application 120 and / or the digital assistant 125. The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a television receiver, a radio broadcast receiver, an e-book device, a gaming device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.). The application 130 can be various types of computing systems / servers that can provide computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0032] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0033] In an embodiment of the present disclosure, the digital assistant 125 is used for interaction with the user. The interactive interface between the user and the digital assistant 125 is presented in the client interface, also referred to as an interactive window or a dialog window. As an intelligent assistant, the digital assistant 125 has intelligent dialogue and information processing capabilities. In the interactive interface, the user inputs an interactive message, and the digital assistant 125 provides a reply message in response to the user input. In the application 120, the digital assistant 125 can be started or awakened by an appropriate means (e.g., a shortcut key, a button, or voice) to present an interactive interface with the user. The interactive interface may include interface elements for information interaction, such as an input box, a message list, a message bubble, and the like.

[0034] In the embodiments of the present disclosure, the digital assistant 125 is used to provide services related to health consultation. Such a digital assistant 125 may sometimes also be referred to as a "digital doctor". Users can use simple text or voice input to send health consultation needs to the digital assistant 125, and can obtain real-time replies from the digital assistant 125 to conduct medical knowledge popular science questions and answers, health consultation, or pan-health management. In this way, the digital assistant 125 can be used to improve the supply of high-quality medical resources efficiently and at low cost, help medical institutions expand the coverage of medical services, and allow users to obtain authoritative and professional guidance at any time.

[0035] In some embodiments, the digital assistant 125 can use the model to determine the response to the user's question. In some embodiments, the model can be trained based on a general corpus and a special corpus for the medical field, so as to achieve accessible, credible, and understandable medical knowledge education and health consultation guidance. Special corpora in the medical field may include, for example, authoritative clinical guidelines, literature, textbooks, clinical doctor-patient communication question and answer data at home and abroad. In some embodiments, the training of medical safety-related mechanisms can be strengthened during model training, so that the model can have the ability to provide credible health consultation advice and pass comprehensive safety testing. In some embodiments, when evaluating the performance of the digital assistant 125, the digital assistant 125 can also be evaluated in terms of communication expression ability, humanistic care and friendliness, so that the digital assistant 125 can achieve empathy with users, understand user needs, and communicate in detail and patiently, which is conducive to alleviating the problems of short communication time and insufficient communication between doctors and patients.

[0036] The model can be deployed on the terminal device 110 or can be accessed by the terminal device 110. Alternatively, the terminal device 110 can communicate with the model to use the model to provide application management services for the user 140. For example, the terminal device 110 can send a prompt word input to the model based on the interaction information between the user 140 and the digital assistant 125, and obtain the processing result of the model. As an alternative or in addition, the digital assistant 125 itself can also be obtained based on the model. The digital assistant 125 can process the conversation message input by the user 140 to obtain the needs of the user 140, such as the questions that the user 140 expects to consult, etc. The model can be configured locally on the terminal device 110 or independently of the terminal device 110 according to the needs, and the present disclosure is not intended to limit this.

[0037] In an embodiment of the present disclosure, in response to a consultation initiation operation, an interactive interface between the user 140 and the digital assistant 125 is provided for health consultation of the user 140. The consultation initiation operation may be an operation initiated by the user 140 in any appropriate manner. The health consultation may be a health consultation with the user 140 as the consultation object, or may be other consultation objects specified by the user 140. In the interactive interface between the user 140 and the digital assistant 125, the user may be supported to input various types of interactive messages by text or voice. Voice input may be converted into a text message. The digital assistant 125 determines the corresponding reply based on the text input by the user.

[0038] In some cases, user 140 may input a file in a non-text message format for consultation. In an embodiment of the present disclosure, users are supported to input messages in multimodal data formats for interaction, including text message formats and non-text message formats. When user 140 inputs a file in a non-text message format, digital assistant 125 can also appropriately determine the reply and provide analysis results for the text. In this way, users can be supported to conduct medical consultation by uploading or photographing existing medical reports or other medical-related information, thereby improving interaction efficiency.

[0039] In the following, for a better understanding of the embodiments of the present disclosure, reference will be made to FIG. 2A to FIG. 2D , FIG. 3A to FIG. 3B as well as FIG. 4A to FIG. 4D The interactive interfaces in these figures may be provided to the user 140 by the terminal device 110. It should be understood that the interactive interfaces in these figures are merely examples, and various interface designs may actually exist. The various graphic elements in the interface may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may also exist. The embodiments of the present disclosure are not limited in this respect.

[0040] Figure 2A The interactive interface 200 may be presented to the user 140 in response to the consultation initiation operation. In the interactive interface 200, the digital assistant 210 may provide a message 210 to guide the user 140 to consult. The user 140 may input a text message 220 through the input box 225 to describe the question to be consulted in text. In some embodiments, the identification information of the current consultation object may also be displayed in the predetermined area 212 of the interactive interface. The consultation object may be the current user 140 or other users associated with the user 140, and there is no limitation here.

[0041] In the embodiments of the present disclosure, in addition to supporting users to input messages by text or voice, users are also supported to input files in non-text message formats. Files in "non-text message formats" may include various types of electronic files, including medical reports in electronic format or electronically scanned or photographed versions of paper medical reports.

[0042] like Figure 2A As shown, user 140 can also upload locally existing image files or video files through "Photo" control 230, take real-time image files or video files through "Photo" control 232, or upload locally existing other files, such as document files, txt files, PDF files, Excel files, or any other appropriate format files through "File" control 234. Figure 2A As shown, if the user selects the “Photo” control 230 , the user may choose to upload one or more local image or video files 236 to input into the interactive interface 200 .

[0043] After receiving the target file input by the user 140, if the target file is in a non-text message format, the terminal device 110 can determine whether the target file meets the health consultation requirements based on the interaction context between the user 140 and the digital assistant 125. Since the reply of the digital assistant 125 is determined by the operation of the model, if all messages or files input by the user are directly provided to the model for processing, when the file input by the user does not meet the requirements, the reply will not meet expectations and no useful feedback will be provided. Unlike users manually entering text messages or voice input, when shooting or uploading files, there may be unintentional mistakes such as wrong selections and inaccurate shooting. For such errors, it is expected that the digital assistant 125 can provide more intelligent feedback to guide users.

[0044] In some embodiments, when determining whether the target file meets the health consultation requirements, it can be determined whether the target file contains medical related information, for example, the target file's file name, metadata, or information parsed from the target file can be used to determine whether it corresponds to medical related information. In the overall context of health consultation, it is expected that the file input by the user 140 should be information associated with the health consultation to avoid the interaction with the digital assistant 125 being directed to an irrelevant direction.

[0045] In some embodiments, alternatively or additionally, when determining whether the target file meets the health consultation requirement, it can be determined that the type of information contained in the target file matches the type of information requested to be provided by the user 140 in the interaction context. For example, in the interaction between the user 140 and the digital assistant 125, the user 140 may actively request to provide a certain type of medical related information, or the digital assistant 125 may request the user 140 to provide a certain type of medical related information.

[0046] In some embodiments, alternatively or additionally, when determining whether the target file meets the health consultation requirements, it can be determined whether the appearance layout of the target file meets the predetermined layout requirements. This is particularly applicable to the case where the target file is generated by shooting or scanning. Due to the shooting or scanning angle of the file, the file may not contain complete information of the medical report, may be distorted, or the light is dim, resulting in unclear display of the content, etc., which will be reflected in the appearance layout of the target file. In order to ensure the accuracy of subsequent analysis, it is expected that the user can modify or provide a more accurate target file.

[0047] In some embodiments, when it is determined that the target file contains medical related information, the type of information contained matches the type of information provided by the requesting user 140 in the interactive context, and / or the appearance layout of the target file meets the predetermined layout requirements, it is determined that the target file meets the health consultation requirements. In this case, the analysis result of the digital assistant 125 for the target file can be determined based on the information included in the target file.

[0048] like Figure 2C As shown in the interactive interface 201 of , the digital assistant 125 provides a message 240 in the interactive interface to request the user to provide the results of the "blood routine test". The user 140 uploaded an image file 242. By parsing the image file 242, it can be determined that the image file 242 contains the blood routine test results, so the file meets the health consultation requirements. Next, based on the information contained in the file and other information provided by the user in the interactive context, the digital assistant 125 provides an analysis result 244. Similarly, in Figure 2D In the interactive interface 202, the user 140 uploads the PDF file 246. By parsing the PDF file 246, it can be determined that the image file 242 contains the blood routine test result, so the file meets the health consultation requirements, and the analysis result 244 can be provided similarly.

[0049] In some embodiments, if it is determined that the target file does not meet the health consultation requirements, the terminal device 110 may provide prompt information to prompt the user 140 to enter the correct file. In some embodiments, the target file is provided as an interactive message in the interactive interface between the user 140 and the digital assistant 125, and the prompt information may also be provided as an interactive message in the interactive interface. Figure 3AAs shown, in the interactive interface 300, the digital assistant 125 provides a message 310 in the interactive interface to request the user to provide the results of the "ultrasound examination". The user 140 uploaded an image file 312. By parsing the image file 312, for example, by extracting the text contained in the image file 312 or recognizing the layout of the image file 312, it can be determined that the image file 312 does not involve "ultrasound examination", and therefore it can be determined that the file does not meet the health consultation requirements. The digital assistant 125 can provide a prompt message 314 to support that the current file does not meet the requirements and prompt the user 140 to enter the correct file. Similarly, in Figure 3B In the interactive interface 301 shown, the digital assistant 125 provides a message 320 in the interactive interface to request the user to provide the test results of "blood routine test". The user 140 uploaded a PDF file 322. By parsing the PDF file 322, for example, by extracting the text contained in the PDF file 322 or based on the file name, it can be determined that the PDF file 322 does not involve "blood routine test", so it can be determined that the file does not meet the health consultation requirements. The digital assistant 125 can provide a prompt message 324 to support that the current file does not meet the requirements and prompt the user 140 to enter the correct file.

[0050] In some embodiments, in the input interface of the target file (e.g., the upload interface), it is possible to automatically identify whether the target file meets the health consultation requirements, and when a file that does not meet the health consultation requirements is identified, a prompt message is provided in the input interface so that the user can change the file to be uploaded in a timely manner. Figure 3C As shown, an input interface 302 may be provided for a user to select one or more files to be uploaded. If the user selects files 340, 342, and 344 and clicks an "upload report" control 346, it may be identified whether the file to be uploaded by the user meets the medical consultation requirements. If it is detected that one or more files to be uploaded do not meet the medical consultation requirements, such as Figure 3C As shown, a prompt message 348 may be provided at the input interface 302 to prompt that part or all of the documents do not meet the medical consultation requirements.

[0051] In some embodiments, the prompt information can also specifically mark the files that do not meet the requirements, which is particularly useful when the user selects multiple files to upload at the same time, so that the user can quickly filter and delete the files that do not meet the requirements. Figure 3C As shown, if it is determined that files 342 and 344 do not meet the medical consultation requirements, prompt information 350 and 352 can be provided at corresponding positions of the files to mark that the two files do not meet the requirements.

[0052] In some embodiments, when a user uploads or takes a photo of a file in real time, prompt information may also be provided to guide the user to upload the correct and relevant health-related files, which can reduce misunderstandings and repeated attempts caused by uploading wrong files. Figure 3D As shown, when a user photographs a paper document, a text prompt message 330 may be provided in the photographing interface 303 to prompt the requirements for the input file. Alternatively or additionally, a photographing prompt box 332 may be provided to guide the user to the correct photographing angle and size, and help the user improve the upload accuracy. Alternatively or additionally, an example viewing control 334 of the photographing template may be provided so that the user can view examples of files that meet the requirements.

[0053] In some embodiments, as described above, the digital assistant 125 determines the response based on the model. Some traditional medical report analysis tools have great limitations, which are mainly reflected in the fact that when analyzing examination reports, they identify key numerical information through rules or template matching and give general analysis results. However, these analysis tools cannot give a detailed explanation of the meaning of the numerical values, the reasons behind them, and the possible consequences, and cannot dynamically answer further questions from users. In the embodiments of the present disclosure, with the help of the digital assistant 125 and the model used by it, it is possible to effectively combine the context of files and text messages, guide users to upload correct pictures / files, and provide detailed medical report analysis, thereby improving the service quality of digital doctors.

[0054] In some embodiments, the model used by the digital assistant 125 includes a deep learning-based model that can process multimodal data input including text input and non-text input. That is, the model can not only support plain text input, but also parse and process files in non-text message formats uploaded by users, such as various medical report images or electronic files.

[0055] In some embodiments, when determining the analysis result of the target file by the digital assistant 125 based on the information extracted from the target file, the information can be extracted from the target file based on the parsing technology corresponding to the format of the target file, and the analysis result for the target file can be determined based on the extracted information and the interaction context between the user 140 and the digital assistant 125. In some embodiments, the extracted information includes text information contained in the target file. Alternatively or additionally, the extracted information includes image information in the target file. For example, for a file involving medical image examination, part or all of the content of the file may involve medical images, which is also important for medical analysis.

[0056] Depending on the format of the target file, different parsing techniques can be used. For example, if the target file includes an image file or a video file, the text information in the image or video file can be extracted based on optical character recognition (OCR) technology. In some embodiments, if the target file includes a non-editable text format such as a PDF file, the text therein can be extracted using corresponding text extraction techniques. In some embodiments, if the target file includes image information related to medical health, the image information of the corresponding area can be directly intercepted.

[0057] In some embodiments, a prompt word input for a model can be generated based on the information extracted from the target file, and the prompt word input is provided to the model to obtain the output of the model. The prompt word input can include text information and image information. Then, the analysis result for the target file can be determined based on the output of the model. In some embodiments, for the information extracted from the target file, key information therein can be screened as the model input, or all information can be provided as the model input.

[0058] In some embodiments, in addition to obtaining a file that meets the requirements to provide an analysis result, the terminal device 110 may also provide a parsed version of the target file in the interactive interface. The parsed version includes information extracted from the target file, such as text information and image information. Figure 2C and Figure 2D In the example of FIG. 2 , for the image file 242 and the PDF file 246 related to blood routine, the parsing results of the files are returned to the interactive interface as a message 248 .

[0059] In some embodiments, the parsed version corresponding to the target file may include a structured representation of the information extracted from the target file. The information in the target file is organized in a structured manner to facilitate user viewing. In some embodiments, the specific format of the information may be based on the information layout format in the target text, which is not limited herein.

[0060] exist Figure 4A In the interactive interface 400 shown, the user triggers the "photograph" control 405 to take a photo of the file to be input. The image file 410 captured by the shooting interface 401 is uploaded to the Figure 4B In the interactive interface 401 shown, by parsing the image file 410, a formatted parsed version 412 of the file is provided in the interactive interface.

[0061] Since the information contained in the parsed version corresponding to the target file is an intermediate product in the process of generating the analysis results of the file, providing the parsed version to the user 140 can facilitate the subsequent use of the target file by the user 140. In addition, by returning the parsing results, the user 140 can also be guided to verify and modify the parsing results in order to obtain more accurate analysis results. For example, if the user 140 determines that a part of the information is parsed incorrectly by viewing the returned parsing results, the user 140 can further enter an interactive message in the interactive interface to request correction of some of the information. In this way, the digital assistant 125 can determine the analysis results for the target file based on the corrected information and the interactive context.

[0062] In some embodiments, for the parsed version corresponding to the provided target file, further user interaction may be supported, including one or more of the following interactive operations on the parsed version: copy operation, export to local operation, share operation, print operation, delete operation, etc. The specific interactive operation may be selected according to actual application needs. In this way, the user 140 can better utilize the target file in subsequent health consultation or other aspects.

[0063] Specifically, in some embodiments, in response to detecting an interactive trigger operation on a parsed version corresponding to a target file, the terminal device 110 may provide at least one interactive option for the parsed version, each interactive option corresponding to an interactive operation on the parsed version. The interactive trigger operation may include appropriate trigger operations such as long press, double click, click, right click, etc. In response to receiving a selection of a target interactive option in at least one interactive option, the terminal device 110 may perform a target interactive operation corresponding to the target interactive option on the parsed version, such as a copy operation, an export to local operation, a share operation, a print operation, or a delete operation. For example, in response to detecting a selection in the interactive interface 400 for a target file, the terminal device 110 may perform a target interactive operation corresponding to the target interactive option on the parsed version, such as a copy operation, an export to local operation, a share operation, a print operation, or a delete operation. Figure 4C The parsed version of 412's long press operation, such as Figure 4D As shown, an interactive option box 414 for the parsed version is presented, which includes a plurality of selectable interactive options, such as delete, export, share, print, delete, etc. The user 140 can select a certain interactive operation as needed.

[0064] According to the embodiments of the present disclosure, combined with file parsing technology, the files provided in the interaction between the user and the digital assistant can be intelligently analyzed to see whether the files involve medical-related information or are related to the consultation context, thereby avoiding confusing the digital assistant's reply and making the digital assistant's analysis more accurate, professional and targeted. By using a model that supports multimodal input, it is possible to more conveniently process various types of user input and provide accurate responses in combination with the context. In some embodiments, users can also be guided to upload files that meet the requirements to improve interaction efficiency and user-friendliness. In addition, in some embodiments, the text uploaded by the user can also be structured as information that can be further interacted with or structured information, which can facilitate users to use directly in subsequent health consultations.

[0065] Figure 5 FIG. 5 is a flowchart of a process 500 for information interaction according to some embodiments of the present disclosure. The process 500 may be implemented at the terminal device 110. Figure 1 Process 500 is described.

[0066] In box 510, the terminal device 110 provides an interactive interface between the user and the digital assistant in response to the consultation initiation operation for health consultation of the user.

[0067] In block 520 , the terminal device 110 receives a target file input by a user, where the target file is in a non-text message format.

[0068] In box 530, the terminal device 110 determines whether the target file meets the health consultation requirements based on the interaction context between the user and the digital assistant.

[0069] In box 540, if the terminal device 110 determines that the target file meets the health consultation requirements, it determines the analysis result of the digital assistant for the target file based on the information included in the target file.

[0070] In some embodiments, determining whether the target file meets the health consultation requirements includes: determining that the target file meets the health consultation requirements by determining at least one of the following: determining that the target file includes medically related information, determining that the type of information contained in the target file matches the type of information requested from the user in the interactive context, and determining that the appearance layout of the target file meets predetermined layout requirements.

[0071] In some embodiments, process 500 further includes: if it is determined that the target file does not meet the health consultation requirements, providing prompt information to prompt the user to input a correct file.

[0072] In some embodiments, providing the prompt information includes: providing the prompt information as an interactive message in an interactive interface. In some embodiments, providing the prompt information includes: providing the prompt information in an input interface of a target file.

[0073] In some embodiments, determining the analysis result of the digital assistant for the target file based on the information extracted from the target file includes: extracting information from the target file based on the parsing technology corresponding to the format of the target file; and determining the analysis result for the target file based on the extracted information and the interaction context between the user and the digital assistant.

[0074] In some embodiments, process 500 further includes: providing a parsed version corresponding to the target file in an interactive interface, the parsed version including information extracted from the target file.

[0075] In some embodiments, the parsed version corresponding to the target file includes a structured representation of information extracted from the target file.

[0076] In some embodiments, process 500 also includes: in response to detecting an interaction trigger operation on a parsed version corresponding to a target file, providing at least one interaction option for the parsed version, each interaction option corresponding to an interaction operation on the parsed version; and in response to receiving a selection of a target interaction option among at least one interaction option, performing a target interaction operation corresponding to the target interaction option on the parsed version.

[0077] In some embodiments, at least one interactive operation corresponding to at least one interactive option includes at least one of the following: a copy operation, an export to local operation, a share operation, a print operation, and a delete operation.

[0078] In some embodiments, the target file includes a file uploaded by a user or captured in real time.

[0079] Figure 6 A schematic structural block diagram of an apparatus 600 for information interaction according to some embodiments of the present disclosure is shown. The apparatus 600 may be implemented as or included in a terminal device 110. Each module / component in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0080] As shown in the figure, the device 600 includes an interface providing module 610, which is configured to provide an interactive interface between the user and the digital assistant in response to a consultation initiation operation for the user's health consultation. The device 600 also includes a file receiving module 620, which is configured to receive a target file input by the user, and the target file is in the format of a non-text message. The device 600 also includes a requirement judgment module 630, which is configured to determine whether the target file meets the health consultation requirements based on the interaction context between the user and the digital assistant. The device 600 also includes an analysis determination module 640, which is configured to determine the analysis results of the digital assistant for the target file based on the information included in the target file if it is determined that the target file meets the health consultation requirements.

[0081] In some embodiments, the requirement judgment module 630 is configured to determine that the target file meets the health consultation requirements by determining at least one of the following: determining that the target file includes medically related information, determining that the type of information contained in the target file matches the type of information requested from the user in the interactive context, and determining that the appearance layout of the target file meets predetermined layout requirements.

[0082] In some embodiments, the device 600 also includes a prompt module configured to: if it is determined that the target file does not meet the health consultation requirements, provide prompt information to prompt the user to input the correct file.

[0083] In some embodiments, the prompt module is configured to: provide the prompt information as an interactive message in the interactive interface. In some embodiments, the prompt module is configured to: provide the prompt information in the input interface of the target file.

[0084] In some embodiments, the analysis and determination module 640 includes: an information extraction module, configured to extract information from the target file based on a parsing technology corresponding to the format of the target file; and an information-based analysis and determination module, configured to determine the analysis results for the target file based on the extracted information and the interaction context between the user and the digital assistant.

[0085] In some embodiments, the apparatus 600 further includes a parsing providing module configured to provide a parsed version corresponding to the target file in an interactive interface, wherein the parsed version includes information extracted from the target file.

[0086] In some embodiments, the parsed version corresponding to the target file includes a structured representation of information extracted from the target file.

[0087] In some embodiments, the device 600 also includes: an interaction option providing module, configured to provide at least one interaction option for the parsed version in response to detecting an interaction trigger operation on the parsed version corresponding to the target file, each interaction option corresponding to an interaction operation on the parsed version; and an interaction execution module, configured to execute a target interaction operation corresponding to the target interaction option on the parsed version in response to receiving a selection of a target interaction option among at least one interaction option.

[0088] In some embodiments, at least one interactive operation corresponding to at least one interactive option includes at least one of the following: a copy operation, an export to local operation, a share operation, a print operation, and a delete operation.

[0089] In some embodiments, the target file includes a file uploaded by a user or captured in real time.

[0090] Figure 7FIG. 7 is a block diagram of an electronic device 700 that can implement one or more embodiments of the present disclosure. It should be understood that Figure 7 The electronic device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 7 The electronic device 700 shown can be used to implement Figure 1 The terminal device 110 or Figure 6 Device 600.

[0091] like Figure 7 As shown, the electronic device 700 is in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of the electronic device 700.

[0092] The electronic device 700 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 can be a volatile memory (e.g., register, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 730 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 700.

[0093] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0094] The communication unit 740 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0095] The input device 750 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 may also communicate with one or more external devices (not shown) through the communication unit 740 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 700, or communicate with any device that allows the electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0096] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary embodiment of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0097] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0098] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0099] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0100] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0101] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

1. A method for information interaction, comprising: In response to the consultation initiation operation, providing an interactive interface between the user and the digital assistant for health consultation of the user; receiving a target file input by the user, wherein the target file is in a non-text message format; Determining whether the target file meets health consultation requirements based on the interaction context between the user and the digital assistant; as well as If it is determined that the target file meets the health consultation requirements, the analysis result of the digital assistant for the target file is determined based on the information included in the target file.

2. The method according to claim 1, wherein determining whether the target file meets the health consultation requirements comprises: The target document satisfies the health consultation requirement by determining at least one of the following: determining that the target file includes medical related information, determining that the information type contained in the target file matches the information type requested from the user in the interaction context, Determine whether the appearance layout of the target file meets the predetermined layout requirement.

3. The method according to claim 1, further comprising: If it is determined that the target file does not meet the health consultation requirements, a prompt message is provided to prompt the user to input a correct file.

4. The method according to claim 1, wherein determining the analysis result of the digital assistant for the target file based on the information extracted from the target file comprises: Extracting the information from the target file based on a parsing technology corresponding to the format of the target file; as well as Based on the extracted information and the interaction context between the user and the digital assistant, an analysis result for the target file is determined.

5. The method according to claim 1, further comprising: A parsed version corresponding to the target file is provided in the interactive interface, wherein the parsed version includes the information extracted from the target file. 6 . The method of claim 5 , wherein the parsed version corresponding to the target file includes a structured representation of the information extracted from the target file.

7. The method according to claim 5 or 6, further comprising: In response to detecting an interaction triggering operation on the parsed version corresponding to the target file, providing at least one interaction option for the parsed version, each interaction option corresponding to an interaction operation on the parsed version; as well as In response to receiving a selection of a target interaction option among the at least one interaction option, a target interaction operation corresponding to the target interaction option is performed on the parsed version.

8. The method according to claim 7, wherein the at least one interactive operation corresponding to the at least one interactive option comprises at least one of the following: a copy operation, an export to local operation, a share operation, a print operation, and a delete operation. 9 . The method according to claim 1 , wherein the target file comprises a file uploaded by the user or photographed in real time.

10. A device for information interaction, comprising: An interface providing module, configured to provide an interactive interface between the user and the digital assistant in response to a consultation initiation operation, for health consultation of the user; A file receiving module, configured to receive a target file input by the user, wherein the target file is in a non-text message format; a requirement determination module, configured to determine whether the target file meets the health consultation requirement based on the interaction context between the user and the digital assistant; as well as The analysis and determination module is configured to determine the analysis result of the digital assistant on the target file based on the information included in the target file if it is determined that the target file meets the health consultation requirement.

11. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 9 when executed by the at least one processing unit.

12. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 9 when executed by a processor.

Images