Information processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an information processing method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of computers. The information processing method disclosed by the invention comprises the following steps: in response to the acquired visual content, displaying the visual content and an information adding area in a first interface; in response to the fact that information is added to the visual content through the information adding area, a first message is displayed in a second interface, the second interface is an interface for conversation between the user and a virtual object, the virtual object is a virtual object providing medical consultation, and the first message comprises the visual content and the added information; and displaying a second message replied by the virtual object based on the first message in the second interface.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to an information processing method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] With the evolution of AI (Artificial Intelligence) technology, some Internet medical platforms have emerged, which can provide users with more convenient and efficient medical health service solutions. The Internet medical platform can use virtual objects based on AI technology to have a conversation with the user and provide medical consultation services. SUMMARY

[0003] According to some embodiments of the present disclosure, an information processing method is provided, including: displaying visual content and an information adding area in a first interface; in response to adding information through the information adding area, displaying a first message in a second interface, wherein the second interface is an interface for a user to have a conversation with a virtual object, and the first message is determined based on the visual content and the added information; and displaying a second message replied by the virtual object based on the first message in the second interface.

[0004] According to some other embodiments of the present disclosure, an information processing device is provided, including: a first display module configured to display visual content and an information adding area in a first interface; a second display module configured to, in response to adding information through the information adding area, display a first message in a second interface, wherein the second interface is an interface for a user to have a conversation with a virtual object, and the first message is determined based on the visual content and the added information; and a third display module configured to display a second message replied by the virtual object based on the first message in the second interface.

[0005] According to some other embodiments of the present disclosure, an electronic equipment is provided, including: a processor; and a memory coupled to the processor, configured to store instructions, which, when executed by the processor, cause the processor to perform the information processing method of any one of the embodiments of the present disclosure.

[0006] According to some other embodiments of the present disclosure, a computer readable storage medium is provided, which stores a computer program, and the program, when executed by a processor, performs the information processing method of any one of the embodiments of the present disclosure.

[0007] Other features, aspects, and advantages of the present disclosure will become apparent from the following detailed description of the exemplary embodiments with reference to the following accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0008] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. It should be understood that the accompanying drawings related to the description below merely relate to some embodiments of the present disclosure and do not limit the present disclosure. In the drawings:

[0009] Figure 1 A flowchart of an information processing method according to some embodiments of the present disclosure is shown;

[0010] Figures 2~12 A schematic diagram of a display interface according to some embodiments of the present disclosure is shown;

[0011] Figure 13 A structural schematic diagram of an information processing apparatus according to some embodiments of the present disclosure is shown;

[0012] Figure 14 A structural schematic diagram of an electronic device according to some embodiments of the present disclosure is shown;

[0013] Figure 15 A structural schematic diagram of an electronic device according to some other embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. It should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments described herein.

[0015] It should be understood that each step described in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. Unless otherwise specified, the relative arrangement of the steps described in these embodiments should be interpreted as merely exemplary and not limiting the scope of the present disclosure.

[0016] The term “comprising” and variations thereof used in the present disclosure mean an open term that includes at least the recited elements / features, but does not exclude other elements / features, i.e., “including but not limited to”. The term “based on” means “based at least in part on”.

[0017] It should be noted that the concepts of “first”, “second”, etc. mentioned in the present disclosure are merely used to distinguish different devices, modules or units, and are not intended to limit the functions performed by these devices, modules or units in a given order or interdependent relationship. Unless otherwise specified, the concepts of “first”, “second”, etc. are not intended to imply a given order or any other manner of given order in time, space, ranking, etc.

[0018] It should be noted that the modification of "one" and "multiple" mentioned in the present disclosure is illustrative rather than restrictive, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly indicated in the context.

[0019] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0020] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0021] The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings, but the present disclosure is not limited to these specific embodiments. The following specific embodiments can be combined with each other, and for the same or similar concepts or processes, some embodiments can not be described again. In addition, in one or more embodiments, specific features, structures or characteristics can be combined by any suitable means from the present disclosure that is clear to those skilled in the art.

[0022] Some Internet medical platforms or applications can use virtual objects (or agents) to have a conversation with users to provide medical consultation services. For example, a user can have a multi-round conversation with a virtual object to describe a disease, symptoms, etc., and the virtual object gives a simple diagnosis result, suggestion, etc. During the multi-round conversation, the user can also send images to the virtual object to reflect the disease, symptoms, etc. In some cases, the user's captured images are difficult to identify accurate parts, symptoms, etc. information. And the user sends text and images to the virtual object through different messages, and the virtual object will give a reply based on the image as soon as it receives the image sent by the user. This conversation mode will lead to inaccurate understanding of the image by the virtual object. If the understanding of the image is biased, the user needs to further input the correction information, which leads to more rounds of conversation and low efficiency.

[0023] To solve the above problems, the disclosure provides an information processing method. In a first interface, visual content and an information adding area are displayed, in response to adding information through the information adding area, a first message is displayed in a second interface in which a user interacts with a virtual object, the first message is determined based on the visual content and the added information, and further, a second message replied by the virtual object based on the first message is displayed in the second interface. Based on the information processing method of the disclosure, in the process of the user interacting with the virtual object providing medical consultation, the information adding area can be provided for the user in the first interface for the visual content, the user can add information to the visual content through the information adding area, assist the virtual object to understand the visual content, and thus improve the accuracy of the second message replied by the virtual object, reduce the number of interactions between the user and the virtual object, and improve the interaction efficiency.

[0024] The information processing method of the disclosure will be described below in conjunction with Figures 1~12 The information processing method of the disclosure will be described below in conjunction with

[0025] Figure 1 The information processing method of the disclosure will be described below in conjunction with Figure 1 As shown in the flowchart of FIG. 1, the information processing method of this embodiment includes steps S102-S106.

[0026] In step S102, visual content and an information adding area are displayed in a first interface.

[0027] The visual content includes at least one of an image and a video. The visual content can be visual content captured by a camera component, or visual content obtained from a storage device such as an internal memory or an external storage device. The first interface is a display interface of the visual content, for example, a preview interface of the visual content, a shooting interface of the visual content, etc., without being limited to the examples. For example, the visual content can be an image of a lesion captured by the user, or a medical record uploaded, etc., without being limited to the examples.

[0028] For example, the information adding area can be an area in the first interface, and the display area of the visual content is different from the information adding area. For example, the information adding area can be an area on the upper layer of the first interface. The information adding area can be displayed in the form of a panel, a window, or a mask, etc., without being limited to the examples.

[0029] In step S104, in response to adding information through the information adding area, a first message is displayed in a second interface.

[0030] The user can add information, e.g., a textual description, to the visual content through the information adding area. The added information can be used to assist the virtual object to understand the visual content. The added information is related to the visual content and is related to the medical consultation. The virtual object is a virtual object that provides medical consultation, and the virtual object can interact (dialogue) with the user to provide medical consultation services, e.g., answer medical and health-related questions raised by the user, provide simple diagnosis results, treatment plans, etc. based on the dialogue with the user.

[0031] In response to adding information to the visual content through the information adding area, the first interface is switched to the second interface, and the first message sent by the user to the virtual object is displayed in the second interface. The second interface is an interface for the user to dialogue with the virtual object, e.g., including an information display area and an input area. The information display area is used to display messages of the interaction between the user and the virtual object, and the input area can be used to input messages.

[0032] The first message is determined based on the visual content and the added information. For example, the first message includes the visual content and the added information. The visual content and the added information can be sent to the virtual object as two consecutive sub-messages in the first message, e.g., the visual content and the added information are displayed in two visual message components. The visual content and the added information can be sent to the virtual object as one first message, e.g., the visual content and the added information are displayed in one visual message component. For example, the visual message component can be displayed as a geometric container, e.g., a message bubble, etc.

[0033] In step S106, the second message replied by the virtual object based on the first message is displayed in the second interface.

[0034] For example, the second message is displayed in the information display area of the second interface. The second message is generated according to the visual content and the added information by using the virtual object or the model corresponding to the virtual object.

[0035] The method of the above embodiment displays the visual content and the information adding area in the first interface, displays the first message in the second interface for the user to dialogue with the virtual object in response to adding information through the information adding area, the first message is determined based on the visual content and the added information, and further displays the second message replied by the virtual object based on the first message in the second interface. In the process of the user interacting with the virtual object that provides medical consultation, the method of the above embodiment can provide the user with an information adding area for the visual content in the first interface, the user can add information to the visual content through the information adding area, assist the virtual object to understand the visual content, and thus improve the accuracy of the second message replied by the virtual object, reduce the number of interactions between the user and the virtual object, and improve the interaction efficiency.

[0036] The following describes how to display information addition areas and some methods for adding information to visual content using information addition areas.

[0037] In some embodiments, displaying visual content and an information addition area in a first interface includes: in response to obtaining visual content, displaying visual content and an information addition control in a first interface; and in response to triggering an operation on the information addition control, displaying an information addition area in a first interface.

[0038] Information addition controls can add areas, buttons, or other forms to information in the first state, and are not limited to the examples given. For instance, in response to a trigger operation on the information addition area in the first state, the information addition area in the second state can be displayed. For example, the first state of the information input area is inactive, and the second state is active; in the second state, the information addition area can add visual content. For example, the information input area may only display inactive input boxes in the first state, and display active input boxes and a virtual keyboard in the second state.

[0039] like Figure 2 As shown, in response to acquiring visual content, the visual content is displayed in the first interface 200, as well as the information addition area 201 of the first state.

[0040] The acquired visual content may include multiple sub-contents, such as multiple images and / or videos. In some embodiments, the visual content includes multiple sub-contents, and displaying the visual content and information adding area in the first interface includes: in response to acquiring each sub-content, displaying a preview of the currently acquired sub-content, thumbnails of one or more already acquired sub-contents, and an information adding control in the first interface, wherein the one or more already acquired sub-contents include the currently acquired sub-content; and in response to a triggering operation on the information adding control, displaying the information adding area in the first interface.

[0041] For example, when the function to retrieve multiple pieces of content is enabled, in response to the retrieval of each piece of content, the first interface displays a preview of the currently retrieved content, thumbnails of one or more retrieved content items, and an information addition control. For instance, the function to retrieve multiple pieces of content can be enabled via a trigger control.

[0042] like Figure 2As shown, when the function to retrieve multiple contents is enabled, in response to the retrieval of each sub-content, the first interface 200 displays the currently retrieved sub-content 202, thumbnails 203 and 204 of the multiple retrieved sub-contents, and the information addition area 201 for the first state. The thumbnails 203 and 204 of the multiple retrieved sub-contents can be displayed in the order of retrieval, with the currently retrieved sub-content 202 corresponding to thumbnail 204. Thumbnail 204 can be displayed with a first effect, indicating that the currently displayed sub-content is the sub-content corresponding to thumbnail 204. Other thumbnails can be displayed with a second effect. For example... Figure 2 As shown, the first interface 200 can also display a content addition control 205. In response to the triggering operation of the content addition control 205, a shooting interface, etc., can be displayed. Subsequent embodiments will describe this in detail.

[0043] In response to a triggering action on the information addition control, an information addition area is displayed in the first interface. Information added through the information addition area can be added to the overall visual content without distinguishing between different sub-content.

[0044] Each sub-content thumbnail can display a corresponding delete control. In response to the triggering of the delete control of a certain thumbnail, the sub-content corresponding to that thumbnail is deleted.

[0045] The method described in the above embodiments can acquire multiple sub-contents as visual content and add areas for displaying information on multiple sub-contents, allowing users to add information on multiple sub-contents. This facilitates the addition of multiple sub-contents and information by users. Users can send multiple sub-contents and related information to a virtual object with a single message, improving the convenience and efficiency of user operations and enhancing the interactive experience.

[0046] In some embodiments, in response to a triggering operation of the information adding control, displaying an information adding area in the first interface includes: in response to a selection operation of a target sub-content among one or more sub-contents that have been acquired and a triggering operation of the information adding control, displaying a preview of the target sub-content and the information adding area in the first interface, wherein the added information includes information added to the target sub-content through the information adding area.

[0047] There can be one or more target sub-contents, and users can select them by triggering their thumbnails. When a user selects a target sub-content and triggers the information addition control, a preview of the target sub-content and the information addition area are displayed. Users can add information to the target sub-content through the information addition area.

[0048] After adding information to the target sub-content, you can also return results such as... Figure 2As shown in the first interface, the user can further select other sub-contents and add information to the selected sub-contents. The user can add different information to each sub-content. The first message can include a plurality of continuous sub-messages, and each sub-message includes a sub-content and added information.

[0049] Based on the method of the above embodiment, the user can select any sub-content as a target sub-content and add information to the target sub-content, improving the flexibility and convenience of user operation, better meeting the needs of the user, and improving the user operation experience.

[0050] In some embodiments, the information adding area includes one or more information options and / or a first input component, and the one or more information options are determined according to the visual content and are related to medical consultation.

[0051] For example, the one or more information options include one or more information options of at least one category of site, lesion, and symptom. The one or more information options are determined according to the visual content, for example, by using a model corresponding to the virtual object to identify the visual content, determining that the visual content includes an image of the affected area, and generating the one or more information options related to the visual content according to the visual content, and providing the one or more information options to the user for selection and confirmation.

[0052] For example, the first input component includes an input box, a virtual keyboard, or the input interaction component includes an input box and a handwriting input area, or the input interaction component includes an input box and a voice input component, etc., not limited to the examples shown. The virtual keyboard, the handwriting input area, the voice input control, etc. are used to receive the user's operation, and the input box is used to display the input information. The one or more information options and the first input component can be displayed in the information adding area at the same time.

[0053] In some embodiments, the information adding area includes one or more information options, and in response to adding information through the information adding area, displaying the first message in the second interface includes: in response to a selection operation on a target option in the one or more information options, displaying the target option in a selected state; and in response to a sending operation, displaying the first message in the second interface, wherein the added information includes information corresponding to the target option.

[0054] The target option can be one or more. In the case where the one or more information options include a plurality of category information options, the user can select one or more target information options for each category. The one or more target options are displayed in a selected state, for example, in a highlighted or other highlighted effect.

[0055] For example, in response to the selection operation on the target option, information corresponding to the target option can also be displayed in the input box of the first input component, and the user can perform modification, addition, deletion, etc. on the information. The information corresponding to the target option can be prompt information generated based on the target option.

[0056] For example, the information addition area includes a sending control, and in response to a triggering operation on the sending control, the first message is displayed in the second interface. The user can add information, send the first message in the first interface displaying the visual content, without switching to the second interface, thereby improving the continuity and efficiency of user operations and improving user experience.

[0057] The method of the above embodiments provides one or more information options for the user in the information addition area for confirmation and selection by the user, thereby improving the convenience and efficiency of user operations. In addition, since the one or more information options are determined and associated with the visual content according to the visual content, the accuracy and effectiveness of the one or more information options are improved, thereby further improving the convenience and efficiency of user operations.

[0058] In some embodiments, the information addition area includes a first input component, and in response to adding information through the information addition area, displaying the first message in the second interface includes: in response to an operation of inputting information in the first input component, displaying the input information in the first input component; and in response to a sending operation, displaying the first message in the second interface, wherein the added information includes the input information.

[0059] For example, the user can input information through a virtual keyboard, a handwriting input area, or a voice input control in the first input component, and the input information is displayed in the input box. The sending control can also be displayed in the input box, and in response to a triggering operation of the sending control, the first message is displayed in the second interface.

[0060] The method of the above embodiments allows the user to flexibly input information in the first input component to add relevant auxiliary information to the visual content, without switching to the second interface, thereby improving the continuity and efficiency of user operations and improving user experience.

[0061] The information addition area can include one or more information options and a first input component, and the user can select one or more target information options and input information through the first input component as information added to the visual content, which is not described in detail. By providing one or more information options and a first input component, the convenience and flexibility of user operations are further improved, and the efficiency and experience of user operations are improved.

[0062] As shown in FIG. 2A, in response to a triggering operation of the user on the information addition area 201 in the first state, the information addition area 201 is displayed in the second state as shown in FIG. 2B. Figure 2 Figure 3 ​The interface shown. (As shown) Figure 2 As shown, the first interface 200 displays a sending control 206. In response to the sending control 206, visual content can be sent directly to the virtual object without adding information.

[0063] like Figure 3 As shown, the first interface 200 displays a second-state information addition area 201, which includes multiple information options, such as information options 301 and 302. Information options 301 and 302 can belong to different categories. For example, when a user triggers information option 301, it is displayed as selected. The information addition area 201 includes an input box 303 and a virtual keyboard 304. The user can input information using the virtual keyboard 304, and the input information is displayed in the input box 303. The information addition area 201 also includes a send control 305, which, in response to the user triggering the send control 305, can display... Figure 4 The interface shown.

[0064] like Figure 4 As shown, the second interface 400 is a dialogue interface between the user and the virtual object. The second interface 400 includes an information display area 401 and an input area 402. The first message may include two consecutive sub-messages 403 and 404, or it may be displayed as a single message. Sub-message 403 includes visual content, and sub-message 404 includes added information. The first message 403 and 404 and the second message 405 are displayed in the information display area 401. The second message 405 may be a response message generated based on the first messages 403 and 404. The first message and the second message may be carried by containers of geometric shapes (visual message components), and are not limited to the examples given.

[0065] The response information generated by the virtual object based on the message sent by the user may sometimes contain errors or inaccuracies, and the user can correct the response information.

[0066] In some embodiments, in response to receiving a first message, the first message is displayed on a second interface, wherein the second interface is an interface for a user to interact with a virtual object, the virtual object is a virtual object providing medical consultation, the first message includes visual content, and the second message includes analysis information generated based on the first message; in response to a modification operation on the analysis information, a third message from the virtual object is displayed on the second interface, wherein the third message includes analysis information regenerated based on the modification information and visual content corresponding to the analysis information.

[0067] The first message can include only visual content. The analysis information is, for example, preliminary judgment information or preliminary diagnosis information obtained according to the first message. The user can directly modify the analysis information, trigger the virtual object to reply to the third message, and the third message includes regenerated analysis information.

[0068] The first message can include visual content and added information as in the foregoing embodiments. In some embodiments, the second message includes analysis information generated based on the first message, and the information processing method further includes: in response to a modification operation on the analysis information, displaying a third message replied by the virtual object in the second interface, wherein the third message includes regenerated analysis information based on modification information corresponding to the analysis information and visual content.

[0069] For example, the analysis information includes one or more editable contents, and the modification on the analysis information is performed by editing the one or more editable contents, thereby improving the convenience and efficiency of the user's modification.

[0070] The method of the foregoing embodiments allows the user to directly modify the analysis information, so that the virtual object replies based on modification information corresponding to the analysis information and visual content, improves the accuracy of the reply of the virtual object, and meets the needs of the user.

[0071] In some embodiments, the analysis information includes one or more editable contents, and in response to a modification operation on the analysis information, displaying a third message replied by the virtual object in the second interface includes: in response to a trigger operation on target content in the one or more editable contents, displaying a second input component; in response to inputting modification information corresponding to the target content through the second input component, displaying the modification information corresponding to the target content in the second input component, wherein the modification information corresponding to the analysis information includes the modification information of the target content; and in response to a confirmation operation, displaying the third message replied by the virtual object in the second interface.

[0072] For example, the one or more editable contents can be displayed in a specified style and as an interactive element. For example, the one or more editable contents can be displayed with highlighting, underlining, or other highlighting effects, or a specified identifier can be displayed beside the one or more editable contents, without being limited to the foregoing examples.

[0073] The one or more editable contents can be determined according to the first message and related to the first message. For example, the one or more editable contents include one or more editable contents in at least one category of a part, a lesion, or a symptom. For example, the one or more editable contents include an editable content “knee joint”, in response to a trigger operation of the user on the editable content “knee joint”, a second input component is displayed, and the user can input “elbow joint” as modification information of “knee joint” in the second input component.

[0074] For example,Figure 5 As shown in FIG. 4, the second message 501 is displayed in the information display area 401 of the second interface 400, and the second message 501 includes editable content 502. In response to a triggering operation on the editable content 502, a third message in which the virtual object replies can be displayed in the second interface, as shown in FIG. 5. Figure 6 As shown in FIG. 5, the input area 503 in the first state can be displayed in the second interface 400. The input area 503 in the first state can also be other forms of controls, such as buttons, and is not limited to the examples shown.

[0075] As shown in FIG. 5, in response to the triggering operation on the editable content 502, the input area 503 in the second state is displayed. The input area 503 in the second state can include a second input component, which includes an input box 601 and a virtual keyboard 602. The user can input modification information of the editable content 502 through the input box 601 and the virtual keyboard 602. The input area or the input area 503 in the second state can include a sending control 603. A triggering operation on the sending control 603 can be a confirmation operation. In response to the triggering operation on the sending control 603, a third message in which the virtual object replies is displayed in the second interface. For example, the second message can no longer be displayed, and the third message is displayed instead of the second message, or the third message can be displayed after the second message. Figure 6

[0076] The method of the above embodiment makes the user more convenient and accurate to modify the second message by setting one or more editable contents. Compared with the way of modifying the second message by inputting information, the accuracy and operation efficiency are improved, the accuracy of the virtual object in understanding the modification information is improved, and thus the accuracy of the third message in which the virtual object replies is improved.

[0077] The following describes how to obtain visual content.

[0078] In some embodiments, an opening control is displayed in the second interface. In response to a triggering operation on the opening control, a shooting interface is displayed, and a real-time preview picture and a shooting control are displayed in the shooting interface. In response to a triggering operation on the shooting control, an image or a video shot is obtained as visual content.

[0079] For example, the user can shoot an image by clicking the shooting control, can shoot a video by long-pressing the shooting control, can start shooting a video by clicking the shooting control and stop shooting the video by clicking the shooting control again, or can switch the type of visual content shot by the shooting control, so as to shoot an image or a video, which is not limited to the examples shown above.

[0080] As shown in FIG. 7, a real-time preview picture and a shooting control 701 are displayed in the shooting interface 700. The user can shoot an image or a video by triggering the shooting control 701. Figure 7 ​​

[0081] In some embodiments, the first content selection region is displayed in the shooting interface, the first content selection region includes one or more thumbnails of candidate visual content and one or more source options; in response to selection of a first target content in the one or more candidate visual content, the first target content is taken as the acquired visual content; or, in response to a triggering operation on a target option in the one or more source options, one or more visual contents corresponding to the target option are displayed, and in response to selection of a second target content in the one or more visual contents corresponding to the target option, the second target content is taken as the acquired visual content.

[0082] As shown in Figure 7 , in the first content selection region 702 of the shooting interface 700, a plurality of source options 703 and 704 and a plurality of thumbnails of candidate visual content, such as the thumbnail 705 of candidate content, are displayed. The user can select visual content through the plurality of source options 703 and 704 or the plurality of thumbnails of candidate visual content as the acquired visual content.

[0083] The content selection region can also be displayed in a separate interface, for example, a content selection control is displayed in the second interface, and in response to a triggering operation on the content selection control, a content selection interface is displayed, the content selection interface includes one or more thumbnails of candidate visual content and one or more source options.

[0084] In some embodiments, a triggering control of a plurality of content acquisition function is displayed in the shooting interface, in response to a triggering operation on the triggering control and a triggering operation on the shooting control, a second content selection region is displayed in the shooting interface, and thumbnails of one or more sub-contents that have been shot are displayed in the second content selection region, and in response to completion of shooting of the plurality of sub-contents, the plurality of sub-contents are taken as the acquired visual content.

[0085] The triggering control of the plurality of content acquisition function can also be provided in other positions or interfaces, for example, the triggering control of the plurality of content acquisition function is directly displayed in the second interface.

[0086] As shown in Figure 7 , a triggering control 706 of a plurality of content acquisition function is displayed in the shooting interface 700, in response to a triggering operation on the triggering control 706 and a triggering operation on the shooting control 701, an interface as shown in Figure 8 is displayed.

[0087] As shown in Figure 8As shown, in the shooting interface 700, a second content selection area 801 is displayed, and thumbnails 802 of one or more sub-contents that have been shot are displayed. The thumbnail of each sub-content can correspond to a display of a deletion control. In response to a triggering operation of the deletion control of a certain thumbnail, the sub-content corresponding to the thumbnail is deleted. The completion control 803 is also displayed in the second content selection area 801. In response to a triggering operation of the completion control 803, the one or more sub-contents are taken as the obtained visual content. For example, in response to the triggering operation of the completion control 803, the interface shown in FIG. 8A can be displayed. The real-time preview screen is also displayed in the shooting interface 700. Figure 2 As shown, in the shooting interface 700, a second content selection area 801 is displayed, and thumbnails 802 of one or more sub-contents that have been shot are displayed. The thumbnail of each sub-content can correspond to a display of a deletion control. In response to a triggering operation of the deletion control of a certain thumbnail, the sub-content corresponding to the thumbnail is deleted. The completion control 803 is also displayed in the second content selection area 801. In response to a triggering operation of the completion control 803, the one or more sub-contents are taken as the obtained visual content. For example, in response to the triggering operation of the completion control 803, the interface shown in FIG. 8A can be displayed. The real-time preview screen is also displayed in the shooting interface 700.

[0088] Based on the method of the above embodiments, the user can shoot multiple sub-contents together to send to the virtual object for identification, and obtain a second message in reply. The user does not need to repeatedly perform a sending operation for multiple sub-contents, thereby improving the convenience and efficiency of user operation, and the virtual object can associate and identify multiple sub-contents, so that the generated second message is more accurate.

[0089] In some embodiments, in response to an operation of starting a real-time call, a video stream of the real-time call is taken as the obtained visual content.

[0090] The user can perform a real-time call with the virtual object. In the real-time call interface of the user and the virtual object, a real-time call screen of the user that is shot can be displayed. The real-time call between the user and the virtual object is similar to the effect of a video call between multiple users. The user can improve the accuracy of medical consultation by performing a video consultation with the virtual object.

[0091] The above multiple embodiments can obtain visual content in multiple ways, and the obtained visual content can include multiple types of content, thereby improving the flexibility and convenience of user operation and improving the accuracy of medical consultation.

[0092] In the process of obtaining visual content, the operation of the user can be guided, thereby improving the quality and accuracy of the obtained visual content. The method of guiding the operation of the user is described below.

[0093] In some embodiments, in response to an operation of starting a real-time call, a video stream of the real-time call is taken as the obtained visual content.

[0094] For example, the guidance information includes information on shooting operation methods such as adjusting shooting distance, focal length, lighting, and stability, with the shooting operation method determined based on the cause of blurriness in the live-line image. For instance, the image or video stream corresponding to the live-line image is identified to determine the cause of blurriness, and guidance information is determined based on the cause of blurriness.

[0095] The system can automatically adjust shooting parameters based on the cause of blurriness in the live image, such as turning on the flash or adjusting the focus. Guidance information includes details about these automatic adjustments. Additionally, guidance information may include prompts for the user to photograph the affected area and information about the location and function of control buttons.

[0096] In response to the shooting operation based on the guidance information, visual content is acquired, and a second message generated based on the visual content can be displayed on the second interface. Alternatively, information can be added to the visual content in conjunction with the aforementioned embodiments, and a second message generated based on the visual content and the added information can be displayed on the second interface.

[0097] The method described in the above embodiments outputs guidance information while displaying the captured real-time image, which can guide users to capture clearer and more accurate visual content, thereby improving the accuracy of subsequent virtual object responses and the accuracy of medical consultations.

[0098] In some embodiments, in response to the operation of activating the camera component, displaying the captured real-time image and outputting guidance information includes: displaying an activation control in a second interface; in response to triggering the activation control, displaying a shooting interface and displaying a real-time preview image in the shooting interface, wherein the operation of activating the camera component includes triggering the activation control, and the captured real-time image includes a real-time preview image; displaying text of guidance information and / or playing audio of guidance information in the shooting interface.

[0099] like Figure 7 As shown, the shooting interface 700 displays a real-time preview and can also display text guidance information 707. It can also play audio guidance information, or display the text guidance information 707 while playing audio.

[0100] In response to a triggering operation on the activation control, a first animation effect can be displayed to prompt entry into the guided shooting mode. In guided shooting mode, guiding information can be output.

[0101] For example, a guided shooting control is displayed in the shooting interface. In response to the triggering operation of the guided shooting control, the guided shooting mode is entered, and the text of the guidance information is displayed and / or the voice of the guidance information is played in the shooting interface.

[0102] The method of the above embodiment displays the real-time preview picture in the shooting interface and displays the text of the guidance information and / or plays the voice of the guidance information, which can guide the user to shoot clearer and more accurate visual content in the user shooting process, thereby improving the accuracy of the subsequent information returned by the virtual object and improving the accuracy of the medical consultation.

[0103] In some embodiments, the guidance information includes one or more pieces of guidance information, the shooting interface includes a shooting control, and the obtaining of the visual content includes: in response to an operation of adjusting shooting based on the one or more pieces of guidance information and one or more triggering operations on the shooting control, obtaining one or more images shot as the visual content, and the first interface includes a preview interface of the one or more images shot.

[0104] The one or more pieces of guidance information can be displayed in multiple times according to the operation of the user. The user can shoot one or more images as the visual content, which can be specifically referred to the foregoing embodiments and will not be described here. After the user shoots the images, a preview interface of the images is displayed, which can be the first interface as shown in Figure 2 .

[0105] The method of the above embodiment displays one or more pieces of guidance information based on the operation of the user adjusting shooting, which can more accurately guide the operation of the user and improve the quality of the one or more images shot, thereby improving the accuracy of the subsequent medical consultation.

[0106] In some embodiments, the shooting interface includes a voice interaction control, and the displaying of the text of the guidance information and / or the playing of the voice of the guidance information in the shooting interface includes: in response to a triggering operation on the voice interaction control, entering a voice interaction mode, displaying the text of the guidance information in the shooting interface and playing the voice of the guidance information.

[0107] As shown in Figure 7 , the voice interaction control 708 is displayed in the shooting interface 700, and in response to a triggering operation on the voice interaction control 708, a second animation effect can be displayed to prompt entering the voice interaction mode. In the voice interaction mode, the text 707 of the guidance information is displayed in the shooting interface and the voice of the guidance information is played, or only the voice of the guidance information is played.

[0108] In some embodiments, in the voice interaction mode, in response to receiving an input voice, a reply voice is output. The reply voice can be generated according to the video stream corresponding to the real-time preview picture and the input voice. The text of the reply voice can also be displayed in the shooting interface.

[0109] For example, the user can have a voice conversation with the virtual object in the voice interaction mode, for example, by describing the symptoms, affected parts, etc. through the voice, and the virtual object generates a reply voice.

[0110] In some embodiments, the reply voice includes information guiding the user to capture multiple sub contents. For example, in the voice interaction mode, the reply voice of the virtual object can guide the user to capture multiple images, videos, etc. of the affected area. The interface for capturing multiple sub contents can be as shown in Figure 8 .

[0111] As shown in Figure 8 , the capture interface 700 can be an interface in the voice interaction mode, and the text 804 of the reply voice can be displayed.

[0112] The method of the above embodiments, in the case that the user triggers the voice interaction control, enters the voice interaction mode, and can guide the user to perform the capture operation through the voice, and the user can also have a voice conversation with the virtual object, improves the convenience and efficiency of the user's medical consultation, and improves the operation experience of the user.

[0113] In some embodiments, one or more pieces of guidance information are determined based on identification of one or more images corresponding to the periodically acquired real-time preview picture, or one or more pieces of guidance information are determined based on identification of a video stream corresponding to the real-time preview picture.

[0114] For example, in the case of previewing the capture, one or more images can be periodically acquired, the model corresponding to the virtual object is used to identify each acquired image, to determine whether the image is clear and the reason for being unclear, and to generate guidance information according to the reason for being unclear. Or the model corresponding to the virtual object is used to identify a video stream corresponding to the real-time preview picture, to determine whether the image is clear and the reason for being unclear, and to generate guidance information according to the reason for being unclear.

[0115] For example, in the voice interaction mode, one or more pieces of guidance information are determined based on identification of a video stream corresponding to the real-time preview picture. In the case that the voice interaction mode is not started, one or more pieces of guidance information are determined based on identification of one or more images corresponding to the periodically acquired real-time preview picture.

[0116] The method of the above embodiments, by identifying one or more images or a video stream corresponding to the real-time preview picture, generates guidance information, which can improve the accuracy of the guidance information, thereby better guiding the user to perform the capture, and improving the accuracy and quality of the visual content.

[0117] In some embodiments, in response to the operation of starting the camera component, the display of the captured real-time picture, and the output of the guidance information includes: displaying a starting control in the second interface; in response to the triggering operation of the starting control, displaying a shooting interface, and displaying a real-time preview picture and a real-time call control in the shooting interface; in response to the triggering operation of the real-time call control, displaying a real-time call interface of the user and the virtual object, and displaying a real-time call picture in the real-time call interface, and playing the voice of the guidance information, wherein the operation of starting the camera component includes the triggering operation of the starting control, and the captured real-time picture includes the real-time call picture.

[0118] As shown in FIG. 7, the real-time call control 901 can also be displayed in the shooting interface 700. In response to the triggering operation of the real-time call control 901, the real-time call interface as shown in FIG. 8 is displayed. The third animation effect can be displayed in response to the triggering operation of the real-time call control 901, and the real-time call interface is switched to. For example, the voice of the guidance information issued by the virtual object is played in the real-time call process, that is, the virtual object guides the user's shooting in the real-time call process. Figure 9 Figure 10 As shown in FIG. 7, the real-time call control 901 can also be displayed in the shooting interface 700. In response to the triggering operation of the real-time call control 901, the real-time call interface as shown in FIG. 8 is displayed. The third animation effect can be displayed in response to the triggering operation of the real-time call control 901, and the real-time call interface is switched to. For example, the voice of the guidance information issued by the virtual object is played in the real-time call process, that is, the virtual object guides the user's shooting in the real-time call process.

[0119] The method of the above embodiments guides the user to shoot in the real-time call process, improves the quality of the real-time call picture, thereby improving the effect of the real-time call and the video consultation, and improving the user experience.

[0120] In some embodiments, the guidance information includes one or more pieces of guidance information, and the one or more pieces of guidance information are determined based on the identification of the video stream corresponding to the picture of the real-time call. In response to the shooting operation based on the guidance information, the visual content is obtained by: in response to the operation of adjusting the shooting based on the one or more pieces of guidance information, obtaining the video stream of the real-time call as the visual content, wherein the first interface includes the real-time call interface.

[0121] For example, the first interface is the real-time call interface. The information adding area can also be displayed in the real-time call process, and the user can add information.

[0122] The method of the above embodiments can more accurately guide the user to perform the shooting operation, further improve the quality of the real-time call picture, thereby improve the effect of the real-time call and the video consultation, and improve the user experience.

[0123] The user can improve the efficiency of medical consultation and improve the user experience through the real-time call with the virtual object. In some embodiments, in response to the operation of starting the real-time call; a real-time call interface of the user and the virtual object is displayed, and a real-time call picture is displayed in the real-time call interface; in the process of the real-time call, the video stream of the user is received, and the reply information generated by the virtual object based on the video stream is output, wherein the reply information includes analysis information generated based on the video stream.​

[0124] As shown in Figure 9 , a real-time call control 901 can be displayed in the shooting interface 700. The real-time call control can also be displayed in the second interface. The user starts the real-time call by triggering the real-time call control. During the real-time call, the user can input a video stream, and the intelligent agent outputs reply information, which can include text and / or voice. For example, the reply information includes analysis information on the disease, treatment plan, etc., without being limited to the examples shown.

[0125] As shown in Figure 10 , a real-time call interface 1000 is displayed in the real-time call interface 1000. The real-time call interface 1000 displays a voice control 1001, and in response to a triggering operation on the voice control 10001, the display of the real-time call picture is stopped, and a preset image or animation can be displayed. The real-time call interface 1000 displays a video control 1002, and in response to the current display of the real-time call picture and the triggering operation on the video control 1002, the real-time call picture is displayed. The real-time call interface 1000 displays an end control 1003, and in response to the triggering operation on the end control 1003, the real-time call is ended. The real-time call interface 1000 can also display a subtitle control 1004, and in response to the triggering operation on the subtitle control 1004, an interface as shown in Figure 11 can be displayed.

[0126] As shown in Figure 11 , in the case of displaying the real-time call picture, in response to the triggering operation on the subtitle control 1004, the real-time call interface 1000 includes a subtitle display area 1101 and a real-time call picture display area 1102. The text of the conversation between the user and the virtual object during the real-time call is displayed in the subtitle display area 1101, and the real-time call picture is displayed in the picture display area 1102. The picture display area 1102 can be displayed on the upper layer of the subtitle display area 1101 in the form of a window, a panel, a mask layer, etc., or it can be a different area in the same layer as the subtitle display area 1101, without being limited to the examples shown.

[0127] In the case of closing the real-time call picture, in response to the triggering operation on the subtitle control 1004, only the subtitle display area 1101 can be displayed in the real-time call interface 1000. In response to the triggering operation on the end control 1003, the real-time call is ended, and the second interface is displayed, and the text of the conversation between the user and the virtual object during the video passing process is displayed in the second interface.

[0128] The method of the above embodiment can provide video consultation services for the user through real-time communication between the user and the virtual object, improve the efficiency and accuracy of the service, and improve the user's experience of medical consultation.

[0129] Visual content captured or uploaded by users may have geometric distortions, orientation deviations, etc., which can be corrected.

[0130] In some embodiments, in response to a correction operation on the visual content, the corrected visual content is displayed, wherein the second message is generated based on the corrected visual content and added information or is generated based on the corrected visual content.

[0131] like Figure 7 As shown, a correction control 709 can be displayed in the shooting interface 700, and the corrected visual content is displayed in response to the triggering operation of the correction control 709.

[0132] like Figure 12 The image shown is a display interface 1200 for the corrected image. Interface 1200 can display an information occlusion control 1201. In response to a trigger operation of the information occlusion control 1201, an occlusion element is displayed. In response to a drag operation on the occlusion element, information within the area corresponding to the drag trajectory is occluded. Interface 1200 can also display one or more candidate questions, for example, candidate question 1202. In response to a trigger operation on candidate question 1202, a second interface is displayed. The second interface displays a fourth message, which includes the corrected image, the candidate question, and a fifth message responding to the fourth message by a virtual object.

[0133] The method described in the above embodiments can correct visual content, improve the accuracy of visual content recognition, thereby improving the accuracy of response information generated by virtual objects and enhancing the user's medical consultation experience.

[0134] This disclosure also provides an information processing apparatus, which is described below in conjunction with... Figure 13 Describe it.

[0135] Figure 13 These are structural diagrams of some embodiments of the information processing apparatus disclosed herein. Figure 13 As shown, the information processing device 130 of this embodiment includes: a first display module 1301, a second display module 1302, and a third display module 1303.

[0136] The first display module 1301 is configured to display visual content and an information addition area in the first interface.

[0137] The second display module 1302 is configured to display a first message in a second interface in response to adding information through an information adding area, wherein the second interface is an interface for the user to interact with a virtual object, and the first message is determined based on visual content and the added information.

[0138] The third display module 1303 is configured to display, in the second interface, a second message in which the virtual object replies to the first message based on the second message.

[0139] The information processing apparatus of the above embodiment displays, in the first interface, visual content and an information adding area, displays, in the second interface in which the user dialogues with the virtual object, a first message in response to adding information through the information adding area, the first message being determined based on the visual content and the added information, and further displays, in the second interface, a second message in which the virtual object replies to the first message based on the second message. In the process in which the user interacts with the virtual object providing medical consultation, the information processing apparatus of the above embodiment can provide the user with an information adding area in the first interface for the visual content, the user can add information for the visual content through the information adding area to assist the virtual object in understanding the visual content, thereby improving the accuracy of the second message replied by the virtual object and reducing the number of interactions between the user and the virtual object to improve the interaction efficiency.

[0140] In some embodiments, the virtual object is a virtual object providing medical consultation, and the information adding area includes one or more information options and / or a first input component, the one or more information options being determined according to the visual content and being related to the medical consultation.

[0141] In some embodiments, the information adding area includes one or more information options, and the second display module 1302 is configured to display, in a selected state, a target option in the one or more information options in response to a selection operation on the target option; and display, in the second interface, the first message in response to a sending operation, wherein the added information includes information corresponding to the target option.

[0142] In some embodiments, the information adding area includes a first input component, and the input information is displayed in the first input component in response to an operation of inputting information in the first input component; and the first message is displayed in the second interface in response to a sending operation, wherein the added information includes the input information.

[0143] In some embodiments, the first message includes the visual content and the added information, the second message includes analysis information generated based on the first message, and the third display module 1303 is configured to display, in the second interface, a third message in which the virtual object replies in response to a modification operation on the analysis information, wherein the third message includes modified information corresponding to the analysis information and the visual content, and the analysis information is regenerated.

[0144] In some embodiments, the analysis information includes one or more editable contents, the third display module 1303 is configured to, in response to a triggering operation on a target content in the one or more editable contents, display a second input component; in response to inputting modification information corresponding to the target content through the second input component, display the modification information corresponding to the target content in the second input component, wherein the modification information corresponding to the analysis information includes modification information of the target content; and in response to a confirmation operation, display a third message in which the virtual object replies in the second interface.

[0145] In some embodiments, the visual content includes a plurality of sub-contents, the first display module 1301 is configured to, in response to obtaining each sub-content, display preview content of the currently obtained sub-content, a thumbnail of one or more sub-contents that have been obtained, and an information adding control in the first interface, wherein the one or more sub-contents that have been obtained include the currently obtained sub-content; and in response to a triggering operation on the information adding control, display an information adding area in the first interface.

[0146] In some embodiments, the first display module 1301 is configured to, in response to a selection operation on a target sub-content in the one or more sub-contents that have been obtained and a triggering operation on the information adding control, display preview content of the target sub-content and the information adding area in the first interface, wherein the added information includes information added for the target sub-content through the information adding area.

[0147] In some embodiments, the information processing apparatus 130 further includes an output module 1304 configured to, in response to an operation of starting the camera component, display a captured real-time picture, and output guide information, wherein the guide information includes information for guiding a shooting operation, the guide information is generated based on the real-time picture; and the first display module 1301 is configured to, in response to the shooting operation based on the guide information, obtain the visual content.

[0148] In some embodiments, the output module 1304 is configured to display a starting control in the second interface; in response to a triggering operation on the starting control, display a shooting interface and display a real-time preview picture in the shooting interface, wherein the operation of starting the camera component includes the triggering operation on the starting control, and the captured real-time picture includes the real-time preview picture; and display text of the guide information and / or play a voice of the guide information in the shooting interface.

[0149] In some embodiments, the guide information includes one or more pieces of guide information, the shooting interface includes a shooting control, and the first display module 1301 is configured to, in response to an operation of adjusting shooting based on the one or more pieces of guide information and one or more triggering operations on the shooting control, obtain one or more images shot as the visual content, wherein the first interface includes a preview interface of the one or more images shot.

[0150] In some embodiments, the photographing interface includes a voice interaction control, and the output module 1304 is configured to, in response to a triggering operation on the voice interaction control, enter a voice interaction mode, display text of the guide information in the photographing interface, and play a voice of the guide information.

[0151] In some embodiments, the one or more pieces of guide information are determined based on identification of one or more images corresponding to the periodically acquired real-time preview picture, or the one or more pieces of guide information are determined based on identification of a video stream corresponding to the real-time preview picture.

[0152] In some embodiments, the output module 1304 is configured to display an opening control in the second interface; in response to a triggering operation on the opening control, display the photographing interface and display the real-time preview picture and a real-time call control in the photographing interface; in response to a triggering operation on the real-time call control, display a real-time call interface of the user and the virtual object, and display a picture of the real-time call in the real-time call interface and play a voice of the guide information, wherein the operation of opening the camera assembly includes the triggering operation on the opening control, and the captured real-time picture includes the picture of the real-time call.

[0153] In some embodiments, the guide information includes one or more pieces of guide information, the one or more pieces of guide information are determined based on identification of a video stream corresponding to the picture of the real-time call, and the first display module 1301 is configured to, in response to an operation of adjusting the photographing based on the one or more pieces of guide information, acquire the video stream of the real-time call as the visual content, wherein the first interface includes the real-time call interface.

[0154] In some embodiments, the output module 1304 is configured to, in response to an operation of opening the real-time call; display a real-time call interface of the user and the virtual object, and display a picture of the real-time call in the real-time call interface; during the process of the real-time call, receive a video stream of the user, and output reply information of the virtual object generated based on the video stream, wherein the reply information includes analysis information generated based on the video stream.

[0155] In some embodiments, the first display module 1301 is further configured to, in response to a correction operation on the visual content, display the corrected visual content, wherein the second message is generated based on the corrected visual content and the added information.

[0156] The present disclosure also provides an electronic device, which will be described below in conjunction with Figure 14 and 15 . Figure 14 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0157] As Figure 14As shown, the electronic device 14 comprises a processor 142; and a memory 141 coupled to the processor 142, for storing instructions, which are executed by the processor 142 to cause the processor 142 to perform the information processing method of any of the embodiments of the present disclosure.

[0158] The electronic device of the above embodiments displays visual content and an information adding area in the first interface, displays a first message in a second interface in which the user dialogues with the virtual object in response to adding information through the information adding area, the first message is determined based on the visual content and the added information, and further displays a second message replied by the virtual object based on the first message in the second interface. In the process of the user interacting with the virtual object providing medical consultation, the electronic device of the above embodiments can provide the user with an information adding area in the first interface for the visual content, the user can add information to the visual content through the information adding area to assist the virtual object in understanding the visual content, thereby improving the accuracy of the second message replied by the virtual object, reducing the number of interactions between the user and the virtual object, and improving the interaction efficiency.

[0159] The memory 141 is used to store one or more computer readable instructions. The memory 141 can include any combination of various forms of computer readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read only memory (ROM), flash memory. The memory 141 may, for example, store an operating system, application programs, a boot loader (BootLoader), a database, and other programs, and can also store various application programs and various data.

[0160] The processor 142 is used to run computer readable instructions to implement the information processing method of any of the above embodiments. The specific implementation of each step of the method can be referred to the above embodiments, and the repeated parts will not be described here.

[0161] The processor 142 can be embodied as various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The central processing unit (CPU) can be X86 or ARM architecture, etc.

[0162] The processor 142 and the memory 141 can communicate with each other directly or indirectly. For example, the processor 142 and the memory 141 can communicate through a network. The network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 142 and the memory 141 can also communicate with each other through a system bus, and the present disclosure does not limit this.

[0163] It should be noted that Figure 14 The components of the electronic device 14 shown are merely exemplary and are not intended to be limiting, and the electronic device 14 can also have other components according to actual application needs. The processor 142 can control other components in the electronic device 14 to perform desired functions.

[0164] The electronic device 14 can be implemented in software, firmware, and / or hardware, and can be integrated into a device in which a related application program is installed.

[0165] Figure 15 A block diagram of an electronic device according to other embodiments of the present disclosure is shown.

[0166] Figure 15 The electronic device 15 shown can be a computer system having a dedicated hardware structure, and can perform corresponding functions when a related application program is installed.

[0167] The electronic device includes, but is not limited to, a mobile terminal such as a smartphone, a notebook computer, a Personal Digital Assistant (PDA), a Tablet Personal Computer (Tablet PC), a PMP (Portable Multimedia Player), a vehicle terminal (e.g., a car navigation terminal), a wearable device, and the like, and a fixed terminal such as a digital television, a desktop computer, and the like.

[0168] As Figure 15 shown, a central processing unit (CPU) 151 performs various processes according to a program stored in a read-only memory (ROM) 152 or a program loaded from a storage portion 158 to a random access memory (RAM) 153. In the RAM 153, data required when the CPU 151 performs various processes, etc. is stored as needed. The central processing unit is merely exemplary, and can be other types of processors such as the various processors described above. The ROM 152, the RAM 153, and the storage portion 158 can be various forms of computer-readable storage media. It should be noted that, although Figure 15 The ROM 152, the RAM 153, and the storage portion 158 are shown separately in the above, but one or more of them can be combined or located in the same or different memory or storage module.

[0169] The CPU 151, the ROM 152, and the RAM 153 are connected to each other via the bus 154. The input / output interface 155 is also connected to the bus 154.

[0170] The following components are connected to the input / output interface 155: an input portion 156, such as a touch panel, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, and the like; an output portion 157, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage portion 158, including a hard disk, a magnetic disk, and the like; and a communication portion 159, including a network interface card, such as a LAN card, a modem, and the like. The communication portion 159 allows communication processing to be performed via a network, such as the Internet. It is readily understood that, although not shown, a memory, such as a random access memory (RAM), a read only memory (ROM), and the like, can be included in the electronic device 15. Figure 15 It is readily understood that, although it is shown in the electronic device 15 that the components communicate via the bus 154, they can also communicate through a network or other means, where the network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0171] A drive 1510 is also connected to the input / output interface 155, as necessary. A removable medium 1511, such as a magnetic disk, a magneto-optical disk, a semiconductor memory, and the like, is attached to the drive 1510, as necessary, so that a computer program read therefrom is installed into the storage portion 158, as necessary.

[0172] In the case where the above-described series of processes are implemented by software, the program constituting the software can be installed from a network, such as the Internet, or a storage medium, such as the removable medium 1511.

[0173] According to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. The present disclosure provides a computer program product including a computer program that, when executed by a processor, implements the information processing method of any embodiment of the present disclosure.

[0174] The computer program product of the above embodiment displays visual content and an information adding area in the first interface, displays a first message in a second interface in which the user dialogues with the virtual object in response to adding information through the information adding area, the first message being determined based on the visual content and the added information, and further displays a second message replied by the virtual object based on the first message in the second interface. In the process in which the user interacts with the virtual object providing medical consultation, the computer program product of the above embodiment can provide the user with an information adding area for the visual content in the first interface, and the user can add information for the visual content through the information adding area to assist the virtual object in understanding the visual content, thereby improving the accuracy of the second message replied by the virtual object, reducing the number of interactions between the user and the virtual object, and improving the interaction efficiency.

[0175] For example, some embodiments of the present disclosure include a computer program product that, when running on a computer, causes the computer to implement the method of any of the preceding embodiments. The computer program product includes computer instructions carried on a computer readable medium, containing program codes for executing the method shown in the flowchart. In such embodiments, the computer instructions can be downloaded and installed from the network through the communication part 159, or installed from the storage part 158, or installed from the ROM 152. When the computer program is executed by the CPU 151, the method of the embodiments of the present disclosure is executed.

[0176] The present disclosure also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements an information processing method.

[0177] The computer readable storage medium of the above embodiment displays visual content and an information adding area in the first interface, displays a first message in a second interface in which the user dialogues with the virtual object in response to adding information through the information adding area, the first message being determined based on the visual content and the added information, and further displays a second message replied by the virtual object based on the first message in the second interface. In the process in which the user interacts with the virtual object providing medical consultation, the computer readable storage medium of the above embodiment can provide the user with an information adding area for the visual content in the first interface, and the user can add information for the visual content through the information adding area to assist the virtual object in understanding the visual content, thereby improving the accuracy of the second message replied by the virtual object, reducing the number of interactions between the user and the virtual object, and improving the interaction efficiency.

[0178] It should be noted that, in the context of the present disclosure, the computer readable medium can be a tangible medium that can contain or store programs for use by or in conjunction with an instruction execution system, device or apparatus.

[0179] The computer readable medium can be a computer readable storage medium or a computer readable signal medium, or any combination thereof.

[0180] The computer readable storage medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this disclosure, a computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. The computer readable storage medium has stored thereon the computer instructions that, when executed by the processor, implement the method of any of the preceding embodiments.

[0181] The computer readable signal medium can include a computer readable program code propagated on or through a carrier wave in a baseband or passed on a carrier, in which the computer readable program code can be embodied. Such propagated computer readable signal medium can take many forms, including but not limited to electro-magnetic, optical or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable signal medium can be transmitted using any suitable medium, including but not limited to wire, cable, wireless, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0182] The computer readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device described above.

[0183] In some embodiments, a computer program is also provided, including instructions, which when executed by a processor, cause the processor to perform the method of any of the preceding embodiments. For example, the instructions can be embodied in a computer program code.

[0184] Computer program code for carrying out operations of the present disclosure can be written in any one or more of a variety of programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0185] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0186] The functions described above can be implemented in one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0187] While certain aspects of the present disclosure have been described with reference to one or more particular embodiments thereof, those skilled in the art will understand that many alternative embodiments can be made therefrom. In general, embodiments of the present disclosure are applicable to any suitable electronic device, system, or architecture. In addition, unless otherwise indicated, the functions performed by the various components described herein can be implemented using electronic components, software, firmware, or any suitable combination thereof. Merely by way of example, various aspects of the present disclosure can be implemented using hardware components, software components, and / or any suitable combination thereof.

Claims

1. An information processing method, comprising: The first interface displays visual content and an area for adding information; In response to adding information through the information adding area, a first message is displayed in the second interface, wherein the second interface is an interface for the user to interact with the virtual object, and the first message is determined based on the visual content and the added information; The second interface displays a second message from the virtual object in response to the first message.

2. The information processing method according to claim 1, wherein, The virtual object is a virtual object that provides medical consultation. The information addition area includes one or more information options and / or a first input component. The one or more information options are determined based on the visual content and are related to the medical consultation.

3. The information processing method according to claim 2, wherein, The information adding area includes one or more information options, and the step of displaying a first message on the second interface in response to adding information through the information adding area includes: In response to a selection operation of a target option among the one or more information options, the target option is displayed in a selected state; In response to the sending operation, the first message is displayed in the second interface, wherein the added information includes information corresponding to the target option.

4. The information processing method according to claim 2, wherein, The information adding area includes the first input component, and the step of displaying the first message on the second interface in response to adding information through the information adding area includes: In response to an operation of inputting information into the first input component, the input information is displayed in the first input component; In response to the sending operation, the first message is displayed in the second interface, wherein the added information includes the input information.

5. The information processing method according to claim 1, wherein, The first message includes the visual content and the added information, the second message includes analysis information generated based on the first message, and the information processing method further includes: In response to the modification operation on the analysis information, a third message from the virtual object is displayed on the second interface, wherein the third message includes analysis information regenerated based on the modification information corresponding to the analysis information and the visual content.

6. The information processing method according to claim 5, wherein, The analysis information includes one or more editable contents, and the third message displayed in the second interface in response to the modification operation of the analysis information includes: In response to a triggering operation on target content in one or more editable contents, a second input component is displayed; In response to inputting modification information corresponding to the target content through the second input component, the modification information corresponding to the target content is displayed in the second input component, wherein the modification information corresponding to the analysis information includes the modification information of the target content; In response to the confirmation operation, the third message replied by the virtual object is displayed in the second interface.

7. The information processing method according to claim 1, wherein, The visual content includes multiple sub-contents, and the display of the visual content and the information addition area in the first interface include: In response to acquiring each sub-content, the preview content of the currently acquired sub-content, the thumbnails of one or more acquired sub-contents, and the information addition control are displayed on the first interface, wherein the one or more acquired sub-contents include the currently acquired sub-content; In response to a trigger operation on the information adding control, the information adding area is displayed in the first interface.

8. The information processing method according to claim 7, wherein, The step of displaying the information adding area in the first interface in response to a trigger operation on the information adding control includes: In response to a selection operation on a target sub-content among one or more acquired sub-contents and a trigger operation on an information addition control, a preview of the target sub-content and the information addition area are displayed on the first interface, wherein the added information includes information added to the target sub-content through the information addition area.

9. The information processing method according to any one of claims 1-8, further comprising: In response to the operation of activating the camera component, the captured real-time image is displayed and guidance information is output, wherein the guidance information includes information for guiding the shooting operation, and the guidance information is generated based on the real-time image; In response to the shooting operation based on the guidance information, the visual content is acquired.

10. The information processing method according to claim 9, wherein, The operation of activating the camera component, displaying the captured real-time image, and outputting guidance information includes: The activation control is displayed in the second interface; In response to the triggering operation of the opening control, a shooting interface is displayed, and a real-time preview screen is displayed in the shooting interface, wherein the operation of opening the camera component includes the triggering operation of the opening control, and the captured real-time screen includes the real-time preview screen; The text of the guidance information is displayed and / or the audio of the guidance information is played in the shooting interface.

11. The information processing method according to claim 10, wherein, The guidance information includes one or more guidance messages, the shooting interface includes shooting controls, and the step of acquiring the visual content in response to a shooting operation based on the guidance information includes: In response to the operation of adjusting the shooting based on one or more guidance information and one or more trigger operations on the shooting control, one or more captured images are acquired as the visual content, wherein the first interface includes a preview interface of the one or more captured images.

12. The information processing method according to claim 10, wherein, The shooting interface includes a voice interaction control, and displaying the text of the guidance information and / or playing the voice of the guidance information on the shooting interface includes: In response to the triggering operation of the voice interaction control, the voice interaction mode is entered, the text of the guidance information is displayed on the shooting interface, and the voice of the guidance information is played.

13. The information processing method according to claim 10, wherein, The one or more guidance messages are determined based on the identification of one or more images corresponding to the periodically acquired real-time preview screen, or the one or more guidance messages are determined based on the identification of the video stream corresponding to the real-time preview screen.

14. The information processing method according to claim 9, wherein, The operation of activating the camera component, displaying the captured real-time image, and outputting guidance information includes: The activation control is displayed in the second interface; In response to the triggering operation of the opening control, the shooting interface is displayed, and a real-time preview and a real-time call control are displayed in the shooting interface; In response to a trigger operation on the real-time call control, a real-time call interface between the user and the virtual object is displayed, and the real-time call screen is displayed in the real-time call interface. The voice of the guidance information is played. The operation of activating the camera component includes triggering the activation control, and the captured real-time screen includes the screen of the real-time call.

15. The information processing method according to claim 14, wherein, The guidance information includes one or more pieces of guidance information, which are determined based on the identification of the video stream corresponding to the real-time call screen. The step of acquiring the visual content in response to the shooting operation based on the guidance information includes: In response to the operation of adjusting the shooting based on one or more guidance information, the video stream of the real-time call is acquired as the visual content, wherein the first interface includes the real-time call interface.

16. The information processing method according to any one of claims 1-8, further comprising: In response to the action of initiating a real-time call; Display the real-time call interface between the user and the virtual object, and display the real-time call screen in the real-time call interface; During the real-time call, the system receives the user's video stream and outputs response information generated by the virtual object based on the video stream, wherein the response information includes analysis information generated based on the video stream.

17. The information processing method according to any one of claims 1-8, further comprising: In response to the correction operation on the visual content, the corrected visual content is displayed, wherein the second message is generated based on the corrected visual content and the added information.

18. An information processing apparatus, comprising: The first display module is configured to display visual content and an information addition area in the first interface; The second display module is configured to display a first message in a second interface in response to adding information through the information adding area, wherein the second interface is an interface for the user to interact with a virtual object, and the first message is determined based on the visual content and the added information. The third display module is configured to display, in the second interface, a second message from the virtual object in response to the first message.

19. An electronic device comprising: processor; as well as A memory coupled to the processor is used to store instructions that, when executed by the processor, cause the processor to perform the information processing method according to any one of claims 1 to 17.

20. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the information processing method according to any one of claims 1 to 17.

Images