Information Processing Method, Apparatus, Electronic Device, and Storage Medium for Display Interface

Abstract

An embodiment of the present application provides a method, apparatus, electronic device, and storage medium for processing information of a display interface. Among them, the method includes: in response to a display instruction of the display interface, determining target information on the display interface; determining key information in the target information; analyzing the key information to determine supplementary information corresponding to the key information; and displaying the supplementary information on the display interface.

Description

Technical Field

[0001] This application relates to computer communication technologies, and in particular, to a method, apparatus, electronic device, and storage medium for processing information on a display interface. Background Art

[0002] The communication mode of online meetings has been increasingly applied. Sharing target information among meeting participants through desktop and application sharing is a common application mode of online meetings. However, different from on-site meetings, when there are many people in an online meeting, the electronic devices of the participants will be muted; thus, when the participants have questions about the content shared by the meeting sharer, they cannot give feedback in time, which affects the experience of the participants. Summary of the Invention

[0003] In view of this, the technical solutions of the embodiments of this application are implemented as follows:

[0004] The embodiments of this application provide a method, apparatus, electronic device, and storage medium for processing information on a display interface. The method includes:

[0005] In response to a display instruction of the display interface, determine the target information on the display interface;

[0006] Determine the key information in the target information;

[0007] Analyze the key information to determine the supplementary information corresponding to the key information;

[0008] On the display interface, display the supplementary information.

[0009] The embodiments of this application provide an apparatus for processing information on a display interface. The apparatus includes:

[0010] A first determination module, configured to determine the target information on the display interface in response to a display instruction of the display interface;

[0011] A second determination module, configured to determine the key information in the target information;

[0012] A third determination module, configured to analyze the key information to determine the supplementary information corresponding to the key information;

[0013] A display module, configured to display the supplementary information on the display interface.

[0014] The embodiments of this application provide an electronic device, which at least includes: a controller and a storage medium configured to store executable instructions, where:

[0015] The controller is configured to execute stored executable instructions, and the executable instructions are configured to execute the information processing method of the display interface described above.

[0016] Correspondingly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are configured to execute the information processing method of the display interface provided above.

[0017] An embodiment of the present application provides an information processing method, device, electronic device, and storage medium for a display interface. By responding to a display instruction of the display interface, target information on the display interface is determined, and then key information of the target information is analyzed to determine supplementary information corresponding to the key information, and the supplementary information is displayed at a corresponding position on the display interface; in this way, during a meeting, the supplementary information corresponding to the key information in the target information displayed on the display interface can be displayed in real time, and without the need for user participation, timely question answering can be provided for the meeting participants, so that the participants can timely obtain relevant knowledge information of the meeting content. Description of the Drawings

[0018] Figure 1 It is a schematic flowchart of the implementation of the information processing method for the display interface provided by the embodiment of the present application;

[0019] Figure 2 It is another schematic flowchart of the implementation of the information processing method for the display interface provided by the embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of the overall framework of the online meeting core word auxiliary interpretation system that combines the content and location information provided by the embodiment of the present application;

[0021] Figure 4 It is a schematic structural diagram of the information processing device for the display interface provided by the embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the composition structure of the electronic device provided by the embodiment of the present application. Detailed Embodiments

[0023] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] In subsequent descriptions, suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of description of the present application, and they have no specific meaning in themselves. Therefore, "module", "component", or "unit" can be used interchangeably.

[0025] The electronic device can be implemented in various forms. For example, the electronic device described in this application may include, such as, mobile phones, tablet computers, laptop computers, palmtop computers, personal digital assistants (PDAs), portable media players (PMPs), desktop computers, servers, and the like.

[0026] In the subsequent description, a mobile electronic device will be taken as an example for illustration. Those skilled in the art will understand that, except for the components specifically for mobile purposes, the structure according to the embodiments of this application can also be applied to fixed-type electronic devices.

[0027] In order to be able to understand the features and technical content of the embodiments of this application in more detail, the implementation of the embodiments of this application will be elaborated in detail below with reference to the accompanying drawings. The accompanying drawings are only for reference and illustration purposes and are not used to limit the embodiments of this application.

[0028] The embodiments of this application provide a method for processing information of a display interface. Figure 1 It is a schematic flowchart of the implementation of the method for processing information of the display interface provided by the embodiments of this application. As Figure 1 shown, the method for processing information of the display interface includes the following steps:

[0029] Step S101: In response to a display instruction of the display interface, determine the target information on the display interface.

[0030] Here, the display interface is the user interface displayed during the process of the user operating the terminal device. For example, it is the interface displayed on the screen of the terminal device during the process of the user running an application on the terminal device. In the embodiments of the present application, the display interface is the interface for all users participating in the meeting to play, display, or share the meeting content on their terminal devices; or it is the interface for the meeting host's terminal device to conduct meeting sharing; or it is the display content on the screen of the meeting sharing device. The display interface can also be referred to as the meeting interface. The way of holding the meeting can be an online video meeting or an offline meeting. The display instruction is the instruction for the meeting host or the moderator to click on the meeting content on the display interface of the terminal device to display the meeting content on the terminal devices of all participants for display; or it is the instruction to display the meeting content on the sharing device within the space where the participants are located. The target information is the words in the display content of the display interface, such as the words in a multimedia file, the content of a text file, or the words in a picture, and these words are related to the meeting theme. In some embodiments, the target information on the display interface can be determined according to the readability of the display content on the display interface. In some examples, the file format of the display content can be determined according to the readability of the display content on the display interface. When the display content is readable, the file format corresponding to the display content is determined. For example, if the file format is a text document (txt) or a word processor document (word), the display content is directly extracted to obtain the target information. When the display content on the display interface is not directly readable, that is, the file format of the display content is determined. For example, when the file format is a picture or a Portable Document Format (PDF), the display content of the display interface is recognized and extracted through an Optical Character Recognition (OCR) algorithm, and then the target information is determined according to the recognized text information. In some embodiments, when the content displayed on the display interface is a video file, the voice information read can be converted into text information through an application installed on the terminal that can read voice information, and then the target information can be determined according to the method of directly reading text information described above; or each frame image of the multimedia video playback is obtained, and then each frame image is recognized through the OCR algorithm to obtain the recognized text information, and further the target information is determined.

[0031] In some embodiments, during the process of text extraction from the display content of the display interface to determine the target information, the location information of the target information can also be determined. Regarding the page where the current display content with the target information is located as an image, determine the horizontal pixel number of the target information from the upper left corner of its page and the page width, the vertical pixel number of the target information from the upper left corner of its page and the page length, and determine [horizontal pixel number / page width, vertical pixel number / page length] as the location information of the target information.

[0032] Step S102: Determine the key information in the target information.

[0033] Here, the key information is the core content in the target information, which can be information with a high degree of relevance to the meeting content or theme, such as the core vocabulary of the meeting content, professional terms in the field corresponding to the meeting theme, etc. In some embodiments, the target information can be input into a pre-trained neural network. The pre-trained neural network is first trained with a large amount of data labeled with whether it is a core word (the input of the neural network is text and position coordinates, and the output is a judgment of whether it is key information 0 (no) / 1 (yes)). After training, the target information or the target information and the position information of the characters in the target information are input into the pre-trained neural network, and the neural network can output the key information in the target information. In some embodiments, after determining the key information, the key information can also be highlighted in the target information.

[0034] Step S103: Analyze the key information to determine the supplementary information corresponding to the key information.

[0035] Here, after determining the key information of the target information, the key information can be searched and matched in a preset database to determine the supplementary information that semantically matches the key information. The supplementary information is information that explains the key information, such as the semantics corresponding to the key information, the application scenario of the key information, the role of the key information, etc. In some embodiments, the key information can be first searched in the database corresponding to the general search engine. If information matching the key information is found, the found information is determined as the supplementary information corresponding to the key information. If no information matching the key information is found in the database corresponding to the general search engine, then search in the professional domain graph. The professional domain graph is a knowledge graph including information related to a specific field corresponding to the key information. It mainly includes: entities, relationships (between entities), attributes (of entities and relationships), etc. If information matching the key information is found in the professional domain graph, the information is determined as the supplementary information of the key information.

[0036] Step S104: Display the supplementary information on the display interface.

[0037] Here, after determining the supplementary information corresponding to the key information of the target information on the display interface, the supplementary information can be displayed on the display interface.

[0038] In some embodiments, the position of the blank area at the key information can be determined according to the position of the key information in the target information, and the supplementary information is displayed in the blank area, or the display interface can be divided into two parts, one part for displaying the meeting display content and the other part for displaying the supplementary information.

[0039] In the embodiments of the present application, by responding to the display instruction of the display interface, the target information on the display interface is determined, and then the key information of the target information is analyzed to determine the supplementary information corresponding to the key information, and the supplementary information is displayed at the corresponding position on the display interface; in this way, during the meeting process, the supplementary information corresponding to the key information in the target information displayed on the display interface can be displayed in real time, and without the participation of the user, timely question answering can be provided for the meeting participants, so that the participants can obtain the relevant knowledge information corresponding to the meeting in a timely manner.

[0040] In some realizable implementation manners, the target information of the display interface can be determined according to the readability of the content presented on the display interface. That is, step S101 can be implemented through the following steps:

[0041] Step S1011: Respond to the display instruction to determine the readability of the content presented on the display interface.

[0042] Here, the content presented on the display interface is the content displayed on the display interface after responding to the display instruction. In some embodiments, the readability of the presented content can be determined according to whether the displayed content can be directly read or directly edited.

[0043] Step S1012: When the content presented on the display interface is readable, determine the file format of the content presented on the display interface.

[0044] Here, readable means that the presented content can be directly read or edited. When the displayed content is readable, the file format corresponding to the displayed content is determined, such as the file format is a text document txt or word.

[0045] Step S1013: Read the content presented on the display interface according to the file format to obtain the target information.

[0046] Here, when the content presented on the display interface is readable and the file format of the presented content is determined, the presented content can be directly read to obtain the target information.

[0047] In the embodiments of the present application, when the display content on the display interface is readable, the file format corresponding to the display content can be determined. Then, according to the file format, the presentation content can be read, so that the target information on the display interface can be accurately determined.

[0048] In some implementable embodiments, when the target information at least includes text information, the text information in the target information can also be obtained by the following method:

[0049] When the content presented on the display interface is unreadable, use optical character recognition on the content presented on the display interface to obtain the text information.

[0050] Here, unreadable means that the content presented on the display interface cannot be directly edited or directly read. When the presented content is unreadable, the file format of the presented content can be determined. For example, when the file format is a picture or a PDF document, the OCR algorithm is used to perform text recognition on the display content of the display interface to obtain the text information corresponding to the presented content, and then the target information can be determined.

[0051] In the embodiments of the present application, according to the readability of the display content on the display interface, a combination of the standard format, that is, the method of direct reading, and the method of OCR recognition and reading can be used to extract the display content on the display interface, realizing the accurate recognition and reading of the display content on the display interface.

[0052] In some implementable embodiments, a trained neural network can be used to determine the key information in the target information; wherein, the trained neural network at least includes: a bidirectional encoding representation model based on a transformer.

[0053] Here, the trained neural network is a bidirectional encoding representation model based on a transformer (Bidirectional Encoder Representation from Transformers, BERT). The model architecture of BERT is based on multi-layer bidirectional transformation decoding, that is, when the model processes a certain word, it can use the information of both the previous word and the subsequent word at the same time, randomly mask some words, and use all the unmasked words for prediction. The BERT neural network will first be trained with a large amount of data marked with whether it is a keyword (the input of the neural network is the target information and the position information, and the output is the judgment of whether it is a keyword 0 (no) / 1 (yes)). After training, the target information or the target information and the position information are jointly input into the neural network, and the neural network can output whether it is a keyword. The meeting documents shared by users will contain many words, and only a part of the words are relatively key in the document and also need to supplement information to help other meeting participants understand.

[0054] In the embodiments of the present application, by combining the target information and the location information and using the trained neural network architecture, the discovery of key words is carried out, improving the accuracy rate.

[0055] In some implementable embodiments, the supplementary information can be determined according to the description information of the key information, that is, the above step S103 can be implemented through the following steps:

[0056] Step S1031: In the first database, search for the description information matching the key information;

[0057] Here, the first database is the database corresponding to the general search engine, such as the database corresponding to Baidu. After determining the key information, the key information will be input into the first database to search for the associated information corresponding to the key information, that is, search for the description information matching the key information.

[0058] Step S1032: Analyze the searched description information and determine that the description information is the supplementary information corresponding to the key information.

[0059] Here, after searching for the description information matching the associated information in the first database, determine the relevance between the description information and the theme of the current meeting. If the relevance is greater than the preset threshold, such as greater than 0.9, then determine the description information as the supplementary information corresponding to the key information.

[0060] In some implementable embodiments, the supplementary information of the key information can also be determined through the following steps:

[0061] Step 1: In response to not searching for the description information, search for the description information matching the key information in the second database;

[0062] Here, in the case where no description information matching the key information is searched in the first database, then determine the field corresponding to the key information and continue to search for the description information in the second database according to the field. The second database is a professional domain graph, and the professional domain graph is a knowledge graph including information related to a specific field corresponding to the key information. The professional domain graph mainly includes: entities, relationships (between entities), attributes (entities and relationships), etc.

[0063] Step 2: Determine the description information matching the key information searched in the second database as the supplementary information corresponding to the key information.

[0064] Here, after searching for the description information matching the associated information in the second database, directly determine the description information as the supplementary information corresponding to the key information.

[0065] In the embodiments of the present application, the supplementary information of the key information can be determined by combining the search in the first database and the retrieval in the second database, and more accurate and professional supplementary information can be obtained, improving the accuracy of determining the supplementary information.

[0066] In some implementable embodiments, after determining the supplementary information corresponding to the key information of the target information on the display interface, the supplementary information can also be displayed on the display interface, that is, the supplementary information can be implemented through the following steps:

[0067] Step 1: In response to the display instruction, determine the position information of the target information on the display interface;

[0068] Here, when receiving the instruction to display the meeting content, determine the current display content of the meeting. Take the page where the target information is located in the current display content as a picture, determine the horizontal pixel number and page width of the target information from the upper left corner of its page, the vertical pixel number and page length of the target information from the upper left corner of its page, and determine the coordinate [horizontal pixel number / page width, vertical pixel number / page length] as the position information of the target information.

[0069] Step 2: Display the supplementary information according to the position information.

[0070] Here, after determining the key information included in the target information, according to the position information of the target information, the position information of the key information can be obtained. After determining the supplementary information of the key information, according to the position information of the key information, determine the area of the key information on the display interface, and display the supplementary information corresponding to each key information in other areas on the display interface.

[0071] In some embodiments, after determining the position information of the key information according to the position information of the target information, determine the blank area within the range of its position area according to the position information of the key information, and display the position information in the blank area. Moreover, the blank area is the area closest to the position of the key information. The blank area is the area where there is no text or picture and there is only a placeholder without any content.

[0072] In the embodiments of the present application, after determining the supplementary information corresponding to the key information of the content displayed on the current display interface, the supplementary information is displayed on the display interface, so that the participants can obtain the supplementary information in a timely manner.

[0073] In some implementable embodiments, the following steps can be used to display the supplementary information according to the position information:

[0074] Step 1: Determine the shared area of the display interface according to the position information.

[0075] Here, the shared area is the area for presenting the meeting content. In some embodiments, the display interface can be divided into at least two parts, one part is the shared area, and the other part is the supplementary area. The supplementary area can also be divided into multiple sub-areas. After determining the position information of the target information, the shared area where the target information is located and the position of the shared area can be determined according to the position information of the target information.

[0076] Step Two: Determine the supplementary area of the display interface according to the position of the shared area.

[0077] Here, the display interface consists of a shared area and a supplementary area. After determining the position of the shared area, the area on the display interface that is outside the shared area and consists of a continuous plurality of blank placeholders is determined as the supplementary area.

[0078] Step Three: Present the supplementary information in the supplementary area.

[0079] Here, the supplementary information is presented in the supplementary area. In some embodiments, in the case of an offline meeting, the supplementary information can be presented in the supplementary area on the display interface of the terminal device of the meeting host. When the terminal device is connected to the shared device, all participants can obtain the supplementary information through the display interface displayed on the shared device. In the case of an online meeting, since the display interfaces of each participant are the same, the supplementary information can be directly presented in the supplementary area on the display interface of each participant.

[0080] In the embodiments of the present application, after determining the supplementary information, the supplementary information is presented in the supplementary area of the display interface, so that the participants can obtain the supplementary information in a timely manner.

[0081] The embodiments of the present application provide a method for processing information of a display interface. Figure 2 For another schematic flowchart of the implementation process of the method for processing information of the display interface provided by the embodiments of the present application, as Figure 2 shown, the method for processing information of the display interface includes the following steps:

[0082] Step S201: Extract the target information and position information of the display interface.

[0083] Here, the accurate extraction of the target information and position information on the display interface is the premise for subsequent key information supplementation. The embodiments of the present application can directly read types in standard rules (such as word, txt, etc.) and types that cannot be directly read (such as pictures, PDFs), and provide two methods, namely extraction according to the standard format and extraction through the OCR recognition algorithm, to determine the target information and position information of the display interface.

[0084] Step S202: Determine the key information based on the neural network.

[0085] Here, the accurate determination of the key information in the meeting materials is a crucial step for information supplementation. In the embodiments of the present application, a typical neural network architecture (such as BERT) is adopted. Using the target information and location information extracted in step S201 as the input, the key information in the meeting materials is determined.

[0086] Step S203: Determine the supplementary information of the key information.

[0087] Here, based on the discovery of the key information in step S202, the key information is supplemented. For the core words in the general domain and the core words in the professional domain, there are two methods: retrieving information through a general domain search engine and retrieving information through a professional domain knowledge graph.

[0088] In the embodiments of the present application, the method for extracting the target information of the display interface by combining standard format reading and OCR recognition can extract the target information in the meeting materials in most scenarios; moreover, by combining the target information and location information and using the BERT neural network architecture, the key information of the target information is determined, resulting in a higher accuracy rate; by combining the general domain search engine and the professional domain knowledge graph for retrieval, the supplementary information of the key information can be discovered more accurately.

[0089] The embodiments of the present application provide an overall framework diagram of an online meeting core word auxiliary interpretation system that combines content and location information. Figure 3 For the schematic diagram of the overall framework diagram of the online meeting core word auxiliary interpretation system provided by the embodiments of the present application, as Figure 3 shown, the online meeting core word auxiliary interpretation system that combines content and location information includes three steps:

[0090] Step S301: Determine the target information and location information of the meeting interface.

[0091] Here, the meeting interface is the interface for playing, displaying, or sharing the meeting content on the terminal devices of all users participating in the meeting; or the interface for meeting display and sharing on the terminal device of the meeting host; or the interface for displaying the meeting content on the screen of the meeting sharing device. The meeting interface is the display interface in the embodiments of the present application. Step S301 can be implemented through the following steps S3011 to S3014:

[0092] Step S3011: Determine whether the content of the current meeting interface can be directly read.

[0093] First, determine whether the display content in the sharing area 32 of the current meeting interface 31 can be directly read. If it can be directly read, execute steps S3012 and S3014; if it cannot be directly read, execute steps S3013 and S3014.

[0094] Step S3012: Directly read according to the file format.

[0095] Step S3013: Perform OCR recognition.

[0096] Here, when the display content in 32 cannot be directly read, OCR recognition is used to determine the text information corresponding to the display content.

[0097] Step S3014: Determine the target information and location information.

[0098] Here, in one example, through steps S3011 to S3014, the target information and location information in the current meeting interface can be determined. For example, Figure 3 as shown in 33: The target information is the text "underfitting", and the location information of the target information is [0.1, 0.3].

[0099] Step S302: Determine the key information according to the target information and location information.

[0100] Here, step S301 can be implemented through the following steps S3021 to S3023:

[0101] Step S3021: Input the target information and location information into the neural network.

[0102] Here, an input sample is constructed with the target information and location information and input into the pre-trained neural network BERT model. In the neural network, word encoding 34 and position encoding 35 are used and concatenated, and then through the decision layer 36, the result of whether it is key information is output.

[0103] Step S3022: Determine whether it is key information.

[0104] Here, when the output result of the neural network is yes, it is determined that the input target information is key information, and then jump to S3031 in step S303. When the output result is no, it is determined that the input target information is non-key information, and then step S3023 is executed, that is, the process ends without further processing. 37 is an example of determining key information. The target information [underfitting] and the location information [0.1, 0.3] are input into the neural network, and the neural network outputs the judgment result of whether it is key information.

[0105] Step S3023: End.

[0106] Step S303: Determine the supplementary information of the key information.

[0107] Here, Step S303 can be implemented through Steps S3031 to S3033:

[0108] Step S3031: Search for the key information in the first database.

[0109] Here, the first database is a public domain search engine, that is, a general search engine, such as Baidu or Google, etc.

[0110] Step S3032: Determine whether supplementary information is found.

[0111] Here, in the case where supplementary information matching the key information can be found in the first database, the supplementary information is displayed in the supplementary area 38 of the conference interface 31. In the case where supplementary information matching the key information is not found in the first database, Step S3033 is executed.

[0112] Step S3033: Search for the key information in the second database.

[0113] Here, the second database is a professional domain graph. The professional domain graph is a database constructed based on professional terms and corresponding information in different technical fields. The descriptive information found in the second database is determined as the supplementary information of the key information, and the supplementary information is displayed in the supplementary area 38 of the conference interface 31. Figure 3 39 in is an example of the supplementary information for the key information "underfitting". In the case where it is not found in the first database, a search is performed in the second database, and finally the supplementary information for the key information underfitting is obtained: An important topic in machine learning is the generalization ability of the model. A model with strong generalization ability is a good model. For a trained model, if it performs poorly on the training set and also performs poorly on the test set, this may be caused by underfitting. Underfitting means that the degree of fitting of the model is not high, the data is far from the fitting curve, or it means that the model does not capture the data features well and cannot fit the data well, and the specific content of the supplementary information for underfitting is displayed in the supplementary area 38.

[0114] In the embodiments of the present application, through the method for extracting target information of the conference interface that combines standard format reading and OCR recognition, the target information in conference materials in most scenarios can be extracted; moreover, by combining the target information and location information and using the BERT neural network architecture, the key information of the target information is determined, resulting in higher accuracy; through the combination of a general domain search engine and a professional domain knowledge graph retrieval, the supplementary information of the key information can be found more accurately.

[0115] The present application provides an information processing device for a display interface.Figure 4 Schematic structural diagram of an information processing device for a display interface provided by an embodiment of the present application, as Figure 4 shown, the information processing device 400 for the display interface includes:

[0116] A first determination module 401, configured to determine target information on the display interface in response to a display instruction for the display interface;

[0117] A second determination module 402, configured to determine key information in the target information;

[0118] A third determination module 403, configured to analyze the key information to determine supplementary information corresponding to the key information;

[0119] A display module 404, configured to display the supplementary information on the display interface.

[0120] In the above device, the first determination module 401 includes:

[0121] A first determination sub-module, configured to determine the readability of the content presented on the display interface in response to the display instruction;

[0122] A second determination sub-module, configured to determine the file format of the content presented on the display interface when the content presented on the display interface is readable;

[0123] A third determination sub-module, configured to read the content presented on the display interface according to the file format to obtain the target information.

[0124] In the above device, the target information at least includes text information, and the device further includes:

[0125] An identification module, configured to perform optical character recognition on the content presented on the display interface to obtain the text information when the content presented on the display interface is unreadable.

[0126] In the above device, the second determination module 402 includes:

[0127] A fourth determination sub-module, configured to determine key information in the target information by using a trained neural network; wherein, the trained neural network at least includes: a bidirectional encoder representation from transformers model.

[0128] In the above device, the third determination module 403 includes:

[0129] A first search sub-module, configured to search for descriptive information matching the key information in a first database

[0130] An analysis sub-module, configured to analyze the retrieved description information to determine that the description information is supplementary information corresponding to the key information.

[0131] In the above device, the third determination module 403 further includes:

[0132] A second search sub-module, configured to search for description information matching the key information in a second database in response to failure to retrieve the description information;

[0133] A fifth determination sub-module, configured to determine the description information matching the key information retrieved in the second database as supplementary information corresponding to the key information.

[0134] In the above device, the device further includes:

[0135] A fifth determination module, configured to determine the position information of the target information on the display interface in response to the display instruction;

[0136] A display module, configured to display the supplementary information according to the position information.

[0137] In the above device, the display module includes:

[0138] A sixth determination sub-module, configured to determine the shared area of the display interface according to the position information;

[0139] A seventh determination sub-module, configured to determine the supplementary area of the display interface according to the position of the shared area;

[0140] A display sub-module, configured to display the supplementary information in the supplementary area.

[0141] Correspondingly, an embodiment of the present application provides an electronic device. Figure 5 The following is a schematic structural diagram of the electronic device provided by the embodiment of the present application. As Figure 5 shown, the electronic device 500 at least includes: a controller 501 and a storage medium 502 configured to store executable instructions, where:

[0142] The controller 501 is configured to execute the stored executable instructions, and the executable instructions are used to implement the information processing method for the provided display interface.

[0143] It should be noted here that: the descriptions of the above storage medium and device embodiments are similar to those of the above method embodiments and have similar beneficial effects to the method embodiments. For the technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0144] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the "in one embodiment" or "in an embodiment" that appears throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics may be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the magnitude of the serial numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0145] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0146] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling or communication connection between the components shown or discussed with each other can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0147] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] In addition, each functional unit in the embodiments of the present application can be all integrated in a processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in a unit; the above integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0149] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes: various media that can store program codes such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs.

[0150] Alternatively, if the above integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence or the part that contributes to the related art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present application. And the foregoing storage medium includes: various media that can store program codes such as removable storage devices, ROM, magnetic disks, or optical discs.

[0151] The above is only the implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An information processing method for a display interface, the method comprising: Responding to a display instruction of the display interface to determine target information on the display interface; Using a trained neural network to determine key information in the target information; wherein, the trained neural network at least includes: a bidirectional encoding representation model based on a transformer; Searching in a first database for descriptive information matching the key information; wherein, the first database includes a database corresponding to a general search engine; Determining the relevance between the descriptive information and the theme of the ongoing meeting, and when the relevance is greater than a preset threshold, determining the descriptive information as supplementary information corresponding to the key information; Responding to the situation that the descriptive information is not found, searching in a second database for descriptive information matching the key information; wherein, the second database includes a professional domain atlas; Determining the descriptive information matching the key information searched in the second database as supplementary information corresponding to the key information; Determining the page on which the target information is located on the current display interface; Determining the horizontal pixel number and page width of the target information from the upper left corner of the page where it is located; Determining the vertical pixel number and page length of the target information from the upper left corner of the page where it is located; Determining the coordinates corresponding to the horizontal pixel number / page width and the vertical pixel number / page length as the position information of the target information; Displaying the supplementary information according to the position information.

2. The method according to claim 1, wherein the responding to a display instruction of the display interface to determine target information on the display interface comprises: Responding to the display instruction to determine the readability of the content presented on the display interface; When the content presented on the display interface is readable, determining the file format of the content presented on the display interface; Reading the content presented on the display interface according to the file format to obtain the target information.

3. The method according to claim 2, wherein the target information at least includes text information, and the method further comprises: When the content presented on the display interface is unreadable, using optical character recognition to recognize the content presented on the display interface to obtain the text information.

4. The method according to claim 1, wherein the displaying the supplementary information according to the position information comprises: Determining a shared area of the display interface according to the position information; Determining a supplementary area of the display interface according to the position of the shared area; Displaying the supplementary information in the supplementary area.

5. An information processing device for a display interface, the device comprising: A first determination module, configured to respond to a display instruction of the display interface to determine target information on the display interface; A second determination module, configured to use a trained neural network to determine key information in the target information, wherein the trained neural network at least includes: a bidirectional encoding representation model based on a transformer; A third determination module, configured to search for description information that matches the key information in the first database; determine the degree of association between the description information and the theme of the currently ongoing meeting, and determine the description information as the supplementary information corresponding to the key information when the degree of association is greater than a preset threshold; in response to the failure to search for the description information, search for description information that matches the key information in the second database; and determine the description information that matches the key information searched in the second database as the supplementary information corresponding to the key information; wherein, the first database includes a database corresponding to a general search engine; and wherein, the second database includes a professional domain graph. A display module, configured to determine the page on which the target information is located on the current display interface; determine the horizontal pixel quantity and the page width of the target information from the upper left corner of the page on which it is located; determine the vertical pixel quantity and the page length of the target information from the upper left corner of the page on which it is located; determine the coordinates corresponding to the horizontal pixel quantity / page width and the vertical pixel quantity / page length as the position information of the target information; and display the supplementary information according to the position information.

6. A computer-readable storage medium storing computer-executable instructions configured to execute the information processing method for a display interface provided in any one of claims 1 to 4 above.

Images