Interaction method, device, electronic device and storage medium based on large model

By using a large-model-based interaction method and structured feedback information, the efficiency of information acquisition and browsing on smart terminal devices is improved, solving the problem of users' inability to conveniently obtain information.

CN118606590BActive Publication Date: 2024-11-26BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411080796.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-26
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

When users browse web pages through smart terminal devices, they find it difficult to conveniently obtain the information they need, requiring multiple interactive operations and taking a long time.

Method used

By using a large-model-based interaction method, the system responds to user interactions, determines relevant page information for the target page, and uses the large model to process the page information to generate structured feedback information, which is then presented to the user to improve information acquisition efficiency.

Benefits of technology

It improved the efficiency of user information acquisition and browsing, enhanced the user's interactive experience, and reduced the frequency of interactive operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118606590B_ABST
    Figure CN118606590B_ABST
Patent Text Reader

Abstract

The present invention provides an interactive method, device, electronic device and storage medium based on a large model, which relates to the field of artificial intelligence technology, especially to the technical fields of intelligent search, deep learning, large models, generative models, etc., and can be applied to application scenarios such as generative search, document intelligent editing, intelligent assistants, Internet e-commerce, and intelligent video browsing. The method includes: responding to the interactive operation of the object on the target page, determining page information related to the target page according to the interactive operation; determining structured feedback information corresponding to the interactive operation of the object according to the page information, wherein the structured feedback information is obtained by processing the page information using the large model; and displaying the structured feedback information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to technical fields such as intelligent search, deep learning, large models, and generative models. It can be applied to application scenarios such as generative search, intelligent document editing, intelligent assistants, Internet e-commerce, and intelligent video browsing. Background Art

[0002] With the popularization of smart terminal devices such as smartphones and tablet computers, users can conveniently browse page information such as text, video, and images through the display screens of smart terminal devices, allowing users to improve the efficiency of obtaining information. Summary of the invention

[0003] The present invention provides an interactive method, device, electronic equipment, storage medium and program product based on a large model.

[0004] According to one aspect of the present invention, there is provided an interaction method based on a big model, comprising: in response to an interaction operation of an object with respect to a target page, determining page information related to the target page according to the interaction operation; determining structured feedback information corresponding to the interaction operation of the object according to the page information, wherein the structured feedback information is obtained by processing the page information using the big model; and displaying the structured feedback information.

[0005] According to another aspect of the present invention, there is provided an interactive device based on a large model, comprising: a page information determination module, for responding to an interactive operation of an object on a target page, and determining page information related to the target page according to the interactive operation; a structured feedback information determination module, for determining structured feedback information corresponding to the interactive operation of the object according to the page information, wherein the structured feedback information is obtained by processing the page information using the large model; and a first display module, for displaying the structured feedback information.

[0006] According to another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method provided by an embodiment of the present invention.

[0007] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method provided according to an embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein the computer program implements the method provided according to an embodiment of the present invention when executed by a processor.

[0009] According to an embodiment of the present invention, page information related to the target page is obtained through interactive operations of browsing objects of the target page, and the structured feedback information obtained by processing the page information according to the large model is displayed to the object, so that structured display of page information can be achieved based on the structured feedback information, so that the object can improve the user's information acquisition efficiency and information browsing efficiency by browsing the structured feedback information, thereby improving the user's interactive experience.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0012] Figure 1 An exemplary system architecture to which a large model-based interaction method and device can be applied according to an embodiment of the present invention is shown;

[0013] Figure 2 A flow chart of an interactive method based on a large model according to an embodiment of the present invention is shown;

[0014] Figure 3 An application scenario diagram of the interaction method based on a large model according to an embodiment of the present invention is shown;

[0015] Figure 4 An application scenario diagram of an interaction method based on a large model according to another embodiment of the present invention is shown;

[0016] Figure 5 An application scenario diagram of an interaction method based on a large model according to another embodiment of the present invention is shown;

[0017] Figure 6 An application scenario diagram of an interaction method based on a large model according to yet another embodiment of the present invention is shown;

[0018] Figure 7 A block diagram showing an interactive device based on a large model according to an embodiment of the present invention; and

[0019] Figure 8 A block diagram of an electronic device suitable for implementing a large model-based interaction method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0021] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0022] The inventors have found that when users browse pages through terminal devices such as smartphones, it is difficult to obtain the required information accurately and conveniently, and usually multiple interactive operations (such as sliding pages) are required to take a long time to obtain the required information.

[0023] The embodiment of the present invention provides an interaction method, device, electronic device, storage medium and program product based on a large model. The interaction method based on a large model includes: responding to an interaction operation of an object on a target page, determining page information related to the target page according to the interaction operation; determining structured feedback information corresponding to the interaction operation of the object according to the page information, wherein the structured feedback information is obtained by processing the page information using the large model; and displaying the structured feedback information.

[0024] According to an embodiment of the present invention, page information related to the target page is obtained through interactive operations of browsing objects of the target page, and the structured feedback information obtained by processing the page information according to the large model is displayed to the object, so that structured display of page information can be achieved based on the structured feedback information, so that the object can improve the user's information acquisition efficiency and information browsing efficiency by browsing the structured feedback information, thereby improving the user's interactive experience.

[0025] Figure 1 An exemplary system architecture to which a large model-based interaction method and apparatus can be applied according to an embodiment of the present invention is shown.

[0026] It should be noted that Figure 1The examples shown are only examples of system architectures to which the embodiments of the present invention can be applied, in order to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, the exemplary system architecture to which the interactive method and apparatus based on the large model can be applied may include a terminal device, but the terminal device may implement the interactive method and apparatus based on the large model provided by the embodiments of the present invention without interacting with the server.

[0027] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0029] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0030] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0031] The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server can also be a server of a distributed system, or a server combined with blockchain.

[0032] It should be noted that the interaction method based on the large model provided in the embodiment of the present invention can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the interaction device based on the large model provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0033] Alternatively, the interaction method based on the big model provided in the embodiment of the present invention can also be generally performed by the server 105. Accordingly, the interaction device based on the big model provided in the embodiment of the present invention can generally be set in the server 105. The interaction method based on the big model provided in the embodiment of the present invention can also be performed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the interaction device based on the big model provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0035] Figure 2 A flow chart of an interaction method based on a large model according to an embodiment of the present invention is shown.

[0036] like Figure 2 As shown, the large model-based interaction method includes operations S210 to S240.

[0037] In operation S210 , in response to an interactive operation of an object on a target page, page information related to the target page is determined according to the interactive operation.

[0038] In operation S220 , structured feedback information corresponding to the interactive operation of the object is determined according to the page information.

[0039] In operation S230 , structured feedback information is displayed.

[0040] According to an embodiment of the present invention, structured feedback information is obtained by processing page information using a large model.

[0041] According to an embodiment of the present invention, the interactive operation for the target page may include any type of operation, for example, it may include gesture interactive operations such as click operations, but is not limited to this, and may also include other types of operations such as voice input operations. The embodiment of the present invention does not limit the specific type of interactive operation, and technical personnel in this field can choose according to actual needs.

[0042] According to an embodiment of the present invention, a large model may refer to a deep learning model with large-scale model parameters, and a large model usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. The large model may include a large-scale language model (LLM), GPT (Generative Pre-trained Transformer), a large visual model, a multimodal large model, and the like. The large model involved in the embodiment of the present invention may be a general large model, or it may be an expert large model obtained after fine-tuning (Fine Tune) based on demand, and the embodiment of the present invention does not limit this.

[0043] According to an embodiment of the present invention, the target page may include the page that the object is browsing. The target page may include any type of page such as a search results page, a video playback page, a details page, a list page, a map navigation page, etc. The embodiment of the present invention does not limit the specific type of the target page.

[0044] According to an embodiment of the present invention, the page information may include information in the target page, such as text, images, videos, and other information of the target page. However, the page information may also include information in an associated page whose similarity with the target page satisfies a page similarity condition. The associated page may be, for example, a jump page linked to the target page, but is not limited thereto, and may also include pages with high similarity to the theme and key information of the target page, such as pages for introducing the same tourist destination, pages associated with the page topic of the target page, and the like.

[0045] According to an embodiment of the present invention, the structured feedback information is obtained after the large model processes the page information. For example, the structured feedback information can be obtained by using the large model to fuse the information content with the same text structure level (or text structure level) in the page information, and the structured feedback information can have a text structure level. The structural hierarchical relationship between multiple structured feedback information can be obtained by aggregating and semantically summarizing the information content in the page information based on the semantic understanding ability of the large model. The structured feedback information can be the key information of each structural level output by the large model when the information content of the page information is fully understood. The key information of the same structural level can be the information content with the same structural level in the aggregated page information. The large model can perform semantic processing processes such as deduplication, semantic aggregation, and redundant information deletion on the information content with the same structural level in the page information of the same structural level, so that the structured feedback information output by the large model can clearly display the key information content in the page information based on the structural level and the aggregated key information.

[0046] It should be noted that the structured feedback information may include data in any format such as text, images, videos, charts, tables, etc., and the embodiments of the present invention do not limit the specific format of the information contained in the structured feedback information. The structural hierarchy involved in the embodiments of the present invention can be understood as a text structure hierarchy. The text structure hierarchy relationship can be expressed in a tree structure or an undirected graph structure or other structural forms.

[0047] According to an embodiment of the present invention, structured feedback information may be information that is required to represent an object's interactive operation on a target page. The structured feedback information may be based on one or more structural levels to display information content such as text, images, videos, and tables in the page information. The structural level may include a title level in the text, such as a structural level represented by a first-level title and a second-level title. For another example, the structural level may also include the execution steps of an operation process. Representing the information content in the page information based on the structural level allows the object to quickly obtain the required information content through the structural level of the structured feedback information, thereby reducing the frequency of interactive operations of the object in the information acquisition process, improving the object's information acquisition efficiency, and thereby improving the user experience.

[0048] In one example, the structured feedback information may include a title and a summary text.

[0049] In one example, the structured information may include a title, summary text, and an image.

[0050] According to an embodiment of the present invention, the interactive operation includes at least one of the following: a pinch operation, a drag operation, a click operation, and a frame selection operation.

[0051] According to an embodiment of the present invention, a pinch operation may represent an operation performed by at least two operation execution bodies contacting an interactive interface by approaching each other. The operation execution body may be used to contact the interactive interface to generate an operation instruction that can be detected by the electronic device. The operation execution body may include a finger, a stylus, etc., and the interactive interface may include a touch-sensitive display screen, an operation panel, etc., for detecting the operation action of the execution body.

[0052] According to an embodiment of the present invention, the pinch operation may include an operation in which two fingers of an object contact a target page displayed in an interactive interface in an open manner, and the two fingers contacting the interactive interface are moved from an open state to a closed state. The movement of the two fingers from an open state to a closed state may be similar to the two-finger pinching method.

[0053] It should be noted that the pinch operation can be performed based on two or more execution bodies, as long as the pinch operation can be simulated.

[0054] According to an embodiment of the present invention, the drag operation may include an interactive operation action of dragging any page element in the target page. The drag operation may be in any direction.

[0055] According to the embodiment of the present invention, the click operation may include any type of click operation, such as a single click (or single click), multiple clicks (such as double clicks), long press clicks, etc. The embodiment of the present invention does not limit the specific execution method of the click operation or the execution duration.

[0056] According to an embodiment of the present invention, the frame selection operation may include an operation of selecting a page area for a target page.

[0057] According to an embodiment of the present invention, any one of a pinch operation, a drag operation, a click operation, and a box selection operation may be selected to determine an interactive operation for a target page. Alternatively, an operation combination consisting of at least two of the pinch operation, the drag operation, the click operation, and the box selection operation may be selected as an interactive operation for a target page.

[0058] According to an embodiment of the present invention, the interactive operation may also include any one or more of a pinch operation, a drag operation, a click operation, and a box selection operation, as well as an operation combination combined with other types of operations. For example, the interactive operation may be an operation combination of a pinch operation and a voice input operation, so that the user can conveniently control the smart terminal device through gestures and voice to execute the method provided by the embodiment of the present invention.

[0059] According to an embodiment of the present invention, the interaction method based on the large model may further include: converting the page display attributes of the target page based on the interaction operation to obtain the target element.

[0060] According to an embodiment of the present invention, converting the page attributes of the target page may include converting any page display attributes of the target page, such as size, shape, display range, display color, etc. For example, the size of the target page may be reduced, and the obtained target element may be a thumbnail of the target page.

[0061] According to an embodiment of the present invention, converting the page attributes of the target page may also include converting any page display attributes such as size, shape, display range, and display color of page elements in the target page. For example, the cover image of the target page may be selected and the cover image may be reduced or enlarged to obtain the target element.

[0062] According to an embodiment of the present invention, the target element and the structured feedback information are displayed on an interactive interface related to the object. The target element can be used as a prompt mark of the target page to facilitate the object to associate the operation intention related to the target page with the structured feedback information, thereby prompting the object to understand the information related to the operation intention more accurately, clearly and quickly under conditions that affect the object's attention such as being interrupted during browsing the target page, so as to improve the efficiency of information browsing.

[0063] According to an embodiment of the present invention, converting the page display attribute of the target page based on the interactive operation to obtain the target element may include: determining a page operation area from the target page based on the interactive operation; and determining the target element according to the page operation area.

[0064] According to an embodiment of the present invention, the page operation area can be determined based on the positional relationship between the operation position or operation area of ​​the interactive operation and the page area in the target page. For example, the page operation area can be determined to be the comment area of ​​the target page based on the fact that the operation position of the interactive operation in the interactive interface is located in the comment area of ​​the target page.

[0065] According to an embodiment of the present invention, determining the target element according to the page operation area may include determining the target element according to any type of page element such as text, image, etc. in the page operation area. It may include using the page element in the page operation area as the target element. Or it may also include converting any page display attribute such as size, shape, display range, display color, etc. of the page element in the page operation area to obtain the target element. The embodiment of the present invention does not limit the specific method of determining the target element.

[0066] For example, the title area displayed on the target page can be used as the page operation area, and a screenshot of the title area can be taken. The obtained target element can be a screenshot of the title area of ​​the target page.

[0067] For another example, the target element may be determined by converting the page display attributes such as converting the text format and enlarging the text size of the title text in the title area.

[0068] Figure 3 An application scenario diagram of the large model-based interaction method according to an embodiment of the present invention is shown.

[0069] like Figure 3 As shown, the application scenario may include a touch display screen 300 of a terminal device. The touch display screen 300 may display a target page 310. The target page 310 may be a recipe page for lotus root and pork ribs soup. The page information of the target page 310 may include a title "Lotus root and pork ribs soup will be more fragrant if made this way", and may also include a text of the steps for making lotus root and pork ribs soup and step display pictures. The step display pictures may include picture 1 and picture 2.

[0070] like Figure 3 As shown, the user object can perform a pinch operation O310 on any area of ​​the target page 310. The touch display screen 300 detects the pinch operation O310 and can send the pinch operation instruction to the processor of the terminal device. The processor executes the large model-based interaction method provided in the embodiment of the present invention in response to the pinch operation instruction corresponding to the pinch operation O310, determines the structured feedback information corresponding to the pinch operation O310 according to the page information of the target page 310, and generates the target element B310 according to the title in the page information. The structured feedback information corresponding to the pinch operation O310 may include the first feedback information J311, the second feedback information J312, and the third feedback information J313. The first feedback information J311 may be step 1 of making lotus root and pork ribs soup. The first feedback information J311 may include the title of step 1 "Blanching to remove the fishy smell", and the step summary of step 1 "Put the pork ribs in cold water, add a little green onion, ginger, and cooking wine, blanch for 5 minutes, remove the floating foam and wash". The second feedback information J312 may include the title of step 2 "Stir-fried Spare Ribs", as well as the step summary of step 2 and the step schematic diagram picture 2. The third feedback information J313 may include the title of step 3 "Simmering on Low Heat", as well as the step summary of step 3 and the step schematic diagram picture 1. The first feedback information J311, the second feedback information J312 and the third feedback information J313 may be displayed on the touch display screen 300 in the form of titles, step summaries and display pictures, so that the user can conveniently browse the production points of each step and the display pictures corresponding to the production points of the step, so that the user object can clearly and conveniently obtain the page content in the target page 310 through simple interactive operations, thereby improving the efficiency of information acquisition and information browsing.

[0071] like Figure 3As shown, the touch display screen 300 can also display a user feedback window J321, which allows the user object to input evaluation content for the structured feedback information generated above, or can also be used to view the evaluation content of other user objects for the structured feedback information of the target page 310, so as to enrich the interaction scene. At the same time, the smart terminal device can also fine-tune the large model by collecting and analyzing the feedback results under the condition of authorization by the user object, thereby improving the prediction accuracy of the structured feedback information.

[0072] It should be noted that Figure 3 The first feedback information J311, the second feedback information J312 and the third feedback information J313 shown can be obtained after processing the page information of the target page 310 based on the large model deployed on the cloud and cached in the storage area of ​​the cloud. The smart terminal device can send an information acquisition request to the cloud based on the pinch operation O310, and then receive the first feedback information J311, the second feedback information J312 and the third feedback information J313 from the cloud storage.

[0073] In one example, the smart terminal device can send page information related to the target page to the cloud server, and the large model deployed on the cloud server processes the page information related to the target page to obtain structured feedback information. The smart terminal device obtains and displays the structured feedback information from the cloud server to meet the user's page information acquisition needs for the target page in real time.

[0074] According to an embodiment of the present invention, determining page information related to a target page based on an interactive operation may include: identifying the intent of an object based on the operation attributes of the interactive operation to obtain the operation intent; and determining at least one page information from a target page area in the target page based on the operation area intent.

[0075] According to an embodiment of the present invention, the operation intention may include an operation area intention, and the operation area intention corresponds to the target page area in the target page. For example, the operation area intention may correspond to the title area of ​​the target page. The operation area intention may indicate the object's intention to obtain information from the target page area in the target page.

[0076] According to an embodiment of the present invention, the target page area may include any type of page area such as a video area, a comment area, a recommendation area, etc., and the embodiment of the present invention does not limit the specific type of the target page area. By performing intent recognition based on the interactive operation of the object, and determining at least one page information from the target page area of ​​the target page, it is possible to more accurately identify the object's need to obtain information from the page information in the target page area based on the interactive operation of the object, and by using a large model to process the page information obtained from the target page area, the object can browse the current page content of concern more conveniently based on the structured attributes of the obtained structured feedback information.

[0077] It should be noted that the structured feedback information may be summary information obtained by semantically summarizing and generalizing the page information. However, it is not limited to this, and may also include the data processing results of using a large language model to perform data processing tasks on the page information. For example, the structured feedback information may also include the detection results of the large model performing an object detection task on the page image, the analysis results of performing a chart analysis task on the chart data, and so on.

[0078] According to an embodiment of the present invention, the operational attributes of an interactive operation may include the type, location, duration, etc. of the interactive operation, or may also include operational information input by the interactive operation, such as voice information input by a semantic input operation, text information input by a text input operation, etc.

[0079] According to an embodiment of the present invention, the intent of an object is identified based on the operational attributes of an interactive operation, which may include detecting attribute information such as the type, location, and duration of the operation, and comparing the obtained attribute detection results with the attribute conditions corresponding to the target page area. In response to the comparison results satisfying the attribute conditions corresponding to the target page area, the operation area intention corresponding to the target page area can be obtained.

[0080] According to an embodiment of the present invention, the operation area intention is determined based on the operation attributes of the interactive operation, and the operation attributes may include: the positional relationship between the operation position of the interactive operation and the regional elements in the target page area. For example, the positional relationship between the interactive operation position and the title element in the title area. For another example, the positional relationship between the interactive operation position and the video frame in the video playback area.

[0081] According to an embodiment of the present invention, the operation attribute may also include the operation area range of the interactive operation, for example, the area range of the sliding operation in the target page, the range of the pinching operation in the target area, and the like.

[0082] According to an embodiment of the present invention, the operation attribute may also include operation attribute information such as text and audio input of the interactive operation. The intention recognition of the object according to the operation attribute of the interactive operation may also include processing the operation attribute information such as text and audio input of the interactive operation based on a neural network algorithm to obtain the operation area intention corresponding to the target page area. The embodiment of the present invention does not limit the specific method of performing the intention recognition.

[0083] It should be noted that intent recognition may be performed based on one or more operation attributes, and the embodiments of the present invention will not be described in detail herein.

[0084] Figure 4 An application scenario diagram of an interaction method based on a large model according to another embodiment of the present invention is shown.

[0085] like Figure 4 As shown, the application scenario may include a touch screen display 400 of a terminal device, and the touch screen display 400 may include a target page 410. The target page 410 may include a video play frame 411 and a video comment area 412. The video play frame 411 may be playing a video of movie A, and the video comment area 412 may display comment information of multiple users on movie A. For example, the video comment area 412 may include comment information of account 1, account 2, account 3, and account 4.

[0086] like Figure 4As shown, the user object can perform a pinch operation O410 on the video comment area 412 of the target page 410. The touch display screen 400 detects that the operation position of the pinch operation O410 is located in the video comment area 412, and it can be obtained that the object's operation area intends to process the page information in the video comment area 412. A plurality of comment information is obtained from the video comment area 412 as the page information corresponding to the pinch operation O410. The plurality of comment information is processed using a large language model, and the plurality of structured feedback information obtained can be the first comment analysis feedback information J411, the second comment analysis feedback information J412, and the third comment analysis feedback information J413. The first comment analysis feedback information J411, the second comment analysis feedback information J412, and the third comment analysis feedback information J413, as well as the target element B410 "A movie screenshot" can be displayed on the touch display screen 400. The first comment analysis feedback information J411 can be a "comment analysis overview", for example, the overall comment content of multiple accounts on movie A can be obtained by analyzing the respective comment content of multiple accounts. The second comment analysis feedback information J412 may include a title "Positive and Negative Comment Data Analysis" and a comparative bar chart. For example, the number of positive comments and the number of negative comments may be represented based on a comparative bar chart, so that the object can more intuitively obtain the proportional relationship between the positive and negative comments for Movie A, thereby improving the efficiency of the relevant object in obtaining the playback effect of Movie A. The third comment analysis feedback information J413 may include a title "Comment User Preference Analysis" and a movie clip heat map. The movie clip heat map can display the playback popularity of Movie A at each moment based on a timeline, thereby facilitating the object to quickly obtain the playback clips with higher popularity in Movie A, thereby improving the efficiency of the playback effect analysis of Movie A.

[0087] According to an embodiment of the present invention, by identifying the operation area intention of the object and determining the page information that the large model needs to process based on the target page area corresponding to the operation area intention, the matching degree between the structured feedback information and the information needs of the object can be further improved, and the display probability of redundant structured feedback information can be reduced, thereby improving the information acquisition efficiency of the object and enhancing the user experience.

[0088] According to an embodiment of the present invention, determining page information related to a target page based on an interactive operation may include: performing intent recognition on an object based on the interactive operation to obtain the operation intent of the object, wherein the operation intent may include information extension intent; and determining associated pages associated with the target page based on the information extension intent.

[0089] According to an embodiment of the present invention, the page information may be determined based on the associated page and the target page. For example, the page text in the target page and the page text in the associated page may be concatenated to obtain the page information. Alternatively, any type of information such as images, videos, audio, etc. of the target page and the associated page may be identified and encoded to obtain the page information.

[0090] According to an embodiment of the present invention, information extension intention can represent that the object needs to obtain other information content besides the page content of the target page. Information extension intention can be understood as the object needs to obtain more extended (or expanded) information for the target page, such as the need to obtain information that is highly similar to the subject of the target page but is not recorded in the target page. Intent recognition can be performed based on the object preference attribute of the object. Under the condition that the similarity between the object preference attribute and the subject of the target page meets the preset similarity threshold, it can be considered that the object has a high degree of concern for the subject of the target page and needs to extend the content information of the target page, thereby meeting the object's need to obtain extended information content, avoiding the object from obtaining extended information content by clicking or searching other pages related to the subject of the target page, and improving the object's information acquisition efficiency.

[0091] According to an embodiment of the present invention, determining the associated pages associated with the target page based on the information extension intention may include retrieving pages with a high similarity to the target page based on the information extension intention. For example, the page theme of the target page may be used for retrieval, and the top k pages in the retrieval ranking results may be used as the associated pages associated with the target page, so that the retrieval function and the large model task execution function may be combined to generate structured feedback information that matches the information acquisition intention of the object, improve the information richness of the structured feedback information, and improve the matching degree between the structured feedback information and the information acquisition needs of the object.

[0092] According to an embodiment of the present invention, performing intent recognition on an object based on interactive operations to obtain the operation intention of the object may also include: performing similarity analysis based on the information type corresponding to the interactive operation and the historical information type corresponding to the historical interactive operations of the object to obtain a type similarity result; and determining the information extension intention when the type similarity result meets a preset information type similarity condition.

[0093] According to an embodiment of the present invention, the information type corresponding to the interactive operation may include the subject type of the object input information carried by the interactive operation, or may also include the page subject type, page topic type, etc. of the target page targeted by the interactive operation. The historical information type may be the page subject type, page topic type, etc. that the object has browsed in the historical period. The type similarity result may represent the degree of similarity between the historical information type and the information type, for example, it may be a similarity value.

[0094] According to an embodiment of the present invention, when the type similarity result meets the preset information type similarity condition, it can indicate that the object has queried or browsed the information type related to the interactive operation in the historical period, and the object has the need to obtain extended information when performing an interactive operation on the target page, thereby obtaining the information extension intention corresponding to the interactive operation.

[0095] According to an embodiment of the present invention, performing a similarity analysis based on the information type corresponding to the information acquisition request and the historical information type corresponding to the historical interaction operation of the object, obtaining a type similarity result may also include: determining a first similarity between the information type and the first historical information type corresponding to the first historical interaction operation; determining a second similarity between the information type and the second historical information type corresponding to the second historical interaction operation; and in response to the first similarity being lower than a preset similarity threshold and the second similarity being higher than or equal to the preset similarity threshold, determining a type similarity result that meets the information type similarity condition.

[0096] According to an embodiment of the present invention, the first historical interaction operation is a historical interaction operation in a first historical period, the second historical interaction operation is a historical interaction operation in a second historical period, and the timing attribute of the first historical period is earlier than the timing attribute of the second historical period. For example, the first historical interaction operation is an interaction operation performed from December 1, 2023 to January 1, 2024, and the second historical interaction operation may be an interaction operation performed from January 1, 2024 to January 7, 2024.

[0097] According to an embodiment of the present invention, when the first similarity is lower than the preset similarity threshold, it can be indicated that the object did not pay attention to the information type of the target page in an earlier historical period, or in a long historical period. When the second similarity is higher than or equal to the preset similarity threshold, it can be indicated that the object is more concerned about the information type of the target page in a recent historical period, or in a recent period. It can be understood that the object has only recently paid attention to the type of the target page, and there is no information accumulation related to the information type of the target page. Therefore, in response to the first similarity being lower than the preset similarity threshold, and the second similarity being higher than or equal to the preset similarity threshold, a type similarity result that satisfies the information type similarity condition is determined, and it can be determined based on the type similarity result that the object needs to perform information expansion, so that the page information can be enriched by determining the associated pages with a high degree of relevance to the target page, and then the structured feedback information generated by the large model can meet the user's information extension needs, and the object can obtain the information it needs to know through a relatively simple operation while reducing the frequency of object operations, thereby improving the efficiency of information acquisition.

[0098] According to an embodiment of the present invention, determining associated pages associated with a target page based on information extension intent may include: determining a first associated page based on the information extension intent and historical interaction operations of the object, wherein the associated pages include the first associated page.

[0099] According to an embodiment of the present invention, in response to the information extension intention, sampling can be performed according to the historical interaction operations of the object in a preset sampling history period to obtain the sampled historical interaction operations. The historical page corresponding to the sampled historical interaction operations is used as the first associated page.

[0100] In one example, N consecutive historical operations before the object performs an interactive operation can be used as sampling operations, and the historical pages corresponding to the N sampling operations can be used as the first associated page, so as to facilitate the object's intention identification through the historical pages recently browsed by the object, so that the information extension intention can more accurately characterize the type of information that the object has recently paid attention to.

[0101] According to an embodiment of the present invention, determining an associated page associated with a target page based on the information extension intention may also include: determining a second associated page based on the information extension intention and page key information of the target page, wherein the associated page includes the second associated page.

[0102] According to an embodiment of the present invention, in response to the information extension intention, the page key information can be used to retrieve candidate pages with a high similarity in information type with the target page, and the top k candidate pages in the retrieved candidate pages are used as the second associated pages. In this way, information extension for the target page can be achieved through the retrieval function. By using a large model to process the page information of the second associated page with a high degree of relevance to the target page, structured feedback information that can meet the user's information extension needs can be obtained, thereby improving the convenience and timeliness of the user's acquisition of information.

[0103] According to an embodiment of the present invention, determining structured feedback information corresponding to an interactive operation of an object based on page information may include: determining a requirement prompt template corresponding to the feedback information requirement type of the object; and using a large model to process the page information and the requirement prompt template to obtain structured feedback information.

[0104] According to an embodiment of the present invention, the operation intention of an object may also include a feedback information requirement type. Intent recognition may be performed based on the operation attribute information input by the interactive operation to obtain the feedback information requirement type of the object. For example, intent recognition may be performed based on a search text (or query) related to the interactive operation to obtain a feedback information requirement type associated with the search text. However, this is not limited to the above. Intent recognition may also be performed based on page information such as page content, page title, page tag, etc. of a target page related to the interactive operation to obtain a feedback information requirement type associated with the search text. The embodiment of the present invention does not limit the specific method for determining the feedback information requirement type.

[0105] According to an embodiment of the present invention, the feedback information demand type can characterize the object's demand intention for the information type of structured feedback information. For example, the type of information that the object needs to obtain may include information query type, transaction demand type, local information acquisition type, text creation type, image creation type, etc. Determine the demand prompt template corresponding to the feedback information demand type, and use the big model to process the page information and the demand prompt template. It can be achieved by using the demand prompt template to control the big model to perform data processing tasks corresponding to the feedback information demand type, so that based on the big model's relatively powerful semantic understanding ability and data processing task execution ability, the page information can be accurately processed according to the object's feedback information demand type, and one or more structured feedback information matching the object's information demand type can be obtained, thereby achieving more accurate and convenient information acquisition, and improving information acquisition efficiency and accuracy when the number of interactive operations is small.

[0106] It should be noted that using the big model to process page information and demand prompt templates may include using the big model to process page information and demand prompt templates in the target page. Alternatively, it may also include using the big model to process page information and demand prompt templates in associated pages. Alternatively, it may also include using the big model to process page information in the target page, page information in associated pages, and demand prompt templates. The page information specifically processed by the big model may be determined based on the extended information intent of the object.

[0107] In an example, the requirement prompt template may be expressed as: "Please analyze the input information enclosed by the following {}, and generate the title of the device operation steps and the specific content of the device operation steps."

[0108] According to an embodiment of the present invention, the feedback information requirement type may be determined based on the following operations: determining an object preference attribute of an object based on an interactive operation; and determining the feedback information requirement type based on the similarity between the object preference attribute and page key information of a target page.

[0109] According to an embodiment of the present invention, the object preference attribute may be any attribute that characterizes the object's information acquisition preference, such as an online shopping preference attribute, an information navigation attribute, a travel preference attribute, etc. The object preference attribute may be determined based on the preference information input by the object. For example, the object may select an attribute tag that meets its information acquisition preference before browsing the target page, so that the object preference information may be determined according to the attribute tag that the object has selected in response to the interactive operation. However, the object preference attribute may also be determined based on the historical interactive operation information (such as historical browsing information, historical text input information, etc.) of the object that performs the interactive operation.

[0110] According to an embodiment of the present invention, the page key information of the target page may include the page title, page topic, most frequently appearing text, etc. of the target page, and may also include preset keywords, images and other information. The page key information may be determined based on actual needs.

[0111] According to an embodiment of the present invention, the similarity between the object preference attribute and the page key information of the target page may include the semantic similarity between the object preference attribute and the page key information. The page key information may be associated with the feedback information demand type. In the case where the page key information includes multiple pieces, the page key information having a similarity with the object preference attribute higher than a preset similarity threshold may be determined as the target page key information. According to the feedback information demand type associated with the target page key information, the feedback information demand type of the object is used. By determining the feedback information demand type based on the similarity between the object preference attribute and the page key information of the target page, the object preference attribute may be more accurately associated with the feedback information demand type, so that the object preference of the object may be accurately met according to the demand prompt template corresponding to the feedback information demand type, thereby improving the matching degree between the structural hierarchical attributes, information content, information type, etc. of the structured feedback demand information and the information demand intention of the object, thereby improving the information acquisition efficiency of the object.

[0112] It should be noted that the information acquisition involved in the method provided in the embodiment of the present invention, including but not limited to object preference attributes, interactive operation input information, etc., is obtained under the condition of obtaining the authorization of the object, and the purpose of obtaining the data is to provide the object with structured feedback information that better meets the user's needs. At the same time, necessary data security measures are adopted in the data processing process, including but not limited to encryption, desensitization and other data security measures to avoid data leakage.

[0113] According to an embodiment of the present invention, using a large model to process page information and a demand prompt template may include: using a target large model determined according to a model calling strategy to process page information and a demand prompt template.

[0114] According to an embodiment of the present invention, the model calling strategy may be determined according to a feedback delay attribute of an object, the feedback delay attribute may be determined according to the amount of information of the page information, and the large model may include a target large model.

[0115] According to an embodiment of the present invention, the feedback delay attribute can represent the delay time of determining structured feedback information based on page information. When the amount of information in the page information is greater than a preset information amount threshold, the feedback delay attribute can represent a longer delay time. To avoid the object waiting time being too long, a target large model adapted to the amount of information in the page information can be selected based on the call strategy. For example, a target large model with a smaller amount of model parameters can be selected to reduce the delay time of generating structured feedback information, thereby improving the efficiency of users browsing to obtain structured feedback information.

[0116] According to an embodiment of the present invention, the interaction method based on the big model may further include: in response to a viewing operation on the structured feedback information, displaying target page content corresponding to the structured feedback information in the target page.

[0117] According to an embodiment of the present invention, the viewing operation can be used to characterize the need of the object to view the details of the structured feedback information. By displaying the target page content corresponding to the structured feedback information in response to the viewing operation, the user can preliminarily understand the key points of information at each structural level in the target page by browsing the structured summary content, and then quickly jump to the target page content that needs to be viewed in detail through the viewing operation, so that the user can quickly browse the structured information, improve the efficiency of information browsing, and flexibly meet the information acquisition needs of the video object.

[0118] It should be noted that the target page content can be any type of page content such as images, videos, text segments, charts, etc.

[0119] According to an embodiment of the present invention, the structured feedback information is determined by the large model processing the associated pages corresponding to the target page. The structured feedback information may correspond to the associated pages. For example, the address introduction summary in the structured feedback information may be generated by processing the museum address information of the associated page.

[0120] According to an embodiment of the present invention, the interaction method based on the big model may further include: in response to a viewing operation on the structured feedback information, displaying an associated page corresponding to the structured feedback information.

[0121] According to an embodiment of the present invention, by responding to a viewing operation to jump to an associated page corresponding to the structured feedback information, the object can quickly jump to an associated page that needs to be viewed in detail through a viewing operation, thereby facilitating users to quickly perform structured browsing of information, improving information browsing efficiency, and flexibly meeting the information acquisition needs of video objects.

[0122] Figure 5 An application scenario diagram of an interaction method based on a large model according to yet another embodiment of the present invention is shown.

[0123] like Figure 5 As shown, the application scenario may include a touch screen 500 of a terminal device, and the touch screen 500 may include a target page 510. The target page 510 may be a news page with the theme of "Tourism in City A is booming". The target page 510 may include news text and news images that introduce the current tourism boom in City A, and the news images may include a picture of Garden A and a picture of Garden B corresponding to Garden A and Garden B, respectively.

[0124] like Figure 5 As shown, the user object can perform an interactive operation O510 on the target page 510, determine the first page information from the target page 510 according to the interactive operation O510, and can also determine the associated page with the target page 510 based on the interactive operation, determine the second page information from the associated page, and obtain multiple structured feedback information by processing the first page information and the second page information using a large model. The multiple structured feedback information may include first feedback information J521, second feedback information J522, third feedback information J523, etc. The first feedback information J521 may be the title of the multiple structured feedback information "A City Tourism Route Recommendation". The second feedback information J522 may be one-day tour scenic spot route recommendation information, for example, it may include the recommended text information of "Museum-A Garden-B Garden-A Restaurant-C Garden", and the scenic spot navigation map information corresponding to the recommended text information. The scenic spot navigation map information may display the navigation route of "Museum-A Garden-B Garden-A Restaurant-C Garden" so that the user can view it in time. The third feedback information J523 may be the scenic spot introduction summary information of the scenic spot "Museum". The third feedback information J523 may include information related to tourism, such as the introduction of the “museum”, tickets, reservation links, address links, and opening hours. In addition, the touch display screen 500 may also include structured feedback information for displaying summary information of scenic spot introductions of other scenic spots, and a target element B520. The target element B520 may be a page screenshot of the target page 510.

[0125] like Figure 5As shown, the user object can perform the viewing operation O520 by clicking on the position on the touch display screen 500 corresponding to the "reservation link" structured content in the third feedback information J523. The processor of the smart terminal device can jump to the museum reservation page 530 in response to the position of the viewing operation O520. This can facilitate the user object to quickly plan the travel and avoid the user planning the travel route by searching for travel guides and related pages, thereby improving the user's information acquisition efficiency and interaction efficiency.

[0126] According to an embodiment of the present invention, the page information includes a target video, and the structured feedback information includes segment description information, which is used to describe a target video segment in the target video. For example, the segment description information may be plot summary information for the video segment, bullet comment summary information, and the like.

[0127] The interactive method based on the large model may also include: in response to a viewing operation on the segment description information, displaying a target video segment corresponding to the segment description information.

[0128] Figure 6 An application scenario diagram of an interaction method based on a large model according to yet another embodiment of the present invention is shown.

[0129] like Figure 6 As shown, the application scenario may include a touch screen display 600 of a terminal device, and the touch screen display 600 may include a target page 610. The target page 610 may be a video playback page with the theme "The only drawback is that it is only available in City A". The target page 610 may include a travel food video in which a video blogger introduces the current travel situation in City A.

[0130] like Figure 6 As shown, the user object can perform an interactive operation O610 on the target page 610, and determine page information from the target page 610 according to the interactive operation O610. The page information can include the theme "The only disadvantage is that it is only available in City A", and can also include video information of a travel food video. The page information can also include an associated page corresponding to the target page 610, such as a navigation map page corresponding to the A City Shengjian Shop involved in the video, etc. By using a large model to process the page information related to the target page 610, multiple structured feedback information can be obtained. The multiple structured feedback information can include first feedback information J621, second feedback information J622, third feedback information J623, etc.

[0131] like Figure 6As shown, the user object can perform the viewing operation O620 by clicking on the position of the third feedback information J623 in the touch display screen 600. The processor of the intelligent terminal device can play the Shengjianpu segment 631 in the travel food video in the jump page 630 of the touch display screen 600 in response to the viewing operation O620, so that the user object can quickly jump to the video segment of interest. At the same time, it can also jump to the associated page 632 corresponding to the third feedback information J623. The associated page 632 can be a navigation page of Shengjianpu in City A. The navigation page may include a navigation map of Shengjianpu. The user can browse the Shengjianpu segment 631 being played by browsing the touch display screen 600, and at the same time come to Shengjianpu for consumption according to the navigation map of Shengjianpu, so as to improve the efficiency of information acquisition and avoid the time waste caused by multiple interactive operations to query information.

[0132] Figure 7 A block diagram of an interaction device based on a large model according to an embodiment of the present invention is shown.

[0133] like Figure 7 As shown, the large model-based interactive device 700 includes: a page information determination module 710 , a structured feedback information determination module 720 and a first display module 730 .

[0134] The page information determination module 710 is used to determine page information related to the target page according to the interactive operation performed by the object on the target page.

[0135] The structured feedback information determination module 720 is used to determine the structured feedback information corresponding to the interactive operation of the object according to the page information, wherein the structured feedback information is obtained by processing the page information using a large model.

[0136] The first display module 730 is used to display structured feedback information.

[0137] According to an embodiment of the present invention, the page information determination module includes: a first operation intention acquisition submodule and an associated page determination submodule.

[0138] The first operation intention obtaining submodule is used to perform intention recognition on the object based on the interactive operation to obtain the operation intention of the object, wherein the operation intention includes information extension intention.

[0139] The associated page determination submodule is used to determine the associated pages associated with the target page based on the information extension intention, wherein the page information is determined based on the associated pages and the target page.

[0140] According to an embodiment of the present invention, the associated page determination submodule includes a first associated page determination unit.

[0141] The first associated page determining unit is used to determine the first associated page based on the information extension intention and the historical interaction operation of the object, and the associated pages include the first associated page.

[0142] According to an embodiment of the present invention, the associated page determination submodule includes a second associated page determination unit.

[0143] The second associated page determining unit is used to determine the second associated page based on the information extension intention and the page key information of the target page, and the associated pages include the second associated page.

[0144] According to an embodiment of the present invention, the first operation intention obtaining submodule includes: a similarity result obtaining unit and an information extension intention determining unit.

[0145] The similarity result obtaining unit is used to perform similarity analysis based on the information type corresponding to the interactive operation and the historical information type corresponding to the historical interactive operation of the object to obtain a type similarity result.

[0146] The information extension intention determination unit is used to determine the information extension intention when the type similarity result meets the preset information type similarity condition.

[0147] According to an embodiment of the present invention, the page information determination module includes: a second operation intention obtaining submodule and a page information determination submodule.

[0148] The second operation intention obtaining submodule is used to perform intention recognition on the object according to the operation attribute of the interactive operation to obtain the operation intention, wherein the operation intention includes the operation area intention, and the operation area intention corresponds to the target page area in the target page.

[0149] The page information determination submodule is used to determine at least one page information from a target page area in a target page according to the operation area intention.

[0150] According to an embodiment of the present invention, the operation area intention is determined based on the operation attributes of the interactive operation, and the operation attributes include at least one of the following: the positional relationship between the operation position of the interactive operation and the regional element in the target page area; the operation area range of the interactive operation.

[0151] According to an embodiment of the present invention, the structured feedback information is determined by the large model processing the associated pages corresponding to the target page.

[0152] According to an embodiment of the present invention, the above-mentioned device further includes a second display module.

[0153] The second display module is used to display the associated page corresponding to the structured feedback information in response to the viewing operation on the structured feedback information.

[0154] According to an embodiment of the present invention, the above-mentioned device further includes a third display module.

[0155] The third display module is used to display the target page content corresponding to the structured feedback information in the target page in response to the viewing operation on the structured feedback information.

[0156] According to an embodiment of the present invention, the page information includes a target video, and the structured feedback information includes segment description information, where the segment description information is used to describe a target video segment in the target video.

[0157] According to an embodiment of the present invention, the above-mentioned device further includes a fourth display module.

[0158] The fourth display module is used to display the target video segment corresponding to the segment description information in response to a viewing operation on the segment description information.

[0159] According to an embodiment of the present invention, the structured feedback information determination module includes: a requirement prompt template determination submodule and a structured feedback information acquisition submodule.

[0160] The demand prompt template determination submodule is used to determine a demand prompt template corresponding to the feedback information demand type of the object, wherein the operation intention of the object also includes the feedback information demand type.

[0161] The structured feedback information acquisition submodule is used to use the large model to process page information and demand prompt templates to obtain structured feedback information.

[0162] According to an embodiment of the present invention, the feedback information requirement type is determined based on the following operations: determining an object preference attribute of an object based on an interactive operation; and determining the feedback information requirement type based on the similarity between the object preference attribute and page key information of a target page.

[0163] According to an embodiment of the present invention, the above-mentioned device further includes a target element obtaining module.

[0164] The target element acquisition module is used to convert the page display attributes of the target page based on the interactive operation to obtain the target element, wherein the target element and the structured feedback information are displayed on an interactive interface related to the object.

[0165] According to an embodiment of the present invention, the target element obtaining module includes: a page operation area determining submodule and a target element determining submodule.

[0166] The page operation area determination submodule is used to determine the page operation area from the target page based on the interactive operation.

[0167] The target element determination submodule is used to determine the target element according to the page operation area.

[0168] According to an embodiment of the present invention, the interactive operation includes at least one of the following: a pinch operation, a drag operation, a click operation, and a frame selection operation.

[0169] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0170] According to an embodiment of the present invention, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the embodiment of the present invention.

[0171] According to an embodiment of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute a method provided according to an embodiment of the present invention.

[0172] According to an embodiment of the present invention, a computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the method provided according to the embodiment of the present invention.

[0173] Figure 8 A block diagram of an electronic device suitable for implementing an interactive method based on a large model according to an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0174] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0175] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0176] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the interaction method based on the large model. For example, in some embodiments, the interaction method based on the large model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the interaction method based on the large model described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the interaction method based on the large model in any other appropriate manner (e.g., by means of firmware).

[0177] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0179] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0181] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0182] A computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises through computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server combined with a blockchain.

[0183] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document is not limited here.

[0184] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An interactive method based on a large model, characterized in that: include: In response to an interactive operation of an object on a target page, determining page information related to the target page according to the interactive operation; Processing the page information and a demand prompt template corresponding to the feedback information demand type of the object by using the big model to obtain structured feedback information, wherein the demand prompt template is used to control the big model to execute a data processing task corresponding to the feedback information demand type, wherein the feedback information demand type is determined based on the intention recognition of the object by the interactive operation, and the feedback information demand type represents the type of information that the object needs to obtain; The structured feedback information is displayed.

2. The method according to claim 1, characterized in that Determining page information related to the target page according to the interactive operation includes: performing intention recognition on the object based on the interactive operation to obtain the operation intention of the object, wherein the operation intention includes information extension intention; and Based on the information extension intention, an associated page associated with the target page is determined, wherein the page information is determined according to the associated page and the target page.

3. The method according to claim 2, characterized in that Based on the information extension intention, determining the associated page associated with the target page includes: Based on the information extension intention and the historical interaction operation of the object, a first associated page is determined, and the associated pages include the first associated page.

4. The method according to claim 2 or 3, characterized in that: Based on the information extension intention, determining the associated page associated with the target page includes: Based on the information extension intention and the page key information of the target page, a second associated page is determined, and the associated pages include the second associated page.

5. The method according to claim 2, characterized in that: Performing intention recognition on the object based on the interactive operation to obtain the operation intention of the object includes: Performing similarity analysis based on the information type corresponding to the interactive operation and the historical information type corresponding to the historical interactive operation of the object to obtain a type similarity result; and In the case where the type similarity result meets a preset information type similarity condition, determining the information extension intention; The information type includes at least one of the following: The subject type of the object input information carried by the interactive operation; The page subject type of the target page; The page topic type of the target page.

6. The method according to claim 1, characterized in that Determining page information related to the target page according to the interactive operation includes: Performing intention recognition on the object according to the operation attribute of the interactive operation to obtain an operation intention, wherein the operation intention includes an operation area intention, and the operation area intention corresponds to a target page area in the target page; According to the operation area intention, at least one piece of page information is determined from a target page area in the target page.

7. The method according to claim 6, characterized in that The operation area intention is determined based on the operation attribute of the interactive operation, and the operation attribute includes at least one of the following: The positional relationship between the operation position of the interactive operation and the regional element in the target page area; The operation area of ​​the interactive operation.

8. The method according to any one of claims 1, 2 or 6, characterized in that: The structured feedback information is determined by the large model processing the associated pages corresponding to the target page; Wherein, the method further comprises: In response to a viewing operation on the structured feedback information, an associated page corresponding to the structured feedback information is displayed.

9. The method according to any one of claims 1, 2 or 6, characterized in that: The method further comprises: In response to a viewing operation on the structured feedback information, target page content corresponding to the structured feedback information in the target page is displayed.

10. The method according to any one of claims 1, 2 or 6, characterized in that: The page information includes a target video, and the structured feedback information includes segment description information, where the segment description information is used to describe a target video segment in the target video; The method further comprises: In response to a viewing operation on the segment description information, a target video segment corresponding to the segment description information is displayed.

11. The method according to claim 1, characterized in that: The feedback information requirement type is determined based on the following operations: determining an object preference attribute of the object based on the interaction operation; and The feedback information requirement type is determined based on the similarity between the object preference attribute and the page key information of the target page.

12. The method according to claim 1, characterized in that The method further comprises: Based on the interactive operation, a page display attribute conversion is performed on the target page to obtain a target element, wherein the target element and the structured feedback information are displayed on an interactive interface related to the object.

13. The method according to claim 12, characterized in that Based on the interactive operation, the page display attribute of the target page is converted to obtain the target element including: Determining a page operation area from the target page based on the interactive operation; and The target element is determined according to the page operation area.

14. The method according to claim 1, characterized in that The interaction operation includes at least one of the following: Pinch operation, drag operation, click operation, and frame selection operation.

15. An interactive device based on a large model, characterized in that: include: A page information determination module, configured to respond to an interactive operation of an object on a target page and determine page information related to the target page according to the interactive operation; a structured feedback information determination module, configured to determine, based on the page information, structured feedback information corresponding to the interactive operation of the object, wherein the structured feedback information is obtained by processing the page information using a large model; and A first display module, used to display the structured feedback information; Wherein, the structured feedback information determination module includes: A requirement prompt template determination submodule, used to determine a requirement prompt template corresponding to the feedback information requirement type of the object; and A structured feedback information acquisition submodule is used to use the large model to process the page information and a demand prompt template corresponding to the feedback information demand type of the object to obtain the structured feedback information, wherein the demand prompt template is used to control the large model to perform a data processing task corresponding to the feedback information demand type, and the feedback information demand type is determined based on the intention recognition of the object through the interactive operation, and the feedback information demand type represents the type of information that the object needs to obtain.

16. The device according to claim 15, characterized in that The page information determination module includes: a first operation intention obtaining submodule, configured to perform intention recognition on the object based on the interactive operation to obtain the operation intention of the object, wherein the operation intention includes information extension intention; and The associated page determination submodule is used to determine the associated pages associated with the target page based on the information extension intention, wherein the page information is determined based on the associated pages and the target page.

17. The device according to claim 16, characterized in that The associated page determination submodule includes: The first associated page determining unit is configured to determine a first associated page based on the information extension intention and the historical interaction operation of the object, wherein the associated pages include the first associated page.

18. The device according to claim 16 or 17, characterized in that The associated page determination submodule includes: The second associated page determining unit is used to determine a second associated page based on the information extension intention and the page key information of the target page, and the associated pages include the second associated page.

19. The device according to claim 16, characterized in that The first operation intention obtaining submodule includes: a similarity result obtaining unit, configured to perform similarity analysis based on the information type corresponding to the interactive operation and the historical information type corresponding to the historical interactive operation of the object to obtain a type similarity result; and An information extension intention determination unit, configured to determine the information extension intention when the type similarity result satisfies a preset information type similarity condition; The information type includes at least one of the following: The subject type of the object input information carried by the interactive operation; The page subject type of the target page; The page topic type of the target page.

20. The device according to claim 15, characterized in that The page information determination module includes: A second operation intention obtaining submodule is used to perform intention recognition on the object according to the operation attribute of the interactive operation to obtain an operation intention, wherein the operation intention includes an operation area intention, and the operation area intention corresponds to a target page area in the target page; The page information determination submodule is used to determine at least one piece of page information from a target page area in the target page according to the operation area intention.

21. The device according to claim 20, characterized in that The operation area intention is determined based on an operation attribute of the interactive operation, where the operation attribute includes at least one of the following: The positional relationship between the operation position of the interactive operation and the regional element in the target page area; The operation area of ​​the interactive operation.

22. The device according to any one of claims 15, 16 or 20, characterized in that The structured feedback information is determined by the large model processing the associated pages corresponding to the target page; Wherein, the device further comprises: The second display module is used to display an associated page corresponding to the structured feedback information in response to a viewing operation on the structured feedback information.

23. The device according to any one of claims 15, 16 or 20, characterized in that The device also includes: The third display module is used to display the target page content corresponding to the structured feedback information in the target page in response to the viewing operation on the structured feedback information.

24. The device according to any one of claims 15, 16 or 20, characterized in that The page information includes a target video, and the structured feedback information includes segment description information, where the segment description information is used to describe a target video segment in the target video; The device also includes: The fourth display module is used to display the target video segment corresponding to the segment description information in response to the viewing operation on the segment description information.

25. The device according to claim 15, characterized in that The feedback information requirement type is determined based on the following operations: determining an object preference attribute of the object based on the interaction operation; and The feedback information requirement type is determined based on the similarity between the object preference attribute and the page key information of the target page.

26. The device according to claim 15, characterized in that The device also includes: The target element acquisition module is used to convert the page display attribute of the target page based on the interactive operation to obtain the target element, wherein the target element and the structured feedback information are displayed on an interactive interface related to the object.

27. The device according to claim 26, characterized in that The target element acquisition module comprises: a page operation area determination submodule, configured to determine a page operation area from the target page based on the interactive operation; and The target element determination submodule is used to determine the target element according to the page operation area.

28. The device according to claim 15, characterized in that The interaction operation includes at least one of the following: Pinch operation, drag operation, click operation, and frame selection operation.

29. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 14.

30. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 14.

31. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to any one of claims 1 to 14.