Content display method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to a content display method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: in a video interaction scene in which at least two interaction objects participate, in a video interaction page of one interaction object, displaying a video picture collected by a login terminal of the other at least one interaction object in real time; responding to an interested event triggered for the environment content in any video picture, and taking the environment content matched with the interested event as target content; and displaying a content picture corresponding to the target content on the video interaction page. By adopting the method, the content display efficiency can be improved and the video interaction continuity can be ensured under the condition of video interaction.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for displaying content. Background Technology

[0002] With the continuous development of technology and the internet, more and more applications have emerged, enabling content and information exchange between different applications. For example, sharing videos, news, and text from application 1 to application 2. However, during video interaction, if participant 1 sees content of interest and wants to share it with participant 2, participant 1 needs to use the sharing function of the application containing the content, while participant 2 needs to open the corresponding application to view the content. This means that sharing content during video interaction disrupts the continuity of the interaction between the two parties and affects the efficiency of content display. Therefore, how to more efficiently display content and ensure the continuity of video interaction is a pressing issue that needs to be addressed. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, apparatus, computer device, storage medium, and computer program product that can improve the efficiency of content display and ensure the continuity of video interaction in the context of video interaction, in order to address the aforementioned technical problems.

[0004] Firstly, this application provides a method for displaying content. The method includes:

[0005] In a video interaction scenario involving at least two interactive objects, the video feed captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects.

[0006] In response to an interest event triggered by environmental content in any video frame, the environmental content matching the interest event is taken as the target content;

[0007] The video interaction page displays the content screen corresponding to the target content.

[0008] Secondly, this application also provides a content display device. The device includes:

[0009] The video display module is used to display, in a video interaction scenario involving at least two interactive objects, the real-time video footage captured by the login terminal of at least one of the other interactive objects on the video interaction page of one of the interactive objects;

[0010] The event triggering module is used to respond to events of interest triggered by environmental content in any video frame, and to take the environmental content that matches the event of interest as the target content.

[0011] The content display module is used to display the content screen corresponding to the target content on the video interaction page.

[0012] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0013] In a video interaction scenario involving at least two interactive objects, the video feed captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects.

[0014] In response to an interest event triggered by environmental content in any video frame, the environmental content matching the interest event is taken as the target content;

[0015] The video interaction page displays the content screen corresponding to the target content.

[0016] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0017] In a video interaction scenario involving at least two interactive objects, the video feed captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects.

[0018] In response to an interest event triggered by environmental content in any video frame, the environmental content matching the interest event is taken as the target content;

[0019] The video interaction page displays the content screen corresponding to the target content.

[0020] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0021] In a video interaction scenario involving at least two interactive objects, the video feed captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects.

[0022] In response to an interest event triggered by environmental content in any video frame, the environmental content matching the interest event is taken as the target content;

[0023] The video interaction page displays the content screen corresponding to the target content.

[0024] The methods, apparatus, computer devices, storage media, and computer program products described above, in video interaction scenarios involving at least two interactive objects, display real-time video footage captured by the login terminals of at least one of the other interactive objects on the video interaction page of one of the interactive objects, ensuring the real-time display of the screen during video interaction. Based on this, in response to an event of interest triggered by environmental content in any video frame, the environmental content matching the event of interest is taken as the target content, and the corresponding content screen is displayed on the video interaction page. This eliminates the need for interactive objects to perform corresponding content sharing operations; the target content is automatically displayed through the environmental content-triggered recognition and content display process. This avoids object operations affecting the continuity of video interaction and eliminates the need for multiple sharing interactions, thereby improving the efficiency of content display. Attached Figure Description

[0025] Figure 1 This is an application environment diagram of the content display method in one embodiment;

[0026] Figure 2 This is a schematic diagram illustrating the technical implementation logic of a content display method in one embodiment.

[0027] Figure 3 This is a flowchart illustrating a method for displaying content in one embodiment;

[0028] Figure 4 This is a schematic diagram of the screen display method of the login terminal of the first interactive object in one embodiment;

[0029] Figure 5 This is a schematic diagram of an interface showing a video feed captured in real time on the login terminal of an interactive object in one embodiment.

[0030] Figure 6 This is a schematic diagram of an embodiment of the interest-triggered operation;

[0031] Figure 7 This is a schematic diagram illustrating an embodiment where environmental content exists in the target video frame and where environmental content does not exist.

[0032] Figure 8 This is a schematic diagram of an interactive operation triggered by environmental content in one embodiment;

[0033] Figure 9 This is a schematic diagram of an interactive operation that zooms in on environmental content using gestures, as shown in one embodiment.

[0034] Figure 10 This is a schematic diagram illustrating interactive operations on a detection control in one embodiment.

[0035] Figure 11 This is a schematic diagram of a content screen displaying the target content in one embodiment;

[0036] Figure 12 This is a schematic diagram of the interface of the content screen corresponding to the target content in one embodiment;

[0037] Figure 13 This is a schematic diagram of the interface of the content screen corresponding to the target content in another embodiment;

[0038] Figure 14 This is a schematic diagram of the interface of the content screen corresponding to the target content in another embodiment;

[0039] Figure 15 Another example is a schematic diagram of an interface displaying target content at a preset angle;

[0040] Figure 16 This is a schematic diagram of an interface displaying a global image of the target content in another embodiment;

[0041] Figure 17 This is a schematic diagram of the interface of a content promotion page displaying target content in one embodiment;

[0042] Figure 18 This is a flowchart illustrating the process of determining target content in one embodiment;

[0043] Figure 19 This is a flowchart illustrating the process of displaying the content screen corresponding to the target content on a video interaction page in one embodiment.

[0044] Figure 20 This is a schematic diagram of an interface in one embodiment where multimedia content is identified as target content, and video footage is displayed and multimedia content is played.

[0045] Figure 21 This is a schematic diagram of an interface for playing multimedia content on a program page in one embodiment.

[0046] Figure 22 This is a schematic diagram of the interface of the program page that invokes the target application on the video interaction page in one embodiment.

[0047] Figure 23 This is a schematic diagram of an interface for playing multimedia content on a target webpage in one embodiment;

[0048] Figure 24 This is a schematic diagram of an interface in one embodiment that displays download prompts for a target application on a video interaction page;

[0049] Figure 25 This is a schematic diagram of an interface displaying playback confirmation information for playing multimedia content through a target application, as shown in one embodiment.

[0050] Figure 26 This is a schematic diagram of the interface for displaying candidate delivery devices for multimedia content in one embodiment.

[0051] Figure 27 This is a schematic diagram of an interface in one embodiment that jumps to a target application to play multimedia content;

[0052] Figure 28 This is a complete flowchart illustrating the method for displaying content in one embodiment;

[0053] Figure 29 This is a structural block diagram of a content display device in one embodiment;

[0054] Figure 30 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] With the continuous development of technology and the internet, more and more applications have emerged, enabling content and information exchange between different applications. For example, sharing videos, news, and text from application 1 to application 2. However, during video interaction, if participant 1 sees content of interest and wants to share it with participant 2, participant 1 needs to use the sharing function of the application containing the content, while participant 2 needs to open the corresponding application to view the content. This means that sharing content during video interaction disrupts the continuity of the interaction between the two parties and affects the efficiency of content display. Therefore, how to more efficiently display content and ensure the continuity of video interaction is a pressing issue that needs to be addressed.

[0057] This application provides a method for improving content display efficiency and ensuring the continuity of video interaction during video interaction. The content display method provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another server.

[0058] Specifically, taking terminal 102 as an example, in a video interaction scenario involving at least two interactive objects, the video feed captured in real-time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects. Responding to an event of interest triggered by environmental content in any of the video feeds, the environmental content matching the event of interest is used as the target content, and the corresponding content is displayed on the video interaction page. Therefore, there is no need for the interactive objects to perform corresponding content sharing operations. The target content is automatically displayed through the process of environmental content triggering recognition and content display. This avoids the impact of object operations on the continuity of video interaction and eliminates the need for multiple sharing interactions, thereby improving the efficiency of content display.

[0059] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers. The content display method provided in the application embodiments can be applied to various scenarios, including but not limited to cloud technology and artificial intelligence.

[0060] Secondly, the technical terms used in this application will be introduced below:

[0061] I. Camera Programming Interface (API).

[0062] A camera API typically refers to a set of programming interfaces that allow developers to control and manage camera hardware within software applications. Through a camera API, functions such as taking photos and videos, adjusting camera settings (such as focus, exposure, and white balance), and accessing the camera's real-time data stream can be implemented.

[0063] II. Convolutional Neural Networks (CNNs).

[0064] CNNs are a type of deep learning architecture typically used to process data with a grid structure, such as images (two-dimensional grids) and audio (one-dimensional grids). CNNs have wide applications in image recognition, video analysis, image classification, and natural language processing. The core concept of a convolutional neural network is the "convolutional layer," which extracts features from the input data through convolution operations.

[0065] III. Image Recognition Model.

[0066] Image recognition models are algorithmic models that utilize machine learning, especially deep learning, to identify and process information in images. These models can recognize objects, faces, scenes, and even the style and emotion of an image. Image recognition models have wide applications in medical image analysis, autonomous driving, security monitoring, social networks, and many other fields.

[0067] IV. Approximate Nearest Neighbor Search (ANN).

[0068] ANN (Abstract Neighbors) is an algorithm for quickly finding the nearest neighbor to a query point in a large dataset. Compared to Exact Nearest Neighbor Search (Exn.), ANN guarantees higher search efficiency. In many practical applications, such as image retrieval, recommender systems, and machine learning, the data volume is often enormous and highly dimensional. In these cases, Exact Nearest Neighbor Search can be very time-consuming; therefore, the ANN algorithm can significantly improve search speed, making its application in real-time systems possible.

[0069] 5. Uniform Resource Identifier (URI).

[0070] A URI scheme is a standard string used to identify resources on the Internet. The purpose of a URI is to provide a simple and flexible way to identify and interact with resources on the network. A URI scheme is an important component of a URI, defining how to specify the type of resource and how to access it.

[0071] The technical implementation logic of this application can be summarized into the following three sides: user side, product side, and backend side, as detailed below. Figure 2The schematic diagram illustrating the technical implementation logic, from the user side, describes a video interaction scenario involving at least two interactive objects, including at least a first interactive object and a second interactive object. The login terminal of the first interactive object displays the video feed captured in real-time by the login terminal of the second interactive object, and similarly, the login terminal of the second interactive object also displays the video feed captured in real-time by the login terminal of the first interactive object. This application describes the process based on the login terminal of the first interactive object. The login terminal of the first interactive object can filter target video frames from the video stream data captured in real-time by the login terminal of the second interactive object, analyze the target video frames for application matching, and then, after completing the matching, display a content screen corresponding to the target content in addition to the displayed video screen. At this time, the content screen is related to the target application used to display the target content.

[0072] On the product side, if it is necessary to display the real-time video feed captured by the login terminal of the second interactive object on the login terminal of the first interactive object, and the target content is considered to be the multimedia content played by the terminal device, then the second interactive object can point the multimedia content played by the terminal device at the camera to display it, so that the multimedia content played by the terminal device is included in the real-time video feed captured by the login terminal of the second interactive object. When the real-time video feed captured by the login terminal of the first interactive object includes the first interactive object, gaze focus detection is performed on the gaze of the first interactive object to obtain the gaze focus result. Then, when the gaze focus result indicates that the gaze of the first interactive object is focused on the environmental content in the video feed, it is determined whether the duration of focusing on the environmental content has reached the preset focus duration. If not, the real-time video feed captured by the login terminal of the second interactive object continues to be displayed on the login terminal of the first interactive object, and no other content is displayed. If not, that is, the duration of focusing on the environmental content has reached the preset focus duration, then the application corresponding to the target content needs to be analyzed. At this time, the product side needs to perform application retrieval and also analyze the application corresponding to the target content and match it with the application on the backend side. Subsequent embodiments will describe this in detail. If no application exists corresponding to the target content, the application will not be displayed on the user's side. Secondly, considering that the target content may be physical content in practical applications, the user's side can also display video footage and the target content displayed at a preset angle, or a global image that fully displays the physical content, or a content promotion page for recommending physical content, or a content detail page for introducing physical content.

[0073] Therefore, if an application corresponding to the target content exists, application identification and matching are required. If the target application exists on the login terminal of the first interactive object, the program page of the target application is invoked. This means the target application can be opened and the user can navigate to the target content, allowing the user to view the video and the target content. Conversely, if the target application does not exist on the login terminal of the first interactive object, the target content can be displayed via a webpage, allowing the user to view the video and the target content displayed on a webpage. Alternatively, a download prompt for the target application can be displayed. If the user needs to download the target application, a download confirmation operation triggered by the download prompt can be initiated. If the target application is successfully downloaded, the program page of the target application is displayed.

[0074] The following details the backend side. The backend monitors object operations and video stream data in real time, filtering target video frames for content detection. Based on the content detection results, it triggers application retrieval. Application retrieval specifically extracts the content features of the multimedia content and determines the target application by comparing these features with the interface features of each candidate application. Therefore, if the target application exists on the login terminal of the first interactive object, it directly navigates to the target content, i.e., pointing to "Open Target Application" on the product side and navigating to the target content. This results in the user side displaying the video and the target content. Similarly, if the target application does not exist on the login terminal of the first interactive object, the target content is synchronized on the webpage, i.e., pointing to "Open Webpage" on the product side and displaying the target content. This results in the user side displaying the video and the target content displayed on the webpage. Furthermore, in practical applications, a download prompt for the target application can be displayed on the login terminal of the first interactive object. If the first interactive object needs to download the target application, a download confirmation operation triggered by the download prompt can be initiated. If the target application is successfully downloaded, the program page of the target application is displayed.

[0075] The following examples illustrate this in detail: In one example, as... Figure 3 As shown, a method for displaying content is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, it can be understood that this method can also be applied to a system including terminal 102 and server 104, and implemented through the interaction between terminal 102 and server 104. In this embodiment, the method includes the following steps:

[0076] Step 302: In a video interaction scenario involving at least two interactive objects, the video footage captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects.

[0077] In this context, the interactive object refers to an object within a video interaction scenario. There must be at least two interactive objects within a video interaction scenario; that is, the interactive objects include at least a first interactive object and a second interactive object, and may also include a third interactive object, a fourth interactive object, and so on, without specific limitations. Based on this, each interactive object has a login terminal; that is, the first interactive object has its own login terminal, and the second interactive object has its own login terminal.

[0078] Since this is a video interaction scenario, the terminal logged into by the participants in the video interaction will inevitably display a video feed of real-time video captures of other participants. This video feed is obtained by parsing the real-time video stream data, which specifically includes video frames captured by an image capture device and audio data captured by an audio capture device. The image capture device can be built into the login terminal of the participants or can be an external device such as a camera capable of image capture. Specifically, this application uses a camera API to control and manage the image capture device to achieve functions such as taking photos, recording videos, and adjusting the settings of the image capture device, and analyzes video frames by accessing the real-time data stream of the image capture device. Similarly, the audio capture device can also be built into the login terminal of the participants or can be an external device such as a microphone capable of audio capture. No specific limitations are made here.

[0079] Specifically, in video interaction scenarios involving at least two interactive objects, the video screen captured in real-time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects. For example, if the interactive objects include a first interactive object and a second interactive object, meaning that at least a first interactive object and a second interactive object are involved in the video interaction, then the login terminal of the first interactive object can display the video screen captured in real-time by the login terminal of the second interactive object. The video screen captured in real-time by the login terminal of the second interactive object can include the second interactive object and the environmental content within the frame of the real-time capture of the login terminal of the second interactive object. The environmental content can be multimedia objects such as video objects, text objects, and graphic objects, or it can be physical content, etc. The video screen can also contain a scene without any objects; no specific limitations are made here.

[0080] Furthermore, the login terminal of the first interactive object can also display the real-time video feed captured by the login terminals of other interactive objects participating in the same video interaction. Since it involves video interaction, in practical applications, to ensure that the first interactive object can know all objects participating in the video interaction, the real-time video feed captured by the login terminal of the first interactive object can also be displayed on the login terminal of the first interactive object.

[0081] Similarly, the login terminal of the second interactive object can also display the video footage captured in real time by the login terminal of the first interactive object. In practical applications, the login terminal of the second interactive object can also display the video footage captured in real time by the login terminal of the second interactive object, as well as the video footage captured in real time by the login terminals of interactive objects participating in the same video interaction. This will not be elaborated further here.

[0082] For ease of understanding, such as Figure 4 The screen display method of the login terminal of the first interactive object is shown. Figure 4 The diagram in Figure (A) shows the real-time video feed 402 captured by the login terminal of the second interactive object displayed on the login terminal of the first interactive object. Figure 4 The diagram in Figure (A) shows the video footage 404 and 406 captured in real time by the login terminal of the second interactive object, displayed in the login terminal of the first interactive object.

[0083] As discussed above, since the interactive objects involved in video interaction can also include multiple objects such as a third interactive object and a fourth interactive object, if the interactive objects involved in video interaction also include a third interactive object and a fourth interactive object, then the real-time video images captured by the login terminal of the second interactive object, the real-time video images captured by the login terminal of the first interactive object, the real-time video images captured by the login terminal of the third interactive object, and the real-time video images captured by the login terminal of the fourth interactive object can be displayed on the login terminal of the first interactive object. Specifically... Figure 5 As shown, the login terminal of the first interactive object displays the real-time video frame 502, the real-time video frame 504, the real-time video frame 506, and the real-time video frame 508 of the login terminal of the fourth interactive object.

[0084] Step 304: In response to an interest event triggered for environmental content in any video frame, the environmental content matching the interest event is taken as the target content.

[0085] Specifically, the "Event of Interest" (EOI) refers to content detection events in the video frame. Content detection events are used to trigger content recognition and detection in the video frame. Therefore, an EOI can be triggered in at least the following ways: the interactive object's gaze is focused on environmental content in the video frame for a preset focus duration; or, the interactive object performs a long press operation on the video frame for a preset duration; or, the interactive object inputs voice interaction content on the video frame; or, the interactive object triggers an interaction operation on the detection control, etc. As shown in the aforementioned embodiments, an EOI can also be a preset gesture operation on the video frame. In practical applications, EOI can also be triggered in real-time based on actual application needs, that is, real-time content detection of the video frame to trigger the EOI. Alternatively, a triggering period can be set, with periodic content detection of the video frame at preset time intervals to trigger the EOI. Therefore, the specific triggering method for EOI needs to be flexibly determined based on the actual situation and application requirements; the aforementioned examples should not be construed as limitations on EOI triggering.

[0086] Secondly, the target content can be multimedia content played by the terminal device, such as video objects, text objects, and image-text objects. The target content can also be physical objects, such as books, water cups, and dolls.

[0087] Specifically, on the video interaction page of the login terminal of any interactive object, based on displaying the video images captured in real time by the login terminals of at least one other interactive object, if the login terminal of the interactive object detects an event of interest triggered by environmental content in any of the video images, the login terminal can respond to the event of interest, determine the environmental content matching the event of interest, and take the environmental content matching the event of interest as the target content.

[0088] The following describes how to determine a target video frame from multiple video frames: In an optional embodiment, when there are at least three interactive objects participating in the video interaction, for example, the interactive objects participating in the video interaction are a first interactive object, a second interactive object, and a third interactive object. Based on this, the triggering method for triggering an interest event for environmental content in any one of the video frames includes: in response to an interest triggering operation triggered for one of the video frames, taking the video frame triggered by the interest triggering operation as the target video frame; and triggering an interest event for the environmental content in the target video frame.

[0089] The interest-triggered action can be a long press operation on a video screen for a preset duration. A long press operation is an action performed by any interactive object on the video screen displayed on the login terminal. In other words, the long press operation needs to be detected by the touch control detection module of the login terminal (which is a touchscreen). Therefore, the preset long press duration can be 3 seconds, 5 seconds, or 7 seconds, etc., and the specific preset long press duration needs to be flexibly determined based on the actual needs of the scenario.

[0090] Specifically, on the video interaction page of any interactive object's login terminal, which displays real-time video footage captured by the login terminals of at least one other interactive object, if there are at least three interactive objects participating in the video interaction, then after detecting an interest trigger operation triggered on one of the video frames, the login terminal responds to the interest trigger operation by first determining the target video frame indicated by the interest trigger operation, and then triggering an interest event for the environmental content in the target video frame. Taking a long-press operation performed by the first interactive object on its login terminal as an example, the login terminal detects a long-press operation on any video frame and times the duration of the long-press operation. If the long-press duration reaches the preset long-press duration, it indicates that the first interactive object has indeed performed an operation on any video frame, rather than a mis-touch or other situation. In this case, the video frame triggered by the interest trigger operation is taken as the target video frame, and an interest event for the environmental content in the target video frame is triggered.

[0091] For example, an interest-triggered operation could be a long press operation on a video screen for a preset duration. On the video interaction page, video screen 1, video screen 2, and video screen 3 are displayed. If the interactive object performs a long press operation on video screen 1 for a preset duration, it can be determined that video screen 1 is the target video screen, and then an interest event is triggered on the environmental content in video screen 1.

[0092] For ease of understanding, based on Figure 5 The example video interaction page, such as Figure 6 As shown, in video screens 502, 504, 506, and 508, the interactive object performs a long press interaction operation on video screen 504 for a preset duration. At this time, it can be determined that video screen 504 is the target video screen.

[0093] The following describes the triggering method for events of interest. In another optional embodiment, the interactive objects participating in the video interaction include at least a first interactive object and a second interactive object. Based on this, when the video interaction page is the display page of the first interactive object, and the video frame whose event of interest is triggered is the target video frame corresponding to the second interactive object, that is, as follows... Figure 6As shown, the video interaction page is the display page of the first interactive object, while the interaction operation of long-pressing for a preset duration is for the video screen 504 corresponding to the second interactive object. At this time, the video screen 504 is the target video screen.

[0094] Based on this, the triggering method for the event of interest includes any of the following: triggering the event of interest when the voice interaction content between the first interactive object and the second interactive object contains preset keywords; triggering the event of interest when a preset gesture operation is triggered on the target video screen; triggering the event of interest when an interactive operation is triggered on the target video screen.

[0095] First, it's understandable that if the target video screen displayed on the login terminal of the first interactive object does not contain any environmental content, no subsequent steps are taken. However, if at least one environmental element is present in the target video screen displayed on the login terminal of the first interactive object, then a response and processing for the event of interest is triggered. For ease of understanding, as follows... Figure 7 As shown, Figure 7 Figure (A) shows an example where the target video screen displayed by the login terminal of the first interactive object does not contain environmental content. Figure 7 Figure (B) illustrates the presence of environmental content 702 and environmental content 704 in the target video screen displayed on the login terminal of the first interactive object. Based on this, if the first interactive object triggers an event of interest on the video screen on the login terminal, it can respond to the aforementioned event of interest to perform content detection and matching recognition, and then use the environmental content that matches the event of interest as the target content.

[0096] The following describes the triggering methods for the aforementioned events of interest. First, we introduce the triggering method when an interactive operation is performed on the target video frame: specifically, when an interactive operation is performed on the display area containing environmental content within the target video frame, the event of interest is triggered. The interactive operation can be a preset gesture on the display area containing the environmental content, a long press on the display area containing the environmental content, etc., and is not limited here. The display area containing the environmental content is a preset area within the video frame, and this display area must include the complete environmental content of the video frame. The shape of the aforementioned preset area can be a quadrilateral or an irregular shape.

[0097] Specifically, the first interactive object performs an interactive operation on a display area containing one of the environmental contents in the target video frame. At this time, the login terminal of the first interactive object receives the interactive operation triggered by that environmental content and responds to the interactive operation. Thus, the login terminal of the first interactive object can identify that environmental content as the target content. For ease of understanding, based on long-press operations and Figure 7The example in Figure (B) will be further introduced, such as... Figure 8 As shown, the video footage captured in real time by the login terminal of the second interactive object displayed by the login terminal of the first interactive object contains environmental content 702 and environmental content 704. The display area 802 contains environmental content 702, and the display area 804 contains environmental content 704. If the first interactive object performs a long press operation on the display area 802, the login terminal of the first interactive object will respond to the aforementioned long press operation and can determine the environmental content 702 in the display area 802 as the target content.

[0098] Secondly, the first interactive object can perform preset gesture operations on the video screen, causing the login terminal of the first interactive object to respond to the aforementioned preset gesture operations for content detection and matching recognition. Therefore, we will now introduce the method of triggering events of interest when a preset gesture operation is triggered on the target video screen:

[0099] The aforementioned preset gesture operation can zoom in or out on a specific environmental content. Specifically, when the first interactive object performs a preset gesture operation on any environmental content in the video frame, the login terminal of the first interactive object receives the preset gesture operation triggered by the environmental content in the target video frame and responds to the aforementioned preset gesture operation. At this time, the login terminal of the first interactive object can directly determine the environmental content targeted by the preset gesture operation as the target content. For ease of understanding, the preset type is described as zooming in and out. Figure 7 The example in Figure (B) will be further introduced, such as... Figure 9 As shown, the video footage captured in real time by the login terminal of the second interactive object displayed by the login terminal of the first interactive object contains environmental content 702 and environmental content 704. If the first interactive object performs a gesture zoom-in interaction operation on the environmental content 704, the login terminal of the first interactive object responds to the gesture zoom-in interaction operation and determines the environmental content 704 as the target content.

[0100] Next, we will introduce how to trigger interest-based events when the voice interaction content between the first and second interaction objects contains preset keywords:

[0101] The voice interaction content refers to the voice messages exchanged between the first and second interaction objects during video interaction. Preset keywords are the voice messages used to trigger content detection. These keywords can be phrases like "content detection," "start recognition," or everyday expressions such as "what are you watching?" No specific limitation is made here. Specifically, the login terminal of the first interaction object can perform real-time voice detection. This means it performs voice detection on the voice interaction content collected in real-time by the login terminal itself. When the detected voice interaction content contains preset keywords (such as "content detection," "start recognition," and "what are you watching" in the aforementioned example), a content detection event for the video screen is triggered.

[0102] The above describes methods for interactive objects to actively trigger events of interest. In practical applications, if an interactive object is interested in a certain environmental content, its gaze will focus. Therefore, the event of interest can be triggered based on the gaze focus result of the interactive object. In one optional embodiment, the triggering method of the event of interest further includes: when the first interactive object is included in the video frame captured in real time by the login terminal of the first interactive object, performing gaze focus detection on the object's gaze to obtain a gaze focus result; when the gaze focus result indicates that the object's gaze is focused on the environmental content in the video frame captured in real time by the login terminal of the second interactive object, and the duration of focusing on the environmental content reaches a preset focus duration, the event of interest is triggered.

[0103] Secondly, gaze-focusing detection is used to identify the focus area of ​​the object's gaze. Therefore, the gaze-focusing result is used at least to characterize the focus area of ​​the first interactive object's gaze, and can further determine the environmental content present in the focus area. Based on this, the preset focus duration can be 3 seconds, 5 seconds, or 7 seconds, etc., and the specific preset focus duration needs to be flexibly determined based on the actual scenario requirements.

[0104] Specifically, the login terminal of the first interactive object can perform content recognition on the real-time captured video stream data to obtain a second content recognition result. The login terminal of the first interactive object can also send the real-time captured video stream data to the server, so that the server can perform content recognition on the video stream data to obtain a second content recognition result. The specific device performing content recognition is not limited here. The purpose of performing content recognition on the video stream data is to identify objects and object types within the video frames of the video stream data. The second content recognition result is used to characterize the objects and object types in the video stream data. Since it is necessary to consider whether the first interactive object's gaze is focused on a certain environmental content in the real-time captured video frame by the login terminal of the second interactive object, the main purpose of determining whether a content detection event for the video frame is triggered in this application is to identify whether the first interactive object exists in the video stream data. Therefore, the second content recognition result is specifically used to characterize whether the first interactive object exists in the video stream data, or whether the first interactive object does not exist.

[0105] Based on this, if the second content recognition result indicates that the first interactive object is included in the real-time video frame captured by the login terminal of the first interactive object, it means that the first interactive object is currently within the real-time video frame captured by the login terminal of the first interactive object. At this point, gaze focus detection can be further performed on the object's gaze to obtain the gaze focus result, that is, to determine the focus area of ​​the object's gaze. It is then necessary to determine whether any environmental content exists in the focus area of ​​the object's gaze. If so, it means that the gaze focus result indicates that the object's gaze is focused on environmental content in the video frame. Then, the duration of the first interactive object focusing on the environmental content is timed. If the duration of focusing on the environmental content reaches a preset focus duration, it means that the first interactive object has focused on the focus area for the preset focus duration, i.e., the first interactive object may be interested in the environmental content within that area. At this point, a content detection event for the video frame is triggered. Specifically, the area for content detection in the video frame at this time is the focus area in the video frame that includes environmental content.

[0106] In practical applications, the event of interest can also be triggered by the interactive object actively performing detection operations. In one optional embodiment, the triggering method of the event of interest further includes: triggering the event of interest when the first interactive object triggers an interactive operation on the detection control.

[0107] The detection control is used to detect interactive operations. Specifically, the detection control can be deployed on the touchscreen of the login terminal of the first interactive object, and it will also be displayed on the login terminal of the first interactive object. Alternatively, the detection control can be an external control on the login terminal of the first interactive object. Specifically, when the first interactive object interacts with the detection control, its login terminal detects the interaction, triggering an interest event (IOEvent) on the real-time video feed captured by the login terminal of the second interactive object. Since the detection control is also displayed on the login terminal of the first interactive object, it can be specifically displayed within the real-time video feed captured by the login terminal of the second interactive object. The IOT event is triggered on the video feed of the login terminal of the participating video interaction object, depending on which video feed the first interactive object interacts with.

[0108] For ease of understanding, such as Figure 10 As shown, the login terminal of the first interactive object displays a video frame 1002 captured in real time by the login terminal of the second interactive object, and a detection control 1004 exists in the video frame 1002. If the first interactive object performs an interactive operation on the detection control 1004, an interest event is triggered for the video frame 1002.

[0109] It is understandable that, since the login terminal of the first interactive object can also display video images captured in real time by the login terminals of other objects participating in video interaction, the aforementioned method can also be used to trigger the detection and judgment of interest events for the video images captured in real time by other terminals, which will not be elaborated here.

[0110] Step 306: On the video interaction page, display the content screen corresponding to the target content.

[0111] The target content can be physical objects, such as books, water cups, and dolls. It can also be multimedia content played on the terminal device, including video, text, and image-text objects; the specific type of the target content is not limited here. Secondly, the content screen is a screen related to the target content. For ease of understanding, taking physical objects as an example, the content screen can be a global image of the physical object, including the complete object, and can include a front view, a top view, and a side view. Alternatively, the content screen can be a partial image of the physical object, excluding the complete object. For example, if the object is a doll, the global image can be a front view of the entire doll, while the partial image can be a front view showing only the doll's head, or a front view showing only half of the doll's body. Considering that physical objects are often purchased online, the content screen can also be a promotional page for the physical object; this is not limited here.

[0112] Taking the target content as multimedia content played by the terminal device as an example, the content screen corresponding to the target content can be multimedia content, or the content screen corresponding to the target content can also be the target application, which is the application that displays multimedia content on the terminal device, or the content screen corresponding to the target content can also be a download prompt message for the target application to which the multimedia content belongs, or the content screen corresponding to the target content can be a selection area for the multimedia content to be displayed, etc. There are no limitations here.

[0113] Specifically, the terminal displays the content screen corresponding to the target content on the video interaction page. This content screen can be displayed simultaneously with the video screen captured in real-time by the logged-in terminal, or they can be displayed at different times; this is not limited here. Taking simultaneous display as an example, the interactive objects with the video interaction include at least a first interactive object and a second interactive object, as well as the video interaction page being the display page for the first interactive object. In this case, the content screen corresponding to the target content can be displayed on the logged-in terminal of the first interactive object, while simultaneously displaying the video screen captured in real-time by the logged-in terminals of other interactive objects.

[0114] It is understandable that the login terminal of the interactive object participating in the video interaction could send the real-time video stream data to the server. The server would then perform content detection and recognition based on the video stream data sent by each login terminal to detect whether the target content exists in the real-time video frame captured by each interactive object's login terminal. If the target content is identified in the video frame, when sending the video stream data of the interactive object's login terminal to the login terminals of other interactive objects, it would also carry relevant information indicating the target content, so that the interactive object's login terminal would simultaneously display the video frame and the content frame corresponding to the target content.

[0115] For ease of understanding, let's take the display of the login terminal of the first interactive object as an example, such as... Figure 11 As shown, Figure 11 The diagram in Figure (A) shows a scenario where the login terminal of the first interactive object simultaneously displays a video feed 1102 captured in real-time by the login terminal of the second interactive object and a content feed 1104 corresponding to the target content. Figure 11 The diagram in Figure (B) shows that the login terminal of the first interactive object simultaneously displays the video screen 1102 captured in real time by the login terminal of the second interactive object, the video screen 1106 captured in real time by the login terminal of the first interactive object, and the content screen 1104 corresponding to the target content.

[0116] Based on this, since the display needs to include the video footage captured in real time by the login terminal of the second interactive object and the content screen corresponding to the target content, the specific method for simultaneously displaying the video footage and the content screen is as follows: The content display method also includes: parsing the video stream data captured in real time by the login terminal of the second interactive object to obtain the video footage corresponding to the video stream data; and performing content matching based on the target content to determine the content screen corresponding to the target content.

[0117] The second video stream data refers to the video stream data captured in real-time by the login terminal of the second interactive object. Specifically, the video stream data includes video frames captured by an image acquisition device and audio data captured by an audio acquisition device. The image acquisition device can be built into the login terminal of the interactive object or can be a camera or other image-capturing device connected externally to the login terminal. Secondly, the content screen is similar to the aforementioned embodiment; the content screen is a screen related to the object type of the target content, and will not be elaborated further here.

[0118] Specifically, the server can parse the video stream data collected in real time from the login terminal of the second interactive object to obtain the video frame corresponding to the video stream data. The server then sends the video frame corresponding to the video stream data to the login terminal of the first interactive object. Similarly, the server performs content matching based on the target content and the object type of the target content to determine the content frame corresponding to the target content. For example, if the target content is physical content, the content frame corresponding to the target content could be a panoramic image of the physical content, a partial image of the physical content, or a promotional page for the physical content, etc. The server then sends the content frame corresponding to the target content to the login terminal of the first interactive object. Based on this, the login terminal of the first interactive object can simultaneously display the video frame and the content frame corresponding to the target content.

[0119] In practical applications, the server can receive real-time video stream data collected by the login terminal of the second interactive object, and then send this data to the login terminal of the first interactive object. The first interactive object's login terminal then parses the video stream data using a similar method to obtain the corresponding video frame, and performs content matching based on the target content and its object type to determine the corresponding content frame. Thus, both the video frame and the content frame corresponding to the target content are simultaneously displayed on the first interactive object's login terminal.

[0120] The following describes the methods for displaying the content screen corresponding to the target content: In a specific embodiment, on the video interaction page, displaying the content screen corresponding to the target content includes any of the following methods: displaying the content screen corresponding to the target content in full-screen mode and displaying the video screen on the content screen corresponding to the target content in a floating window mode; displaying the video screen in full-screen mode and displaying the content screen corresponding to the target content on the video screen in a floating window mode; displaying the video screen through a first floating window and displaying the content screen corresponding to the target content through a second floating window.

[0121] The following sections describe the specific display methods for the content screen corresponding to the target content:

[0122] 1. Display the content screen corresponding to the target content in full-screen mode, and display the video screen on top of the content screen corresponding to the target content in a floating window mode. The floating window mode means the video screen floats on top of other screens. Specifically, when displaying the content screen corresponding to the target content, it can be displayed in full-screen mode on the login terminal of the interactive object to ensure a clear and intuitive presentation of the corresponding content screen to the interactive object. Since the aforementioned content screen is already displayed in full-screen mode, to ensure that the video screen can also be displayed simultaneously, the video screen is displayed in a floating window mode on the content screen corresponding to the target content. For ease of understanding, the login terminal of the first interactive object is used as the display reference, such as... Figure 12 As shown, in the login terminal of the first interactive object, the content screen 1202 corresponding to the target content is displayed in full screen mode, and the video screen 1204 is displayed on the content screen 1202 corresponding to the target content in floating window mode.

[0123] 2. Display the video in full-screen mode, and then display the content screen corresponding to the target content on top of the video screen in a floating window. Specifically, when displaying both the video screen and the content screen simultaneously, the video screen can be displayed in full-screen mode on the login terminal of the interactive object. To ensure that the content screen corresponding to the target content can also be displayed simultaneously, the video screen is displayed on top of the content screen corresponding to the target content in a floating window. For ease of understanding, the login terminal of the first interactive object is used as the display reference, such as... Figure 13 As shown, in the login terminal of the first interactive object, the video screen 1302 is displayed in full screen mode, and the content screen 1304 corresponding to the target content is displayed on the video screen 1302 in floating window mode.

[0124] 3. Display the video image through a first floating window and the content image corresponding to the target content through a second floating window. Specifically, considering that the object can perform other interactive processing simultaneously while performing video interaction, displaying the image entirely in full-screen mode may not meet other interactive needs. This application also provides a display method in the login terminal of the interactive object, where the video image is displayed through a first floating window and the content image corresponding to the target content is displayed through a second floating window. For ease of understanding, the login terminal of the first interactive object is used as the display reference, such as... Figure 14 As shown, in the login terminal of the first interactive object, a video screen 1402 is displayed through a first floating window, and a content screen 1404 corresponding to the target content is displayed through a second floating window. It is understandable that, in the presence of multiple interactive objects, similar display methods can be used... Figure 11 The actual simultaneous display is shown. Furthermore, the corresponding examples in the embodiments of this application are used to understand this solution, but should not be construed as specific limitations on this solution.

[0125] By displaying video and content images simultaneously using the aforementioned methods, it is possible to meet various display needs in practical applications and improve the feasibility and flexibility of content display.

[0126] The following describes how to display the content screen corresponding to the target content when the target content is physical content. In one specific embodiment, displaying the content screen corresponding to the target content on the video interaction page includes at least one of the following methods: displaying the target content shown at a preset angle on the video interaction page; displaying a global image of the target content on the video interaction page; displaying a content promotion page for the target content on the video interaction page; or displaying a content details page for the target content on the video interaction page.

[0127] The following sections describe the methods for displaying content on the screen:

[0128] 1. On the video interaction page, the target content is displayed at a preset angle. The preset angle can be any angle, such as 90°, and is not limited here. The target content displayed at the preset angle can be a partial image of a real object, where the partial image does not include the complete object. For example, if the object is a doll, the partial image could be a front view showing only the doll's head, or a front view showing only half of the doll's body, or a side view of the doll.

[0129] Specifically, on the video interaction page, the target content is displayed at a preset angle. As described above, the target content at the preset angle can be displayed in full screen on the video interaction page, and the video feed can be displayed on the corresponding content screen as a floating window. Alternatively, the video feed can be displayed in full screen, and the target content at the preset angle can be displayed on the video feed as a floating window. Alternatively, the video feed can be displayed through a first floating window, and the target content at the preset angle can be displayed through a second floating window.

[0130] To facilitate understanding, we will use the following example: displaying the target content in full-screen mode and then showing the video feed as a floating window on top of the target content. The target content will be specifically a color palette. Figure 15 As shown, a partial image 1502 of the physical content of the drama is displayed in full screen mode, and a video screen 1504 is displayed on the partial image 1502 of the physical content in a floating window mode.

[0131] 2. On the video interaction page, display a global image of the target content. This global image includes the complete physical content and may include a front view, a top view, and a side view of the physical content. Specifically, the display method of the global image of the target content on the video interaction page is similar to the aforementioned embodiment and will not be repeated here. For ease of understanding, an example is provided where the content screen corresponding to the target content is displayed in full-screen mode, and the video screen is displayed on top of the content screen corresponding to the target content in a floating window. The target content is specifically a color palette, as an example. Figure 16 As shown, a global image 1602 displaying the complete physical content is shown in full-screen mode, and a video image 1602 is displayed on the global image 1602 displaying the complete physical content in a floating window mode.

[0132] 3. On the video interaction page, display the content promotion page for the target content. Considering that physical products can usually be purchased online, the content screen corresponding to the target content can also be a content promotion page for the physical product. Therefore, the video screen and the content promotion page recommending the physical product can be displayed simultaneously on the login terminal of the interactive object. The content promotion page is used to recommend physical products; therefore, it includes the physical product itself and descriptive text describing the product. In practical applications, since the content promotion page is used to recommend physical products, after the first interactive object interacts with the displayed content promotion page, it indicates that the interactive object may be interested in the physical product. At this point, it can also redirect to the shopping platform indicated by the content promotion page. Specific details regarding subsequent steps are not limited here.

[0133] To facilitate understanding, we will use a full-screen video display as an example, with the target content displayed as a floating window on top of the video screen. The target content will be specifically a color palette as an example. Figure 17 As shown, the video screen 1702 is displayed in full screen mode, and the content promotion page 1704 recommending physical products is displayed on the video screen 1702 in a floating window.

[0134] 4. On the video interaction page, display the content details page of the target content. Similar to the content promotion page, the content details page of the target content does not redirect to promotional content; it only displays the content details specific to the target content. The specific display method is similar to the aforementioned embodiments and will not be repeated here.

[0135] It is understood that the corresponding examples in the embodiments of this application are used to understand this solution, but should not be construed as specific limitations on this solution.

[0136] In the aforementioned content display method, in video interaction scenarios involving at least two interactive objects, the video feed captured in real-time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects to ensure the real-time display of the screen during video interaction. Based on this, in response to an event of interest triggered by environmental content in any video frame, the environmental content matching the event of interest is taken as the target content, and the corresponding content screen is displayed on the video interaction page. This eliminates the need for interactive objects to perform corresponding content sharing operations. The target content is automatically displayed through the environmental content trigger recognition and content display process, thus avoiding the impact of object operations on the continuity of video interaction and eliminating the need for multiple sharing interactions, thereby improving the efficiency of content display.

[0137] The following describes how to determine the target content. In a specific embodiment, such as... Figure 18 As shown, in response to an interest event triggered for environmental content in any video frame, the environmental content matching the interest event is taken as the target content, including:

[0138] Step 1802: In response to an interest event triggered for environmental content in any video frame, determine the target video frame indicated by the interest event.

[0139] Specifically, in the video interaction page of any interactive object's login terminal, based on displaying video frames captured in real time by the login terminals of at least one other interactive object, if the login terminal of an interactive object detects an event of interest triggered by environmental content in any of the video frames, the login terminal responds to the event of interest by first determining the target video frame indicated by the event of interest. For example, if video frames 1, 2, and 3 are displayed on the video interaction page, and the event of interest is triggered by environmental content in video frame 1, then video frame 1 is the target video frame. The method for determining the target video frame is similar to that in the aforementioned embodiments and will not be repeated here.

[0140] Step 1804: Filter out the target video frames from the video stream data containing the target video images.

[0141] Specifically, the login terminal of the interactive object obtains video stream data containing the target video frame. That is, the login terminal needs to send a data acquisition request to the server. This request requests the video stream data containing the target video frame, which is being captured in real-time by the login terminal. Then, the login terminal filters out the target video frames from the video stream data containing the target video frame.

[0142] Since the login terminal of the interactive object matching the target video image can call the camera API to collect video stream data in real time, and analyzing the video stream data can display the corresponding video image on any interactive object's login terminal, not all video images (i.e., video frames in the video stream data) are usable. Therefore, it is necessary to extract keyframes from the video stream data to filter out the target video frame. This can be achieved by first preprocessing consecutive video frames in the video stream data, specifically by using image enhancement algorithms (such as histogram equalization) to improve the image quality of each video frame, and then implementing video frame selection logic to process only the video frames with significant variations, thereby reducing the computational burden. Based on this, the specific steps for filtering the target video frame from the video stream data containing the target video image are: preprocessing multiple video frames in the video stream data to improve the image quality of each video frame, and then continuously comparing the preprocessed video frames to filter out the video frames with significant variations, thus identifying the video frames with significant variations as the target video frame.

[0143] The following details how to compare and filter video frames that show significant changes after image preprocessing:

[0144] 1. Calculate the image difference between two consecutive preprocessed video frames. If the image difference exceeds a preset difference threshold, determine that the next frame in the two consecutive preprocessed video frames is the one with the greater change.

[0145] 2. Generate image sequences for the same object in the video frames after image preprocessing, and then analyze the motion trajectory of the same object in the image sequence to estimate the motion pattern between the video frames after image preprocessing. If the vector change of the motion pattern representing the optical flow of the object is greater than the change threshold, it indicates that the change between frames is large. In other words, video frames with vector changes greater than the change threshold are selected as video frames with large changes.

[0146] 3. When a static background exists in the video frame, each preprocessed video frame is compared with the static background to detect changes in the video frame. If the difference between the video frame and the static background exceeds a preset difference threshold, the video frame is determined to be a video frame with large changes.

[0147] 4. Compare the color histograms of two consecutive preprocessed video frames. If the difference between the color histograms exceeds a preset difference threshold, the video frame is determined to be a video frame with large changes.

[0148] In practical applications, all video frames in the video stream data can be directly identified as target video frames. Alternatively, the similarity between consecutive video frames in the video stream data can be calculated, and if the similarity of multiple consecutive video frames is greater than a preset similarity threshold, one of these frames can be selected as the target video frame. Therefore, image enhancement algorithms can be used to improve the image quality of each video frame in the video stream data, and then target video frames can be extracted and filtered using methods described above (such as color histogram-based methods, motion vector-based methods, and Local Binary Pattern (LBP) methods). After extracting the target video frames, they can be further preprocessed to facilitate subsequent content recognition. Preprocessing can include scaling the target video frames to the size expected by the pre-trained model, such as 224x224 pixels, and then converting the scaled target video frames to grayscale to reduce computational complexity. Additionally, the pixel values ​​of the target video frames can be normalized to between 0 and 1 to improve the stability of data processing.

[0149] It is understood that the specific method for selecting target video frames from video stream data needs to be flexibly determined based on the actual situation and scenario requirements, and should not be construed as a specific limitation on this application.

[0150] Step 1806: Perform environmental content recognition on the target video frame and use the environmental content represented by the content recognition result as the target content.

[0151] Environmental content recognition includes target video frame recognition and object matching. Specifically, environmental content recognition identifies objects present in the target video frame and the object type of each object, then matches the objects with candidate objects corresponding to their object types. The target content can be an object belonging to a preset object type, or the object that occupies the largest proportion of the content detection frame in the target video frame; there are no specific limitations here.

[0152] Specifically, the login terminal of the interactive object can perform environmental content recognition on the target video frame to obtain a first environmental content recognition result for the target video frame. Then, based on the first environmental content recognition result, it is determined that the video screen includes the target content. That is, environmental content recognition is first performed on the target video frame to obtain the target video frame. As mentioned above, when each login terminal collects video stream data in real time and transmits it accordingly, in order to improve the image quality of each video frame in the video stream data, image enhancement algorithms can be used to perform image enhancement processing on each video frame in the video stream data. Then, the target video frame is extracted and filtered using the methods described above (e.g., color histogram-based methods, motion vector-based methods, LB methods, etc.). Environmental content recognition of target video frames can be performed using image recognition models. Considering the stability and efficiency of the model, the environmental content recognition of target video frames by the login terminal of the interactive object includes: scaling the extracted target video frames to a size supported by the image recognition model, such as 224x224 pixels, and then converting the scaled target video frames to grayscale to reduce the computational complexity of the image recognition model. The pixel values ​​of the target video frames can also be normalized to between 0 and 1 to improve stability.

[0153] After the aforementioned processing, the preprocessed target video frame is used to extract features using an image recognition model (such as a convolutional neural network). Since a convolutional neural network consists of multiple alternating convolutional and pooling layers, each convolutional layer applies a set of convolutional filters to the input image. Each filter captures a specific pattern or feature in the image, such as edges, textures, color blocks, or corners. The pooling layer reduces the spatial size of the image while retaining important feature information. These features can be considered a high-level representation of the image, capturing its main information and can be used for subsequent environmental content recognition. That is, the image recognition model can extract image features from the target video frame, thereby obtaining the objects included in the target video frame, the detection boxes to which the objects belong, and the object types. This completes the first step of environmental content recognition. Considering different object types, a candidate object database for each object type can be built in advance. The object with the highest similarity to the target video frame is selected from this database as the object in the detection box. Then, the object in the detection box with the largest content proportion in the target video frame is identified as the target content. Alternatively, you can directly select an object belonging to a preset object type and filter it based on similarity from the candidate object database of the preset object type to obtain the target content of the preset object type.

[0154] Since the similarity between objects is considered during the filtering process, and similarity can be obtained by comparing image features with the features of candidate objects in the candidate object database, the most similar objects can be found. At this time, some efficient search algorithms can be used, such as approximate nearest neighbor search, as well as some optimized data structures, such as high-dimensional indexed tree structures (K-dimension tree, KD tree), ball tree, etc. In addition, hashing techniques (such as Locality-Sensitive Hashing (LSH)) can be considered to further improve the speed of query filtering, without making specific limitations here.

[0155] It is understood that the corresponding examples in the embodiments of this application are used to understand this solution, but should not be construed as specific limitations on this solution.

[0156] In the above embodiments, by filtering target video frames, it is possible to avoid filtering and content detection of all video frames, thereby ensuring the reliability and efficiency of content detection. This allows for more accurate and efficient determination of target content through content recognition, thereby improving the reliability and efficiency of displaying the content screen corresponding to the target content.

[0157] The following describes how to display the content screen corresponding to the target content when the target content is multimedia content played by the terminal device. In one embodiment, such as... Figure 19 As shown, the video screen displays a terminal device playing multimedia content, and the target content is the multimedia content played by the terminal device.

[0158] In other words, if a terminal device exists within the video frame and is playing multimedia content, then the target content can be the multimedia content being played by the terminal device. This target content can be a video object, a text object, an image / text object, etc. For example, if the multimedia content is a video object, then the multimedia content could be "TV series A, episode B, minute C". If the multimedia content is a text object, then the multimedia content could be "article A, page B".

[0159] Based on this, the video interaction page displays the content screen corresponding to the target content, including:

[0160] Step 1902: On the video interaction page, play multimedia content.

[0161] Specifically, multimedia content is played on the video interaction page of the login terminal of the interactive object. In practical applications, video footage and multimedia content can be simultaneously displayed on the video interaction page of the login terminal of the interactive object. Taking the login terminal of the first interactive object as the display reference, the video footage captured in real-time by the login terminal of the second interactive object can be displayed on the video interaction page of the login terminal of the first interactive object. Since the video footage captured in real-time by the login terminal of the second interactive object shows the terminal device playing multimedia content, multimedia content can then be played on the video interaction page of the login terminal of the first interactive object. For easier understanding, please refer to [link to relevant documentation]. Figure 20 , Figure 20 As shown in (A): In the login terminal of the first interactive object, a video frame 2002 captured in real time by the login terminal of the second interactive object is displayed. At this time, the video frame 2002 contains a terminal device 2006 playing multimedia content 2004. The multimedia content 2004 is then identified as the target content. Figure 20 As shown in (B): In the login terminal of the first interactive object, a video screen 2002 is displayed and multimedia content 2004 is played.

[0162] In one specific embodiment, the terminal device plays multimedia content through a target application; that is, the multimedia content is played on the terminal device through the target application. The method for identifying how the terminal device plays multimedia content through the target application includes: determining candidate applications that match the category of the multimedia content; extracting the content features of the multimedia content; performing similarity analysis between the content features and the corresponding interface features of each candidate application to obtain feature similarity; and selecting candidate applications whose feature similarity meets the matching requirements as the target application.

[0163] Extracting content features from multimedia content and performing similarity analysis requires an image recognition model. This application uses a CNN (Convolutional Neural Network) model, primarily for feature extraction. A CNN model consists of alternating convolutional and pooling layers. Each convolutional layer applies a set of convolutional filters to the input image, capturing a specific pattern or feature, such as edges, textures, color blocks, or corners. Pooling layers reduce the spatial size of the image while preserving important feature information. These features can be considered a high-level representation of the image, capturing its main information, thus enabling the extraction of content features from multimedia content.

[0164] Then, for each content sample and the application sample to which it belongs, the sample features corresponding to the content sample can include the application sample's color scheme, the shape and position of buttons in the application sample, the font and size of text in the application sample, the layout style of UI components in the application sample, the application logo of the application sample, and application-specific interface elements, etc. Then, by performing data augmentation on each content sample, more content samples can be generated, thereby improving the model's generalization ability. During training, methods for data augmentation of content samples include, but are not limited to, rotation, scaling, cropping, flipping, and color transformation. For example, a screenshot of an application sample can be used as a content sample, and this content sample can be rotated and scaled to obtain new content samples.

[0165] Therefore, an image recognition model (primarily a classifier) ​​is trained based on the aforementioned features to identify the application to which the image recognition model belongs. This eliminates the need to train a complete model from scratch, instead leveraging the knowledge already learned by the pre-trained model to improve training efficiency and recognition accuracy. Furthermore, since the pre-trained models are trained on a large amount of image data, they can recognize a wide variety of image features, which improves the model's generalization ability, enabling it to better handle various types of images.

[0166] Based on this, since there are content samples and application samples to which the content samples belong, an application feature library can be constructed for different application types. This library contains interface features for different application types. Therefore, in practical applications, by comparing the content features with the interface features of different application types in the application feature library, the target application to which the target content belongs can be found. At this point, some efficient search algorithms can be used, such as approximate nearest neighbor search, and some optimized data structures, such as KD-trees and ball trees. Furthermore, hashing techniques (such as LSH) can be considered to further improve the query and filtering speed; specific limitations are not specified here.

[0167] Specifically, the server first identifies candidate applications that match the category of the multimedia content. That is, if the multimedia content is classified as text, the candidate application is one that can display text; similarly, if the multimedia content is classified as video, the candidate application is one that can play video. Based on this, the server extracts the content features of the multimedia content using a trained image recognition model. Then, it extracts the interface features corresponding to each candidate application from an application feature library. The server then performs a similarity analysis between the content features and the corresponding interface features of each candidate application to obtain feature similarity scores. The candidate application corresponding to the interface feature with the highest similarity score to the content features is selected as the target application to which the target content belongs. For example, given candidate applications A1, A2, and A3, the server extracts interface feature B1 corresponding to candidate application A1, interface feature B2 corresponding to candidate application A2, and interface feature B3 corresponding to candidate application A3. Calculate the feature similarity between the content feature and interface features B1, B2, and B3 respectively. If the feature similarity between the content feature and interface feature B1 is 40%, the feature similarity between the content feature and interface feature B2 is 80%, and the feature similarity between the content feature and interface feature B3 is 60%, it can be determined that the feature similarity value with interface feature B2 is the largest (i.e., 80%). Therefore, the candidate application A2 corresponding to interface feature B2 is determined as the target application to which the target content belongs.

[0168] Based on this, on the video interaction page, multimedia content is played, including: on the video interaction page, the program page of the target application is invoked; on the program page, multimedia content is played.

[0169] Specifically, the login terminal of the interactive object, on the video interaction page, invokes the program page of the target application, and then plays multimedia content on the program page. That is, the target application's program page is first invoked on the video interaction page of the login terminal of the interactive object, and then multimedia content is played on the program page. For ease of understanding, as follows... Figure 21 As shown, multimedia content 2104 is played on program page 2102.

[0170] It is understandable that the terminal device plays multimedia content through the target application and needs to invoke the target application's program page to play the multimedia content. Therefore, it is necessary to consider whether the login terminal of the interactive object has the target application. The following describes two cases: the login terminal of the interactive object has the target application and the login terminal does not have the target application.

[0171] In one specific embodiment, activating the program page of the target application on the video interaction page includes: searching for the target application among existing applications; and if the target application is found, activating the program page of the target application on the video interaction page.

[0172] Specifically, the terminal searches for the target application among its downloaded existing applications. The search method can be based on the target application's program identifier or program name; this is not limited here. Based on this, if the target application is found among the terminal's existing applications, its program page is invoked on the video interaction page. The invoked program page can be displayed in full-screen mode or as a floating window on the video interaction page; this is not limited here. For ease of understanding, as follows... Figure 22 As shown, on the video interaction page, program page 2202 of the target application is invoked.

[0173] In an optional embodiment, if the target application does not exist in the existing applications, a target webpage is invoked on the video interaction page, and multimedia content is played on the target webpage. The target webpage is related to the target application; that is, the target webpage is used to play multimedia content specific to the target application. Specifically, if the target application does not exist in the existing applications, it means that the target application cannot be invoked to play content. In this case, a target webpage related to the target application can be invoked, and then multimedia content can be played on the target webpage. Invoking the target webpage can be done in full-screen mode or as a floating window on the video interaction page; there is no limitation here.

[0174] For ease of understanding, such as Figure 23 As shown, Figure 23 The diagram in Figure (A) illustrates how the target webpage 2302 is invoked on the video interaction page. Based on this, multimedia content can be further played on the target webpage 2302, such as... Figure 23 The diagram in (B) shows the playback of multimedia content 2304 on the target webpage 2302.

[0175] In an optional embodiment, the content display method further includes: displaying download prompts for the target application on the video interaction page when the target application does not exist in existing applications.

[0176] Specifically, if the target application is not present in existing applications, a download prompt for the target application is displayed on the video interaction page of the login terminal of the interacting object. Since the target application is not present in existing applications, both the video feed and the target content displayed on the webpage can be displayed simultaneously. Considering the potential for webpage display latency and the possibility that it may not fully support the relevant functions of the target application, a download prompt for the target application can be displayed to ensure the reliability of the target content display. This prompt instructs the interacting object to download the target application. In other words, even if the target application is not present in the login terminal of the interacting object, a download prompt for the target application is displayed. For ease of understanding, as follows... Figure 24 As shown, the video interaction page displays a download prompt message 2402 for the target application.

[0177] Based on this, on the video interaction page, the program page of the target application is invoked, including: responding to a program download confirmation operation triggered by the download prompt information, and displaying the program page of the target application if the target application is successfully downloaded. Specifically, when the interactive object confirms the download of the target application, that is, when the interactive object triggers a program download confirmation operation in response to the download prompt information, the login terminal of the interactive object responds to the program download confirmation operation, and displays the program page of the target application if the target application is successfully downloaded.

[0178] In practical applications, such as meeting scenarios or some video interaction scenarios, there is no need to automatically push and display content. Therefore, content can be displayed through playback confirmation information. In one specific embodiment, on the video interaction page, the program page of the target application is invoked, including: displaying playback confirmation information for playing multimedia content through the target application on the video interaction page; and in response to a confirmation operation triggered by the playback confirmation information, invoking the program page of the target application on the video interaction page.

[0179] The playback confirmation information is used to confirm whether to play multimedia content through the target application. In other words, the playback confirmation information enables the interactive object to determine whether to play multimedia content through the target application. The playback confirmation information can be displayed in the notification area, which is used to display notifications for the application or the system, and also supports minimizing the application. In this application, the notification area is specifically used to display the playback confirmation information. Specifically, on the video interaction page of the interactive object's login terminal, the playback confirmation information for playing multimedia content through the target application is displayed. That is, during video interaction, the interactive object does not directly launch the target application and play multimedia content through it; instead, it first displays the playback confirmation information, which is the corresponding text of the playback confirmation information displayed in the notification area. The corresponding text of the playback confirmation information can be something like "Multimedia content will be played for you soon," etc., and is not limited here. For ease of understanding, as... Figure 25 As shown, the login terminal of the interactive object displays a playback confirmation message 2502 indicating that multimedia content is played through the target application, and the playback confirmation message 2502 is located in the notification prompt area 2504.

[0180] Based on this, once the interactive object determines that multimedia content will be played through the application, it can confirm the playback confirmation information displayed on the login terminal. This confirmation operation can be a long press, a click, or a swipe, etc., and is not limited here. At this time, the login terminal of the interactive object responds to the confirmation operation triggered by the playback confirmation information, launching the target application's program page on the video interaction page, and then playing the multimedia content through the target application's program page. The method of launching the program page and playing the multimedia content is similar to that described in the previous embodiments and will not be repeated here.

[0181] The following describes how to determine the playback confirmation information. In one specific embodiment, on the video interaction page, displaying playback confirmation information for multimedia content played through the target application includes: generating a deep link indicating the target application and the multimedia content, and displaying playback confirmation information for the multimedia content based on the deep link.

[0182] The playback confirmation information is used to confirm whether to play multimedia content through the target application. Since the target content is the multimedia content played by the terminal device, meaning the multimedia content must be specifically displayed within the target application on the terminal device, as described in the previous embodiments, determining the target application playing the multimedia content on the terminal device also involves determining the multimedia content. Specifically, this requires the server to generate a deep link indicating the target application and the multimedia content. After determining the target application playing the multimedia content on the terminal device using a similar method, a dynamic deep link is generated. A deep link refers to a link service provided by the linking website that allows the user to access content on the linked website without leaving the linking website's page. In this case, the address bar displays the URL of the linking website, not the URL of the linked website. In this application, the deep link indicates the multimedia content within the target application. Therefore, a URI scheme needs to be defined for each supported application to implement in-application navigation; thus, the target application must be an application that supports the defined URI scheme. At this point, a deep link generation request is initiated to the target application. The deep link generation request includes content parameters indicating multimedia content. Then, the content parameters indicating multimedia content are extracted from the deep link generation request. Based on the content parameters indicating multimedia content and the predefined URL structure, a deep link is constructed. Finally, the generated deep link is returned to the login terminal of the interactive object.

[0183] Therefore, after the login terminal of the interactive object obtains the deep link, it can display playback confirmation information for the multimedia content based on the deep link. Specifically, the login terminal displays a notification area containing playback confirmation information based on the deep link. At this point, triggering the playback confirmation information in the notification area on the video interaction page, which includes a deep link indicating the target application and the multimedia content, allows the deep link to be triggered, thereby launching the target application's program page and playing the multimedia content.

[0184] Based on this, if the interactive object initiates a trigger operation in response to the playback confirmation information, the program page of the target application will be invoked on the video interaction page of the interactive object's login terminal, and multimedia content will be played on the program page. Initiating a trigger operation in response to the playback confirmation information is determined by at least one of the following: the interactive object's gaze is focused on the playback confirmation information, and the duration of focusing on the playback confirmation information reaches a preset focusing duration; or, the interactive object selects voice in response to the playback confirmation information; or, the display duration of the playback confirmation information reaches a preset display duration; or, the interactive object interacts with the playback confirmation information. Therefore, the following describes how to determine the trigger operation for the playback confirmation information. In one optional embodiment, the method for determining the trigger operation for the playback confirmation information includes: performing gaze focus detection on the interactive object's gaze to obtain a gaze focus result; and triggering the trigger operation for the playback confirmation information when the gaze focus result indicates that the interactive object's gaze is focused on the playback confirmation information, and the duration of focusing on the playback confirmation information reaches a preset focusing duration.

[0185] Secondly, gaze focus detection is used to identify the focus area of ​​the object's gaze. Therefore, the gaze focus result is at least used to characterize the focus area of ​​the interactive object's gaze. Based on this, the preset focus duration can be 3 seconds, 5 seconds, or 7 seconds, etc., and the specific preset focus duration needs to be flexibly determined based on the actual scenario requirements. Specifically, if the interactive object is included in the video frame captured in real time by the login terminal, it means that the interactive object is in the video frame captured in real time by the login terminal. At this time, gaze focus detection can be further performed on the interactive object's gaze to obtain the gaze focus result, that is, to determine the focus area of ​​the interactive object's gaze. Then, it is necessary to determine whether the focus area of ​​the object's gaze is the playback confirmation information, that is, to determine whether the interactive object's gaze is focused on the playback confirmation information. If so, that is, the gaze focus result characterizes that the interactive object's gaze is focused on the playback confirmation information, then the time for the interactive object to focus on the playback confirmation information is started. When the time for focusing on the playback confirmation information reaches the preset focus duration, the trigger operation for the playback confirmation information is triggered, thereby waking up the program page of the target application and playing multimedia content on the program page.

[0186] In one alternative embodiment, determining the triggering operation for the playback confirmation information includes: triggering the triggering operation for the playback confirmation information when it is detected that the interactive object selects voice for the playback confirmation information.

[0187] The object selection voice is the voice issued by the interactive object to trigger the playback confirmation information. The object selection voice can be "agree to display" or "content can be displayed," etc., without limitation here. Specifically, the login terminal of the interactive object can perform voice detection in real time. After the login terminal of the interactive object displays the playback confirmation information indicating the target content, if the preset information of the object selection voice (such as "agree to display" and "content can be displayed" in the previous example) is detected, it is determined to trigger the trigger operation for the playback confirmation information, thereby waking up the program page of the target application and playing multimedia content on the program page.

[0188] In one optional embodiment, the method for determining the trigger operation for the playback confirmation information includes: recording the information display duration of the playback confirmation information; and triggering the trigger operation for the playback confirmation information when the information display duration reaches a preset display duration.

[0189] The preset long-press duration can be 3 seconds, 5 seconds, or 7 seconds, etc., and the specific preset display duration needs to be flexibly determined based on the actual scenario requirements. Specifically, after the login terminal of the interactive object displays the playback confirmation information indicating the target content, the display duration of the playback confirmation information begins to be recorded. When the display duration of the playback confirmation information reaches the preset display duration, a trigger operation for the playback confirmation information is triggered, thereby activating the program page of the target application and playing multimedia content on the program page. In this case, when the playback confirmation information indicating the target content is displayed on the login terminal of the interactive object, a notification related to the preset display duration can be displayed simultaneously. For example, taking a preset long-press duration of 3 seconds as an example, the notification related to the preset display duration is: "The target content will be displayed for you soon, countdown 3 seconds."

[0190] In one alternative embodiment, determining the triggering operation for the playback confirmation information includes: triggering the triggering operation for the playback confirmation information when an interaction object is detected to interact with the playback confirmation information.

[0191] Interacting with the playback confirmation information can include any of the following: long-pressing, clicking, or swiping. Specifically, when the login terminal of the interacting object detects that the object has interacted with the playback confirmation information, it can also trigger a corresponding action to open the target application's program page and play multimedia content on that page.

[0192] Furthermore, considering practical applications, multimedia content can also be displayed using a delivery device. Therefore, it is necessary to select a delivery device capable of delivering content for displaying the target content. This will be described below. In an optional embodiment, the content display method further includes: displaying device identifiers of candidate delivery devices capable of delivering multimedia content on a video interaction page; responding to a selection operation on a device identifier, selecting the candidate delivery device represented by the selected device identifier as the target delivery device; and delivering the multimedia content to the target delivery device.

[0193] The device identifier uniquely identifies the delivery device, which is a terminal capable of projecting content, such as a projector or television. The device identifier can be the device name or device number of the delivery device, etc., without limitation. Specifically, the delivery selection area displays the device identifiers of candidate delivery devices that can project multimedia content, as well as the communication connection status between the candidate delivery devices and the logged-in terminal of the interactive object.

[0194] Specifically, considering practical application scenarios, when a projection device exists near the login terminal of the interactive object, candidate projection devices suitable for displaying multimedia content are first selected. Then, the device identifiers of these candidate projection devices are displayed in the projection selection area on the video interaction page. In other words, the multimedia content is not directly displayed at this point; instead, the device identifiers of the candidate projection devices are shown. This allows the interactive object to choose whether to play the multimedia content through the projection device or on the video interaction page of the login terminal. For ease of understanding, as follows... Figure 26 As shown, the video interaction page of the login terminal of the interactive object displays the projection selection area 2602, and the device identifier 2604 of the candidate projection device that can project and display multimedia content is displayed in the projection selection area 2602.

[0195] Based on this, if an interactive object wishes to play multimedia content through a delivery device, it can select a device identifier from the displayed identifiers. This selection can be a long press, a click, or other similar operation. This allows the logged-in terminal to respond to the device identifier selection and use the candidate delivery device represented by the selected identifier as the target delivery device, thus delivering the multimedia content to the target delivery device for playback. For example, if device identifiers C1, C2, and C3 are displayed, where C1 represents delivery device D1, C2 represents delivery device D2, and C3 represents delivery device D3, and the interactive object selects device identifier C2, then device D2 becomes the target delivery device. The multimedia content is then delivered to and played through device D2.

[0196] The following details how to filter candidate delivery devices and display terminal identifiers: In one specific embodiment, on the video interaction page, the device identifiers of candidate delivery devices that can deliver multimedia content are displayed, including: searching for available delivery devices that are connected to the display terminal of the video interaction page; using the searched available delivery devices as candidate delivery devices for delivering multimedia content; and displaying the device identifiers of the candidate delivery devices on the video interaction page.

[0197] Specifically, after determining the multimedia content and the target application, the login terminal of the interactive object can detect available devices near it. The login terminal is the display terminal of the video interaction page, and available devices near it can be those with signal strength greater than a signal strength threshold. Therefore, to ensure normal communication, the display terminal also needs to detect the connection status between each available device and itself, and identify the available devices connected to it as available projection devices. These available projection devices can be one or more.

[0198] Based on this, the available delivery devices found are selected as candidate delivery devices for multimedia content, and their device identifiers are displayed on the video interaction page. The specific method is similar to the aforementioned embodiments and will not be repeated here. It is understood that if no available device is available near the display terminal of the video interaction page, or if the connection between the available device and the display terminal of the video interaction page is not established, the content will be displayed using the display method described in the aforementioned embodiments.

[0199] Secondly, considering the target application, if the interactive object wants to directly jump to the target application to play multimedia content, the interactive object can directly interact with the multimedia content, jump directly to the target application, and display the multimedia content through the target application. In this case, the video screen can be displayed as a floating window. Alternatively, the interactive object can interact with the target application, jumping to and displaying the target application, and the video screen can also be displayed as a floating window on the page where the multimedia content is played through the target application. The aforementioned interactive operations can include single-click operations, double-click operations, and swipe operations, etc., and are not limited here.

[0200] For ease of understanding, let's take the display of video footage in the login terminal of the first interactive object as an example, such as... Figure 27 As shown, Figure 27 The diagram in Figure (A) shows the login terminal of the first interactive object, displaying video screen 2702 and multimedia content 2704. If the login terminal of the first interactive object performs an interactive operation on the multimedia content 2704, the following can be displayed: Figure 27 The diagram in Figure (B) shows the display method of multimedia content 2704 displayed by the target application in the login terminal of the first interactive object, and the video screen 2702 is displayed on the multimedia content 2704 displayed and played by the target application in a floating window.

[0201] In this embodiment, the target content is mainly multimedia content. However, for target content of different object types, such as when the target content is physical content, the content screen corresponding to the target content will be different. However, the display method of displaying the target content in full-screen / floating window mode is similar to that described in the previous embodiment, and will not be repeated here.

[0202] In the above embodiments, when a terminal device playing multimedia content is displayed in the video frame, the multimedia content can be identified and the target application playing the multimedia content can be determined. Taking into account the multimedia content and the target application, the multimedia content is displayed on the login terminal of the interactive object, ensuring reliable content display in various real-world scenarios, thereby improving the reliability and flexibility of content display.

[0203] Based on the detailed description of the foregoing embodiments, the complete flow of the method for demonstrating the content in the embodiments of this application will be described below. In one embodiment, such as Figure 28 As shown, a method for displaying content is provided, which can be applied to... Figure 1Taking server 104 as an example, it can be understood that this method can also be applied to terminal 102, and also to a system including terminal 102 and server 104, and is implemented through the interaction between terminal 102 and server 104. In this embodiment, the method includes the following steps:

[0204] Step 2801: In a video interaction scenario involving at least two interactive objects, the video footage captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects; the interactive objects participating in the video interaction include at least a first interactive object and a second interactive object, the video interaction page is the display page of the first interactive object, and the video footage displays a terminal device playing multimedia content.

[0205] Step 2802: In response to an interest-triggered operation triggered for one of the video frames, the video frame triggered by the interest-triggered operation is taken as the target video frame, which is the video frame corresponding to the second interactive object.

[0206] Step 2803: If the language interaction content between the first interactive object and the second interactive object contains preset keywords, trigger an interest event.

[0207] Step 2804: When a preset gesture operation is triggered on the target video screen, an event of interest is triggered.

[0208] Step 2805: When an interactive operation is triggered on the target video screen, an event of interest is triggered.

[0209] Step 2806: Filter out the target video frames from the video stream data containing the target video images.

[0210] Step 2807: Perform environmental content recognition on the target video frame, and take the environmental content represented by the content recognition result as the target content; the target content is the multimedia content played by the terminal device.

[0211] Step 2808: Identify candidate applications that match the category of multimedia content; extract the content features of the multimedia content; perform similarity analysis between the content features and the corresponding interface features of each candidate application to obtain feature similarity; and select candidate applications whose feature similarity meets the matching requirements as target applications.

[0212] Step 2809: Locate the target application in existing applications.

[0213] Step 2810: If the target application is found, display playback confirmation information on the video interaction page, indicating that multimedia content can be played through the target application.

[0214] Step 2811: In response to the confirmation operation triggered by the playback confirmation information, the program page of the target application is invoked on the video interaction page; on the program page, multimedia content is played.

[0215] Step 2812: If the target application does not exist in existing applications, display a download prompt for the target application on the video interaction page.

[0216] Step 2813: In response to the program download confirmation operation triggered by the download prompt information, if the target application is successfully downloaded, display the program page of the target application; on the program page, play multimedia content.

[0217] Step 2814: On the video interaction page, display device identifiers of candidate delivery devices that can be used to display multimedia content; in response to the selection operation for the device identifier, select the candidate delivery device represented by the selected device identifier as the target delivery device; and deliver the multimedia content to the target delivery device.

[0218] It should be understood that the specific implementation methods of steps 2801 to 2814 are similar to those of the aforementioned embodiments, and will not be repeated here.

[0219] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0220] Based on the same inventive concept, this application also provides a content display device for implementing the above-mentioned content display method. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more content display device embodiments provided below can be found in the limitations of the content display method described above, and will not be repeated here.

[0221] In one embodiment, such as Figure 29 As shown, a content display device is provided, including: a video display module 2902, an event triggering module 2904, and a content display module 2906, wherein:

[0222] The video display module 2902 is used to display the real-time video footage captured by the login terminal of at least one other interactive object on the video interaction page of one of the interactive objects in a video interaction scenario involving at least two interactive objects.

[0223] The event triggering module 2904 is used to respond to an interest event triggered for environmental content in any video frame, and to take the environmental content matching the interest event as the target content.

[0224] The content display module 2906 is used to display the content screen corresponding to the target content on the video interaction page.

[0225] In one embodiment, the video frame displays a terminal device playing multimedia content, and the target content is the multimedia content played by the terminal device.

[0226] The content display module is specifically used to play multimedia content on the video interaction page.

[0227] In one embodiment, the terminal device plays multimedia content through a target application;

[0228] The content display module is specifically used to launch the target application's program page on the video interaction page; and to play multimedia content on the program page.

[0229] In one embodiment, the content display module is specifically used to search for the target application among existing applications; if the target application is found, the program page of the target application is invoked on the video interaction page.

[0230] In one embodiment, the content display device further includes an information display module;

[0231] The information display module is used to display download prompts for the target application on the video interaction page when the target application is not present in existing applications.

[0232] The content display module is specifically used to respond to the program download confirmation operation triggered by the download prompt information, and to display the program page of the target application when the target application is successfully downloaded.

[0233] In one embodiment, the content display module is specifically used to display playback confirmation information for playing multimedia content through the target application on the video interaction page; and to invoke the program page of the target application on the video interaction page in response to a confirmation operation triggered by the playback confirmation information.

[0234] In one embodiment, the content display module is specifically used to generate a deep link indicating the target application and the multimedia content, and to display playback confirmation information for the multimedia content based on the deep link. The playback confirmation information is used to confirm whether to play the multimedia content through the target application.

[0235] In one embodiment, the content display device further includes an application determination module;

[0236] The application identification module is used to identify candidate applications that match the category of multimedia content; extract the content features of the multimedia content; perform similarity analysis between the content features and the corresponding interface features of each candidate application to obtain feature similarity; and select candidate applications whose feature similarity meets the matching requirements as target applications.

[0237] In one embodiment, the content display device further includes a content delivery module;

[0238] The content delivery module is used to display device identifiers of candidate delivery devices that can deliver multimedia content on the video interaction page; in response to the selection operation of the device identifier, the candidate delivery device represented by the selected device identifier is selected as the target delivery device; and the multimedia content is delivered to the target delivery device.

[0239] In one embodiment, the content delivery module is specifically used to search for available delivery devices that are connected to the display terminal of the video interaction page; to use the searched available delivery devices as candidate delivery devices for delivering multimedia content; and to display the device identifier of the candidate delivery devices on the video interaction page.

[0240] In one embodiment, the event triggering module is specifically configured to respond to an event of interest triggered for environmental content in any video frame, determine the target video frame indicated by the event of interest, filter out the target video frame from the video stream data containing the target video frame, perform environmental content recognition on the target video frame, and take the environmental content represented by the content recognition result as the target content.

[0241] In one embodiment, the event triggering module is specifically used to, when there are at least three interactive objects participating in the video interaction, respond to an interest triggering operation triggered for one of the video frames, take the video frame triggered by the interest triggering operation as the target video frame, and trigger an interest event for the environmental content in the target video frame.

[0242] In one embodiment, the interactive objects involved in video interaction include at least a first interactive object and a second interactive object;

[0243] The event triggering module is specifically used to trigger the event of interest when the video interaction page is the display page of the first interactive object and the video frame of the second interactive object is the target video frame of the second interactive object; and when the voice interaction content between the first interactive object and the second interactive object contains preset keywords.

[0244] In one embodiment, the interactive objects involved in video interaction include at least a first interactive object and a second interactive object;

[0245] The event triggering module is specifically used when the video interaction page is the display page of the first interactive object, and the video frame of the triggered interest event is the target video frame corresponding to the second interactive object; and when a preset gesture operation is triggered on the target video frame, the interest event is triggered.

[0246] In one embodiment, the interactive objects involved in video interaction include at least a first interactive object and a second interactive object;

[0247] The event triggering module is specifically used when the video interaction page is the display page of the first interactive object, and the video frame whose interest event is triggered is the target video frame corresponding to the second interactive object; when an interactive operation is triggered on the target video frame, the interest event is triggered.

[0248] In one embodiment, the content display module is specifically used to display target content at a preset angle on a video interaction page.

[0249] In one embodiment, the content display module is specifically used to display a global image of the target content on a video interaction page.

[0250] In one embodiment, the content display module is specifically used to display a content promotion page for the target content on a video interaction page.

[0251] In one embodiment, the content display module is specifically used to display the content details page of the target content on the video interaction page.

[0252] In one embodiment, the content display module is specifically used to display the content screen corresponding to the target content in full-screen mode, and to display the video screen on the content screen corresponding to the target content in a floating window mode.

[0253] In one embodiment, the content display module is specifically used to display the video screen in full-screen mode and to display the content screen corresponding to the target content on the video screen in a floating window mode.

[0254] In one embodiment, the content display module is specifically used to display the video image through a first floating window and the content image corresponding to the target content through a second floating window.

[0255] In one embodiment, the computer device can be either a server or a terminal. This embodiment uses a computer device as a terminal as an example, and its internal structure diagram can be as follows: Figure 30 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a content display method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0256] Those skilled in the art will understand that Figure 30 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0257] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0258] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0259] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0260] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0261] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based content display logic devices, etc., and are not limited to these.

[0262] The technical features in the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0263] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for displaying content, characterized in that, The method includes: In a video interaction scenario involving at least two interactive objects, the video feed captured in real time by the login terminal of at least one of the other interactive objects is displayed on the video interaction page of one of the interactive objects. In response to an event of interest triggered by environmental content in any of the video frames, the environmental content matching the event of interest is taken as the target content; The video interaction page displays the content screen corresponding to the target content.

2. The method according to claim 1, characterized in that, The video frame displays a terminal device playing multimedia content, and the target content is the multimedia content played by the terminal device. The step of displaying the content screen corresponding to the target content on the video interaction page includes: The multimedia content is played on the video interaction page.

3. The method according to claim 2, characterized in that, The terminal device plays the multimedia content through the target application; Playing the multimedia content on the video interaction page includes: On the video interaction page, the program page of the target application is invoked; The multimedia content is played on the program page.

4. The method according to claim 3, characterized in that, The step of activating the program page of the target application on the video interaction page includes: Locate the target application in existing applications; If the target application is found, the program page of the target application is invoked on the video interaction page.

5. The method according to claim 3, characterized in that, The method further includes: If the target application is not present in existing applications, a download prompt for the target application will be displayed on the video interaction page. The step of activating the program page of the target application on the video interaction page includes: In response to the program download confirmation operation triggered by the download prompt information, if the target application is successfully downloaded, the program page of the target application is displayed.

6. The method according to claim 3, characterized in that, The step of activating the program page of the target application on the video interaction page includes: On the video interaction page, playback confirmation information is displayed for playing the multimedia content through the target application; In response to a confirmation operation triggered by the playback confirmation information, the program page of the target application is invoked on the video interaction page.

7. The method according to claim 6, characterized in that, The video interaction page displays playback confirmation information for playing the multimedia content through the target application, including: A deep link is generated to indicate that the target application is connected to the multimedia content, and playback confirmation information for the multimedia content is displayed based on the deep link. The playback confirmation information is used to confirm whether to play the multimedia content through the target application.

8. The method according to claim 3, characterized in that, The method further includes: Identify candidate applications that match the category of the multimedia content; Extract the content features of the multimedia content; The content features are compared with the interface features corresponding to each candidate application to obtain the feature similarity. Candidate applications whose feature similarity meets the matching requirements are selected as target applications.

9. The method according to claim 1, characterized in that, The method further includes: On the video interaction page, device identifiers of candidate delivery devices that can deliver and display the multimedia content are displayed; In response to the selection operation for the device identifier, the candidate delivery device represented by the selected device identifier is selected as the target delivery device; The multimedia content is delivered to the target delivery device.

10. The method according to claim 9, characterized in that, The step of displaying device identifiers for candidate devices that can display the multimedia content on the video interaction page includes: Search for available projection devices that are connected to the display terminal of the video interaction page; The available delivery devices found in the search will be used as candidate delivery devices for the multimedia content; The device identifier of the candidate deployment device is displayed on the video interaction page.

11. The method according to claim 1, characterized in that, The step of responding to an interest event triggered for any of the environmental content in the video frame, and using the environmental content matching the interest event as the target content, includes: In response to an event of interest triggered for environmental content in any of the video frames, the target video frame indicated by the event of interest is determined; Filter out the target video frames from the video stream data containing the target video images; Environmental content recognition is performed on the target video frame, and the environmental content represented by the content recognition result is taken as the target content.

12. The method according to any one of claims 1 to 11, characterized in that, When there are at least three interactive objects participating in the video interaction, the triggering method for triggering an interest event for any environmental content in the video frame includes: In response to an interest-triggered operation triggered for one of the video frames, the video frame triggered by the interest-triggered operation is taken as the target video frame; Trigger an interest event for the environmental content in the target video frame.

13. The method according to any one of claims 1 to 11, characterized in that, The interactive objects participating in the video interaction include at least a first interactive object and a second interactive object; When the video interaction page is the display page of the first interactive object, and the video frame whose interest event is triggered is the target video frame corresponding to the second interactive object, the triggering method of the interest event includes any one of the following methods: When the voice interaction content between the first interactive object and the second interactive object contains preset keywords, an interest event is triggered. When a preset gesture operation is triggered in response to the target video frame, an interest event is triggered; When an interactive operation is triggered in response to the target video frame, an event of interest is triggered.

14. The method according to any one of claims 1 to 11, characterized in that, Displaying the content screen corresponding to the target content on the video interaction page includes at least one of the following methods: The target content is displayed on the video interaction page at a preset angle; On the video interaction page, a global image of the target content is displayed; On the video interaction page, a content promotion page for the target content is displayed; The video interaction page displays the content details page of the target content.

15. The method according to any one of claims 1 to 11, characterized in that, The display of the content screen corresponding to the target content on the video interaction page includes any one of the following methods: The content screen corresponding to the target content is displayed in full-screen mode, and the video screen is displayed on the content screen corresponding to the target content in a floating window mode; The video screen is displayed in full screen mode, and the content screen corresponding to the target content is displayed on the video screen in a floating window mode; The video is displayed through a first floating window, and the content corresponding to the target content is displayed through a second floating window.

16. A content display device, characterized in that, The device includes: The video display module is used to display, in a video interaction scenario involving at least two interactive objects, the real-time video footage captured by the login terminal of at least one of the other interactive objects on the video interaction page of one of the interactive objects; The event triggering module is used to respond to an event of interest triggered by environmental content in any of the video frames, and to take the environmental content matching the event of interest as the target content. The content display module is used to display the content screen corresponding to the target content on the video interaction page.

17. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 15.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 15.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 15.

Images