Explanation role display method and device, storage medium and electronic equipment

Abstract

The invention discloses an explanation role display method and device, a storage medium and electronic equipment. The method can be applied to the map field. The method comprises the following steps: displaying a collected live-action image; an object explanation resource matched with the target object is obtained, the target object is associated with the real scene image, and the object explanation resource is used for explaining the target object; a target explanation role generated based on the object explanation resource is displayed in the real scene image, and the target explanation role is configured with a moving following mode matched with the object type of the target object; and in response to the change of the relative position between the image acquisition equipment and the target object, updating and displaying the target explanation role in the real scene image according to the movement following mode. The technical problem that the explanation effect of an existing explanation mode on the target object is poor is solved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method and device for displaying an explanation role, a storage medium and an electronic device. Background Art

[0002] Nowadays, with the development of tourism, users have an increasing demand for explanations of scenic spots. There are usually two ways to explain scenic spots. One is to scan the QR code related to the scenic spot and play the explanation audio related to the scenic spot; the other is to use a specific explanation device to play the corresponding explanation audio according to the user's current location.

[0003] The existing explanation methods can only explain the relevant scenic spots through audio, the explanation content is not rich enough, and there is a lack of interaction with users, so there is a technical problem of poor explanation effect.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for displaying an explanation role, a storage medium and an electronic device, so as to at least solve the technical problem that the existing explanation method has a poor explanation effect on the target object.

[0006] According to one aspect of an embodiment of the present invention, a method for displaying an explanation role is provided, comprising: displaying a captured real-scene image; acquiring an object explanation resource matching a target object, wherein the target object is associated with the real-scene image, and the object explanation resource is used to explain the target object; displaying a target explanation role generated based on the object explanation resource in the real-scene image, wherein the target explanation role is configured with a mobile following mode matching an object type of the target object; and updating and displaying the target explanation role in the real-scene image according to the mobile following mode in response to a change in the relative position between an image acquisition device and the target object.

[0007] According to another aspect of an embodiment of the present invention, a display device for an explanation role is also provided, including: a first display unit, used to display a captured real-scene image; an acquisition unit, used to acquire an object explanation resource matching a target object, wherein the target object is associated with the real-scene image, and the object explanation resource is used to explain the target object; a second display unit, used to display a target explanation role generated based on the object explanation resource in the real-scene image, wherein the target explanation role is configured with a mobile following mode matching the object type of the target object; and an updating unit, used to update and display the target explanation role in the real-scene image according to the mobile following mode in response to a change in the relative position between the image acquisition device and the target object.

[0008] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for displaying an explanation role when running.

[0009] According to another aspect of the embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method for displaying a narration role.

[0010] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method for displaying a narration role through the computer program.

[0011] In an embodiment of the present invention, a captured real-scene image is displayed; an object explanation resource matching a target object is acquired, wherein the target object is associated with the real-scene image, and the object explanation resource is used to explain the target object; a target explanation character generated based on the object explanation resource is displayed in the real-scene image, wherein the target explanation character is configured with a mobile following mode matching an object type of the target object; in response to a change in the relative position between an image acquisition device and the target object, the target explanation character is updated and displayed in the real-scene image according to the mobile following mode, thereby realizing explanation of the target object through AR (Augmented Reality) technology.

[0012] In the above-mentioned implementation manner, when the target object is collected in the current collection interface, object explanation resources matching the target object are obtained, and based on the explanation resources, an explanation role is superimposed and displayed in the real-life image, so that the object information related to the target object is explained by the explanation role, thereby improving the interactivity between the user and the explanation process and improving the explanation effect; in addition, in the above-mentioned implementation manner, the following method of the explanation role can also be determined according to the object type of the target object, so as to realize switching of different following methods of the explanation role in different explanation environments, and then in the process of the image acquisition device moving, further improve the interactivity and fun of the explanation process, solve the technical problem of poor explanation effect existing in the existing method, and improve the explanation effect of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0014] Figure 1 is a schematic diagram of a hardware environment of an optional method for displaying a role of an explanation according to an embodiment of the present invention;

[0015] Figure 2 is a flow chart of an optional method for displaying a narration role according to an embodiment of the present invention;

[0016] Figure 3 is a schematic diagram of an optional method for displaying a narration role according to an embodiment of the present invention;

[0017] Figure 4 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0018] Figure 5 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0019] Figure 6 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0020] Figure 7 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0021] Figure 8 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0022] Fig. 9 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0023] Fig.10 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0024] Fig.11 is a schematic diagram of another optional method for displaying a narration role according to an embodiment of the present invention;

[0025] Fig.12 is a schematic structural diagram of an optional display device for explaining roles according to an embodiment of the present invention;

[0026] Fig.13 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The following is an explanation of the technical terms involved in this application:

[0030] AR: AR (full name Augmented Reality) is a computer technology that combines virtual information with the real world to create an enhanced reality experience. AR can capture images and sounds of the real world by using cameras, sensors and other devices, and combine this information with computer-generated virtual content to enhance the user's sensory experience;

[0031] THREE.js: THREE.js is a JavaScript-based 3D graphics library that provides a series of tools and APIs for creating and displaying 3D graphics scenes. Based on the WebGL renderer, this library can provide developers with a simpler and more efficient way to develop 3D graphics;

[0032] Green screen video: Also known as Chroma Keying, it is a video special effects technology used to replace the background in a shot video with another background or image in post-production. This technology is widely used in movies, TV shows, games, and advertisements. It can achieve realistic visual effects without actually shooting exterior scenes or special effects scenes. Green and blue are commonly used background colors because they have the largest color difference in human vision and are easiest to identify and replace.

[0033] Streaming media: refers to a transmission method that provides multimedia content such as audio and video to users through the Internet or other computer networks. Unlike traditional downloading methods, streaming media can play content while it is being transmitted, so users can watch or listen while downloading. Streaming media technology has been widely used in music, movies, TV programs, live broadcasts and other fields.

[0034] According to one aspect of an embodiment of the present invention, a method for displaying an explanation role is provided. As an optional implementation, the method for displaying an explanation role can be applied to, but is not limited to, Figure 1 In the display system of the explanation role composed of the server 102 and the terminal device 104 shown in FIG. Figure 1 As shown, the server 102 is connected to the terminal device 104 through the network 110. The above network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The above terminal device may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a handheld computer, a MID (Mobile Internet Devices, mobile Internet devices), a PAD, a desktop computer, a smart TV, a vehicle-mounted device, etc. The above terminal device may be installed with a client with shooting and image acquisition functions, such as a picture recognition client, an instant messaging client, a travel sharing client, a browser client, a video playback client, etc. The above terminal device is also provided with a display, a processor and a memory. The display can be used to display the above program interface with shooting and image acquisition functions, and display the current collected real scene picture. The processor can be used to pre-process the real scene picture to be identified before identification, for example, the acquired real scene picture is converted into a format; the memory is used to store the real scene picture to be identified.

[0035] It is understandable that after the currently collected real-life picture is acquired in the above-mentioned terminal device 104, the target object included in the real-life picture is identified. Then, the terminal device 104 can send the object information of the identified target object to the server 102 through the network 110. When the server 102 receives the object information of the target object, it queries the object explanation resource matching the target object in the database. The terminal device 104 can receive the object explanation resource returned by the server 102 through the network 110, and based on the object explanation resource, the target explanation role for explaining the object information associated with the target object is superimposed on the real-life picture. The server 102 can be a single server, or a server cluster composed of multiple servers, or a cloud server. The above-mentioned server includes a database and a processing engine. Among them, the above-mentioned database may include a media resource library corresponding to multiple different target objects, which is used to match the corresponding object explanation resource to the target object.

[0036] According to one aspect of an embodiment of the present invention, the display system of the above-mentioned explanation role can also perform the following steps: the terminal device 104 executes step S102 to display the real-life image that has been collected; then step S104 is executed, and the terminal device 104 sends a resource acquisition request to the server 102 through the network 110; the server 102 executes step S106, and the server 102 sends an object explanation resource to the terminal device 104 through the network 110; then, the terminal device 104 executes steps S108 to S112 to obtain an object explanation resource that matches the target object, wherein the target object is associated with the real-life image, and the object explanation resource is used to explain the target object; the target explanation role generated based on the object explanation resource is displayed in the real-life image, wherein the target explanation role is configured with a mobile following mode that matches the object type of the target object; in response to a change in the relative position between the image acquisition device and the target object, the target explanation role is updated and displayed in the real-life image according to the mobile following mode.

[0037] It is understandable that, when the terminal device 104 is a device with sufficient computing processing capabilities and storage capabilities, the resource acquisition process related to the above steps S104 and S106 can also be performed inside the terminal device 104.

[0038] In an embodiment of the present invention, a captured real-scene image is displayed; an object explanation resource matching a target object is acquired, wherein the target object is associated with the real-scene image, and the object explanation resource is used to explain the target object; a target explanation character generated based on the object explanation resource is displayed in the real-scene image, wherein the target explanation character is configured with a mobile following mode matching an object type of the target object; in response to a change in the relative position between an image acquisition device and the target object, the target explanation character is updated and displayed in the real-scene image according to the mobile following mode, thereby realizing explanation of the target object through AR (Augmented Reality) technology.

[0039] In the above-mentioned implementation manner, when the target object is collected in the current collection interface, object explanation resources matching the target object are obtained, and based on the explanation resources, an explanation role is superimposed and displayed in the real-life image, so that the object information related to the target object is explained by the explanation role, thereby improving the interactivity between the user and the explanation process and improving the explanation effect; in addition, in the above-mentioned implementation manner, the following method of the explanation role can also be determined according to the object type of the target object, so as to realize switching of different following methods of the explanation role in different explanation environments, and then in the process of the image acquisition device moving, further improve the interactivity and fun of the explanation process, solve the technical problem of poor explanation effect existing in the existing method, and improve the explanation effect of the target object.

[0040] The above is only an example and is not limited in this embodiment.

[0041] As an optional implementation, Figure 2 As shown, the above-mentioned method for displaying the explanation role includes the following steps:

[0042] S202, displaying the acquired real scene image;

[0043] S204, obtaining an object explanation resource matching the target object, wherein the target object is associated with the real scene image, and the object explanation resource is used to explain the target object;

[0044] S206, displaying a target explanation character generated based on the object explanation resource in the real scene image, wherein the target explanation character is configured with a moving follow mode matching the object type of the target object;

[0045] S208, in response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile follow mode.

[0046] It should be noted that the real scene image in the above step S202 may be a real scene image obtained by an image acquisition device. The above image acquisition device may include but is not limited to a terminal device with an image acquisition function, for example, a smart phone device, a tablet device, or a head-mounted smart terminal, for example, a smart glasses device (equipped with a camera to capture scene images in real scenes), a VR head display device, etc. In this embodiment, the device type of the above image acquisition device is not limited.

[0047] Furthermore, a target application may be run in the above-mentioned image acquisition device, and in the application, when the image acquisition device is in an image acquisition state, a currently acquired real-scene image may be displayed in real time.

[0048] It should be noted that the real scene image may be an image displayed in the acquisition interface of the image acquisition device. The acquisition interface may be an application program interface in the target application that has the function of acquiring real-time images. Correspondingly, when the image acquisition interface is displayed, the camera component of the current terminal device is called to acquire the real-time scene image, and the acquired real-time scene image is displayed in the current interface.

[0049] As an optional method, in the case where the image acquisition device is a smart phone, the application with the function of acquiring real-time images may include but is not limited to an instant messaging application, an image recognition application, a travel sharing application, a video playback application, a browser application, etc. running on the smart phone. Taking the browser program as an example, in response to the selection operation of the shooting control on the right side of the browser search box, the image acquisition interface is displayed; in another method, in response to the input operation of "photo search" in the browser search box, the image acquisition interface is displayed; taking the instant messaging application as an example, in response to the selection operation of the object explanation applet in the instant messaging program, the image acquisition interface is displayed in the object explanation applet.

[0050] It is understandable that the above-mentioned real scene image can be a real-time scene obtained by shooting by the current user, or a picture obtained by real-time acquisition of the real scene when the current user turns on the explanation mode. For example, the user is currently in front of a landmark building in a scenic spot, and opens the mobile phone to capture a picture of the landmark building in the current environment through the mobile phone camera. In this embodiment, the above-mentioned target object can be the above-mentioned landmark building in the current environment.

[0051] The target object in the above step S204 is explained below. It is necessary or explained that the above target object can be an object in the current environment, the object can be a physical object to be explained, and can also be any physical object in the current environment. For example, when the current environment is a scenic spot with a larger range, the above target object can be a landmark scenic spot object in the above scenic spot, such as a sculpture in a park; it can also be any physical object in the above scenic spot; when the current environment is a relatively small indoor environment, the indoor environment includes multiple objects to be explained, and the above target object can be any object among the multiple objects to be explained. For example, when the current environment is a museum, the above target object can be an exhibit object in the museum exhibition hall. In this embodiment, the object type of the above target object is not limited.

[0052] Furthermore, in the above step S204, the object explanation resources matching the above target object may include but are not limited to media resources for explaining the target object, such as video resources, audio resources, etc.

[0053] In the case where the above-mentioned object explanation resource is an object explanation video, the above-mentioned video can be a video explaining the above-mentioned target object, and can also be a video explaining the object environment in which the above-mentioned target object is located. For example, in the case where the above-mentioned target object is a scenic spot object in a scenic spot environment, the above-mentioned object explanation video can be an explanation video that directly introduces the scenic spot object, and can also be an explanation video that introduces the scenic spot environment; in the case where the target object is an exhibit object in a museum exhibition hall environment, the above-mentioned object explanation video can be an explanation video that introduces the exhibit object. In an optional implementation, the above-mentioned object explanation video can also be a live video, and correspondingly, the host object in the above-mentioned live video can be a specific type of the above-mentioned reference explanation role. In this implementation, the specific explanation content included in the above-mentioned explanation video is not limited.

[0054] It should be noted that, in this embodiment, the object explanation video may include an explanation character for explanation, and the explanation character may be a real person character or a virtual character. In this embodiment, the role type of the explanation character is not limited.

[0055] Corresponding to different object types, the object information associated with the target object may refer to different contents. For example, when the target object is a physical object in a scenic area environment, the object information may be the introduction information of the scenic area environment object; when the target object is an exhibit object in a pavilion environment, the object information may be the introduction information of the exhibit object. In this embodiment, the content referred to by the object information is not limited.

[0056] In the above step S206, the above target explanation role can be an explanation role generated based on the above object explanation video. The target explanation role can be a real person character, a virtual character, or an animation character. In this embodiment, the role type of the above target explanation role is not limited.

[0057] In the case where the object explanation resource is an object explanation video, a method for generating the target explanation role based on the object explanation video may be to generate the target explanation role based on a reference explanation role included in the object explanation video, wherein the reference explanation role is an object role in the object explanation video used to explain the target object.

[0058] The method of generating the target explanation character based on the reference explanation character in the object explanation video may include but is not limited to obtaining the target explanation character by cutting out the reference explanation character in the object explanation video; obtaining a second explanation element by cutting out the reference explanation character in the object explanation video and adding modification elements; generating a virtual character based on the reference explanation character, wherein the similarity between the character actions and character expressions of the virtual character and those of the reference explanation character is greater than or equal to a similarity threshold.

[0059] It can be understood that after the target explanation role is superimposed and displayed in the real scene image, an explanation interface in which the target explanation role explains the object information associated with the target object can be displayed in the image acquisition interface.

[0060] Furthermore, in the above step S208, after the above target explanation character is superimposed and displayed in the above real-life image, when the relative position between the image acquisition device and the target object changes, the target explanation character's movement following mode can be determined according to the object type of the target object, and the target explanation character can be updated and displayed in the real-life image.

[0061] In the above implementation, the mobile following mode of the above target explanation role corresponds to the object type of the target object. The corresponding manner may include but is not limited to, when the target object is a physical object in a scenic environment, the target explanation role is configured as a first mobile following mode; when the target object is a physical object in a pavilion environment, the target explanation role is configured as a second mobile following mode; in another corresponding manner, when the target object is a physical object in an open environment (e.g., outdoors), the target explanation role is configured as a first mobile following mode; when the target object is a physical object in a closed environment (e.g., indoors), the target explanation role is configured as a second mobile following mode, wherein the target explanation role in the first mobile following mode can be displayed at a fixed position in the real scene image according to a fixed size; and the target explanation role in the second mobile following mode can be displayed in real time at an active position in the real scene image according to a changing size.

[0062] It should be noted that in various implementations of the present application, the acquisition of relevant data of the user requires the permission of the user, and the acquisition method and the processing method of the relevant data comply with the provisions of relevant normative requirements.

[0063] Four specific implementation scenarios to which the above implementation method can be applied are described below.

[0064] In the first implementation scenario, the above implementation method can be applied in a scenic spot, that is, in the scenic spot, the user can open the mobile phone and trigger the explanation program through a target applet in an instant messaging application, that is, start the image acquisition function of the mobile phone and collect the real-life image in the current scenic environment in real time. In an optional manner, when the target scenic spot to be explained appears in the real-life image, the acquisition of the object explanation video is triggered; in another optional manner, when it is detected that the current location of the mobile phone is a location in the target area, the acquisition of the object explanation video is triggered.

[0065] When an explanation video of the target scenic spot is obtained, a target explanation character is generated for superimposition display in the real scene image according to the reference explanation character in the explanation video, so as to explain the target scenic spot in the real scene image through the second explanation image. It is understandable that in the scenic spot, the user can hold the mobile phone while playing, although the position of the mobile phone changes, it is in the area near the target scenic spot. Therefore, even if the mobile phone used by the user is not aimed at the target scenic spot, the target explanation character can still be displayed in the real scene image;

[0066] In the second implementation scenario, the above implementation method can be applied in an exhibition hall, that is, in the exhibition hall, the user can open the mobile phone and trigger the explanation program through a target applet in an instant messaging application, that is, start the image acquisition function of the mobile phone to collect the real-life image in the current exhibition hall environment in real time. When the target exhibit object to be explained appears in the real-life image, the acquisition of the object explanation video corresponding to the exhibit object is triggered; then, a target explanation role is generated for superimposed display in the real-life image based on the reference explanation role in the explanation video, so as to explain the target exhibit in the real-life image through the target explanation role. In this implementation, when the user uses the mobile phone to aim at the reference exhibit object, that is, when the above-mentioned target exhibit object leaves the current real-life image, the explanation video corresponding to the reference exhibit object can continue to be acquired, so as to display the third explanation role corresponding to the reference exhibit object in the current real-life image, and explain the reference exhibit object through the third explanation role;

[0067] It should be noted that the above-mentioned embodiments of the present application can also be applied to non-scenic spot explanation scenarios, for example, it can be a scenario of explaining the usage of equipment objects, or it can be a scenario of explaining and displaying goods in a shopping mall.

[0068] Specifically, in the third implementation scenario, the above implementation method can be applied in the scenario of equipment usage introduction. For example, in an indoor environment where multiple mechanical equipment are placed, the user can turn on the mobile phone to trigger the equipment explanation program. When the target equipment object to be explained appears in the real-life image, the acquisition of the equipment explanation video corresponding to the equipment object is triggered; then, a target explanation role is generated for superimposed display in the real-life image based on the reference explanation role in the equipment explanation video, so as to explain the usage of the target equipment in the real-life image through the target explanation role. In this implementation, when the user uses the mobile phone to aim at the second equipment object, that is, when the above-mentioned target equipment object leaves the current real-life image, the explanation video corresponding to the second equipment object can continue to be acquired, so as to display the third explanation role corresponding to the second equipment object in the current real-life image, and explain the usage of the second equipment object through the third explanation role.

[0069] In the fourth implementation scenario, the above implementation method can be applied in a commodity promotion scenario. For example, in a shopping mall environment where multiple jewelry products are placed, the user can open the mobile phone to trigger the commodity introduction program. When the target jewelry object to be introduced appears in the real scene image, the acquisition of the recommended video corresponding to the jewelry object is triggered; then, the target explanation role that is superimposed and displayed in the real scene image is generated according to the reference explanation role in the recommended video, so that the characteristics, advantages, materials and other commodity attributes of the current jewelry are explained in the real scene image through the target explanation role. In addition, the target explanation role can also be displayed at the location where the jewelry object is located, so as to achieve the display effect of recommending the jewelry object while the target explanation role wears the jewelry object. In this implementation, when the user uses the mobile phone to aim at the second jewelry object, that is, when the above-mentioned target jewelry object leaves the current real scene image, the recommended video corresponding to the second jewelry object can continue to be acquired, so as to display the third explanation role corresponding to the second jewelry object in the current real scene image, and explain the commodity attributes of the second jewelry object through the third explanation role.

[0070] Furthermore, in the above-mentioned commodity promotion scene, the obtained recommended video can also be a live video. Specifically, when the target jewelry object to be introduced appears in the real-scene image, the acquisition of the live recommended video corresponding to the jewelry object is triggered; then, a target explanation role is generated for superimposed display in the real-scene image according to the host object in the live recommended video, so that the characteristics, advantages, materials and other commodity attributes of the current jewelry can be explained in the real-scene image through the target explanation role corresponding to the host object. In addition, the target explanation role can also be displayed at the location where the jewelry object is located, so as to achieve the display effect of the target explanation role wearing the jewelry object and recommending the jewelry object. In this embodiment, when the user uses the mobile phone to aim at the second jewelry object, that is, when the above-mentioned target jewelry object leaves the current real-scene image, the live recommended video corresponding to the second jewelry object can be continuously obtained, so as to display the third explanation role corresponding to the second jewelry object in the current real-scene image, and explain the commodity attributes of the second jewelry object through the third explanation role in the new live recommended video. Optionally, it can also be displayed in real time in the current real-scene image based on the relevant video elements in the above-mentioned live recommended video, for example, the live barrage information, live gift information, live preferential activity information, etc. can be displayed in real time.

[0071] The following combination Figure 3 A specific application scenario and implementation process are described. In this embodiment, a new method of real-time AR video explanation of scenic spots is provided. When a user arrives within a certain range of a specific scenic spot in a scenic spot, he can pick up his mobile phone to see a real-person video explanation of the scenic spot, providing a virtual "tour guide" for the user.

[0072] Compared with the self-service voice broadcast and explanation services currently provided by most scenic spots, video explanations can add visual experience to the auditory experience. In addition to the real-person explanation, relevant information of the scenic spot can also be displayed in the scene through video. The content displayed can be richer and more varied, and the amount of information and interactivity provided are stronger, bringing a better user experience.

[0073] When the scenic spot being explained is a larger area, the video model will be set to follow the rotation and movement of the camera. When the user needs to look around at the scenic spot, the video will also follow the camera and be displayed on the screen in real time.

[0074] Specifically, Figure 3 In (a), when the user holds the image acquisition device 301 and aims at the target object 302 in the target scenic area, the real scene image corresponding to the target object 302 is displayed on the current acquisition interface of the image acquisition device 301;

[0075] Then, if Figure 3 As shown in FIG. 3 (b), the image acquisition device 301 obtains an object explanation video 304 that matches the target object 302 , and the object explanation video 304 includes a reference explanation character 305 that explains the target object 302 .

[0076] Then, a target explanation character 306 is generated based on the reference explanation character 305 , and the target explanation character 306 is superimposed and displayed on the real-scene image, thereby achieving real-time explanation of the target object 302 according to the target explanation character 306 in the image acquisition interface.

[0077] It should be noted that, according to the object type of the target object 302 (the scenic spot object in the scenic area), the following mode of the target explanation character 306 can be determined as the first mobile following mode, that is, the target explanation character 306 can be displayed in the real scene image even when the position of the image acquisition device changes.

[0078] Then, if Figure 3 As shown in FIG. 5( c ), in response to the user moving the image acquisition device 301, when the target object 302 leaves the real scene image currently acquired by the image acquisition device 301, the target explanation character 306 is kept displayed in the real scene image, that is, the target explanation character 306 is kept explaining the target object 302 (in Figure 3 (c) is not shown in the figure).

[0079] In an embodiment of the present invention, a captured real-scene image is displayed; an object explanation resource matching a target object is acquired, wherein the target object is associated with the real-scene image, and the object explanation resource is used to explain the target object; a target explanation character generated based on the object explanation resource is displayed in the real-scene image, wherein the target explanation character is configured with a mobile following mode matching an object type of the target object; in response to a change in the relative position between an image acquisition device and the target object, the target explanation character is updated and displayed in the real-scene image according to the mobile following mode, thereby realizing explanation of the target object through AR (Augmented Reality) technology.

[0080] In the above-mentioned implementation manner, when the target object is collected in the current collection interface, object explanation resources matching the target object are obtained, and based on the explanation resources, an explanation role is superimposed and displayed in the real-life image, so that the object information related to the target object is explained by the explanation role, thereby improving the interactivity between the user and the explanation process and improving the explanation effect; in addition, in the above-mentioned implementation manner, the following method of the explanation role can also be determined according to the object type of the target object, so as to realize switching of different following methods of the explanation role in different explanation environments, and then in the process of the image acquisition device moving, further improve the interactivity and fun of the explanation process, solve the technical problem of poor explanation effect existing in the existing method, and improve the explanation effect of the target object.

[0081] In an optional implementation, in response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile following mode includes:

[0082] S1, when the target explanation character is in the first moving follow mode, taking the first display position of the target explanation character before the relative position between the image acquisition device and the target object changes as the updated second display position; and displaying the updated target explanation character at the second display position;

[0083] S2, when the target explanation character is in the second mobile following mode, the updated target explanation character is displayed at a third image position associated with the target object in the real scene image, wherein the distance between the third image position and the image position where the target object is located is less than or equal to an intervention target distance threshold.

[0084] It can be understood that, in this embodiment, when the target explanation character is in the first moving follow mode, it is determined that the target explanation character is kept displayed at a fixed position in the real scene image, that is, the display position of the target explanation character may not change with the change of the relative position between the image acquisition device and the target object;

[0085] For example, when the user is in a scenic spot and the real-scene image captured by the image acquisition device is a real-scene image, even if the user moves the image acquisition device, that is, the display position of the target object in the real-scene image changes accordingly according to the movement, the target explanation character can still be displayed at a fixed position in the real-scene image.

[0086] In an optional implementation, the above-mentioned displaying the updated target explanation character at the second display position includes: displaying the target explanation character at the second display position according to a fixed image display area.

[0087] like Figure 4 In the figure (a), when the real scene image captured by the current image acquisition device includes a target object 401, and the target object 401 is a scenic spot object in a scenic area, a target explanation character 402 explaining the target object 401 can be displayed in the current real scene image; then, when the user moves the holding image acquisition device, that is, moves away from the target object, the real scene image is displayed as follows Figure 4 As shown in FIG. (b), in response to the change in the position of the image acquisition device, the real-life image currently being acquired is displayed in real time, that is, the target object 403 at the right edge of the acquisition interface is displayed in the current real-life image. In this embodiment, even if the relative position between the image acquisition device and the target object changes, the target explanation character 402 is still displayed at the lower left of the real-life image displayed in the acquisition interface. It can be understood that Figure 4 The position of the target explanation character 402 in FIG. (b) can be Figure 4 The image position of the target explanation character 402 is the same as in FIG.

[0088] When the target explanation character is in the second moving follow mode, the target explanation character may be determined to be displayed at a position associated with the target object in the image. For example, the associated third image position may be the edge of the image area corresponding to the target object, or above, below or at other positions of the image area. In this embodiment, the specific method for determining the third image position is not limited.

[0089] In an optional embodiment, the above-mentioned third image position associated with the target object in the real-life image displays an updated target explanation character, including: determining a current display area corresponding to the current relative distance between the image acquisition device and the target object; and displaying the target explanation character at the third image position according to the current display area, wherein the current display area decreases as the current relative distance increases.

[0090] like Figure 5 In the figure (a), when the real scene image captured by the current image acquisition device includes a target object 501, and the target object 501 is an exhibit object in the exhibition hall, a target explanation character 502 explaining the target object 501 can be displayed in the current real scene image, and the target explanation character 502 can be displayed at the upper edge of the target object 501; then, when the user moves the holding image acquisition device, that is, moves away from the exhibit object, the real scene image is displayed as follows Figure 5 As shown in FIG. (b), in response to the change in the position of the image acquisition device, the real-scene image currently acquired is displayed in real time, that is, the target object 503 at the right edge of the acquisition interface is displayed in the current real-scene image. In this embodiment, according to the change in the relative position between the image acquisition device and the target object, the target explanation character 504 is synchronously updated and displayed, that is, the target explanation character 504 is kept displayed at the upper edge of the updated target object 503. At the same time, the image size of the target explanation character 504 is synchronously adjusted according to the changed relative distance.

[0091] Through the above-mentioned implementation mode of the present application, when the target explanation character is in the first mobile following mode, the first display position of the target explanation character before the relative position between the image acquisition device and the target object changes is used as the updated second display position; the updated target explanation character is displayed at the second display position; when the target explanation character is in the second mobile following mode, the updated target explanation character is displayed at the third image position associated with the target object in the real-life image, and then according to the object type in the actual acquisition environment, the corresponding explanation requirements are determined, and then different mobile following modes are configured for the explanation character, so as to dynamically adjust the display position of the explanation character in different explanation environments, thereby improving the effect of displaying the explanation character.

[0092] In an optional implementation, before displaying the updated target explanation character at the second display position, the method further includes:

[0093] S1, obtaining a first reference image region that does not include a target object in a real scene image;

[0094] S2, superimposing and displaying the target explanation character at a first display position determined from the first reference image area.

[0095] It can be understood that, in this embodiment, before the target explanation character is displayed in the real scene image, the display position for displaying the target explanation character can be determined from the first reference image area that does not include the target object, thereby preventing the target explanation character from blocking the target object to be explained.

[0096] Optionally, before the target explanation character is superimposed and displayed at the first display position, the display area for displaying the target explanation character can be determined based on the area of ​​the first reference image area that does not include the target object. For example, a target display area that is smaller than the area of ​​the first reference image area can be determined as the display area of ​​the target explanation character, thereby avoiding the target explanation character from obstructing the target object to be explained.

[0097] It can be understood that in this embodiment, the initial display position and initial display area of ​​the target explanation character in the first mobile following mode in the real-scene image can be determined by the above-mentioned embodiment, and when the relative position between the image acquisition device and the target object changes, the initial display position and initial display area are still maintained to display the target explanation character in the real-scene image.

[0098] Through the above-mentioned implementation mode of the present application, a first reference image area that does not include the target object in the real-scene image is obtained; the target explanation character is superimposed and displayed at the first display position determined from the first reference image area, so as to quickly determine the initial display position of the target explanation character, thereby avoiding the target explanation character from blocking the target object to be explained, and improving the display effect of displaying the target explanation character.

[0099] In an optional implementation, before the third image position associated with the target object in the real scene image displays the updated target explanation character, the method further includes:

[0100] S1, obtaining a second reference image region in the real scene image whose image distance to the target object is less than or equal to the target distance threshold;

[0101] S2, superimposing and displaying the target explanation character at a fourth display position determined from the second reference image area.

[0102] It can be understood that in this embodiment, when the target explanation character is in the second moving following mode, the initial display position of the target explanation character can be determined by determining a fourth display position in the display area associated with the target object.

[0103] In an optional manner, the fourth display position may be an image position that generates an interactive effect with the target object. For example, the image position may be an image position at the edge of the target object, an image position above the target object, or an image position to the side of the target object; in a specific scenario, when the target object is a hand sculpture, the fourth image position may be the position of the palm of the hand sculpture, thereby achieving a display effect in which the explaining character explains the hand sculpture at the palm position of the hand sculpture.

[0104] Furthermore, in this embodiment, the initial display position of the target explanation character in the first mobile following mode in the real-scene image can be determined by the above-mentioned embodiment, and when the relative position between the image acquisition device and the target object changes, the display position of the target explanation character is dynamically adjusted according to the relative position, so as to achieve that the change in the relative position of the target explanation character and the target object in the real-scene image is less than the target threshold.

[0105] Through the above-mentioned implementation mode of the present application, a second reference image area whose image distance with the target object in the real scene image is less than or equal to the target distance threshold is obtained; the target explanation character is superimposed and displayed at the fourth display position determined from the second reference image area, and then the fourth display position that can generate a sense of interaction with the target object is selected to display the target explanation character, and at the same time, when the relative position between the image acquisition device and the target object changes, the display position of the target explanation character is dynamically adjusted, so as to maintain the sense of interaction between the target explanation character and the target object, and improve the display effect of displaying the target explanation character

[0106] In an optional implementation manner, before displaying the target explanation character generated based on the object explanation resource in the real scene image, the method further includes:

[0107] S1, obtain the annotation information of the target object;

[0108] S2-1, when the annotation information indicates that the target object is a scenic spot object, determining to configure a first moving following mode for the target explanation character;

[0109] S2-2: When the annotation information indicates that the target object is an exhibit object, determine to configure a second movement following mode for the target explanation role.

[0110] In an optional embodiment, after displaying the target explanation character generated based on the object explanation resource in the real-scene image, the method further includes: when the target explanation character is in the first mobile following mode, in response to a change in the device position of the image acquisition device, when the target explanation character does not include the target object in the real-scene image, keeping displaying the target explanation character in the real-scene image.

[0111] In an optional implementation manner, after displaying the target explanation role generated based on the object explanation resource in the real scene image, the method further includes:

[0112] S1, when the target explanation character is in the second moving follow mode, in response to a change in the device position of the image acquisition device, when the real scene image does not include the target object, canceling the display of the target explanation character in the real scene image;

[0113] S2, when the real scene image displays the reference object, obtaining a reference object explanation resource matching the reference object, wherein the reference object explanation resource is used to explain the reference object;

[0114] S3, displaying a first explanation character generated based on the reference object explanation resource in the real scene image, wherein the first explanation character is configured with a second moving follow mode.

[0115] It is understandable that in this embodiment, in the case of the target object captured, the object type of the target object can be obtained, and then in the case of the target object being a scenic spot object, the first moving following mode is configured for the target explanation role; and in the case of the target object being an exhibit object, the second moving following mode is configured for the target explanation role. Thus, when the object to be explained is a large-scale object to be explained, the explanation role is still displayed in the real-time image according to the movement of the image acquisition device; when the object to be explained is a small-scale specific object to be explained, the object to be explained and the explanation role corresponding to the object to be explained are dynamically displayed and adjusted according to the movement of the image acquisition device.

[0116] The following combination Figure 6 A specific implementation method is described.

[0117] In the case where the above-mentioned target object is an exhibit object, such as Figure 6 As shown in FIG. 5( a ), the target object 601 displayed in the current real-time image is a “crown” exhibit. In the process of displaying the target explanation character 602 explaining the target object 601 in the real-time image, the display position of the target explanation character 602 can be determined as the position below the target object 601, thereby forming a display effect that the target explanation character 602 is wearing the “crown” exhibit target object 601 and explaining the target object 601.

[0118] Then, if Figure 6As shown in FIG. (b), in response to the change in the relative position between the image acquisition device and the target object, that is, the current user points the image acquisition device at another exhibit object, the target object 601 is canceled from the current real scene image, and another exhibit object target object 603 is displayed in the real scene image. Then, the target explanation character 602 corresponding to the target object 601 is canceled from the current real scene image, and the target explanation character 604 corresponding to the target object 603 is displayed. Figure 6 As shown in Figure (b), the target object 603 is a necklace exhibit object, and the target explanation character 604 is displayed at the position where the target object 603 is located, so as to display a screen in which the target explanation character 604 is wearing the necklace exhibit, i.e., the target object 603, and explaining the target object 603.

[0119] In another optional implementation, the explanation character may be set as a cartoon image displayed at a fixed position of the target object, and by setting the position, the cartoon character may appear to be standing near the target object and performing a series of actions.

[0120] It is understandable that in this embodiment, the above-mentioned cartoon image can be the original cartoon image in the original explanation video, and there is no need to load the model or render the model animation separately, so the production cost of displaying the animation effect in the real scene image is lower.

[0121] In an optional implementation manner, before displaying the target explanation character generated based on the object explanation resource in the real scene image, the method further includes one of the following:

[0122] Method 1: when the object explanation resource is an object explanation video including a reference explanation role, a reference explanation role obtained by clipping from the object explanation video is used as a target explanation role, wherein the reference explanation role is used to explain the target object in the object explanation video;

[0123] Method 2: when the object explanation resource is an object explanation video including a reference explanation role, obtaining the role action features of the reference explanation role; and generating the target explanation role according to the role action features;

[0124] Method three: when the object explanation resource is the object explanation audio, obtain the target explanation image that matches the object explanation audio, and generate the target explanation role according to the target explanation image.

[0125] Method 4: When the object explanation resource is an object explanation video including a reference explanation role, a decorative element is added to the role image where the reference explanation role is located; and the reference explanation role to which the decorative element is added is used as the target explanation role.

[0126] In this embodiment, four methods of generating a target explanation role according to a reference explanation role are provided.

[0127] In the first method, the reference explanation character obtained by cutting out the object explanation video can be directly used as the target explanation character;

[0128] In the second method mentioned above, a virtual target explanation character generated based on the reference explanation character can be driven in the real-life image according to the character action features of the reference explanation character, thereby achieving the technical effect of generating a cartoon character based on the real character and explaining through the cartoon character.

[0129] The following combination Figure 7 The above implementation is described in detail. Figure 7 As shown, the real-life image 701 currently captured by the image acquisition device includes a target object 702. When the target object 702 is detected, an object explanation video 703 corresponding to the target object 702 is obtained, wherein a reference explanation character 704 is included. Further, a cartoon-like target explanation character 705 is generated based on the reference explanation character, and the actions and expressions of the cartoon-like target explanation character 705 can be driven based on the role action features of the reference explanation character 704. Then, the target explanation character 705 is superimposed and displayed in the real-life image 701 to obtain a real-life image displayed in real time in the acquisition interface.

[0130] In the third method, when the target explanation resource is an audio resource, a corresponding explanation role can be obtained from a plurality of preset candidate explanation roles based on the audio resource. For example, when the frequency of the audio corresponding to the audio resource is lower than the target threshold (indicating the voice of a male role), a male explanation role is obtained from a plurality of candidate explanation roles; when the frequency of the audio corresponding to the audio resource is higher than the target threshold (indicating the voice of a female role), a female explanation role is obtained from a plurality of candidate explanation roles.

[0131] In the fourth method, after obtaining the reference explanation character through cutout, decorative elements can be added to the reference explanation character to obtain the target explanation character. The decorative elements in this embodiment may include but are not limited to headdress elements, jewelry elements, tie elements, clothing elements, etc. used to decorate human characters, and may also be color elements (changing the display color of the character), collar elements, etc. used to decorate animal characters. In this embodiment, the specific types of the above decorative elements are not limited;

[0132] Through the above-mentioned implementation mode of the present application, when the reference explanation character obtained by cutting out the object explanation video is used as the target explanation character, the consistency of the explanation action of the explanation character and the explanation content can be ensured; when the role action features of the reference explanation character are obtained; the target explanation character is generated according to the role action features, or decorative elements are added to the role image where the reference explanation character is located; and the reference explanation character with added decorative elements is used as the target explanation character, the display effect of the explanation character can be improved.

[0133] In an optional implementation, the acquiring of the object explanation resource matching the target object includes at least one of the following:

[0134] Method 1: when the object explanation resource is an object explanation video including a reference explanation role, obtaining an object explanation video matching the target object, wherein the object explanation video is a pre-stored explanation video;

[0135] Method 2: when the object explanation resource is an object explanation video including a reference explanation role, obtain a live broadcast link that matches the target object; obtain a live broadcast explanation video that matches the target object based on the live broadcast link, and generate a target explanation role based on the live broadcast explanation role in the live broadcast explanation video.

[0136] It can be understood that, in this embodiment, the acquired object explanation video can be a pre-stored video of a fixed length, or a live video in a live stream pulled in real time.

[0137] For example, in a commodity promotion scene, a live video can be obtained to explain the product to be promoted. Specifically, when the target jewelry object to be introduced appears in the real-life image, the acquisition of the live recommendation video corresponding to the jewelry object is triggered; then, a target explanation role is generated for superimposed display in the real-life image according to the host object in the live recommendation video, so that the characteristics, advantages, materials and other commodity attributes of the current jewelry can be explained in the real-life image through the target explanation role corresponding to the host object. In addition, the target explanation role can also be displayed at the location where the jewelry object is located, so as to achieve the display effect of the target explanation role wearing the jewelry object and recommending the jewelry object. In this embodiment, when the user uses the mobile phone to aim at the second jewelry object, that is, when the above-mentioned target jewelry object leaves the current real-life image, the live recommendation video corresponding to the second jewelry object can continue to be obtained, so as to display the third explanation role corresponding to the second jewelry object in the current real-life image, and explain the commodity attributes of the second jewelry object through the third explanation role in the new live recommendation video. Optionally, it can also be displayed in real time in the current real-life image based on the relevant video elements in the above-mentioned live recommendation video, for example, the live barrage information, live gift information, live discount event information, etc. are displayed in real time.

[0138] In an optional implementation, when the target explanation role generated based on the object explanation resource is superimposed and displayed in the real scene image, at least one of the following is also included:

[0139] Method 1: Overlaying and displaying the object text corresponding to the object explanation resource in the real scene image;

[0140] In an optional implementation, after the object text corresponding to the object explanation resource is superimposed and displayed in the real scene image, the method further includes: in response to a first touch operation on the object text, copying the object text and displaying corresponding copy prompt information;

[0141] Method 2: Overlaying and displaying media elements associated with object explanation resources in the real scene image;

[0142] In an optional implementation, after the media element associated with the object explanation video is superimposed and displayed in the real scene image, the method further includes: in response to a second touch operation on the media element, adjusting the display position of the media element in the real scene image according to the second touch operation;

[0143] Method three: Overlay the media jump link associated with the object explanation resource in the real-scene image.

[0144] In an optional embodiment, after superimposing and displaying a media jump link associated with an object explanation video in a real-life image, the method further includes: in response to a third touch operation on the media jump link, jumping and displaying a target web page associated with the target object, wherein the target web page displays media resources associated with the target object.

[0145] It is understandable that, in this embodiment, a variety of ways of further displaying the real scene image while displaying the target explanation character are provided.

[0146] In the first method mentioned above, the target explanation character is displayed in the real-scene image, and the object text corresponding to the object information can also be displayed synchronously, for example, it can be the subtitle text matching the object information in the object explanation video; optionally, in the case where the above-mentioned object explanation video is a live video, the above-mentioned object text can also be the barrage text displayed in the live video; in this embodiment, the text type of the above-mentioned object text is not limited.

[0147] For example Figure 8 As shown in FIG. 8( a ), in the process of displaying the explanation character explaining the target object, the subtitle text in the original explanation video, that is, the object text 801, can be displayed in the real scene image; Figure 8As shown in FIG. 8( b ), when the original explanation video is a live video, the barrage text in the original live video, i.e., the object text 802 , can also be displayed in the real-scene image while displaying the explanation character explaining the target object.

[0148] In addition, when the object text is displayed in the real scene image, the object text can be automatically copied in response to a first touch operation on the object text, so that the user can share the object text or perform further search operations based on the object text.

[0149] In the second method mentioned above, the target explanation character is displayed in the real-scene image, and the media elements associated with the object explanation video can also be displayed synchronously, for example, it can be animation elements, picture elements, etc. displayed in the object explanation video during the process of explaining the target object, and then the media elements used to assist the explanation in the object explanation video are synchronously displayed in the real-scene image; optionally, in the case that the above-mentioned object explanation video is a live video, the above-mentioned media elements can also be gift media elements, expression media elements, etc. displayed in the live video; in this embodiment, the types of the above-mentioned media elements are not limited.

[0150] In addition, when the above-mentioned media elements are displayed in the real-scene image, in response to the second touch operation on the media elements, further editing operations can be performed on the media elements displayed in the real-scene image. For example, when the displayed media elements block the target object, the display position of the media elements is adjusted to enhance the video display effect of explaining the target object in the real-scene image.

[0151] In the third method, the target explanation character is displayed in the real scene image, and a media jump link associated with the explanation video can also be displayed synchronously. For example, the media jump link can be a video webpage, news information webpage, or Baidu Encyclopedia webpage related to the target object.

[0152] In addition, when a media jump link associated with the explanation video is displayed in the real-scene image, in response to a third touch operation on the jump link, the target webpage associated with the target object may be automatically jumped and displayed.

[0153] Through the above-mentioned implementation manner of the present application, by displaying an explanation character for explaining the target object in the real scene image, other related display elements can be displayed in the real scene image at the same time, thereby improving the effect of the explanation character's explanation of the target object.

[0154] In an optional embodiment, after updating and displaying the target explanation character in the real scene image according to the mobile following mode in response to the change in the relative position between the image acquisition device and the target object, the method further includes at least one of the following:

[0155] S1, when the image acquisition device stops acquiring real-scene images, determining a first explanation progress of the target explanation role on the target object;

[0156] S2, in response to the image acquisition device resuming acquisition of the real-scene image, resuming display of the target explanation role in the real-scene image, and displaying the target explanation role to continue explaining the target object according to the second explanation progress.

[0157] It can be understood that in this embodiment, when the user captures a real-life image through an image capture device, the corresponding explanation character can be displayed in real time on the real-life image to explain the target object; and when the user turns off the capture function of the image capture device (i.e., pauses the capture), the target object can continue to be explained according to the second explanation progress when the capture is resumed next time.

[0158] In an optional implementation manner, before the display of the target explanation role continuing to explain the target object according to the second explanation progress, the method further includes one of the following:

[0159] Method 1: Determine the first explanation progress as the second explanation progress;

[0160] Method 2: obtaining a first time point when the image acquisition device stops acquiring real-scene images and a second time point when the image acquisition device resumes acquiring real-scene images, and determining a second explanation progress according to the first explanation progress and the time difference between the second time point and the first time point.

[0161] In this embodiment, two methods for determining the second explanation progress are provided. In the first method, at the time point when the acquisition of the target object is resumed, the explanation of the target object can be continued according to the explanation progress corresponding to the time point when the image acquisition was stopped last time;

[0162] For example, if the duration of the explanation resource for explaining the target object is 30 minutes, assuming that the time point at which the user pauses the acquisition of real-scene images is 15:00, and the resource playback progress corresponding to this time point is 15 minutes (that is, the explanation is 50% completed), the explanation resource can be played according to the original progress at the time point at which the real-scene image acquisition is resumed at 15:15, and the corresponding explanation object can be displayed.

[0163] In the second manner, the second explanation progress may be determined according to the time difference between the time point of resuming collection and the time point of last stopping collection.

[0164] For example, if the duration of the explanation resource for explaining the target object is 30 minutes, assuming that the time point when the user pauses the acquisition of real-scene images is 15:00, and the resource playback progress corresponding to this time point is 15 minutes (i.e., the explanation is 50% complete), the time point for resuming the acquisition of real-scene images is 15:15. It can be determined that the time difference between the second time point and the first time point is 15 minutes, corresponding to 50% of the resource playback progress of the explanation resource, that is, 15 minutes is added to the original 15-minute progress, and it is determined that the playback of the target explanation resource has been completed. Then, the second explanation progress is determined to be the 0th minute, that is, the explanation resource is restarted at the time node when the image acquisition is resumed, and the corresponding explanation object is displayed.

[0165] Through the above-mentioned implementation mode of the present application, when the image acquisition device stops acquiring real-scene images, the first explanation progress of the target explanation character on the target object is determined; in response to the image acquisition device resuming acquisition of real-scene images, the target explanation character is restored to be displayed in the real-scene images, and the target explanation character is displayed to continue explaining the target object according to the second explanation progress, so that when the acquisition is paused, the target object is explained according to the new explanation progress, thereby improving the explanation efficiency of the target object.

[0166] In an optional implementation, displaying a target explanation role generated based on an object explanation resource in a real scene image includes:

[0167] S1, when the object explanation resource is an object explanation video including a reference explanation role, the object explanation video is cut out and processed according to the image area occupied by the reference explanation role to obtain a target explanation role;

[0168] S2, creating a virtual three-dimensional space in the current interface, and displaying the target explanation character in a virtual plane determined in the virtual three-dimensional space, wherein the virtual plane is displayed superimposed on the real scene image displayed in the current interface.

[0169] In this embodiment, the current interface may be an interface displayed in a browser or an interface displayed in an application applet. In the interface, the Three.js engine may be called to create a virtual three-dimensional space in the interface.

[0170] It should be noted that the above-mentioned Three.js is a 3D engine running in the browser, which can be used to create various three-dimensional scenes, including cameras, lights and shadows, materials and other objects.

[0171] After the virtual three-dimensional space is created by Three.js in the above interface, a virtual plane in the above virtual three-dimensional space can be created by using the plane buffer geometry (PlaneGeometry, a class for generating plane geometry) in Three.js.

[0172] In an optional implementation, the above-mentioned creating a virtual three-dimensional space in the current interface and displaying the target explanation character in a virtual plane determined in the virtual three-dimensional space includes:

[0173] S1, obtaining a target video frame obtained by processing a current video frame of an object explanation video, wherein the target video frame is a video frame obtained by processing a streaming video file of the object explanation video, the current video frame includes displaying a target explanation character, and transparently displays an image area that does not include the target explanation character;

[0174] S2, draw the target video frame in the virtual plane.

[0175] In this implementation, the drawImage method of canvas may be used to draw the target explanation character displayed in the virtual plane based on the processed video frames.

[0176] In addition, in this embodiment, the object explanation video obtained is specifically a streaming media video. There is no need to wait for the entire video to be downloaded. The video is loaded and played instantly, thereby significantly improving the playback speed and fluency of the explanation video.

[0177] In an optional implementation, the above-mentioned step of performing post-processing on the object explanation video according to the image area occupied by the reference explanation role to obtain the target explanation role includes:

[0178] S1, obtaining the average color value of the first color channel and the second color channel of each fragment in the current video frame of the object explanation video;

[0179] S2, obtaining the color value difference between the color value of the third color channel of each fragment and the average color value, wherein the area color of the image area to be processed that does not include the reference explanation character in the object explanation video is set to the color corresponding to the third color channel;

[0180] S3-1, when the color value difference of the fragment is greater than the first color value threshold, setting the fragment as a transparent fragment;

[0181] S3-2, when the color value difference of the fragment is less than the second color value threshold, setting the fragment as an opaque fragment, wherein the second color value threshold is less than the first color value threshold;

[0182] S3-3, when the color value difference of the fragment is greater than or equal to the second color value threshold and less than or equal to the first color value threshold, setting the fragment as a semi-transparent fragment according to the semi-transparency parameter associated with the color value difference;

[0183] S4, taking the explanation character in the target video frame obtained after processing the current video frame as the target explanation character.

[0184] It should be noted that, in this embodiment, the object explanation video may be, but is not limited to, a green screen video, a red screen video, or a blue screen video. Specifically, when the object explanation video is a green screen video, the area corresponding to the reference explanation character in each video frame of the object explanation video is displayed in normal colors, and the image area except the area corresponding to the reference explanation character in each video frame is set to green, that is, the color value of the green color channel of each pixel in the area is 255, and the color value of the red color channel and the color value of the blue color channel are both 0.

[0185] In a specific way, the video component is used in the image acquisition interface to directly read the video in the form of streaming media, and the video starts to play after the plane model is built. The drawImage method of canvas is used to draw the current video screen during each rendering process. In Android models, WeChat applet can directly use canvas texture, convert the canvas with the drawn video screen into CanvasTexture, and pass it into the fragment shader of ShaderMaterial; while in IOS models, WeChat applet does not support the fragment shader to directly read CanvasTexture. Here, the canvas with the drawn video screen is converted into an image, and TextureLoader is used to load the image and convert it into a texture and pass it into the fragment shader.

[0186] The specific process of green screen removal is as follows Fig. 9 As shown in the figure, the texture of the transmitted video image is used to remove the green screen background in the fragment shader. Here, the transparency is calculated. The opacity of the green screen part is 0, and the opacity of the main video content is 1. The corresponding transparency is calculated at the intersection of the green screen and the main video, so that the main video disappears smoothly at the edge and the green screen is completely removed.

[0187] The steps are as follows: first, calculate the average color value of the R channel (representing the red color value component) and the B channel (representing the blue color value component) of the video picture texture used by the current fragment;

[0188] Next, find the difference between the G channel (representing the green color value component) and the average color value of the R and B channels;

[0189] Based on the established threshold of 0.25, when the above difference is less than 0, the intensity of the green color of the fragment is very low and it is identified as the main body of the video. At this time, the opacity is 1; when the difference is greater than the threshold of 0.25, the intensity of the offset green color is very high and it is identified as a green screen. At this time, the opacity is 0 and it is completely eliminated;

[0190] In addition, the area between the thresholds is the junction of the green screen and the main body of the video. The built-in smoothstep function of glsl is used to smoothly calculate the opacity. Here it is a semi-transparent state, and then the obtained opacity value is cubed to make the semi-transparent part more transparent, which has a better effect on green removal.

[0191] In an optional implementation, in response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile following mode includes:

[0192] When the target explanation character is in the first moving follow mode, in response to the image acquisition device moving, the virtual plane is controlled to move in the virtual three-dimensional space following the current position and rotation angle of the virtual device, including: controlling the distance between the virtual plane in the virtual three-dimensional space and the virtual device to remain at a fixed distance, and controlling the virtual plane in the virtual three-dimensional space to remain perpendicular to the current orientation of the virtual device; wherein the virtual device is a mapping of the image acquisition device in the virtual three-dimensional space;

[0193] When the target explanation character is in the second movement following mode, in response to the image acquisition device moving, the virtual plane is controlled to be fixed at the target space position in the virtual three-dimensional space.

[0194] The following specifically describes the process of calculating the position of the virtual plane when the target explanation character is in the first moving follow mode.

[0195] When the scenic spot being explained is a large area, the user needs to look around the scenic spot. At this time, in the above virtual space, the virtual camera mapped by the user's image acquisition device will also rotate synchronously, and the video plane model (that is, the above virtual plane used to display streaming video) will rotate with the virtual camera. The following rotation is divided into two aspects. On the one hand, the horizontal position (x-axis coordinate and z-axis coordinate) of the plane model rotates around the virtual camera, and the rotation arc is the arc y of the camera itself rotating around the y-axis. Since the distance d between the virtual camera and the video model remains unchanged, the horizontal position of the video model after rotation is calculated as:

[0196] x = camera x-axis coordinate + d*sin y;

[0197] z = camera z-axis coordinate + d*cos y;

[0198] The vertical position coordinate y will be consistent with the height of the virtual camera; on the other hand, the plane model itself also needs to rotate y arcs around its own y axis so that the video content faces the camera and the content is fully displayed. The schematic diagram of the specific model position calculation is as follows Fig.10 shown.

[0199] Each time the video plane model texture is rendered, the position of the plane model and the rotation arc around its own y-axis are adjusted in the above manner according to the position and rotation arc of the camera. This allows the video model (i.e. the virtual plane showing the streaming video after the cutout processing) to move in all directions following the camera's perspective, so that the video explanation screen can always be displayed on the screen.

[0200] It can be understood that, in the case of the second mobile following mode, Fig.11 As shown, when the camera in the virtual three-dimensional space rotates or moves (ie, the image acquisition device in the real space rotates or moves), the plane model used to display the processed streaming media video is still kept unchanged at the original position.

[0201] It should be noted that the mini program in the application does not support THREE.js to directly render the video. Through the above implementation of this application, a technical method is provided to process and render the streaming video through the video component of the mini program, THREE.js and the drawImage method of canvas when the video texture cannot be used, thereby minimizing the occupied resources, and the video picture quality is high and the playback is smooth and instant. In addition, the adaptation problem of the texture input of the mini program on different platforms is solved; in addition, the above implementation also provides a method for displaying and removing the green screen video in the AR scene, so as to improve the display effect of the narration character in the real scene image; at the same time, it also provides a follow-up display solution that keeps the video plane model always displayed in front of the screen under specific needs.

[0202] In the above-mentioned implementation manner, when the target object is captured in the acquisition interface of the image acquisition device, an explanation video of the object matching the target object is obtained, and the explanation role in the explanation video is superimposed and displayed in the real-life image, so that the object information related to the target object is explained by the explanation role, thereby improving the interactivity between the user and the explanation process and improving the explanation effect; in addition, in the above-mentioned implementation manner, the following mode of the explanation role can also be determined according to the object type of the target object, so as to realize switching of different following modes of the explanation role in different explanation environments, and then in the process of the image acquisition device moving, the interactivity and fun of the explanation process are further improved, thereby solving the technical problem of poor explanation effect in the existing method and improving the explanation effect on the target object.

[0203] It should be noted that in the above-mentioned implementation modes of the present application, the acquisition of relevant data of the user requires the permission of the user, and the acquisition method and the processing method of the relevant data comply with the provisions of the relevant normative requirements.

[0204] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0205] According to another aspect of the embodiment of the present invention, there is also provided a display device for explaining a role for implementing the display method of explaining a role. Fig.12 As shown, the device comprises:

[0206] The first display unit 1202 is used to display the acquired real scene image;

[0207] An acquisition unit 1204 is used to acquire an object explanation resource that matches the target object, wherein the target object is associated with the real scene image, and the object explanation resource is used to explain the target object;

[0208] The second display unit 1206 is used to display the target explanation role generated based on the object explanation resource in the real scene image, wherein the target explanation role is configured with a moving follow mode matching the object type of the target object;

[0209] The updating unit 1208 is used to update the target explanation character in the real scene image according to the mobile following mode in response to the change of the relative position between the image acquisition device and the target object.

[0210] Optionally, in this embodiment, the embodiments to be implemented by the above-mentioned various unit modules can refer to the above-mentioned various method embodiments, which will not be repeated here.

[0211] In an optional implementation, the updating unit 1208 includes:

[0212] A first updating module is used to, when the target explanation character is in the first moving following mode, use the first display position of the target explanation character before the relative position between the image acquisition device and the target object changes as the updated second display position; and display the updated target explanation character at the second display position;

[0213] The second updating module is used to display the updated target explanation character at a third image position associated with the target object in the real scene image when the target explanation character is in the second moving following mode, wherein the distance between the third image position and the image position where the target object is located is less than or equal to an intervention target distance threshold.

[0214] Optionally, the first updating module is further used to: display the target explanation role at the second display position according to a fixed image display area;

[0215] Optionally, the above-mentioned second updating module is also used to: determine the current display area corresponding to the current relative distance based on the current relative distance between the image acquisition device and the target object; display the target explanation role at the third image position according to the current display area, wherein the current display area decreases as the current relative distance increases.

[0216] Optionally, the first updating module is further used to: obtain a first reference image region that does not include the target object in the real scene image; and superimpose and display the target explanation character at a first display position determined from the first reference image region.

[0217] Optionally, the second updating module is further used to: obtain a second reference image area in the real scene image whose image distance to the target object is less than or equal to a target distance threshold; and superimpose and display the target explanation character at a fourth display position determined from the second reference image area.

[0218] Optionally, the above-mentioned second display unit is also used to: obtain annotation information of the target object; when the annotation information indicates that the target object is a scenic spot object, determine to configure a first mobile following mode for the target explanation role; when the annotation information indicates that the target object is an exhibit object, determine to configure a second mobile following mode for the target explanation role.

[0219] Optionally, the second display unit is further used to: when the target explanation character is in the first moving following mode, in response to a change in the device position of the image acquisition device, keep displaying the target explanation character in the real scene image when the target object is not included in the real scene image.

[0220] Optionally, the above-mentioned second display unit is also used to: when the target explanation character is in the second mobile following mode, in response to the change of the device position of the image acquisition device, when the target object is not included in the real-scene image, cancel the display of the target explanation character in the real-scene image; when the real-scene image displays a reference object, obtain a reference object explanation resource matching the reference object, wherein the reference object explanation resource is used to explain the reference object; and display a first explanation character generated based on the reference object explanation resource in the real-scene image, wherein the first explanation character is configured with the second mobile following mode.

[0221] Optionally, the second display unit is further used for one of the following:

[0222] In the case where the object explanation resource is an object explanation video including a reference explanation role, the reference explanation role obtained by clipping from the object explanation video is used as the target explanation role, wherein the reference explanation role is used to explain the target object in the object explanation video;

[0223] When the object explanation resource is an object explanation video including a reference explanation role, obtaining the role action features of the reference explanation role; generating a target explanation role according to the role action features;

[0224] When the object explanation resource is the object explanation audio, a target explanation image matching the object explanation audio is obtained, and a target explanation role is generated according to the target explanation image.

[0225] Optionally, the second display unit is further used for at least one of the following:

[0226] In the case where the object explanation resource is an object explanation video including a reference explanation role, obtaining an object explanation video matching the target object, wherein the object explanation video is a pre-stored explanation video;

[0227] In the case where the object explanation resource is an object explanation video including a reference explanation role, a live broadcast link matching the target object is obtained; a live broadcast explanation video matching the target object is obtained based on the live broadcast link, and a target explanation role is generated based on the live broadcast explanation role in the live broadcast explanation video

[0228] Optionally, the second display unit is further used for at least one of the following:

[0229] Overlaying and displaying object text corresponding to the object explanation resource in the real scene image;

[0230] Overlaying and displaying media elements associated with object explanation resources in the real scene image;

[0231] A media jump link associated with the object explanation resource is superimposed and displayed in the real scene image.

[0232] Optionally, after the object text corresponding to the object information is superimposed and displayed in the real scene image, the method further includes: in response to a first touch operation on the object text, copying the object text and displaying corresponding copy prompt information;

[0233] Optionally, after the media element associated with the object explanation video is superimposed and displayed in the real scene image, the method further includes: in response to a second touch operation on the media element, adjusting a display position of the media element in the real scene image according to the second touch operation;

[0234] Optionally, after superimposing and displaying the media jump link associated with the object explanation video in the real-scene image, the method further includes: in response to a triggering operation on the media jump link, jumping and displaying a target web page associated with the target object, wherein the target web page displays media resources associated with the target object.

[0235] Optionally, the above-mentioned updating unit is also used to: when the image acquisition device stops acquiring real-scene images, determine a first explanation progress of the target explanation role on the target object; in response to the image acquisition device resuming acquisition of real-scene images, resume displaying the target explanation role in the real-scene images, and display the target explanation role continuing to explain the target object according to the second explanation progress.

[0236] Optionally, the updating unit is further used for one of the following:

[0237] Determine the first explanation progress as the second explanation progress;

[0238] A first time point when the image acquisition device stops acquiring real-scene images and a second time point when the image acquisition device resumes acquiring real-scene images are acquired, and a second explanation progress is determined according to the first explanation progress and the time difference between the second time point and the first time point.

[0239] Optionally, the second display unit includes:

[0240] An acquisition module, for obtaining a target explanation role by performing post-processing on the object explanation video according to an image area occupied by the reference explanation role when the object explanation resource is an object explanation video including a reference explanation role;

[0241] The display module is used to create a virtual three-dimensional space in the current interface and display the target explanation role in a virtual plane determined in the virtual three-dimensional space, wherein the virtual plane is superimposed on the real scene image displayed in the current interface.

[0242] Optionally, the above-mentioned display module is used to: obtain a target video frame obtained after processing a current video frame of the object explanation video, wherein the target video frame is a video frame obtained after processing a streaming media video file of the object explanation video, the current video frame includes displaying a target explanation character, and transparently displays an image area that does not include the target explanation character; and draw the target video frame in a virtual plane.

[0243] Optionally, the above-mentioned acquisition module is used to: obtain the average color value of the first color channel and the second color channel of each fragment in the current video frame of the object explanation video; obtain the color value difference between the color value of the third color channel of each fragment and the average color value, wherein the area color of the image area to be processed that does not include the reference explanation character in the object explanation video is set to the color corresponding to the third color channel; when the color value difference of the fragment is greater than the first color value threshold, the fragment is set as a transparent fragment; when the color value difference of the fragment is less than the second color value threshold, the fragment is set as an opaque fragment, wherein the second color value threshold is less than the first color value threshold; when the color value difference of the fragment is greater than or equal to the second color value threshold and less than or equal to the first color value threshold, the fragment is set as a semi-transparent fragment according to the semi-transparency parameter associated with the color value difference; and the explanation character in the target video frame obtained after processing the current video frame is used as the target explanation character.

[0244] Optionally, the updating unit is used to: when the target explanation character is in the first moving following mode, in response to the movement of the image acquisition device, control the virtual plane in the virtual three-dimensional space to move following the current position and rotation angle of the virtual device, including: controlling the distance between the virtual plane and the virtual device in the virtual three-dimensional space to remain fixed, and controlling the virtual plane in the virtual three-dimensional space to remain perpendicular to the current orientation of the virtual device; wherein the virtual device is a mapping of the image acquisition device in the virtual three-dimensional space; and when the target explanation character is in the second moving following mode, in response to the movement of the image acquisition device, controlling the virtual plane in the virtual three-dimensional space to be fixed at the target space position.

[0245] Optionally, in this embodiment, the embodiments to be implemented by the above-mentioned various unit modules can refer to the above-mentioned various method embodiments, which will not be repeated here.

[0246] According to another aspect of the embodiment of the present invention, there is also provided an electronic device for implementing the above-mentioned display method of the explanation role, which may be Fig.13 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Fig.13As shown, the electronic device includes a memory 1302 and a processor 1304. The memory 1302 stores a computer program, and the processor 1304 is configured to execute the steps in any of the above method embodiments through the computer program.

[0247] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0248] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0249] S1, displaying the acquired real scene image;

[0250] S2, obtaining an object explanation resource matching the target object, wherein the target object is associated with the real scene image, and the object explanation resource is used to explain the target object;

[0251] S3, displaying a target explanation character generated based on the object explanation resource in the real scene image, wherein the target explanation character is configured with a moving follow mode matching the object type of the target object;

[0252] S4, in response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile follow mode.

[0253] Alternatively, a person skilled in the art may understand that: Fig.13 The structure shown is for illustration only, and the electronic device may also be a vehicle-mounted terminal, a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Fig.13 The structure of the electronic device is not limited. Fig.13 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.13 Different configurations are shown.

[0254] Among them, the memory 1302 can be used to store software programs and modules, such as the program instructions / modules corresponding to the display method and device of the explanation role in the embodiment of the present invention. The processor 1304 executes various functional applications and data processing by running the software programs and modules stored in the memory 1302, that is, realizing the above-mentioned display method of the explanation role. The memory 1302 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1302 may further include a memory remotely arranged relative to the processor 1304, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1302 may be specifically, but not limited to, used to store information such as a map recognition model for identifying entities, event information corresponding to entity objects, and so on. As an example, if Fig.13 As shown, the memory 1302 may include, but is not limited to, the first display unit 1202, the acquisition unit 1204, the second display unit 1206, and the update unit 1208 in the display device for explaining the role. In addition, other module units in the display device for explaining the role may also be included but are not limited to, which will not be repeated in this example.

[0255] Optionally, the transmission device 1306 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1306 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1306 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0256] In addition, the electronic device further includes: a display 1308, and a connection bus 1310, which is used to connect various module components in the electronic device.

[0257] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0258] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program / instruction, the computer program / instruction comprising a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0259] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0260] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method for displaying an explanation role.

[0261] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0262] S1, displaying the acquired real scene image;

[0263] S2, obtaining an object explanation resource matching the target object, wherein the target object is associated with the real scene image, and the object explanation resource is used to explain the target object;

[0264] S3, displaying a target explanation character generated based on the object explanation resource in the real scene image, wherein the target explanation character is configured with a moving follow mode matching the object type of the target object;

[0265] S4, in response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile follow mode.

[0266] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0267] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the above methods of various embodiments of the present invention.

[0268] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0269] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0270] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0271] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0272] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for displaying an explanation role, characterized in that: include: Display the collected real scene images; Acquire an object explanation resource matching a target object, wherein the target object is associated with the real scene image, and the object explanation resource is used to explain the target object; Displaying a target explanation character generated based on the object explanation resource in the real scene image, wherein the target explanation character is configured with a moving follow-up mode matching the object type of the target object; In response to a change in the relative position between the image acquisition device and the target object, the target explanation character is updated and displayed in the real scene image according to the movement following mode.

2. The method according to claim 1, characterized in that In response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile following mode includes: When the target explanation character is in the first moving follow mode, the first display position of the target explanation character before the relative position between the image acquisition device and the target object changes is used as the updated second display position; and the updated target explanation character is displayed at the second display position; When the target explanation character is in the second mobile following mode, the updated target explanation character is displayed at a third image position associated with the target object in the real scene image, wherein the distance between the third image position and the image position where the target object is located is less than or equal to an intervention target distance threshold.

3. The method according to claim 2, characterized in that The step of displaying the updated target explanation character at the second display position includes: Displaying the target explanation character at the second display position according to a fixed image display area; The updated target explanation character is displayed at a third image position associated with the target object in the real-scene image, comprising: determining a current display area corresponding to the current relative distance between the image acquisition device and the target object; and displaying the target explanation character at the third image position according to the current display area, wherein the current display area decreases as the current relative distance increases.

4. The method according to claim 2, characterized in that: Before displaying the updated target explanation character at the second display position, the method further includes: Acquire a first reference image region that does not include the target object in the real scene image; The target explanation character is displayed in a superimposed manner at the first display position determined from the first reference image area.

5. The method according to claim 2, characterized in that: Before displaying the updated target explanation character at the third image position associated with the target object in the real scene image, the method further includes: Acquire a second reference image region in the real scene image, the image distance between which and the target object is less than or equal to a target distance threshold; The target explanation character is displayed in a superimposed manner at a fourth display position determined from the second reference image area.

6. The method according to claim 2, characterized in that Before displaying the target explanation character generated based on the object explanation resource in the real scene image, the method further includes: Acquire the labeling information of the target object; When the annotation information indicates that the target object is a scenic spot object, determining to configure the first moving following mode for the target explanation character; When the annotation information indicates that the target object is an exhibit object, it is determined to configure the second movement following mode for the target explanation role.

7. The method according to claim 6, characterized in that After displaying the target explanation role generated based on the object explanation resource in the real scene image, the method further includes: When the target explanation character is in the first movement following mode, in response to a change in the device position of the image acquisition device, the target explanation character is kept displayed in the real scene image when the target object is not included in the real scene image.

8. The method according to claim 6, characterized in that After displaying the target explanation role generated based on the object explanation resource in the real scene image, the method further includes: When the target explanation character is in the second moving following mode, in response to a change in the device position of the image acquisition device, when the real scene image does not include the target object, canceling the display of the target explanation character in the real scene image; In the case where the real scene image displays a reference object, obtaining a reference object explanation resource matching the reference object, wherein the reference object explanation resource is used to explain the reference object; A reference explanation character generated based on the reference object explanation resource is displayed in the real scene image, wherein the reference explanation character is configured with the second movement following mode.

9. The method according to claim 1, characterized in that: Before displaying the target explanation character generated based on the object explanation resource in the real scene image, the method further includes one of the following: In the case where the object explanation resource is an object explanation video including a reference explanation role, the reference explanation role obtained by clipping from the object explanation video is used as the target explanation role, wherein the reference explanation role is used to explain the target object in the object explanation video; In the case where the object explanation resource is an object explanation video including a reference explanation role, obtaining the role action features of the reference explanation role; generating the target explanation role according to the role action features; In the case where the object explanation resource is an object explanation audio, a target explanation image matching the object explanation audio is obtained, and the target explanation role is generated according to the target explanation image.

10. The method according to claim 9, characterized in that The acquiring of the object explanation resource matching the target object comprises at least one of the following: In the case where the object explanation resource is an object explanation video including a reference explanation role, obtaining the object explanation video matching the target object, wherein the object explanation video is a pre-stored explanation video; In the case where the object explanation resource is an object explanation video including a reference explanation role, a live broadcast link matching the target object is obtained; a live broadcast explanation video matching the target object is obtained based on the live broadcast link, and the target explanation role is generated based on the live broadcast explanation role in the live broadcast explanation video.

11. The method according to claim 9, characterized in that After the target explanation character is updated and displayed in the real scene image in response to the change in the relative position between the image acquisition device and the target object according to the mobile following mode, the method further includes at least one of the following: When the image acquisition device stops acquiring the real scene image, determining a first explanation progress of the target explanation role on the target object; In response to the image acquisition device resuming acquisition of the real-scene image, the target explanation character is restored to be displayed in the real-scene image, and the target explanation character is displayed to continue explaining the target object according to the second explanation progress.

12. The method according to claim 11, characterized in that Before displaying the target explanation character to continue explaining the target object according to the second explanation progress, the method further includes one of the following: determining the first explanation progress as the second explanation progress; A first time point at which the image acquisition device stops acquiring the real-scene image and a second time point at which the image acquisition device resumes acquiring the real-scene image are acquired, and the second explanation progress is determined according to the first explanation progress and a time difference between the second time point and the first time point.

13. The method according to claim 1, characterized in that The step of displaying a target explanation character generated based on the object explanation resource in the real scene image comprises: In the case where the object explanation resource is an object explanation video including a reference explanation role, the object explanation video is subjected to post-processing by cutting out the image area occupied by the reference explanation role to obtain the target explanation role; A virtual three-dimensional space is created in the current interface, and the target explanation character is displayed in a virtual plane determined in the virtual three-dimensional space, wherein the virtual plane is displayed superimposed on the real scene image displayed in the current interface.

14. The method according to claim 13, characterized in that The step of creating a virtual three-dimensional space in the current interface and displaying the target explanation character in a virtual plane determined in the virtual three-dimensional space includes: Acquire a target video frame obtained by processing a current video frame of the object explanation video, wherein the target video frame is a video frame obtained by processing a streaming video file of the object explanation video, the current video frame includes displaying the target explanation character, and transparently displays an image area that does not include the target explanation character; The target video frame is drawn in the virtual plane.

15. The method according to claim 13, characterized in that The step of performing post-processing on the object explanation video according to the image area occupied by the reference explanation character to obtain the target explanation character includes: Obtain the average color value of the first color channel and the second color channel of each fragment in the current video frame of the object explanation video; Obtaining a color value difference between the color value of the third color channel of each fragment and the average color value, wherein the area color of the image area to be processed that does not include the reference explanation character in the object explanation video is set to a color corresponding to the third color channel; When the color value difference of the fragment is greater than a first color value threshold, setting the fragment as a transparent fragment; When the color value difference of the fragment is less than a second color value threshold, setting the fragment as an opaque fragment, wherein the second color value threshold is less than the first color value threshold; When the color value difference of the fragment is greater than or equal to the second color value threshold and less than or equal to the first color value threshold, setting the fragment as a semi-transparent fragment according to a semi-transparency parameter associated with the color value difference; The explanation character in the target video frame obtained after processing the current video frame is used as the target explanation character.

16. The method according to claim 13, characterized in that In response to the change in the relative position between the image acquisition device and the target object, updating and displaying the target explanation character in the real scene image according to the mobile following mode includes: When the target explanation character is in the first moving follow mode, in response to the movement of the image acquisition device, the virtual plane is controlled to move in the virtual three-dimensional space following the current position and rotation angle of the virtual device, including: controlling the distance between the virtual plane and the virtual device in the virtual three-dimensional space to remain fixed, and controlling the virtual plane in the virtual three-dimensional space to remain perpendicular to the current orientation of the virtual device; wherein the virtual device is a mapping of the image acquisition device in the virtual three-dimensional space; When the target narrator character is in the second moving follow mode, in response to the image acquisition device moving, the virtual plane is controlled to be fixed at the target space position in the virtual three-dimensional space.

17. A display device for explaining roles, characterized in that: include: A first display unit, used to display the acquired real scene image; An acquisition unit, configured to acquire an object explanation resource matching a target object, wherein the target object is associated with the real scene image, and the object explanation resource is used to explain the target object; A second display unit is used to display a target explanation character generated based on the object explanation resource in the real scene image, wherein the target explanation character is configured with a moving follow mode matching the object type of the target object; An updating unit is used to update and display the target explanation character in the real scene image according to the mobile following mode in response to a change in the relative position between the image acquisition device and the target object.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 16 when executed.

19. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

20. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 16 through the computer program.

Images