Content navigation method, device, system, computer equipment and storage medium
The guided content is obtained and displayed in real time through AR devices, which solves the problem of difficulty in updating the scenic spot guide signs, realizes rapid update and personalized display of guided information, and improves user experience and management efficiency.
Patent Information
- Application Number
- CN202110235626.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-03
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-03-03
AI Technical Summary
It is difficult to update and maintain existing guide signs in scenic spots and other scenes, and it is impossible to replace or add guide information in time, which affects the user experience.
The scene video frame images are collected through the AR device, sent to the server to obtain the guide content, and displayed it in real time on the device, dynamically adjust the content abundance according to the user's shooting angle change rate, and supports a variety of operation instructions, such as voice playback and sharing.
It realizes rapid updates and personalized display of guided content, improves user experience, saves data traffic, and simplifies the management and maintenance of guided information.
Smart Images

Figure CN112947756B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of augmented reality (AR) technology, and in particular to a content navigation method, device, system, computer equipment and storage medium. Background Art
[0002] In scenic spots, resorts and other scenes, guide signs are usually used to show tourists the introduction information of each guide object in the scene. Such guide signs have fixed content, and when updating or installing, the guide signs need to be discarded and replaced with new ones manually, which is not conducive to adding and updating guide information in a timely manner. Summary of the invention
[0003] The embodiments of the present disclosure at least provide a content navigation method, apparatus, system, computer equipment, and storage medium.
[0004] In a first aspect, an embodiment of the present disclosure provides a content navigation method, which is applied to an augmented display (AR) device, and the content navigation method includes: acquiring a video frame image of a target scene in which the AR device is located; sending the video frame image to a server; receiving target AR navigation content returned by the server based on the video frame image; the target AR navigation content is used to indicate navigation information in the target scene; and associating and displaying a target navigation object in the video frame image and the target AR navigation content.
[0005] In this way, by setting AR guide content for each target guide object in the target scene, when guiding the user, the target AR guide content is determined for the AR device directly from the AR guide contents corresponding to the target scene based on the video frame image of the target scene where the AR device is located. This content guidance method can facilitate timely updating of the guide content of each guide object in the target scene.
[0006] In an optional embodiment, the target AR guide content is determined based on the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; wherein, for any guide object in the target scene, different shooting angle change rate intervals correspond to different AR guide contents of the any guide object.
[0007] In this way, when the shooting angle change rate is small, it is considered that the user has a higher interest in the current object displayed in the video frame image, so more and richer guide content can be chosen to be displayed to him; when the shooting angle change rate is large, it is considered that the user has little interest in the current object displayed in the video frame image, so only preliminary guide content needs to be displayed to him, reducing the transmission of the target AR guide content between the AR device and the server, and saving the data traffic consumed by the transmission.
[0008] In an optional implementation, it also includes: responding to a trigger instruction input by a user for the target AR guide content, and using the target AR guide content to perform an operation corresponding to the trigger instruction; the trigger instruction includes at least one of the following: a voice playback instruction, a sharing instruction, a screenshot instruction, a translation instruction, an enlarged display instruction, a screen recording instruction, a close instruction, and an evaluation instruction.
[0009] In this way, users can flexibly operate the target AR guide content, which is convenient for users.
[0010] In an optional implementation, for the case where the trigger instruction includes the voice playback instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: playing the voice content in the target AR guide content, and / or, broadcasting the text content in the target AR guide content by voice; for the case where the trigger instruction includes the sharing instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: generating sharing information based on the target AR guide content, and publishing the sharing information to the target sharing platform corresponding to the sharing instruction; for the case where the trigger instruction includes the translation instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: translating the target AR guide content into a target language; for the case where the trigger instruction includes the In the case of the zoom display instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: zooming in and displaying the target AR guide content in the interactive interface of the AR device; in the case where the trigger instruction includes the close instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: closing the target AR guide content; in the case where the trigger instruction includes the evaluation instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: generating an evaluation request for the target AR guide content based on the evaluation content input by the user, and sending the evaluation request to the server; receiving and displaying the evaluation result fed back by the server based on the evaluation request; the evaluation result includes the target AR guide content updated based on the evaluation content.
[0011] In an optional embodiment, the associated display of the target guide object in the video frame image and the target AR guide content includes: determining the display position of the target AR guide content based on the position of the target guide object corresponding to the target AR guide content in the video frame image, and displaying the target AR guide content based on the display position; or displaying the target AR guide content at a preset position in the interactive interface of the AR device; or receiving the first posture information sent by the server; the first posture information is used to characterize the posture of the target AR guide content relative to the AR device; based on the first posture information, displaying the target AR guide content; or determining the user's line of sight position in the video frame image, and based on the line of sight position, determining the target guide object the user is looking at; and displaying the target AR guide content corresponding to the target guide object the user is looking at.
[0012] In this way, the display position of the target AR guide content can be flexibly determined to avoid occlusion of the target object by the guide content.
[0013] In an optional embodiment, when there are multiple target AR guide contents corresponding to the same target guide object, the associated display of the target guide object and the target AR guide contents in the video frame image includes: displaying the multiple target AR guide contents in sequence according to a preset order of the multiple target AR guide contents; or, displaying the multiple target AR guide contents according to the length of time the user gazes at the target guide object; or, determining the display order of the multiple target AR guide contents according to the user attributes of the user, and displaying the multiple target AR guide contents according to the display order; wherein the user attributes include: at least one of age, gender, expression, and interest preference.
[0014] In this way, the display order of the target AR guide content is determined for different users in a targeted manner, which is more targeted to each user and the display of the target AR guide content is more flexible.
[0015] In a second aspect, an embodiment of the present disclosure provides a content navigation method, which is applied to a server, and the content navigation method includes: receiving a video frame image sent by an augmented reality (AR) device; determining, based on the video frame image, second posture information of the AR device in a scene coordinate system corresponding to a target scene; determining target AR navigation content based on the second posture information; and sending the target AR navigation content to the AR device.
[0016] In an optional embodiment, after determining the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, the method further includes: determining the target guide object included in the video frame image based on the video frame image; determining the third posture information of the AR guide content corresponding to the target guide object in the scene coordinate system; determining the target AR guide content based on the second posture information includes: determining the target AR guide content based on the second posture information and the third posture information.
[0017] In an optional implementation, determining the target AR guide content based on the second posture information includes: determining a plurality of alternative AR guide contents based on the second posture information; determining a shooting angle change rate between the current video frame image and a historical video frame image of the current video frame image; determining, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; and determining, from the plurality of alternative AR guide contents, the alternative AR guide content corresponding to the target shooting angle change rate interval as the target AR guide content.
[0018] In an optional implementation, the method further includes: determining first posture information based on the second posture information and the third posture information; the first posture information represents the posture of the target AR guide content relative to the AR device; and sending the first posture information to the AR device.
[0019] In an optional implementation, determining the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image includes: performing key point recognition on the video frame image to obtain a first key point in the video frame image; based on the first key point, determining a second key point matching the first key point from a high-precision three-dimensional map of the target scene, and determining the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0020] In an optional embodiment, after performing key point recognition on the video frame image to obtain the first key point in the video frame image, the method further includes: determining the target pixel point in the video frame image corresponding to the first key point; determining the two-dimensional coordinate value of the target pixel point in the image coordinate system corresponding to the video frame image; and determining the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system, including: determining the second posture information of the AR device in the scene coordinate system based on the two-dimensional coordinate value and the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0021] In an optional implementation, determining the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image includes: performing scene key point recognition on the video frame image to determine the target pixel point corresponding to at least one scene key point in the video frame image; performing depth value prediction on the video frame image to determine the depth value of the target pixel point in the video frame image; and determining the second posture information of the AR device based on the depth value of the target pixel point.
[0022] In an optional implementation, the target AR guide content may be expressed in at least one of the following forms: text, animation, image, and voice.
[0023] In a third aspect, the embodiments of the present disclosure further provide a content navigation device, which is applied to an augmented reality (AR) device, and includes: an acquisition module, which is used to acquire a video frame image of a target scene in which the AR device is located; a first sending module, which is used to send the video frame image to a server; a first receiving module, which is used to receive target AR navigation content returned by the server based on the video frame image; the target AR navigation content is used to indicate the navigation information in the target scene; and an association module, which is used to associate and display the target navigation object in the video frame image with the target AR navigation content.
[0024] In an optional embodiment, the target AR guide content is determined based on the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; wherein, for any guide object in the target scene, different shooting angle change rate intervals correspond to different AR guide contents of the any guide object.
[0025] In an optional embodiment, it also includes a response module, which is used to: respond to a trigger instruction input by a user for the target AR guide content, and use the target AR guide content to perform an operation corresponding to the trigger instruction; the trigger instruction includes at least one of the following: a voice playback instruction, a sharing instruction, a screenshot instruction, a translation instruction, an enlarged display instruction, a screen recording instruction, a close instruction, and an evaluation instruction.
[0026] In an optional embodiment, for the case where the trigger instruction includes the voice playback instruction, when the target AR guide content is used to perform the operation corresponding to the trigger instruction, the response module is used to: play the voice content in the target AR guide content, and / or, announce the text content in the target AR guide content by voice; for the case where the trigger instruction includes the sharing instruction, when the target AR guide content is used to perform the operation corresponding to the trigger instruction, the response module is used to: generate sharing information based on the target AR guide content, and publish the sharing information to the target sharing platform corresponding to the sharing instruction; for the case where the trigger instruction includes the translation instruction, when the target AR guide content is used to perform the operation corresponding to the trigger instruction, the response module is used to: translate the target AR guide content into a target language; for the case where the trigger instruction includes the In the case of the zoom display instruction, when the response module uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: in the interactive interface of the AR device, zoom in and display the target AR guide content; in the case where the trigger instruction includes the close instruction, when the response module uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: close the target AR guide content; in the case where the trigger instruction includes the evaluation instruction, when the response module uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: generate an evaluation request for the target AR guide content based on the evaluation content input by the user, and send the evaluation request to the server; receive and display the evaluation result fed back by the server based on the evaluation request; the evaluation result includes the target AR guide content updated based on the evaluation content.
[0027] In an optional embodiment, when the association module associates and displays the target guide object in the video frame image with the target AR guide content, it is used to: determine the display position of the target AR guide content based on the position of the target guide object corresponding to the target AR guide content in the video frame image, and display the target AR guide content based on the display position; or display the target AR guide content at a preset position in the interactive interface of the AR device; or receive the first posture information sent by the server; the first posture information is used to characterize the posture of the target AR guide content relative to the AR device; based on the first posture information, display the target AR guide content; or determine the user's line of sight position in the video frame image, and based on the line of sight position, determine the target guide object the user is looking at; and display the target AR guide content corresponding to the target guide object the user is looking at.
[0028] In an optional embodiment, when there are multiple target AR guide contents corresponding to the same target guide object, the association module, when associating and displaying the target guide object in the video frame image and the target AR guide contents, is used to: display the multiple target AR guide contents in sequence according to a preset order of the multiple target AR guide contents; or, display the multiple target AR guide contents according to the length of time the user gazes at the target guide object; or, determine the display order of the multiple target AR guide contents according to the user attributes of the user, and display the multiple target AR guide contents according to the display order; wherein the user attributes include: at least one of: age, gender, expression, and interest preference.
[0029] In a fourth aspect, the embodiments of the present disclosure further provide a content navigation device, which is applied to a server, and includes: a second receiving module, which is used to receive a video frame image sent by an augmented reality (AR) device; a first determination module, which is used to determine, based on the video frame image, a second posture information of the AR device in a scene coordinate system corresponding to a target scene; a second determination module, which is used to determine a target AR navigation content based on the second posture information; and a second sending module, which sends the target AR navigation content to the AR device.
[0030] In an optional implementation, a third determination module is further included, which is used to: determine the target guide object included in the video frame image based on the video frame image; determine the third pose information of the AR guide content corresponding to the target guide object in the scene coordinate system; the determination of the target AR guide content based on the second pose information includes: determining the target AR guide content based on the second pose information and the third pose information.
[0031] In an optional implementation, when the third determination module determines the target AR guide content based on the second posture information, it is used to: determine a plurality of alternative AR guide contents based on the second posture information; determine the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; determine, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; and determine, from the plurality of alternative AR guide contents, the alternative AR guide content corresponding to the target shooting angle change rate interval as the target AR guide content.
[0032] In an optional implementation, it also includes a third sending module, which is used to: determine the first posture information based on the second posture information and the third posture information; the first posture information represents the posture of the target AR guide content relative to the AR device; and send the first posture information to the AR device.
[0033] In an optional embodiment, when the first determination module determines the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, it is used to: perform key point recognition on the video frame image to obtain a first key point in the video frame image; based on the first key point, determine a second key point matching the first key point from a high-precision three-dimensional map of the target scene, and determine the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0034] In an optional implementation, a fourth determination module is further included, which is used to: determine the target pixel point corresponding to the first key point in the video frame image; determine the two-dimensional coordinate value of the target pixel point in the image coordinate system corresponding to the video frame image; and determine the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system, including: determining the second posture information of the AR device in the scene coordinate system based on the two-dimensional coordinate value and the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0035] In an optional implementation, when the fourth determination module determines the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, it is used to: perform scene key point recognition on the video frame image to determine the target pixel point corresponding to at least one scene key point in the video frame image; perform depth value prediction on the video frame image to determine the depth value of the target pixel point in the video frame image; and determine the second posture information of the AR device based on the depth value of the target pixel point.
[0036] In an optional implementation, the target AR guide content may be expressed in at least one of the following forms: text, animation, image, and voice.
[0037] In a fifth aspect, an optional implementation of the present disclosure further provides a content navigation system, including an AR device and a server; wherein:
[0038] The AR device is used to collect a video frame image of a target scene where the AR device is located; send the video frame image to a server; receive target AR guide content returned by the server based on the video frame image; the target AR guide content is used to indicate the guide information in the target scene; and associate and display the target guide object in the video frame image with the target AR guide content;
[0039] The server is used to receive a video frame image sent by an augmented reality (AR) device; determine second posture information of the AR device in a scene coordinate system corresponding to a target scene based on the video frame image; determine target AR guide content based on the second posture information; and send the target AR guide content to the AR device.
[0040] In a sixth aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions execute any possible implementation of the first aspect or the second aspect mentioned above.
[0041] In a seventh aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, any possible implementation of the first aspect or the second aspect described above is executed.
[0042] For a description of the effects of the above-mentioned content navigation apparatus, system, computer device, and computer-readable storage medium, please refer to the description of the above-mentioned content navigation method, which will not be repeated here.
[0043] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0045] Figure 1 A flow chart of a content navigation method provided by an embodiment of the present disclosure is shown;
[0046] Figure 2 A flowchart showing another content navigation method provided by an embodiment of the present disclosure is shown;
[0047] Figure 3 A specific example diagram showing content displayed by an AR device when a target AR navigation content and a target AR navigation object are displayed in association with each other in a content navigation method provided by an embodiment of the present disclosure is shown;
[0048] Figure 4 A specific example diagram showing content displayed by an AR device when a target AR navigation content is displayed in an interactive interface according to first pose information in a content navigation method provided by an embodiment of the present disclosure is shown;
[0049] Figure 5 A specific example diagram showing content displayed by an AR device when text navigation content is displayed in a text box format in a content navigation method provided by an embodiment of the present disclosure;
[0050] Figure 6 A schematic diagram of a content navigation device provided by an embodiment of the present disclosure is shown;
[0051] Figure 7 A schematic diagram showing another content navigation device provided by an embodiment of the present disclosure;
[0052] Figure 8 A schematic diagram of a content navigation system provided by an embodiment of the present disclosure is shown;
[0053] Fig. 9 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0055] According to research, the current content navigation method usually includes setting up physical signboards at the scene of the tour or visit, marking text, images and other information on the signboards, and users obtain navigation information through the signboards. The information of this content navigation method is relatively fixed and cannot be flexibly changed. When adding new content or replacing existing content, it is necessary to manually remove the original signboard and replace it with a new one. This method is not conducive to the addition and timely update of navigation information.
[0056] Based on the above research, the present disclosure provides a content navigation method, device, system, computer equipment and storage medium, which collects video frame images of the scene where the AR device is located, sends the video frame images to a server, and then accepts and displays the target AR navigation content returned by the server based on the video frame images, abandoning the traditional multiple replacement method, and can facilitate timely updating of the navigation content of each navigation object in the target scene.
[0057] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.
[0058] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0059] To facilitate understanding of this embodiment, a content navigation method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the content navigation method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: an AR device or a server or other processing device. The AR device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the content navigation method can be implemented by a processor calling a computer-readable instruction stored in a memory.
[0060] The content navigation method provided by the embodiment of the present disclosure is described below with the server as the execution subject.
[0061] See also Figure 1 FIG. 1 is a flowchart of a content navigation method provided by an embodiment of the present disclosure, wherein the method includes steps S101 to S104, wherein:
[0062] S101: receiving a video frame image sent by an augmented reality (AR) device;
[0063] S102: Determine second position information of the AR device in a scene coordinate system corresponding to the target scene based on the video frame image;
[0064] S103: Determine target AR guide content based on the second posture information;
[0065] S104: Sending target AR guide content to the AR device.
[0066] The disclosed embodiment processes the received video frame images shot by the AR device to determine the second position information of the AR device in the target scene, thereby determining the target AR guide content according to the second position information, and sending the target AR guide content to the AR device. Since the AR guide content is set for each target guide object in the target scene, when guiding the user, the target AR guide content is determined for the AR device directly based on the video frame image of the target scene where the AR device is located, from each AR guide content corresponding to the target scene. This content navigation method can facilitate timely updating of the guide content of each guide object in the target scene.
[0067] The above S101 to S104 are described in detail below.
[0068] With respect to the above S101, the AR device, for example, is a device that can use an image acquisition device to acquire an image of a target scene and can display AR information. On the AR device, for example, an image acquisition device and an interactive interface can be set; after the image acquisition device is started, the image within the shooting field of the image acquisition device can be acquired in real time to form a video stream; each frame image in the video stream is displayed in real time in the interactive interface.
[0069] The video frame images sent by the AR device may include, for example, video frame images captured by the user in the target scene; wherein the target scene includes, for example, but is not limited to, at least one of the following: outdoor scenes such as scenic spots, playgrounds, and sports fields, and closed places such as exhibition halls, offices, restaurants, and residences.
[0070] When a user carries an AR device to a target scene, the user can use the image acquisition device in the AR device to shoot the target scene to obtain a video stream. The AR device can sample video frame images from the video stream and send the video frame images to the server.
[0071] With respect to the above S102, the server may use the video frame image to determine the specific position and posture of the AR device in the target scene, that is, the second posture information of the AR device in the target scene.
[0072] Specifically, the second posture information includes the first three-dimensional coordinate value of the optical center of the image acquisition device deployed on the AR device in the scene coordinate system, and the optical axis orientation information of the image acquisition device; the optical axis orientation information may include, for example: the deflection angle and pitch angle of the optical axis of the image acquisition device in the scene coordinate system established based on the target scene; or the optical axis orientation information is, for example, a vector in the scene coordinate system.
[0073] In a possible implementation, when determining the second pose information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image acquired by the AR device, for example, the following method can be adopted: performing key point recognition on the video frame image to obtain the first key point in the video frame image; based on the first key point, determining the second key point matching the first key point from a high-precision three-dimensional map of the target scene, and determining the second pose information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0074] In a specific implementation, the first key point includes, for example, at least one of the following: a key point representing contour information of an object contour, a key point representing color block information on the surface of an object, and a key point representing a texture change on the surface of an object.
[0075] After the key point recognition is performed on the video frame image and the first key point in the video frame image is obtained, the matching second key point will be determined from the pre-built high-precision 3D map of the target scene. At this time, the object represented by the determined second key point is the same object as the object represented by the first key point. Furthermore, the 3D coordinate value of the second key point in the high-precision 3D map is the 3D coordinate value of the first key point in the high-precision 3D map.
[0076] Here, the high-precision three-dimensional map of the target scene can be obtained, for example, by any of the following methods: Simultaneous Localization and Mapping (SLAM) modeling, Structure-From-Motion (SFM) modeling.
[0077] Exemplarily, when constructing a high-precision three-dimensional map of the target scene, a three-dimensional coordinate system is established with a preset coordinate point as the origin; wherein the preset coordinate point can be a coordinate point of a building in the target scene or a coordinate point where a camera device is located when the camera captures the target scene; the camera captures video frame images, and constructs a high-precision three-dimensional map of the target scene by tracking a sufficient number of key points in the camera video frame; wherein the key points in the constructed high-precision three-dimensional map of the target scene also include the key point information of the above-mentioned objects, that is, the second key points.
[0078] The first key point is matched with the high-precision 3D map of the target scene, the second key point is determined, and the 3D coordinate value (x1, y1, z1) of the second key point in the high-precision 3D map of the target scene is read. Then, based on the 3D coordinate value of the second key point, the second posture information of the AR device in the scene coordinate system is determined.
[0079] Specifically, when determining the second pose information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point of the target, for example, by utilizing the principle of camera imaging, the second pose information of the AR device in the high-precision three-dimensional map is restored according to the three-dimensional coordinate value of the second key point of the target in the high-precision three-dimensional map.
[0080] Here, when using the camera imaging principle to restore the second posture information of the AR device in the high-precision three-dimensional map, a camera coordinate system can be constructed using the AR device; wherein the origin of the camera coordinate system is the point where the optical center of the image acquisition device in the AR device is located; the z-axis is the straight line where the optical axis of the image acquisition device is located; the plane perpendicular to the optical axis and where the optical center is located is the plane where the x-axis and y-axis are located; the depth detection algorithm can be used to determine the depth value corresponding to each pixel point in the video frame image; after determining the target pixel point in the video frame image, the depth value of the target pixel point in the camera coordinate system can be obtained. h; that is, the three-dimensional coordinate value of the first key point corresponding to the target pixel point in the camera coordinate system can be obtained; then, the three-dimensional coordinate value of the first key point in the camera coordinate system and the three-dimensional coordinate value of the first key point in the scene coordinate system are used to restore the coordinate value of the origin of the camera coordinate system in the scene coordinate system, that is, the position information of the AR device in the second pose information in the scene coordinate system, and the z-axis of the camera coordinate system is used to determine the angle of the z-axis in the scene coordinate system relative to each coordinate axis of the scene coordinate system, so as to obtain the posture information of the AR device in the second pose information in the scene coordinate system.
[0081] For example, the three-dimensional coordinate value of the target pixel in the camera coordinate system is expressed as (x2, y2, h).
[0082] Based on the acquired three-dimensional coordinate value (x1, y1, z1) of the second key point of the target in the scene coordinate system and the determined three-dimensional coordinate value (x2, y2, h) of the target pixel point in the camera coordinate system, the second pose information of the AR device in the scene coordinate system is determined according to the mapping relationship (x1, y1, z1) → (x2, y2, h).
[0083] In another possible implementation, when determining the second pose information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image acquired by the AR device, the video frame image can be firstly subjected to scene key point recognition to determine the target pixel point corresponding to at least one scene key point in the video frame image, and the video frame image can be subjected to depth value prediction to determine the depth value corresponding to each pixel point in the video frame image, and then the second pose information of the AR device can be determined based on the depth value corresponding to the target pixel point in each pixel point.
[0084] Among them, the scene key point can be a pre-set key point in the scene where the AR device is located, such as a table corner, a desk lamp, a potted plant, etc. The depth value of the target pixel point can be used to indicate the distance between the scene key point corresponding to the target pixel point and the AR device.
[0085] The position coordinates of the scene key points in the scene coordinate system are pre-set and fixed. By determining the target pixel point corresponding to at least one scene key point in the video frame image, the orientation information of the AR device in the scene coordinate system can be determined; based on the depth value of the target pixel point corresponding to the scene key point, the posture information of the AR device in the scene coordinate system can be determined, that is, the second posture information of the AR device can be determined.
[0086] With respect to the above S103, the target AR guide content determined based on the second posture information is, for example, the target AR content that can be displayed in the interactive interface of the AR device determined from the pre-set AR content.
[0087] In one possible implementation, when determining the target AR guide content based on the second posture information, for example, the following method can be adopted: based on the video frame image, determine the target guide object included in the video frame image; determine the third posture information of the AR guide content corresponding to the target guide object in the scene coordinate system; based on the second posture information and the third posture information, determine the target AR guide content.
[0088] Exemplarily, the guide objects may include objects in the target scene, such as scenic spots, museum exhibits, functional buildings, etc. For at least some of the guide objects in the target scene, AR guide content corresponding to the objects in the target scene may be pre-set.
[0089] In a specific implementation, the target navigation object refers to at least part of the navigation objects contained in the video frame image acquired by the AR device. The target navigation object can be obtained, for example, in the following manner: performing key point recognition on the video frame image to obtain a first key point, based on the first key point, determining a target second key point matching the first key point from the second key point of the high-precision three-dimensional map of the target scene, and determining the navigation object to which the second key point belongs as the target navigation object.
[0090] Then, based on the target navigation object and the association relationship between the target navigation object and the AR navigation content, the AR navigation content associated with the target navigation object is determined.
[0091] In a possible implementation, all AR guide contents associated with the target guide object may be used as target AR guide contents.
[0092] In another possible implementation, the target AR guide content may be determined from all AR guide contents associated with the target guide object based on certain screening criteria.
[0093] For example, when setting AR navigation content for a navigation object, third position information within the target scene may be determined for the AR navigation content.
[0094] Among them, the third posture information of the AR guide content in the target scene, for example, is the third posture information of the AR guide content in the scene coordinate system, and the third posture information includes, for example: the AR guide content in the scene coordinate system, the three-dimensional coordinate value of the AR guide content, and the posture in the scene coordinate system.
[0095] Then, after the target navigation object is determined, the target AR navigation content is determined based on the second pose information and the third pose information of the AR navigation content corresponding to the target navigation object in the scene coordinate system.
[0096] In another embodiment of the present disclosure, after determining the target AR guide content, the server can also determine the first pose information based on the second pose information and the third pose information of the AR guide content corresponding to the target guide object in the scene coordinate system. The first pose information is used to determine the display position and display posture of the AR device displaying the target AR guide content. Then, the server sends this first pose information to the AR device, so that the AR device can display the target AR content based on the first pose information, which can make the AR guide content have a richer display form.
[0097] Alternatively, the target AR barrage can be determined according to the shooting angle change rate, which can be specifically determined in the following manner: based on the second posture information, multiple alternative AR guide contents are determined; the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image is determined; from multiple shooting angle change rate intervals, the target shooting angle change rate interval to which the shooting angle change rate belongs is determined; and the alternative AR guide content corresponding to the target shooting angle change rate interval is determined as the target AR guide content.
[0098] When determining multiple alternative AR guide contents based on the second posture information, since the second posture information includes the posture information of the AR device in the target scene, the second posture information can be used to determine the target guide object that can be photographed by the AR device in the target scene, and the AR guide contents corresponding to the target guide objects can be used as alternative AR guide contents.
[0099] Among them, the alternative AR guide content corresponding to the target guide object may include brief AR guide content such as name introduction. For example, in the case where the target guide object is a landscape tower, the corresponding AR guide content may include the name of "landscape tower". When displaying the AR guide content of the target guide object, compared with other AR guide content, "landscape tower" is a more concise and summarized AR guide content. Therefore, when the user holds an AR device to shoot the target AR guide object landscape tower, when the shooting time is short, or when the landscape tower is photographed but not the main object of concern, only AR guide content such as "landscape tower" can be displayed.
[0100] In addition, when the target navigation object is a landscape tower, the corresponding AR navigation content can also include detailed introduction information such as "The landscape tower was built in XXX years, and the designer who designed and built the landscape tower is XXXX..." When the user uses an AR device to shoot the landscape tower, if the shooting time is long, or when the landscape tower is the main object of attention, the detailed introduction information corresponding to the target navigation object can be displayed while displaying a concise and summarized AR navigation content such as "landscape tower".
[0101] Specifically, when determining the shooting angle change rate between the historical video frame images corresponding to the current video frame image, the shooting angle change of the user for the target navigation object within a certain time interval can be determined, and the set time interval is, for example, 1 second or 2 seconds. Multiple shooting angle change rate intervals can also be set. Taking the time interval of 1 second as an example, for example, the shooting angle change rate for the target navigation object can fall within multiple shooting angle change rate intervals of [0 degrees / second, 10 degrees / second], [10 degrees / second, 20 degrees / second], ..., [170 degrees / second, 180 degrees / second]. Since the greater the shooting angle change rate, the lower the user's attention to the target navigation object, for the target navigation object, for the shooting angle change rate intervals with larger values such as [160 degrees / second, 170 degrees / second], [170 degrees / second, 180 degrees / second], the corresponding AR navigation content set includes concise and summarized AR navigation content, such as "landscape tower"; for the shooting angle change rate intervals with smaller values such as [0 degrees / second, 10 degrees / second], [10 degrees / second, 20 degrees / second], the corresponding AR navigation content set includes more detailed introduction information corresponding to the target navigation object, such as "the landscape tower was built in XXX years, and the designer who designed and built the landscape tower is XXXX..."
[0102] In this way, when the shooting angle change rate is small, it is considered that the user has a higher interest in the current object displayed in the video frame image, so more and richer guide content can be chosen to be displayed to him; when the shooting angle change rate is large, it is considered that the user has little interest in the current object displayed in the video frame image, so only preliminary guide content needs to be displayed to him, reducing the transmission of the target AR guide content between the AR device and the server, and saving the data traffic consumed by the transmission.
[0103] With respect to the above S104 , the server sends the determined target AR tour guide content to the AR device, so that the AR device displays the target AR tour guide content to the user.
[0104] Next, the content navigation method provided by the embodiment of the present disclosure is described by taking an AR device as an example.
[0105] See also Figure 2FIG. 1 is a flowchart of another content navigation method provided by an embodiment of the present disclosure, the method comprising:
[0106] S201: Collecting video frame images of a target scene where the AR device is located;
[0107] S202: Sending video frame images to a server;
[0108] S203: receiving target AR navigation content returned by the server based on the video frame image; the target AR navigation content is used to indicate navigation information in the target scene;
[0109] S204: Associatively displaying the target navigation object and the target AR navigation content in the video frame image.
[0110] The detailed process of S201 to S203 can be found in the above Figure 1 The corresponding embodiments will not be described in detail here.
[0111] In view of the above S204, when the target navigation object and the target AR navigation content in the video frame image are displayed in association, any of the following methods (a1) to (a4) may be adopted:
[0112] (a1): Based on the position of the target AR guide object corresponding to the target AR guide content in the video frame image, determine the display position of the target AR guide content, and display the target AR guide content based on the display position.
[0113] In this way, after determining the target AR tour content, the AR device can directly associate and display the target AR tour content and the target tour object.
[0114] Specifically, the AR device can identify the target navigation object from the acquired video frame image, and determine the position of the target navigation object in the video frame image, and then based on the position, associate and display the corresponding target AR navigation content and the target navigation object.
[0115] When the target AR guide content and the target AR guide object are associated and displayed, for example Figure 3 As shown, the determined target guided objects include restaurants, bars, coffee shops, and architectural attractions; Figure 3 Among them, 31, 32, and 33 correspond to target AR guide contents with directional properties, indicating restaurants, bars, and coffee shops; for architectural attractions, highlighted AR content can be added to the architectural attractions, and a text AR guide content 34 can be associated with the target AR guide content 34 corresponding to the architectural attractions and displayed in association with the architectural attractions.
[0116] In addition, see above Figure 3Corresponding example, when the AR device displays the target AR guide content, the target AR guide content may be rendered on the interactive interface in a variety of content display forms, for example, a display form that highlights the target guide object (a highlight or different color aperture with a prominent effect and surrounding the outline of the target guide object), a display form that points to the target guide object through a certain pattern, etc.
[0117] (a2): Display the target AR guide content at a preset position in the interactive interface of the AR device.
[0118] Among them, the preset position may include any area where the target AR guide content can be displayed without blocking the captured video frame image. For example, the video frame image can be displayed in the upper three-quarters of the display space on the interactive interface, and then the target AR guide content can be displayed in the lower one-quarter of the display space.
[0119] (a3): Receive the first position information sent by the server; the first position information is used to represent the position of the target AR guide content relative to the AR device; based on the first position information, display the target AR guide content.
[0120] At this time, the first position information received from the server and the camera imaging principle can be used to determine the display position and display posture of the target AR guide content in the interactive interface of the AR device; wherein the display posture refers to the posture of the target AR guide content displayed in the interactive interface of the AR device. Specifically, when the user uses the AR device to shoot the target guide object, the display posture of the corresponding target AR guide content changes according to the change of the shooting angle of the target guide object by the AR device. The display position refers to the position of the target AR guide content when it is rendered in the interactive interface.
[0121] like Figure 4 As shown, a specific example of displaying the target AR guide content in the interactive interface according to the first position information is provided. In this example, 41 represents the display form of the target AR guide content when the target guide object in the figure is a "building". The target AR guide content can be displayed in different directions according to the first position information.
[0122] (a4): Determine the user's line of sight position in the video frame image, and based on the line of sight position, determine the target navigation object that the user is looking at; and display the target AR navigation content corresponding to the target navigation object that the user is looking at.
[0123] Specifically, eye tracking technology can be used to track the user's line of sight. When a person's eyes look in different directions, subtle changes in the eyes can produce extractable features. Then, when the computer extracts this feature through image capture or scanning, it can track the changes in the eyes in real time and then determine the user's line of sight position based on the image. After determining the line of sight position, the AR device can determine the target navigation object that the user is looking at based on the line of sight position. Then, the AR device can display the target AR navigation content corresponding to the target navigation object that the user is looking at.
[0124] In another embodiment of the present disclosure, when displaying the target AR guide content, the determined target AR guide content can be simultaneously displayed in the interactive interface; in addition, when the number of target AR guide content is greater than one, when displaying the target AR guide content through the AR device, for example, any one of the following methods (b1) to (b3) can be adopted:
[0125] (b1): Display multiple target AR guide contents in sequence according to the preset order of the multiple target AR guide contents.
[0126] Exemplarily, the preset order may be determined in the following manner:
[0127] ①: According to the first position information of the AR guide content, determine the distance of the target AR guide content greater than one relative to the AR device, and determine the preset order of the corresponding target AR guide content when displaying according to the order from near to far.
[0128] ②: Predetermine the key target AR guide content among multiple target AR guide contents. For example, you can first determine multiple key target guide objects in the target scene. The key target guide objects may include, for example, the "treasure of the town hall" in the exhibition hall, or the "escape sign" that needs to be paid special attention to. Then, you can determine the key target AR guide content corresponding to the key target guide object. At this time, the preset order corresponds to the priority display of the key target AR guide content, and then the display of other target AR guide content.
[0129] (b2): Display multiple target AR navigation contents according to the length of time the user gazes at the target navigation object.
[0130] For example, eye tracking technology can be used to determine the target navigation object that the user is continuously looking at in the target scene, and the target navigation object that the user is continuously looking at is used as the key target navigation object, and the corresponding target AR navigation content is displayed first. At this time, the preset order corresponds to the priority display of the key target AR navigation content, and then the display of other target AR navigation content.
[0131] (b3): Determine the display order of the multiple target AR guide contents according to the user attributes of the user, and display the multiple target AR guide contents according to the display order; wherein the user attributes include at least one of age, gender, expression, and interest preference.
[0132] Exemplarily, the user attributes of the user can be obtained first. For example, when the user uses this content navigation method, the user questionnaire can be filled out first. In the user questionnaire, for example, information such as age, gender, and interest preference can be included. After collecting the user's attribute information, the user's preference can be determined. For example, for user A, it can be determined that his preference is biased towards modern art decoration exhibits, while for user B, it can be determined that he prefers weapons exhibits with historical backgrounds. When user A enters the target scene, it can be determined that when the target AR navigation content is displayed to user A, the display order corresponds to the priority display of the target AR navigation content corresponding to the modern art decoration exhibits in the target navigation object, and then the target AR navigation content corresponding to other target navigation objects. When user B enters the target scene, it can be determined that when the target AR navigation content is displayed to user B, the display order corresponds to the priority display of the target AR navigation content corresponding to the weapons exhibits with historical backgrounds in the target navigation object, and then the target AR navigation content corresponding to other target navigation objects.
[0133] In addition, the user's expression when viewing the target navigation object in the target scene can also be determined, and then according to the recognized user's expression on different target navigation objects, it is determined whether to give priority to pushing the corresponding target navigation content to the user. For example, when the user views the exhibition sculpture in the target navigation object, it can be detected that the user's expression is interested, and the display order is to give priority to displaying the target AR navigation content corresponding to the target navigation object, and then display the target AR navigation content corresponding to other target navigation objects.
[0134] AR tour content includes, for example, at least one of the following: text AR tour content, animation AR tour content, image AR tour content, and voice AR tour content. In a specific implementation, taking a venue as a tour object, each type of content corresponds to the tour information pre-set for the venue, such as venue introduction information, venue route indication information, venue historical background introduction tour information, venue historical figure introduction tour information, etc.
[0135] Taking the display of text AR guide content as an example, when the target AR guide content is text AR guide content, a pop-up window containing text guide information can be directly displayed in the AR device interaction interface, or displayed in the form of a text box on the video frame image based on the relative position relationship with the target guide object.
[0136] For example Figure 3 and Figure 5 As shown, two specific examples of directly displaying text AR guide content are provided. In this example, Figure 3 34 means that the text guide content is displayed in the form of a text box. Figure 5 51 indicates the AR guide content of directly displaying a pop-up window containing text guide information.
[0137] In another embodiment of the present disclosure, the navigation method can also: respond to the user's triggering of the target AR navigation content, and use the target AR navigation content to perform an operation corresponding to the triggering. The triggering includes at least one of the following: voice playback, sharing, screenshot, translation, magnified display, screen recording, closing, and positioning.
[0138] For example, when the AR device associates and displays the target navigation object and the corresponding target AR navigation content, it can also provide the user with corresponding trigger controls for the target AR navigation content. When the user triggers the corresponding trigger control as needed, the AR device can directly perform the corresponding operation based on the user's trigger.
[0139] In addition, the AR device can also respond to the trigger instruction input by the user to the target AR guide content, and use the target AR guide content to perform the operation corresponding to the trigger instruction. Among them, the trigger instruction includes at least one of the following: voice playback instruction, sharing instruction, screenshot instruction, translation instruction, zoom display instruction, screen recording instruction, close instruction and evaluation instruction.
[0140] Specifically, when the trigger instruction includes a voice playback instruction, when the AR device uses the target AR guide content to perform an operation corresponding to the voice playback instruction, it can play the voice content in the target AR guide content, and / or broadcast the text content in the target AR guide content through voice.
[0141] For example, when the target AR tour content includes textual tour explanation information of the scenic spot, the target AR tour content may include, for example, a textual information about the historical development of the scenic spot. The AR device may determine the corresponding voice information based on the textual information in the target AR tour content and play the voice information; or, directly set the voice information, and after the user triggers the voice play command, the AR device may directly play the set voice information.
[0142] When the trigger instruction includes a sharing instruction, when the AR device uses the target AR guide content to perform an operation corresponding to the sharing instruction, it can generate sharing information based on the target AR guide content and publish the sharing information to the target sharing platform corresponding to the sharing instruction.
[0143] Exemplarily, when the user inputs an operation corresponding to a sharing instruction for the target AR tour content, the AR device can generate sharing information containing the content in the target AR tour content based on the target AR tour content. For example, when the target AR tour content includes explanation information about a sculpture in the target scene, the explanation information and the captured image can be used to generate sharing information, and then the generated sharing information can be published to the target sharing platform corresponding to the sharing instruction. The target sharing platform is not limited to web pages or application software. In addition, after the user in the target sharing platform triggers the sharing information, the detailed information in the sharing information can be viewed and / or jumped to other platforms where the sharing information can be viewed. Specifically, it can be determined according to the actual situation, and will not be repeated here.
[0144] In the case where the trigger instruction includes a translation instruction, the AR device may translate the target AR guide content into the target language when performing an operation corresponding to the translation instruction using the target AR guide content.
[0145] For example, since there may be tourists from all over the world browsing in the scenic area, there may be some tourists who need guided content corresponding to different languages. In order to display the target AR guided content more intuitively on the interactive interface, the target AR guided content in one language can be displayed on the interactive interface, and a translation trigger control that can perform language translation can be displayed on the interactive interface. When a user who needs translation triggers the translation trigger control, he can select a suitable translation language. After the AR device obtains the translation language selected by the user, the translated target AR guided content is displayed in the interactive interface, so that users with different language needs can read the target AR guided content on the interactive interface.
[0146] In the case where the trigger instruction includes an enlarged display instruction, the AR device may enlarge and display the target AR guide content in the interactive interface when using the target AR guide content to perform an operation corresponding to the enlargement instruction.
[0147] For example, when displaying a large number of small exhibits such as jewelry, the AR device can display target AR guide content determined separately for multiple jewelry in the interactive interface when photographing multiple jewelry. When there are many target AR guide contents that can be displayed, the user can zoom in on any target AR guide content that interests the user, and then the AR device displays the zoomed target AR guide content on the interactive interface for the user to view.
[0148] In addition, enlarging the target AR guide content can also make it easier for users with poor eyesight to view, further improving the user experience.
[0149] In the case where the trigger instruction includes a closing instruction, the AR device may close the target AR guide content in the interactive interface when using the target AR guide content to perform an operation corresponding to the closing instruction.
[0150] For example, for a tour guide who has visited the target scene multiple times, or a user such as the staff in the target scene who does not need to watch the target AR tour content, they can trigger a close trigger control on the interactive interface to close part of the target AR tour content displayed on the interactive interface, or close all of the target AR tour content that is displayed.
[0151] In the case where the trigger instruction includes an evaluation instruction, when the AR device uses the target AR guide content to perform the operation corresponding to the evaluation instruction, it can generate an evaluation request for the target AR guide content based on the evaluation content input by the user, and send the evaluation request to the server. The evaluation content may include comments on the target AR guide content itself, such as "this guide content is very valuable and worth learning", or "there are historical time errors in the guide content".
[0152] After receiving the evaluation request, the server can determine the corresponding evaluation result and send the evaluation result to the AR device. For example, after receiving the comment content "This tour guide content is very valuable and worth learning", the server can determine through the relevant reviewers that there is no problem with the target AR tour guide content reflected in the comment content, so the target AR tour guide content can be retained without change; after receiving the comment content "There is a historical time error in the tour guide content", the relevant reviewers confirm that there are problems that need to be corrected in the target AR tour guide content, and the target AR tour guide content can be withdrawn and corrected, and then the target AR tour guide content can be published. Here, models such as convolutional neural networks can also be used to process the corresponding target AR tour guide content based on the comment content, instead of manual processing.
[0153] The various trigger instructions listed above are merely examples, and no limitation is made herein on the trigger instructions and the actual processing procedures corresponding to the trigger instructions.
[0154] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0155] Based on the same inventive concept, a content navigation device corresponding to the content navigation method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned content navigation method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0156] See also Figure 6 , which is a schematic diagram of a content navigation device provided by an embodiment of the present disclosure, and is applied to an augmented reality (AR) device, the device comprises: a collection module 61, a first sending module 62, a first receiving module 63 and an association module 64; wherein,
[0157] The acquisition module 61 is used to acquire the video frame image of the target scene where the AR device is located; the first sending module 62 is used to send the video frame image to the server; the first receiving module 63 is used to receive the target AR guide content returned by the server based on the video frame image; the target AR guide content is used to indicate the guide information in the target scene; the association module 64 is used to associate and display the target guide object in the video frame image and the target AR guide content.
[0158] In an optional embodiment, the target AR guide content is determined based on the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; wherein, for any guide object in the target scene, different shooting angle change rate intervals correspond to different AR guide contents of the any guide object.
[0159] In an optional implementation, a response module 65 is further included, which is used to: respond to a trigger instruction input by a user for the target AR guide content, and use the target AR guide content to perform an operation corresponding to the trigger instruction; the trigger instruction includes at least one of the following: a voice playback instruction, a sharing instruction, a screenshot instruction, a translation instruction, an enlarged display instruction, a screen recording instruction, a close instruction, and an evaluation instruction.
[0160] In an optional implementation, in the case where the trigger instruction includes the voice playback instruction, when the target AR guide content is used to perform the operation corresponding to the trigger instruction, the response module 65 is used to: play the voice content in the target AR guide content, and / or, announce the text content in the target AR guide content by voice; in the case where the trigger instruction includes a sharing instruction, when the target AR guide content is used to perform the operation corresponding to the trigger instruction, the response module 65 is used to: generate sharing information based on the target AR guide content, and publish the sharing information to the target sharing platform corresponding to the sharing instruction; in the case where the trigger instruction includes the translation instruction, when the target AR guide content is used to perform the operation corresponding to the trigger instruction, the response module 65 is used to: translate the target AR guide content into a target language; in the case where the trigger instruction includes the In the case of an enlarged display instruction, when the response module 65 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: enlarge and display the target AR guide content in the interactive interface of the AR device; in the case where the trigger instruction includes the close instruction, when the response module 65 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: close the target AR guide content; in the case where the trigger instruction includes the evaluation instruction, when the response module 65 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: generate an evaluation request for the target AR guide content based on the evaluation content input by the user, and send the evaluation request to the server; receive and display the evaluation result fed back by the server based on the evaluation request; the evaluation result includes the target AR guide content updated based on the evaluation content.
[0161] In an optional embodiment, when the association module 64 associates and displays the target guide object in the video frame image with the target AR guide content, it is used to: determine the display position of the target AR guide content based on the position of the target guide object corresponding to the target AR guide content in the video frame image, and display the target AR guide content based on the display position; or display the target AR guide content at a preset position in the interactive interface of the AR device; or receive the first posture information sent by the server; the first posture information is used to characterize the posture of the target AR guide content relative to the AR device; based on the first posture information, display the target AR guide content; or determine the user's line of sight position in the video frame image, and based on the line of sight position, determine the target guide object the user is looking at; and display the target AR guide content corresponding to the target guide object the user is looking at.
[0162] In an optional embodiment, when there are multiple target AR guide contents corresponding to the same target guide object, the association module 64, when associating and displaying the target guide object and the target AR guide contents in the video frame image, is used to: display the multiple target AR guide contents in sequence according to a preset order of the multiple target AR guide contents; or, display the multiple target AR guide contents according to the length of time the user stares at the target guide object; or, determine the display order of the multiple target AR guide contents according to the user attributes of the user, and display the multiple target AR guide contents according to the display order; wherein the user attributes include: at least one of: age, gender, expression, and interest preference.
[0163] See also Figure 7 FIG. 1 is a schematic diagram of another content navigation device provided by an embodiment of the present disclosure, which is applied to a server. The device includes: a second receiving module 71, a first determining module 72, a second determining module 73 and a second sending module 74; wherein,
[0164] The second receiving module 71 is used to receive the video frame image sent by the augmented reality AR device; the first determining module 72 is used to determine the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image; the second determining module 73 is used to determine the target AR guide content based on the second posture information; the second sending module 74 is used to send the target AR guide content to the AR device.
[0165] In an optional implementation, a third determination module 75 is further included, which is used to: determine the target guide object included in the video frame image based on the video frame image; determine the third pose information of the AR guide content corresponding to the target guide object in the scene coordinate system; the determination of the target AR guide content based on the second pose information includes: determining the target AR guide content based on the second pose information and the third pose information.
[0166] In an optional implementation, when the third determination module 75 determines the target AR guide content based on the second posture information, it is used to: determine a plurality of alternative AR guide contents based on the second posture information; determine the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; determine, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; and determine, from the plurality of alternative AR guide contents, the alternative AR guide content corresponding to the target shooting angle change rate interval as the target AR guide content.
[0167] In an optional implementation, it also includes a third sending module 76, which is used to: determine the first posture information based on the second posture information and the third posture information; the first posture information represents the posture of the target AR guide content relative to the AR device; and send the first posture information to the AR device.
[0168] In an optional embodiment, when determining the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, the first determination module 72 is used to: perform key point recognition on the video frame image to obtain a first key point in the video frame image; based on the first key point, determine a second key point matching the first key point from a high-precision three-dimensional map of the target scene, and determine the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0169] In an optional implementation, a fourth determination module 77 is further included, which is used to: determine the target pixel point corresponding to the first key point in the video frame image; determine the two-dimensional coordinate value of the target pixel point in the image coordinate system corresponding to the video frame image; and determine the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system, including: determining the second posture information of the AR device in the scene coordinate system based on the two-dimensional coordinate value and the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0170] In an optional implementation, the fourth determination module 77, when determining the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, is used to: perform scene key point recognition on the video frame image to determine the target pixel point corresponding to at least one scene key point in the video frame image; perform depth value prediction on the video frame image to determine the depth value of the target pixel point in the video frame image; and determine the second posture information of the AR device based on the depth value of the target pixel point.
[0171] In an optional implementation, the target AR guide content may be expressed in at least one of the following forms: text, animation, image, and voice.
[0172] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0173] The present disclosure also provides a content navigation system, including an AR device and a server. Figure 8, which is a schematic diagram of a content navigation system provided by an embodiment of the present disclosure, including an AR device 82 held by a user 81, and a server 83; wherein,
[0174] The AR device 82 is used to collect a video frame image of a target scene where the AR device is located; send the video frame image to a server; receive a target AR guide content returned by the server based on the video frame image; the target AR guide content is used to indicate the guide information in the target scene; and associate and display the target guide object in the video frame image with the target AR guide content;
[0175] The server 83 is used to receive the video frame image sent by the augmented reality AR device; based on the video frame image, determine the second posture information of the AR device in the scene coordinate system corresponding to the target scene; based on the second posture information, determine the target AR guide content; and send the target AR guide content to the AR device.
[0176] In an optional embodiment, the target AR guide content is determined based on the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; wherein, for any guide object in the target scene, different shooting angle change rate intervals correspond to different AR guide contents of the any guide object.
[0177] In an optional embodiment, the AR device 82 is also used to: respond to a trigger instruction input by the user for the target AR guide content, and use the target AR guide content to perform an operation corresponding to the trigger instruction; the trigger instruction includes at least one of the following: a voice playback instruction, a sharing instruction, a screenshot instruction, a translation instruction, an enlarged display instruction, a screen recording instruction, a close instruction, and an evaluation instruction.
[0178] In an optional implementation, in the case where the trigger instruction includes the voice playback instruction, when the AR device 82 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: play the voice content in the target AR guide content, and / or, announce the text content in the target AR guide content by voice; in the case where the trigger instruction includes the sharing instruction, when the AR device 82 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: generate sharing information based on the target AR guide content, and publish the sharing information to the target sharing platform corresponding to the sharing instruction; in the case where the trigger instruction includes the translation instruction, when the AR device 82 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: translate the target AR guide content into a target language; in the case where the trigger instruction includes the In the case of an enlarged display instruction, when the AR device 82 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: enlarge and display the target AR guide content in the interactive interface of the AR device 82; in the case where the trigger instruction includes the close instruction, when the AR device 82 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: close the target AR guide content; in the case where the trigger instruction includes the evaluation instruction, when the AR device 82 uses the target AR guide content to perform the operation corresponding to the trigger instruction, it is used to: generate an evaluation request for the target AR guide content based on the evaluation content input by the user, and send the evaluation request to the server; receive and display the evaluation result fed back by the server based on the evaluation request; the evaluation result includes the target AR guide content updated based on the evaluation content.
[0179] In an optional embodiment, when the AR device 82 associates and displays the target guide object in the video frame image with the target AR guide content, it is used to: determine the display position of the target AR guide content based on the position of the target guide object corresponding to the target AR guide content in the video frame image, and display the target AR guide content based on the display position; or display the target AR guide content at a preset position in the interactive interface of the AR device 82; or receive the first posture information sent by the server; the first posture information is used to characterize the posture of the target AR guide content relative to the AR device 82; based on the first posture information, display the target AR guide content; or determine the user's line of sight position in the video frame image, and based on the line of sight position, determine the target guide object the user is looking at; and display the target AR guide content corresponding to the target guide object the user is looking at.
[0180] In an optional embodiment, when there are multiple target AR guide contents corresponding to the same target guide object, the AR device 82, when displaying the target guide object and the target AR guide contents in the video frame image in an associated manner, is used to: display the multiple target AR guide contents in sequence according to a preset order of the multiple target AR guide contents; or, display the multiple target AR guide contents according to the length of time the user stares at the target guide object; or, determine the display order of the multiple target AR guide contents according to the user attributes of the user, and display the multiple target AR guide contents according to the display order; wherein the user attributes include: at least one of: age, gender, expression, and interest preference.
[0181] In an optional embodiment, after determining the second posture information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, the server 83 is further used to: determine the target guide object included in the video frame image based on the video frame image; determine the third posture information of the AR guide content corresponding to the target guide object in the scene coordinate system; when the server 83 determines the target AR guide content based on the second posture information, it is used to: determine the target AR guide content based on the second posture information and the third posture information.
[0182] In an optional implementation, when the server 83 determines the target AR guide content based on the second posture information, it is used to: determine a plurality of alternative AR guide contents based on the second posture information; determine the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; determine, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; and determine, from the plurality of alternative AR guide contents, the alternative AR guide content corresponding to the target shooting angle change rate interval as the target AR guide content.
[0183] In an optional implementation, the server 83 is further used to: determine the first posture information based on the second posture information and the third posture information; the first posture information represents the posture of the target AR guide content relative to the AR device; and send the first posture information to the AR device.
[0184] In an optional implementation, when the server 83 determines the second pose information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, it is used to: perform key point recognition on the video frame image to obtain a first key point in the video frame image; based on the first key point, determine a second key point matching the first key point from a high-precision three-dimensional map of the target scene, and determine the second pose information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0185] In an optional implementation, after the server 83 performs key point recognition on the video frame image and obtains the first key point in the video frame image, it is further used to: determine the target pixel point in the video frame image corresponding to the first key point; determine the two-dimensional coordinate value of the target pixel point in the image coordinate system corresponding to the video frame image; when the server 83 determines the second posture information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the second key point in the scene coordinate system, it is used to: determine the second posture information of the AR device in the scene coordinate system based on the two-dimensional coordinate value and the three-dimensional coordinate value of the second key point in the scene coordinate system.
[0186] In an optional implementation, when the server 83 determines the second pose information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, it is used to: perform scene key point recognition on the video frame image to determine the target pixel point corresponding to at least one scene key point in the video frame image; perform depth value prediction on the video frame image to determine the depth value of the target pixel point in the video frame image; and determine the second pose information of the AR device based on the depth value of the target pixel point.
[0187] In an optional implementation, the target AR guide content may be expressed in at least one of the following forms: text, animation, image, and voice.
[0188] The present disclosure also provides a computer device, such as Fig. 9 FIG. 1 is a schematic diagram of a computer device structure provided in an embodiment of the present disclosure, including:
[0189] A processor 91 and a memory 92; the memory 92 stores machine-readable instructions executable by the processor 91, and the processor 91 is used to execute the machine-readable instructions stored in the memory 92. When the machine-readable instructions are executed by the processor 91, the processor 91 performs the following steps:
[0190] Capture a video frame image of a target scene where the AR device is located; send the video frame image to a server; receive target AR guide content returned by the server based on the video frame image; the target AR guide content is used to indicate the guide information in the target scene; and associate and display the target guide object in the video frame image with the target AR guide content.
[0191] Alternatively, the processor 91 executes the following steps:
[0192] Receive a video frame image sent by an AR device; determine second posture information of the AR device in a scene coordinate system corresponding to a target scene based on the video frame image; determine target AR guide content based on the second posture information; and send the target AR guide content to the AR device.
[0193] The above-mentioned memory 92 includes internal memory 921 and external memory 922; the memory 921 here is also called internal memory, which is used to temporarily store the calculation data in the processor 91 and the data exchanged with the external memory 922 such as the hard disk. The processor 91 exchanges data with the external memory 922 through the internal memory 921.
[0194] The specific execution process of the above instructions can refer to the steps of the content navigation method described in the embodiment of the present disclosure, which will not be repeated here.
[0195] The present disclosure also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the content navigation method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.
[0196] The computer program product of the content navigation method provided in the embodiment of the present disclosure carries a program code, and the instructions included in the program code can be used to execute the steps of the content navigation method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0197] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0198] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0199] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0200] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0201] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0202] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A content navigation method, characterized in that: Applied to augmented reality (AR) devices, the content navigation method includes: Collecting a video frame image of a target scene where the AR device is located; Sending the video frame image to a server; Receiving target AR navigation content returned by the server based on the video frame image; the target AR navigation content is used to indicate navigation information in the target scene; the server is used to determine second position information of the AR device in a scene coordinate system corresponding to the target scene based on the video frame image; and determining the target AR navigation content based on the second position information; Associatively displaying the target navigation object in the video frame image and the target AR navigation content; The target AR guide content is determined based on a shooting angle change rate between a current video frame image and a historical video frame image of the current video frame image; Among them, for any guide object in the target scene, different shooting angle change rate intervals correspond to different AR guide contents of the any guide object.
2. The content navigation method according to claim 1, characterized in that: Also includes: In response to a trigger instruction input by a user to the target AR guide content, performing an operation corresponding to the trigger instruction using the target AR guide content; The trigger instruction includes at least one of the following: a voice playback instruction, a sharing instruction, a screenshot instruction, a translation instruction, an enlarged display instruction, a screen recording instruction, a close instruction, and an evaluation instruction.
3. The content navigation method according to claim 2, characterized in that: In the case where the trigger instruction includes the voice play instruction, the using the target AR guide content to perform the operation corresponding to the trigger instruction includes: playing the voice content in the target AR guide content, and / or broadcasting the text content in the target AR guide content by voice; In the case where the trigger instruction includes the sharing instruction, the using the target AR tour content to perform the operation corresponding to the trigger instruction includes: generating sharing information based on the target AR tour content, and publishing the sharing information to a target sharing platform corresponding to the sharing instruction; In the case where the trigger instruction includes the translation instruction, the using the target AR guide content to perform the operation corresponding to the trigger instruction includes: translating the target AR guide content into a target language; In the case where the trigger instruction includes the zoom display instruction, the using the target AR guide content to perform the operation corresponding to the trigger instruction includes: zooming in and displaying the target AR guide content in the interactive interface of the AR device; In the case where the trigger instruction includes the closing instruction, the using the target AR guide content to perform the operation corresponding to the trigger instruction includes: closing the target AR guide content; In the case where the trigger instruction includes the evaluation instruction, the use of the target AR guide content to perform the operation corresponding to the trigger instruction includes: generating an evaluation request for the target AR guide content based on the evaluation content input by the user, and sending the evaluation request to the server; receiving and displaying the evaluation result fed back by the server based on the evaluation request; the evaluation result includes the target AR guide content updated based on the evaluation content.
4. The content navigation method according to any one of claims 1 to 3, characterized in that: The associated display of the target navigation object and the target AR navigation content in the video frame image includes: Determine a display position of the target AR guide content based on a position of a target guide object corresponding to the target AR guide content in the video frame image, and display the target AR guide content based on the display position; or Displaying the target AR navigation content at a preset position in the interactive interface of the AR device; or Receiving first pose information sent by the server; the first pose information is used to represent the pose of the target AR guide content relative to the AR device; based on the first pose information, displaying the target AR guide content; or Determine the sight line position of the user in the video frame image, and based on the sight line position, determine the target navigation object that the user is gazing at; and display the target AR navigation content corresponding to the target navigation object that the user is gazing at.
5. The content navigation method according to any one of claims 1 to 3, characterized in that: In the case that there are multiple target AR guide contents corresponding to the same target guide object, the associated display of the target guide object and the target AR guide contents in the video frame image includes: Displaying the multiple target AR guide contents in sequence according to the preset order of the multiple target AR guide contents; Alternatively, displaying a plurality of the target AR navigation contents according to the time the user gazes at the target navigation object; Alternatively, the display order of the multiple target AR guide contents is determined according to the user attributes of the user, and the multiple target AR guide contents are displayed according to the display order; wherein the user attributes include at least one of age, gender, expression, and interest preference.
6. A content navigation method, characterized in that: Applied to a server, the content navigation method comprises: Receive video frame images sent by augmented reality AR devices; Based on the video frame image, determining second posture information of the AR device in a scene coordinate system corresponding to the target scene; Determining target AR guide content based on the second posture information; Sending the target AR tour content to the AR device; The determining target AR navigation content based on the second posture information includes: Based on the second posture information, determining a plurality of candidate AR guide contents; Determine a shooting angle change rate between a current video frame image and a historical video frame image of the current video frame image; Determining, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; The candidate AR guide content corresponding to the target shooting angle change rate interval is determined as the target AR guide content.
7. The content navigation method according to claim 6, characterized in that: After determining the second position information of the AR device in the scene coordinate system corresponding to the target scene based on the video frame image, the content navigation method further includes: Based on the video frame image, determining the target navigation object included in the video frame image; Determine the third position information of the AR guide content corresponding to the target guide object in the scene coordinate system; The determining target AR navigation content based on the second posture information includes: The target AR guide content is determined based on the second posture information and the third posture information.
8. The content navigation method according to claim 6 or 7, characterized in that: The method further comprises: Determine first posture information based on the second posture information and the third posture information; the first posture information represents the posture of the target AR guide content relative to the AR device; Send the first posture information to the AR device.
9. The content navigation method according to claim 6 or 7, characterized in that: The determining, based on the video frame image, second posture information of the AR device in a scene coordinate system corresponding to the target scene includes: Performing key point recognition on the video frame image to obtain a first key point in the video frame image; Based on the first key point, a second key point matching the first key point is determined from a high-precision three-dimensional map of the target scene, and based on the three-dimensional coordinate value of the second key point in the scene coordinate system, second posture information of the AR device in the scene coordinate system is determined.
10. The content navigation method according to claim 9, characterized in that: After performing key point recognition on the video frame image to obtain the first key point in the video frame image, the content navigation method further includes: Determine a target pixel point corresponding to the first key point in the video frame image; Determine the two-dimensional coordinate value of the target pixel point in the image coordinate system corresponding to the video frame image; The determining, based on the three-dimensional coordinate value of the second key point in the scene coordinate system, second pose information of the AR device in the scene coordinate system includes: Based on the two-dimensional coordinate value and the three-dimensional coordinate value of the second key point in the scene coordinate system, second posture information of the AR device in the scene coordinate system is determined.
11. The content navigation method according to claim 6 or 7, characterized in that: The determining, based on the video frame image, second posture information of the AR device in a scene coordinate system corresponding to the target scene includes: Performing scene key point recognition on the video frame image to determine a target pixel point corresponding to at least one scene key point in the video frame image; Predicting the depth value of the video frame image to determine the depth value of the target pixel point in the video frame image; Based on the depth value of the target pixel point, second posture information of the AR device is determined.
12. The content navigation method according to any one of claims 6 to 7, characterized in that: The target AR navigation content may be expressed in at least one of the following forms: Text, animation, images and voice.
13. A content navigation device, characterized in that: Applied to augmented reality (AR) equipment, the content navigation device comprises: An acquisition module, used to acquire video frame images of a target scene where the AR device is located; A first sending module, used for sending the video frame image to a server; A first receiving module is configured to receive target AR navigation content returned by the server based on the video frame image; the target AR navigation content is used to indicate navigation information in the target scene; the server is configured to determine second position information of the AR device in a scene coordinate system corresponding to the target scene based on the video frame image; and determine the target AR navigation content based on the second position information; An association module, used for associating and displaying the target navigation object in the video frame image and the target AR navigation content; The target AR guide content is determined based on a shooting angle change rate between a current video frame image and a historical video frame image of the current video frame image; Among them, for any guide object in the target scene, different shooting angle change rate intervals correspond to different AR guide contents of the any guide object.
14. A content navigation device, characterized in that: Applied to a server, the content navigation device comprises: A second receiving module is used to receive the video frame image sent by the AR device; A first determination module is used to determine second posture information of the AR device in a scene coordinate system corresponding to a target scene based on the video frame image; A second determination module, configured to determine target AR navigation content based on the second posture information; A second sending module, configured to send the target AR tour guide content to the AR device; The second determining module is specifically used for: Based on the second posture information, determining a plurality of candidate AR guide contents; Determine a shooting angle change rate between a current video frame image and a historical video frame image of the current video frame image; Determining, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; The candidate AR guide content corresponding to the target shooting angle change rate interval is determined as the target AR guide content.
15. A content navigation system, characterized in that: include: Augmented reality AR devices and servers; The AR device is used to collect a video frame image of a target scene where the AR device is located; send the video frame image to a server; receive target AR guide content returned by the server based on the video frame image; the target AR guide content is used to indicate the guide information in the target scene; and associate and display the target guide object in the video frame image with the target AR guide content; The server is used to receive a video frame image sent by an augmented reality (AR) device; based on the video frame image, determine the second posture information of the AR device in the scene coordinate system corresponding to the target scene; based on the second posture information, determine the target AR guide content; and send the target AR guide content to the AR device; the server is specifically used to determine a plurality of alternative AR guide contents based on the second posture information; determine the shooting angle change rate between the current video frame image and the historical video frame image of the current video frame image; determine, from a plurality of shooting angle change rate intervals, a target shooting angle change rate interval to which the shooting angle change rate belongs; and determine the alternative AR guide content corresponding to the target shooting angle change rate interval as the target AR guide content.
16. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the content navigation method as described in any one of claims 1 to 5, or executes the content navigation method as described in any one of claims 6 to 12.
17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a computer device, the computer device executes the content navigation method according to any one of claims 1 to 5, or executes the content navigation method according to any one of claims 6 to 12.
