Remote navigation method, device, system, medium and program product

Abstract

The application discloses a remote navigation method, device, system, medium and program product, and belongs to the technical field of data processing. The method comprises the following steps: in the case that a video call is established between a first terminal and a second terminal through a VoNR+ platform, acquiring media data from the first terminal and / or the second terminal through the VoNR+ platform, copying and forwarding the media data to a media platform; analyzing and processing the media data through the media platform, obtaining analysis result data and sending the analysis result data to a navigation application platform; generating an augmented reality (AR) guide figure through the navigation application platform, generating interaction signaling according to the acquired path related information and the analysis result data, sending the AR guide figure and the interaction signaling to the VoNR+ platform, and the interaction signaling comprises control data of the AR guide figure; generating display media data based on the media data, the AR guide figure and the interaction signaling through the VoNR+ platform, and sending the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in a navigation place according to the display media data and the AR guide figure performs actions according to the control data. The application can reduce the navigation difficulty and improve the navigation effect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a remote navigation method, device, system, medium and program product. BACKGROUND

[0002] With the popularity of terminal devices and the rapid development of communication technology, the functions of terminal devices have also been greatly expanded. For example, users can use terminal devices for navigation to find destinations more easily and improve travel efficiency. However, the operation methods of various functions in navigation are relatively complex and difficult for some users, such as the elderly and children. For example, navigation can guide through map, compass and other guidance forms, but the map, compass and other guidance forms are difficult for users such as the elderly and children to use, and the navigation effect is poor, which is difficult to meet the navigation use requirements of the elderly and children. SUMMARY

[0003] The embodiments of the present application provide a remote navigation method, device, system, medium and program product, which can reduce the difficulty of navigation and improve the navigation effect.

[0004] In a first aspect, the embodiments of the present application provide a remote navigation method applied to a remote navigation system, the remote navigation system including a voice over new radio (VoNR+) platform, a media platform and a navigation application platform. The method includes: in a case where a video call is established between a first terminal and a second terminal through the VoNR+ platform, acquiring media data from the first terminal and / or the second terminal through the VoNR+ platform, and copying and forwarding the media data to the media platform; analyzing and processing the media data through the media platform to obtain analysis result data and sending the analysis result data to the navigation application platform; generating an augmented reality (AR) guide figure through the navigation application platform, and generating interaction signaling according to the acquired path related information and the analysis result data, and sending the AR guide figure and the interaction signaling to the VoNR+ platform, the interaction signaling including control data of the AR guide figure; generating display media data based on the media data, the AR guide figure and the interaction signaling through the VoNR+ platform, and sending the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure that acts according to the control data in the navigation place according to the display media data.

[0005] In a second aspect, the embodiments of the present application provide a remote navigation device, comprising: a voice over new radio (VoNR+) module, configured to acquire media data from a first terminal and / or a second terminal and copy and forward the media data to a media module, in a case that the first terminal and the second terminal establish a video call through the VoNR+ module; the media module, configured to analyze and process the media data to obtain analysis result data and send the analysis result data to a navigation application module; the navigation application module, configured to generate an augmented reality (AR) guide figure and generate interaction signaling according to acquired path-related information and the analysis result data, and send the AR guide figure and the interaction signaling to the VoNR+ module, the interaction signaling comprising control data of the AR guide figure; and the VoNR+ module is further configured to generate display media data based on the media data, the AR guide figure and the interaction signaling, and send the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in a navigation site according to the display media data and perform actions according to the control data.

[0006] In a third aspect, the embodiments of the present application provide a remote navigation system, comprising: a voice over new radio (VoNR+) platform, configured to acquire media data from a first terminal and / or a second terminal and copy and forward the media data to a media platform, in a case that the first terminal and the second terminal establish a video call through the VoNR+ platform; the media platform, configured to analyze and process the media data to obtain analysis result data and send the analysis result data to a navigation application platform; the navigation application platform, configured to generate an augmented reality (AR) guide figure and generate interaction signaling according to acquired path-related information and the analysis result data, and send the AR guide figure and the interaction signaling to the VoNR+ module, the interaction signaling comprising control data of the AR guide figure; and the VoNR+ platform is further configured to generate display media data based on the media data, the AR guide figure and the interaction signaling, and send the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in a navigation site according to the display media data and perform actions according to the control data.

[0007] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the remote navigation method of the first aspect.

[0008] In a fifth aspect, the embodiments of the present application provide a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the remote navigation method of the first aspect.

[0009] The embodiment of the application provides a remote navigation method, device, system, medium and program product. A video call between two terminals is established through a VoNR+ platform, media data is obtained from the terminals, the media data is analyzed and processed by a media platform, analysis result data obtained through the analysis and processing is transmitted to a navigation application platform, the navigation application platform generates an AR road guide person image, control data for controlling the behavior of the AR road guide person image is generated according to path related information and the analysis result data, and interactive signaling is generated according to the control data. The interactive signaling and the AR road guide person image are sent to the VoNR+ platform. The VoNR+ platform can generate display media data of a real-time real environment in which the AR road guide person image performing an action can be displayed based on the media data, the AR road guide person image and the control data in the interactive signaling, and the display media data is sent to the two terminals, so that the two terminals can display the AR road guide person image performing the action according to the control data in the real navigation place, real-time navigation of the AR road guide person image on the terminal of the user is realized, the AR road guide person action is clear, and the user receiving the navigation does not need to perform complex operations, the user receiving the navigation only needs to follow the indication of the AR road guide person image. The remote navigation method is simple to use, reduces the navigation difficulty, is more friendly to the users such as the old and the young, can meet the navigation use requirements of the users such as the old and the young, and improves the navigation effect. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments of the application. Those skilled in the art can also obtain other drawings according to these drawings without any creative effort.

[0011] Figure 1 An architecture schematic diagram of an example of the remote navigation system provided by the embodiments of the application;

[0012] Figure 2 A flowchart of the remote navigation method provided by an embodiment of the application;

[0013] Figure 3 A schematic diagram of an example of the interactive signaling provided by the embodiments of the application;

[0014] Figure 4 A schematic diagram of an example of the navigation interface displayed by the terminal provided by the embodiments of the application;

[0015] Figure 5 A schematic diagram of another example of the navigation interface displayed by the terminal provided by the embodiments of the application;

[0016] Figure 6 A schematic diagram of an example of the navigation interface displayed by the terminal in an automatic mode provided by the embodiments of the application;

[0017] Figure 7 FIG. 1 is a schematic diagram of an example of a navigation interface displayed by a terminal according to an embodiment of the present application;

[0018] Figure 8 FIG. 2 is a schematic diagram of an example of a navigation interface displayed by a terminal according to an embodiment of the present application;

[0019] Figure 9 FIG. 3 is a schematic diagram of an example of a navigation interface displayed by a terminal according to an embodiment of the present application;

[0020] Figure 10 FIG. 4 is a schematic diagram of another example of a navigation interface according to an embodiment of the present application;

[0021] Figure 11 FIG. 5 is a flowchart of an example of a remote navigation process according to an embodiment of the present application;

[0022] Figure 12 FIG. 6 is a schematic diagram of a remote navigation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of the various aspects of the present application will be described in detail below with reference to the drawings. The following detailed description is merely intended to explain the present application, and is not intended to limit the present application. The present application can be implemented without some of the specific details, which are well known to those skilled in the art. The following description of the embodiments is merely provided to give a better understanding of the present application by showing examples of the present application.

[0024] It should be noted that the terms such as first and second, etc., are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the elements defined by the statement "comprise" do not exclude the presence of other identical elements in the processes, methods, articles or devices including the elements.

[0025] With the popularity of terminal devices and the rapid development of communication technology, the functions of terminal devices have also been greatly expanded. For example, users can use terminal devices for navigation, find destinations more easily, and improve travel efficiency. However, the operation methods of various functions in navigation are relatively complex for some users, such as the elderly and children, and the operation is relatively difficult. For example, the travel route is pre-recorded into the terminal device, and when traveling, the terminal device matches the positioning information with the pre-recorded travel route, and guides through voice prompts, but the actual travel conditions are complex, there are difficulties in positioning, there are many obstacles on the travel route, and there may be a situation of close straight-line distance but need to detour, which may also cause inaccurate voice prompts, and the travel environment is often noisy, and the effect of voice prompts is also poor. For users such as the elderly and children, the navigation effect is poor, and it is difficult to meet the navigation use needs of users such as the elderly and children. For another example, the position information of the other party can be shared through the call between terminal devices, and then guided through a map, a compass, and the like. However, the map, the compass, and the like are difficult for users such as the elderly and children to use, and the navigation effect is poor, and it is difficult to meet the navigation use needs of users such as the elderly and children.

[0026] The present application provides a remote navigation method, device, system, medium and program product. The virtual guide person, i.e. AR guide person, is generated by using the Augmented Reality (AR) technology in the video call of the terminal through the platform for implementing the 5G Enhanced Voice over New Radio (VoNR+) technology, i.e. VoNR+ platform, media platform and navigation application platform, and the motion of the AR guide person, such as walking, turning, raising hands and the like, is controlled in combination with the real-time real environment photographed by the terminal, so as to realize the navigation guidance. The AR guide person can be combined with the real-time real environment, and the motion of the AR guide person is simple and easy to understand, without the need for the user to perform complex operations, which reduces the difficulty of navigation, and the navigation effect for users such as the elderly and children is better, and the navigation use needs of users such as the elderly and children can be met.

[0027] The remote navigation method, device, system, medium and program product provided by the present application are described below respectively.

[0028] For the sake of understanding, the remote navigation system applying the remote navigation method in the embodiments of the present application is described first. Figure 1 The architecture schematic diagram of an example of the remote navigation system provided by the embodiments of the present application is shown in Figure 1 As shown in the figure, the remote navigation system can include a VoNR+ platform 11, a media platform 12 and a navigation application platform 13.

[0029] The VoNR+ platform 11 enables 5G VoNR+ new calling functions, which can include ultra-high-definition voice calls, video calls, and related services based on the 5G network. 5G VoNR+ new calling provides richer media capabilities and real-time interactive capabilities. In some examples, the VoNR+ platform 11 may include a VoNR+ media plane and VoNR+ capability network elements. The VoNR+ media plane provides unified media management functions; for example, it can specifically provide, but is not limited to, audio and video media stream duplication, media stream forwarding, and video compositing and rendering. The VoNR+ capability network elements provide functions such as event notification, call control, and data channel (DC) processing. The VoNR+ platform 11 can communicate and interact with terminals, such as... Figure 1 As shown, the VoNR+ platform 11 can communicate and interact with terminals A1 and A2. Terminals A1 and A2 can establish ultra-high-definition voice calls and video calls through the VoNR+ platform 11. The VoNR+ platform 11 can be implemented through one or more electronic devices, for example, through one or more servers. The type and number of electronic devices forming the VoNR+ platform 11 are not limited here.

[0030] Media platform 12 can communicate and interact with VoNR+ platform 11. VoNR+ platform 11 can transmit media data such as audio and video media streams to media platform 12. Media platform 12 can analyze and process the media data. For example, media platform 12 can perform speech processing, word recognition, semantic analysis, and other analysis on audio streams, and image segmentation, object detection, and other analysis on video streams to obtain analysis result data, which can represent the results of the analysis and processing. Media platform 12 can be implemented through one or more electronic devices, and the type and number of electronic devices forming media platform 12 are not limited herein.

[0031] The navigation application platform 13 can communicate with the media platform 12 and the VoNR+ platform 11. The navigation application platform 13 can communicate with the terminal A1 and the terminal A2 to receive a remote navigation request initiated by the terminal A1 or the terminal A2. The navigation application platform 13 defines a lightweight navigation control protocol based on 5G VoNR+ communication, and can use a ProtoBuff (ProtocolBuffers) serialization protocol to define interactive signaling of a lightweight remote procedure call (RPC) negotiation. Specifically, the RPC can use a gRPC framework. The navigation application platform can generate an AR guide avatar for remote navigation, and the motion control of the AR guide avatar can be based on the control data in the above-mentioned interaction information. The navigation application platform 13 and the VoNR+ platform 11 can interact through the above-mentioned interactive signaling to transmit the control data corresponding to the AR guide avatar to the VoNR+ platform 11, so that the VoNR+ platform 11 can perform a mixing operation and the like, and transmit the obtained display media data including the AR guide avatar to the terminal A1 and the terminal A2, so that the terminal A1 and the terminal A2 can display a navigation image including the AR guide avatar and a display environment.

[0032] The terminal A1 and the terminal A2 can be devices with 5G VoNR+ audio and video call capabilities. After receiving the display media data fed back by the VoNR+ platform 11, the terminal A1 and the terminal A2 can perform control and rendering corresponding to the display media data, and further display a navigation image including the AR guide avatar and a display environment.

[0033] The data transmission between the terminal A1, the terminal A2, the VoNR+ platform 11, the media platform 12, and the navigation application platform 13 can be performed through several data transmission channels. As shown in Figure 1 The data transmission channel can include a data channel 14, a media channel 15, and a signaling channel 16. The data channel 14 can transmit display media data between the VoNR+ platform 11 and the terminal A1 and the terminal A2, and the data channel 14 can provide a high real-time single-stream or multi-stream data interaction channel. The media channel can transmit media data such as audio and video media streams. The signaling channel can transmit interactive signaling and the like.

[0034] The first aspect of the present application provides a remote navigation method, which can be applied to a remote navigation system including a VoNR+ platform, a media platform, and a navigation application platform. The VoNR+ platform, the media platform, and the navigation application platform can be referred to the related description in the above-mentioned embodiments, which will not be repeated here. Figure 2 The flowchart of the remote navigation method provided by an embodiment of the present application is shown in Figure 2 The remote navigation method can include steps S201 to S204.

[0035] In step S201, in the case that the first terminal and the second terminal establish a video call through the VoNR+ platform, media data is obtained from the first terminal and / or the second terminal through the VoNR+ platform, and the media data is copied and forwarded to the media platform.

[0036] One of the first terminal and the second terminal is a navigated terminal, and the other is a navigation guide terminal. In the case that the first terminal is the navigated terminal, the second terminal is the navigation guide terminal; in the case that the first terminal is the navigation guide terminal, the second terminal is the navigated terminal. The navigated terminal is the terminal of the user who requests navigation help, and the navigation guide terminal is the terminal of the user who can assist navigation. In some examples, a call initiated by the first terminal can be received through the VoNR+ platform, and a call is initiated to the second terminal to establish a video call between the first terminal and the second terminal. That is, the first terminal initiates a call to the VoNR+ platform, establishes a data transmission channel with the VoNR+ platform, the VoNR+ platform initiates a call to the second terminal, establishes a data transmission channel with the second terminal, the second terminal answers the call, and the first terminal and the second terminal successfully establish a video call. The first terminal can be a navigated terminal, or a navigation guide terminal, that is, the video call can be initiated by the navigated terminal, or the navigation guide terminal, which is not limited here.

[0037] The media data can include audio data, video data, and other media data. The navigated terminal can provide video data, and the navigation guide terminal can provide audio data. The first terminal and / or the second terminal can transmit the media data to the VoNR+ platform. In some examples, a remote navigation request of the first terminal or the second terminal is received through the navigation application platform, and in response to the remote navigation request, a user authorization request is obtained, and the user authorization request is sent to the first terminal and / or the second terminal through the VoNR+ platform; if authorization consent information sent by the first terminal and / or the second terminal is received through the VoNR+ platform, the media data is obtained from the first terminal and / or the second terminal through the VoNR+ platform. That is, the remote navigation request can be initiated by the navigated terminal or the navigation guide terminal to the navigation application platform, the navigation application platform issues a user authorization request to the VoNR+ platform, the user authorization request can include user agreement, privacy policy, and other description content; the VoNR+ platform sends the user authorization request to the navigated terminal; if the user agrees to the authorization, the authorization consent input of the navigated terminal is input, and the authorization consent information of the navigated terminal is fed back to the VoNR+ platform; the VoNR+ platform has the permission to obtain the media data from the first terminal and / or the second terminal. The VoNR+ platform copies and forwards the obtained media data to the media platform, so that the media platform processes the media data.

[0038] In step S202, the media data is analyzed by the media platform to obtain analysis result data and send to the navigation application platform.

[0039] The analysis result data obtained by the media platform analyzing the media data can provide assistance for the navigation application platform to provide navigation services. The media platform can analyze the media data according to the type of the media data.

[0040] In some examples, when the media data includes voice data sent by the navigation guidance terminal, the voice data is analyzed by the media platform to obtain analysis result data. The voice analysis includes one or more of the following: noise reduction processing, feature extraction, word recognition, semantic analysis. The voice data can be implemented as an audio stream, and the audio stream can be processed by noise reduction, feature extraction, etc., to identify the words and perform semantic analysis, so as to obtain data that can represent the analysis result of the voice data. For example, the voice data is "XX, turn left from here", and a series of voice analysis can be performed to obtain analysis result data, which can include data representing the meaning of "turn left", so that the navigation application platform can generate control data to control the AR signpost figure to turn left in the subsequent process.

[0041] In some examples, when the media data includes video data sent by the navigation terminal, the video data is analyzed by the media platform to obtain analysis result data. The image analysis includes one or more of the following: feature extraction, target detection, image segmentation, image classification, image environment analysis. The video data can be implemented as a video stream, and the video stream can be processed by feature extraction, target detection algorithm, image segmentation algorithm, etc. to detect the terrain and objects such as roads, buildings, pedestrians and vehicles in the video stream, determine the type, position and motion of the objects, etc., to obtain analysis result data that can represent the type, position and motion of the objects, etc. The user movement information such as the moving direction and speed of the navigation terminal can also be determined by feature extraction and image classification of the images in the video stream to obtain analysis result data that can represent the user movement information. The light and shadow in the video stream can also be analyzed by image environment analysis to determine the influence of weather, surrounding environment and other factors, to obtain analysis result data that can represent the environmental factors in the video stream, and the navigation strategy can be adjusted according to the environmental factors.

[0042] In the above examples, the navigation terminal is one of the first terminal and the second terminal, and the navigation guidance terminal is the other of the first terminal and the second terminal.

[0043] In step S203, the AR guide figure is generated by the navigation application platform, and the interaction signaling is generated according to the obtained path-related information and analysis result data, and the AR guide figure and the interaction signaling are sent to the VoNR+ platform.

[0044] The AR guide figure can be a default figure, or can be obtained by custom setting by the user of the first terminal or the user of the second terminal. The user can set the characteristics of the AR guide figure, such as gender, face shape, dress, hairstyle, etc., through the first terminal or the second terminal, to increase the immersion and personalized experience of navigation. The navigation application platform can receive the AR guide figure setting information from the first terminal or the second terminal, or receive the AR guide figure setting information from the first terminal or the second terminal through the VoNR+ platform, to generate the AR guide figure.

[0045] The path-related information is related information required to obtain the path of the terminal being navigated. The path-related information can indicate the position of the terminal being navigated, the position of the destination, the navigation site, etc. The path-related information can be obtained from the information related to the position of the terminal and the position of the destination sent by the first terminal or the second terminal. The path-related information and the analysis result data can be integrated to generate the control data of the AR guide figure, and the control data of the AR guide figure is used to control the behavior of the AR guide figure. The interaction signaling can be generated according to the control data of the AR guide figure, and the interaction signaling includes the control data of the AR guide figure.

[0046] In some examples, the interaction signaling can include a signaling head part and a signaling data part. The signaling head part includes control bits, and the signaling data part includes control data corresponding to the control bits. The control bits can be used to represent the control type of the behavior of the AR guide figure, and the control data can be the value of the control type of the behavior of the AR guide figure. One control bit can correspond to one control data, or can correspond to multiple control data, which is not limited herein. For example, the control bits can include, but are not limited to, one or more of the following: navigation mode, movement instruction of the AR guide figure, body action of the AR guide figure, prompt given by the AR guide figure, and whether the position of the AR guide figure is reset. Correspondingly, the control data can include, but is not limited to, one or more of the following: navigation mode data, movement instruction data of the AR guide figure, skeletal animation data of the AR guide figure, prompt data of the AR guide figure, and position reset data of the AR guide figure. The navigation mode data can include manual mode identifier and automatic mode identifier. The movement instruction data of the AR guide figure can include data indicating the distance and speed of the movement of the AR guide figure. The skeletal animation data of the AR guide figure can indicate the body action of the AR guide figure, such as, can indicate the body action of stopping, turning, and greeting. The prompt data of the AR guide figure can indicate the prompt information or warning information given by the AR guide figure to the user. The position reset data of the AR guide figure is used to indicate whether to reset the position of the AR guide figure displayed in the first terminal and the second terminal. The signaling head part in the above example can also include one or more of the following: video call identifier, data source address, data destination address, and verification data. For example, Figure 3 An example of the interaction signaling provided by the embodiments of the present application is shown in the schematic diagram as Figure 3 As shown in the schematic diagram, the signaling head part of the interaction signaling includes a video call identifier, i.e., Call ID, a data source address, a data destination address, control bits, and verification data, and the signaling data part includes control data. The video call identifier is a unique identifier for each call. The data source address can indicate the source address of the data. The data destination address can indicate the destination address of the data. The verification data can include a verification code generated for the signaling head part and the signaling data part, and specifically, the verification data can be implemented as Cyclic Redundancy Check (CRC).

[0047] In step S204, based on the media data, the AR guide figure, and the interaction signaling, the VoNR+ platform generates display media data and sends the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in the navigation site according to the display media data and the control data.

[0048] The VoNR+ platform can determine the behavior of the guide figure in the real-time real environment according to the guide figure image and the interaction signaling, and the real-time real environment is the image corresponding to the media data. The VoNR+ platform can perform a mixed flow operation on the AR guide figure image performing the action according to the control data in the interaction signaling and the media data, to generate display media data. The display media data can display the real-time real environment including the AR guide figure performing the action. The VoNR+ platform sends the display media data to the first terminal and the second terminal. The first terminal and the second terminal display images according to the received display media data, and specifically display the AR guide figure performing the action according to the control data in the real navigation place, thereby realizing real-time navigation of the AR guide figure on the terminal of the user.

[0049] In the embodiments of the present application, the VoNR+ platform establishes a video call between two terminals, thereby obtaining media data from the terminals, and a media platform analyzes and processes the media data. The analysis result data obtained by the analysis and processing is transmitted to a navigation application platform. The navigation application platform generates an AR guide figure, and generates control data that can control the behavior of the AR guide figure according to path-related information and analysis result data, and generates interaction signaling according to the control data. The interaction signaling and the AR guide figure are sent to the VoNR+ platform. The VoNR+ platform can generate display media data that can display a real-time real environment including the AR guide figure performing the action based on the media data, the AR guide figure, and the control data in the interaction signaling, and send the display media data to the two terminals, so that the two terminals can display the AR guide figure performing the action according to the control data in the real navigation place, thereby realizing real-time navigation of the AR guide figure on the terminal of the user. The AR guide figure action is clear and lifelike, and the user receiving the navigation only needs to follow the indication of the AR guide figure, which is simple to use and more friendly to users such as the elderly and children, can meet the navigation use requirements of users such as the elderly and children, and improves the navigation effect.

[0050] In some embodiments, the above-mentioned analysis result data can include one or more of the following: voice semantic data, image object type data, image object position data, image object motion data, user movement data, and environmental factor data.

[0051] The voice semantic data is data representing semantics of voice data in the media data. The image object type data is data representing types of objects detected in the image, for example, representing types of objects such as roads, vehicles, buildings, obstacles, steps, etc. detected in the image. The image object type data includes plane object type data representing types of detected plane objects such as straight roads, crossroads, etc. and non-plane object type data representing types of detected non-plane objects such as steps, door thresholds, etc. The identification of the types of non-plane objects can assist in generating control data for actions such as stepping up, stepping down, crossing a door threshold, etc. of the AR guide avatar in subsequent processes. The detection of the types of non-plane objects can realize navigation in non-plane environments such as upstairs and downstairs. Through the accurate mimic human body actions of the AR guide avatar, the user is guided to walk in the corresponding direction, realizing navigation in non-plane environments. The image object position data is data representing positions of objects detected in the image. The image object motion data is data representing motion conditions of objects detected in the image, for example, representing whether the objects are in a static state or a motion state. The user movement data is data representing movement conditions of the user of the terminal being navigated, for example, including data such as the movement direction and speed of the user. The environmental factor data is data representing environmental factors of the real environment photographed by the terminal being navigated. The navigation application platform can obtain the control data in the above embodiments by using the pre-trained control instruction model according to the analysis result data.

[0052] In some embodiments, the destination to be navigated by the user is far away and not within the shooting range of the terminal being navigated, so that positioning information needs to be obtained to assist in obtaining the planned path of navigation. The related content of the terminal being navigated can be referred to the related content of the above embodiments, which will not be repeated here. The path-related information includes the positioning information of the terminal being navigated and the destination position information. The positioning information of the terminal being navigated can indicate the position of the terminal being navigated, and the destination position information can indicate the position of the destination to be navigated. Specifically, in the case that the destination indicated by the destination position information is beyond the shooting range of the terminal being navigated, the navigation application platform can determine a first planned path according to the positioning information and the destination position information. The navigation application platform generates control data according to the first planned path and the analysis result data, and the control data is used to control the actions of the AR guide avatar. The navigation application platform fills the control data according to the predetermined format of the interactive signaling to generate the interactive signaling.

[0053] The destination position information indicates a destination beyond the shooting range of the navigation terminal, i.e., the destination is not in the range displayed by the navigation terminal. The navigation application platform determines a first planned path from the position of the navigation terminal to the position of the destination according to the position of the navigation terminal indicated by the positioning information and the position of the destination indicated by the destination position information. In combination with the analysis result data and the first planned path, control data for controlling the AR guide avatar to walk along the first planned path can be obtained. For example, if the analysis result data includes the moving speed of the user, the control data can control the AR guide avatar to walk along the first planned path at the moving speed of the user to match the moving speed of the user, so that the navigation is more in line with the needs of the user. The first terminal and the second terminal can display the AR guide avatar making actions, for example, Figure 4 An example of a navigation interface displayed by a terminal according to an embodiment of the present application is shown in the schematic diagram as Figure 4 The navigation interface includes a real environment including a building B1, a landscape model B2, a landscape model B3, and a ground B4. An AR guide avatar C1 is integrated into the real environment. The AR guide avatar C1 points out a direction with a finger to prompt the user to move in the direction. The navigation application platform fills the obtained control data into the corresponding positions in the interactive signaling and sends the interactive signaling to the VoNR+ platform. The navigation mode using positioning information is suitable for places where the destination is far away from the navigation terminal, located outdoors, and has less shielding, etc., and the positioning error is small.

[0054] In some embodiments, the position of the destination required by the user for navigation is close and within the shooting range of the navigation terminal. In this case, the navigation application platform can obtain the planned path required for navigation through image segmentation, object detection, etc. The specific content of the navigation terminal can be referred to the related description in the above embodiments, which will not be repeated here. The path-related information can include an image of a navigation site captured by the navigation terminal and destination position information. The navigation site is the real environment in which the user is located, and the destination position information can indicate the position of the destination. Specifically, in the case where the destination indicated by the destination position information is located within the shooting range of the navigation terminal, the navigation application platform can perform image segmentation and object detection on the captured image of the navigation site based on the destination in the captured image of the navigation site and the position of the navigation terminal; the navigation application platform determines a second planned path according to the image blocks obtained by image segmentation and the objects in the image blocks obtained by object detection; the navigation application platform generates control data for controlling the actions of the AR guide avatar according to the second planned path and the analysis result data; and the navigation application platform fills the control data according to the predetermined format of the interactive signaling to generate the interactive signaling.

[0055] The destination indicated by the destination position information is located in the shooting range of the terminal being navigated, i.e., the destination is in the range displayed by the first terminal and the second terminal. The navigation application platform can divide the image of the navigation site into a plurality of image blocks according to the destination and the position of the terminal being navigated. The number and size of the image blocks can be determined according to the destination and the position of the terminal being navigated. For example, the image can be divided into 3x3 image blocks, or the image can be divided into 4x3 image blocks, which are not limited herein. The object detection can determine the objects in the image, which can include obstacles and roads. The planned path needs to avoid obstacles and travel along the roads. The second planned path is a planned path from the shooting position of the terminal being navigated to the destination in the image obtained by shooting, which is determined according to the image blocks and the objects in the image blocks. The control data for controlling the AR guide figure to walk along the second planned path can be obtained in combination with the analysis result data and the second planned path. For example, if the analysis result data includes the moving speed of the user, the control data can control the AR guide figure to walk along the second planned path at the moving speed of the user, so as to match the moving speed of the user and make the navigation more in line with the needs of the user. The first terminal and the second terminal can display the destination and the AR guide figure making actions, for example, Figure 5 Another example of the navigation interface displayed by the terminal provided by the embodiments of the present application is shown in the schematic diagram as Figure 5 The navigation interface includes a real environment, which includes a building B1, a landscape model B2, a landscape model B3, and a ground B4. The navigation interface also includes a destination indication icon B5, which is integrated into the real environment with an AR guide figure C1. The AR guide figure C1 points out a direction with a finger to prompt the user of the direction in which the user should go forward. The navigation application platform fills the obtained control data into the corresponding positions in the interactive signaling and sends the interactive signaling to the VoNR+ platform. The navigation method using image segmentation and object detection is suitable for a destination that is close to the terminal being navigated, located in an indoor, underground, or other difficult-to-position site.

[0056] The navigation mode in the embodiments of the present application can include an automatic mode. In the automatic mode, the interactive signaling is automatically generated by the navigation application platform according to the analysis result data obtained by the media platform analysis processing, and the navigation process can not require the navigation guide terminal to specifically give an indication, and the AR guide figure can be displayed to the navigated terminal and the navigation guide terminal. The automatic mode can be set as a default mode, that is, after starting navigation, the automatic mode is entered by default. In the automatic mode, the automatic mode can be temporarily suspended, and the navigation guide terminal can give a precise instruction to temporarily manually control the AR guide figure. Specifically, in the case where the navigation mode is the automatic mode, a pause instruction sent by the first terminal or the second terminal is received through the VoNR+ platform; the interactive information is generated by the navigation application platform according to the pause instruction and the analysis result data, and the AR guide figure and the interactive signaling are sent to the VoNR+ platform. The pause instruction is used to indicate that the automatic mode is temporarily stopped, and the user can issue the pause instruction by operating the navigated terminal or the navigation instruction terminal. For example, Figure 6 An example of a navigation interface displayed by a terminal in the automatic mode provided by the embodiments of the present application is shown in the schematic diagram, Figure 7 An example of a navigation interface displayed by a terminal after the automatic mode is suspended provided by the embodiments of the present application is shown in the schematic diagram, Figure 6 As shown in the figure, the navigation interface in the automatic mode includes a pause control B6. If the user clicks the pause control B6, the terminal (the first terminal or the second terminal) can send a pause instruction, which can be sent to the VoNR+ platform and then sent to the navigation application platform. The navigation application platform temporarily changes the navigation mode data in the control data to data indicating the manual mode according to the pause instruction, so as to fill the control data into the interactive signaling. The VoNR+ platform receives the AR guide figure and the interactive signaling, and generates display media data according to the AR guide figure and the interactive signaling, and sends the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the navigation interface as shown in the figure, Figure 7 Figure 7 The navigation interface in the automatic mode in the suspended state includes an automatic mode control B7. If the user clicks the automatic mode control B7, the navigation mode can return to the automatic mode. In the automatic mode in the suspended state, the user of the navigation guide terminal can issue a precise body action instruction for controlling the AR guide figure through voice, such as “quick, quick”, “turn right here”, “please turn your head”, etc. The voice data can be processed by the media platform, the VoNR+ platform, the navigation application platform, etc., to generate interactive signaling including bone animation data of the AR guide figure indicating actions such as accelerating the moving speed, turning right, turning the head, etc. The VoNR+ platform fuses the AR guide figure performing the above actions with the media data of the real environment, and sends to the first terminal and the second terminal, so that the AR guide figure displayed by the first terminal and the second terminal performs the above actions. ​

[0057] The navigation mode in the embodiments of the present application can include a manual mode. In the manual mode, the user can operate the navigation guidance terminal, the navigation guidance terminal can issue corresponding instructions according to the input of the user, the navigation application platform can generate interaction signaling according to the corresponding instructions of the manual input of the user of the navigation guidance terminal, and the guiding action given by the AR road guide avatar is controlled by the user of the navigation guidance terminal. Specifically, in the case where the navigation mode is the manual mode, the VoNR+ platform can receive the manual control instructions sent by the first terminal or the second terminal, and send the manual control instructions to the navigation application platform; the navigation application platform can generate interaction signaling according to the manual control instructions, and send the AR road guide avatar and the interaction signaling to the VoNR+ platform. The user can issue the manual control instructions by operating the navigation guidance terminal, and the manual control instructions can include control instructions for controlling the AR road guide avatar to move forward, backward, left or right, or control instructions for controlling the AR road guide avatar to perform a specific action, such as a stop action, a greeting action, a reaching action, etc. For example, Figure 8 An example of a navigation interface displayed by a terminal in the manual mode provided by the embodiments of the present application is shown in the schematic diagram of Figure 8 As shown in the schematic diagram, the user can issue control instructions for controlling the AR road guide avatar to move forward, backward, left or right by operating the direction moving control B9 in the navigation interface, and the user can issue control instructions for controlling the AR road guide avatar to perform a specified precise action by operating the action selection control B8 in the navigation interface. Each small square in the action selection control B8 can indicate a precise action. The first terminal and the second terminal can not only display in portrait mode, but also display in landscape mode. Figure 9 An example of a navigation interface displayed by a terminal in the manual mode provided by the embodiments of the present application is shown in the schematic diagram of Figure 9 The functions of the action selection control B8 and the direction moving control B9 shown in the schematic diagram are the same as those of the action selection control B8 and the direction moving control B9 in Figure 8 The functions of the action selection control B8 and the direction moving control B9 shown in the schematic diagram are the same as those of the action selection control B8 and the direction moving control B9 in

[0058] In some embodiments, the first terminal or the second terminal can also give a text prompt, so that the navigation interface displayed by the first terminal and the second terminal displays the text prompt. Specifically, the text data sent by the first terminal or the second terminal can be received by the VoNR+ platform, and the text data is sent to the navigation application platform; the interactive information is generated by the navigation application platform according to the text data, and the AR road guide avatar and the interactive signaling are sent to the VoNR+ platform. The text data can represent the text input by the user. The user can input the text by operating the text input control of the navigation interface. For example, as shown in Figure 6 to Figure 9 the user can click the text input space B10 of the navigation interface, thereby inputting the text in the pop-up input box, and the first terminal or the second terminal can transmit the text data corresponding to the input text to the navigation application platform through the VoNR+ platform, and the navigation application platform fills the text data as the prompt data of the AR road guide avatar into the interactive signaling, and sends the AR road guide avatar and the interactive signaling to the VoNR+ platform. The VoNR+ platform fuses the AR road guide avatar sending the prompt information with the media data of the real environment, and sends it to the first terminal and the second terminal, so that the AR road guide avatar displayed by the first terminal and the second terminal sends the prompt information. The text data can prompt the bus route, train number, etc. For example, Figure 10 the schematic diagram of another example of the navigation interface provided by the embodiments of the present application is shown in Figure 10 the AR road guide avatar in the navigation interface sends the prompt information of "take XX bus". Through the input of the text data, the user of the navigation guidance terminal can initiate the prompt to the user of the navigation terminal in real time through the AR road guide avatar, and synchronize important information to the user of the navigation terminal, so that the prompt is more intuitive and easy to understand.

[0059] In some embodiments, the position of the AR road guide avatar can change, and it can appear in a position inconvenient for the user to operate in the navigation interface, or it can be out of the navigation interface. For this case, the position of the AR road guide avatar needs to be reset. Specifically, when the VoNR+ platform receives a reset instruction or the media platform determines that the AR road guide avatar is located outside the image displayed by the first terminal and the second terminal, the navigation application platform generates interactive signaling including position reset data indicating the position of the AR road guide avatar in the image, and sends the AR road guide avatar and the interactive signaling to the VoNR+ platform. The user can issue a reset instruction by operating the first terminal or the second terminal, and the reset instruction is used to indicate the reset of the position of the AR road guide avatar in the navigation interface, for example, the position of the AR road guide avatar is reset in the middle of the navigation interface. After the position of the AR road guide avatar in the navigation interface is reset, the planned path can be determined again, and navigation is performed. The content of determining the planned path and navigation can be referred to the related description in the above embodiments, and will not be described here. For example, Figure 6 to Figure 9As shown, the user can click the reset control B11 in the navigation interface, and the first terminal or the second terminal can send a reset instruction to the navigation application platform through the VoNR+ platform. The navigation application platform fills the position reset data indicating the reset of the position of the AR guide figure into the interaction signaling according to the reset instruction. Alternatively, when the media platform detects that the figure of the AR guide is located outside the navigation interface, a trigger instruction is sent to the navigation application platform, so that the navigation application platform generates the interaction signaling including the position reset data indicating the reset of the position of the AR guide figure. The navigation application platform sends the AR guide figure and the interaction signaling to the VoNR+ platform, and the VoNR+ platform fuses the AR guide figure after the reset position indicated by the execution control data with the media data of the real environment, and sends it to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure after the reset position.

[0060] To facilitate description, the process of remote navigation in the above embodiments is described below through specific interaction between the terminal, the VoNR+ platform, the media platform, and the navigation application platform. Figure 11 A flowchart of an example of the remote navigation process provided by the embodiments of the present application is shown in the figure, wherein the VoNR+ platform includes a VoNR+ capability network element and a VoNR+ media plane, and the terminal includes a calling terminal and a called terminal. In this example, the calling terminal is a navigation guide terminal, and the called terminal is a navigation terminal. In other scenarios, the calling terminal can also be a navigation terminal, and the called terminal can also be a navigation guide terminal. As shown in the figure, Figure 11 The remote navigation process can include steps d1 to d14.

[0061] In step d1, the calling terminal initiates a call to the VoNR+ capability network element.

[0062] In step d2, the VoNR+ capability network element initiates a call to the called terminal.

[0063] In step d3, the called terminal requests to establish a data channel connection with the VoNR+ capability network element.

[0064] In step d4, the VoNR+ capability network element feeds back to the called terminal that the connection is successful.

[0065] In step d5, the called terminal answers the call, and the call between the called terminal and the calling terminal is answered successfully through the VoNR+ capability network element.

[0066] In step d6, the called terminal sends media data to the VoNR+ media plane.

[0067] In step d7, the calling terminal initiates a remote navigation request to the navigation application platform.

[0068] In step d8, the navigation application platform issues a user agreement and a privacy policy to the VoNR+ media surface.

[0069] In step d9, the VoNR+ media surface initiates a remote navigation request, a user agreement and a privacy policy to the called terminal.

[0070] In step d10, the called terminal feeds back consent information.

[0071] In step d11, the VoNR+ media forwards media data to the media platform.

[0072] In step d12, the media platform analyzes and processes the media data, and sends analysis result data to the navigation application platform.

[0073] In step d13, the navigation application platform obtains navigation materials and control instructions according to the analysis result data, and sends them to the VoNR+ media surface. The navigation materials can include AR guide figure images, and the control instructions can be realized through control data in interactive signaling.

[0074] In step d14, the VoNR+ media surface sends display media data obtained according to the navigation materials and the control instructions to the called terminal.

[0075] The specific content of steps d1 to d14 can be referred to the related description in the above embodiments, which will not be repeated here.

[0076] The remote navigation method provided by the embodiments of the present application can provide navigation including lifelike AR guide figure images, so as to make the immersive experience of navigation more immersive, and can provide different modes such as automatic mode and manual mode for navigation guidance. In the short-distance navigation or manual mode, the positioning information of the device is not needed, so as to avoid the influence of positioning difficulties such as weak signal and building obstruction, and the navigation can also be applicable to non-planar sites, so that the user can select different modes according to different environments, and the practicability and applicability of remote navigation are improved.

[0077] The second aspect of the present application provides a remote navigation device. Figure 12 The structure diagram of the remote navigation device provided by an embodiment of the present application is shown in Figure 12 The remote navigation device 300 can include a VoNR+ module 301, a media module 302 and a navigation application module 303.

[0078] The VoNR+ module 301 can be used to obtain media data from the first terminal and / or the second terminal in the case that the first terminal and the second terminal establish a video call through the VoNR+ module, and copy and forward the media data to the media module 302.

[0079] The media module 302 can be configured to analyze the media data and obtain analysis result data, and send the analysis result data to the navigation application module 303.

[0080] The navigation application module 303 can be configured to generate an augmented reality (AR) guide figure, and generate interaction signaling according to the obtained path-related information and the analysis result data, and send the AR guide figure and the interaction signaling to the VoNR+ module 301, wherein the interaction signaling comprises control data of the AR guide figure.

[0081] The VoNR+ module 301 can also be configured to generate display media data based on the media data, the AR guide figure and the interaction signaling, and send the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure according to the display media data, and the AR guide figure performs actions according to the control data in the navigation site.

[0082] In some embodiments, the path-related information comprises positioning information of the terminal to be navigated and destination location information, and the terminal to be navigated is one of the first terminal and the second terminal. The navigation application module 303 can be specifically configured to: in a case where the destination indicated by the destination location information is beyond the shooting range of the terminal to be navigated, determine a first planned path according to the positioning information and the destination location information; generate control data according to the first planned path and the analysis result data, the control data being used to control actions of the AR guide figure; and fill the control data according to a predetermined format of the interaction signaling, to generate the interaction signaling.

[0083] In some embodiments, the path-related information comprises an image of the navigation site shot by the terminal to be navigated and destination location information, and the terminal to be navigated is one of the first terminal and the second terminal. The navigation application module 303 can be specifically configured to: in a case where the destination indicated by the destination location information is within the shooting range of the terminal to be navigated, perform image segmentation and object detection on the shot image of the navigation site based on the destination in the shot image of the navigation site and the position of the terminal to be navigated; determine a second planned path according to image blocks obtained by the image segmentation and objects obtained by object detection in the image blocks; generate control data according to the second planned path and the analysis result data, the control data being used to control actions of the AR guide figure; and fill the control data according to a predetermined format of the interaction signaling, to generate the interaction signaling.

[0084] In some embodiments, the interaction signaling comprises a signaling head part and a signaling data part, the signaling head part comprises a control bit, and the signaling data part comprises control data corresponding to the control bit.

[0085] In some examples, the signaling head part further comprises one or more of the following: a video call identifier, a data source address, a data destination address, and verification data.

[0086] In some embodiments, the control data includes one or more of the following: navigation mode data, movement instruction data of the AR guide figure, skeleton animation data of the AR guide figure, prompt data of the AR guide figure, position resetting data of the AR guide figure.

[0087] In some embodiments, the media module 302 can be specifically configured to: in a case where the media data includes voice data sent by the navigation guide terminal, perform voice analysis processing on the voice data to obtain analysis result data, the voice analysis processing including one or more of the following: noise reduction processing, feature extraction, word recognition, semantic analysis; in a case where the media data includes video data sent by the navigation terminal, perform image analysis processing on the video data to obtain analysis result data, the image analysis processing including one or more of the following: feature extraction, target detection, image segmentation, image classification, image environment analysis. The navigation terminal is one of the first terminal and the second terminal, and the navigation guide terminal is the other of the first terminal and the second terminal.

[0088] In some embodiments, the analysis result data includes one or more of the following: voice semantic data, image object type data, image object position data, image object motion data, user movement data, environmental factor data. The image object type data includes planar object type data and non-planar object type data.

[0089] In some embodiments, the VoNR+ module 301 can be further configured to: in a case where the navigation mode is an automatic mode, receive a pause instruction sent by the first terminal or the second terminal.

[0090] The navigation application module 303 can be further configured to generate interaction information according to the pause instruction and the analysis result data, and send the AR guide figure and the interaction information to the VoNR+ module 301.

[0091] In some embodiments, the VoNR+ module 301 can be further configured to: in a case where the navigation mode is a manual mode, receive a manual control instruction sent by the first terminal or the second terminal, and send the manual control instruction to the navigation application module 303.

[0092] The navigation application module 303 can be further configured to generate interaction information according to the manual control instruction, and send the AR guide figure and the interaction information to the VoNR+ module 301.

[0093] In some embodiments, the VoNR+ module 301 can be further configured to: receive text data sent by the first terminal or the second terminal, and send the text data to the navigation application module 303.

[0094] The navigation application module 303 can also be configured to generate interaction information according to the text data, and send the AR road guide figure and the interaction signaling to the VoNR+ module 301.

[0095] In some embodiments, the navigation application module 303 can also be configured to, in a case where the VoNR+ module 301 receives a reset instruction or the media module 302 determines that the AR road guide figure is located outside the images displayed by the first terminal and the second terminal, generate interaction signaling including position reset data indicating a position of resetting the position of the AR road guide figure in the image, and send the AR road guide figure and the interaction signaling to the VoNR+ module 301.

[0096] In some embodiments, the VoNR+ module 301 can also be configured to, before obtaining the media data from the first terminal and / or the second terminal, receive a call initiated by the first terminal, and initiate a call to the second terminal to establish a video call between the first terminal and the second terminal.

[0097] In some embodiments, the navigation application module 303 can also be configured to receive a remote navigation request of the first terminal or the second terminal, and in response to the remote navigation request, obtain a user authorization request, and send the user authorization request to the first terminal and / or the second terminal through the VoNR+ module 301.

[0098] The VoNR+ module 301 can be specifically configured to, if receiving the authorization consent information sent by the first terminal and / or the second terminal, obtain the media data from the first terminal and / or the second terminal.

[0099] It should be noted that the remote navigation device 300 is a device corresponding to the remote navigation method in the above-mentioned embodiments, and all implementation manners in the above-mentioned method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0100] The third aspect of the present application provides a remote navigation system, which includes a VoNR+ platform, a media platform and a navigation application platform. The specific content of the VoNR+ platform, the media platform and the navigation application platform can be referred to the related description in the above-mentioned embodiments. The remote navigation system executes the remote navigation method in the above-mentioned embodiments, and can achieve the same technical effects. To avoid repetition, it will not be described here.

[0101] The fourth aspect of the present application provides a computer readable storage medium, which stores computer program instructions. The computer program instructions are executed by a processor to implement the remote navigation method in the above embodiments and achieve the same technical effects. To avoid repetition, details are not described herein. The computer readable storage medium can include a non-transitory computer readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, without limitation.

[0102] The fifth aspect of the present application provides a computer program product, which includes a computer program. The computer program is executed by a processor to implement the remote navigation method in the above embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0103] It should be noted that each of the above-described embodiments of the present application is described in a progressive manner. Identical or similar parts among the embodiments can be cross-referenced. Each embodiment focuses on the differences from other embodiments. For the device embodiments, system embodiments, computer readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description of the method embodiments. The present application is not limited to the specific steps and structures described above and shown in the drawings. Those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application. Moreover, detailed descriptions of known methods and techniques are omitted herein for the sake of brevity.

[0104] The above describes aspects of the present application with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus enable the implementation of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. The processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It should also be understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware, or a combination of special-purpose hardware and computer instructions.

[0105] It should be understood by those skilled in the art that the above embodiments are exemplary but not limiting. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Other changed embodiments of the disclosed embodiments can be understood and implemented by those skilled in the art based on the drawings, the specification and the claims. In the claims, the term "comprising" does not exclude other devices or steps; the numerical term "one" does not exclude a plurality; the terms "first", "second" are used to designate names and not to indicate any particular order. Any reference signs in the claims should not be understood as limiting the scope of protection. The functions of multiple parts appearing in the claims can be implemented by a single hardware or software module. The fact that certain technical features appear in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A method of remote navigation, characterized in that, The application is applied to a remote navigation system, and the remote navigation system comprises a voice over new radio (VoNR+) platform, a media platform and a navigation application platform; The method comprises: In a case where a video call is established between a first terminal and a second terminal through the VoNR+ platform, media data is acquired from the first terminal and / or the second terminal through the VoNR+ platform, and the media data is copied and forwarded to the media platform; The media data comprises audio data provided by a navigation guide terminal and video data provided by a navigation terminal, and the analysis result comprises analysis result data obtained by performing voice analysis processing on the audio data and analysis result data obtained by performing image analysis processing on the video data; An augmented reality (AR) guide figure is generated through the navigation application platform, and interactive signaling is generated according to acquired path-related information and the analysis result data, and the AR guide figure and the interactive signaling are sent to the VoNR+ platform, wherein the interactive signaling comprises control data of the AR guide figure; The VoNR+ platform generates display media data based on the media data, the AR guide figure and the interactive signaling, and sends the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in a navigation site according to the display media data and perform actions according to the control data.

2. The method of claim 1, wherein, The path-related information comprises positioning information of a navigation terminal and destination location information, and the navigation terminal is one of the first terminal and the second terminal; The generation of the interactive signaling according to the acquired path-related information and the analysis result data comprises: In a case where a destination indicated by the destination location information is beyond a shooting range of the navigation terminal, a first planned path is determined according to the positioning information and the destination location information; The control data is generated according to the first planned path and the analysis result data, and the control data is used to control actions of the AR guide figure; The control data is filled in a predetermined format of the interactive signaling to generate the interactive signaling.

3. The method of claim 1, wherein, The path-related information comprises an image of a navigation site and destination location information, and the navigation terminal is one of the first terminal and the second terminal; The generation of the interactive signaling according to the acquired path-related information and the analysis result data comprises: In a case where a destination indicated by the destination location information is within a shooting range of the navigation terminal, image segmentation and object detection are performed on the shot image of the navigation site based on the destination and the position of the navigation terminal in the shot image of the navigation site; A second planned path is determined according to image blocks obtained by the image segmentation and objects obtained by object detection in the image blocks; The control data is generated according to the second planned path and the analysis result data, and the control data is used to control actions of the AR guide figure; Filling the control data according to a predetermined format of the interactive signaling, and generating the interactive signaling.

4. The method of claim 1, wherein, The interactive signaling comprises a signaling head part and a signaling data part, The signaling head part comprises control bits, and the signaling data part comprises the control data corresponding to the control bits.

5. The method of claim 4, wherein, The signaling head part further comprises one or more of the following: A video call identifier, a data source address, a data destination address, and verification data.

6. The method of claim 1, wherein, The control data comprises one or more of the following: Navigation mode data, movement instruction data of the AR guide figure, skeletal animation data of the AR guide figure, prompt data of the AR guide figure, and position resetting data of the AR guide figure.

7. The method of claim 1, wherein, The analysis and processing of the media data by the media platform to obtain analysis result data and send the analysis result data to the navigation application platform comprises In a case where the media data comprises voice data sent by a navigation guide terminal, performing voice analysis and processing on the voice data by the media platform to obtain the analysis result data, the voice analysis and processing comprising one or more of the following: noise reduction processing, feature extraction, word recognition, and semantic analysis; In a case where the media data comprises video data sent by a navigated terminal, performing image analysis and processing on the video data by the media platform to obtain the analysis result data, the image analysis and processing comprising one or more of the following: feature extraction, target detection, image segmentation, image classification, and image environment analysis. The navigated terminal is one of the first terminal and the second terminal, and the navigation guide terminal is the other of the first terminal and the second terminal.

8. The method of claim 1, wherein, The analysis result data comprises one or more of the following: Voice semantic data, image object type data, image object position data, image object motion data, user movement data, and environmental factor data. The image object type data comprises planar object type data and non-planar object type data.

9. The method of claim 1, wherein, Further comprising: In a case where the navigation mode is an automatic mode, receiving, by the VoNR+ platform, a pause instruction sent by the first terminal or the second terminal; Generating, by the navigation application platform, interactive information according to the pause instruction and the analysis result data, and sending, to the VoNR+ platform, the AR guide figure and interactive signaling.

10. The method of claim 1, wherein, Further comprising: In a case where the navigation mode is a manual mode, receiving, by the VoNR+ platform, a manual control instruction sent by the first terminal or the second terminal, and sending the manual control instruction to the navigation application platform; Generating, by the navigation application platform, interactive signaling according to the manual control instruction, and sending, to the VoNR+ platform, the AR guide figure and the interactive signaling.

11. The method of claim 1, wherein, Further comprising Receiving, by the VoNR+ platform, text data sent by the first terminal or the second terminal, and sending the text data to the navigation application platform; Generating, by the navigation application platform, interactive information according to the text data, and sending, to the VoNR+ platform, the AR guide figure and the interactive signaling.

12. The method of claim 1, wherein, Further comprising: In the case that the VoNR+ platform receives a reset instruction or the media platform determines that the AR guide figure is located outside the images displayed by the first terminal and the second terminal, the navigation application platform generates interactive signaling including position reset data indicating the position of the AR guide figure in the image, and sends the AR guide figure and the interactive signaling to the VoNR+ platform.

13. The method of claim 1, wherein, Before the media data is acquired from the first terminal and / or the second terminal by the VoNR+ platform, the method further comprises: The VoNR+ platform receives a call initiated by the first terminal, and initiates a call to the second terminal to establish a video call between the first terminal and the second terminal.

14. The method of claim 1, wherein, Before the media data is acquired from the first terminal and / or the second terminal by the VoNR+ platform, the method further comprises: The navigation application platform receives a remote navigation request of the first terminal or the second terminal, and in response to the remote navigation request, acquires a user authorization request, and sends the user authorization request to the first terminal and / or the second terminal through the VoNR+ platform; The media data is acquired from the first terminal and / or the second terminal by the VoNR+ platform, comprising: If the authorization consent information sent by the first terminal and / or the second terminal is received by the VoNR+ platform, the media data is acquired from the first terminal and / or the second terminal by the VoNR+ platform.

15. A remote navigation device, characterized in that Comprising: The enhanced new radio bearer voice (VoNR+) module is configured to acquire media data from the first terminal and / or the second terminal in the case that the first terminal and the second terminal establish a video call through the VoNR+ module, and copy and forward the media data to the media module; The media module is configured to analyze and process the media data to obtain analysis result data and send the analysis result data to the navigation application module; the media data includes audio data provided by a navigation guide terminal and video data provided by a navigated terminal; the analysis result includes analysis result data obtained by performing speech analysis processing on the audio data, and analysis result data obtained by performing image analysis processing on the video data; The navigation application module is configured to generate an augmented reality (AR) guide figure, and generate interactive signaling according to the acquired path-related information and the analysis result data, and send the AR guide figure and the interactive signaling to the VoNR+ module; the interactive signaling includes control data of the AR guide figure; The VoNR+ module is further configured to generate display media data based on the media data, the AR guide figure, and the interactive signaling, and send the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in the navigation site according to the control data based on the display media data.

16. A remote navigation system characterized by Comprising: The VoNR+ platform is used for establishing a video call between the first terminal and the second terminal, acquiring media data from the first terminal and / or the second terminal, copying and forwarding the media data to a media platform, and the media platform is used for analyzing and processing the media data to obtain analysis result data and sending the analysis result data to a navigation application platform, wherein the media data includes audio data provided by a navigation guidance terminal and video data provided by a navigated terminal, and the analysis result includes analysis result data obtained by performing voice analysis processing on the audio data and analysis result data obtained by performing image analysis processing on the video data. The navigation application platform is used for generating an augmented reality (AR) guide figure, generating interaction signaling according to the acquired path-related information and the analysis result data, and sending the AR guide figure and the interaction signaling to the VoNR+ module, wherein the interaction signaling includes control data of the AR guide figure. The VoNR+ platform is further used for generating display media data based on the media data, the AR guide figure, and the interaction signaling, and sending the display media data to the first terminal and the second terminal, so that the first terminal and the second terminal display the AR guide figure in a navigation site according to the display media data and perform actions according to the control data. The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the remote navigation method in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer program is executed by a processor to implement the remote navigation method in any one of claims 1 to 14.

18. A computer program product, characterised in that, ​

Images