A communication method, terminal and edge computing network device

Abstract

The present invention provides a communication method, a terminal and an edge computing network device, which belong to the field of communication technology, wherein the communication method applied to a first terminal includes: when the first terminal and a second terminal conduct a video call based on an operator call network, a first AR image is displayed on a video call interface, and the first AR image is an AR image determined according to information related to the second terminal. The present invention adds an AR function on the basis of a video call based on an operator call architecture to realize an AR video call function, that is, adding an AR video call function to an existing video call, which can bring a brand new call experience to users.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a communication method, a terminal and an edge computing network device. Background Art

[0002] The video call function in existing operator calls mainly transmits video images and voice to the other party in real time through the front or rear camera, realizing a simple video call function without adding other functions. Summary of the invention

[0003] In view of this, the present invention provides a communication method, a terminal and an edge computing network device, which are used to solve the problem of the single function of the video call function of the current operator calls.

[0004] In order to solve the above technical problem, in a first aspect, the present invention provides a communication method, which is applied to a first terminal, and the method includes:

[0005] When the first terminal and the second terminal conduct a video call based on the operator call network, a first AR image is displayed on the video call interface, and the first AR image is an AR image determined according to information related to the second terminal.

[0006] Optionally, before displaying the first AR image on the video call interface, the method further includes:

[0007] Receiving information of the first AR image sent by the edge computing network device;

[0008] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0009] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0010] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0011] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0012] In a second aspect, the present invention further provides a communication method, which is applied to an edge computing network device, the edge computing network device is an edge computing network device currently accessed by a second terminal, and the second terminal and the first terminal conduct a video call based on an operator call network, the method comprising:

[0013] Determine information of the first AR image according to information related to the second terminal;

[0014] The information of the first AR image is sent to the first terminal.

[0015] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0016] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0017] The determining the information of the first AR image according to the information related to the second terminal includes:

[0018] The AR navigation is planned according to the current location information of the first terminal and the second terminal, the map information and the current location information of the first terminal.

[0019] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0020] The determining the information of the first AR image according to the information related to the second terminal includes:

[0021] Matching the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0022] The AR navigation is planned according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

[0023] Optionally, the method further includes:

[0024] Sending information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0025] Optionally, before sending the information of the second AR image to the second terminal, the method further includes:

[0026] Acquire a first video image sent by the first terminal;

[0027] extracting a user portrait of the first terminal from the first video image;

[0028] The user portrait of the first terminal is superimposed on the real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0029] In a third aspect, the present invention further provides a communication method, applied to a second terminal, the method comprising:

[0030] When receiving an AR video call request based on the operator's call network sent by the first terminal, obtaining information related to the second terminal;

[0031] Send information related to the second terminal.

[0032] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0033] Optionally, the method further includes:

[0034] Receiving information of a second AR image sent by the edge computing network device;

[0035] The second AR image is displayed on the video call interface, and the second AR image includes the real-time moving position of the first terminal and the path navigation from the first terminal to the position of the second terminal.

[0036] In a fourth aspect, the present invention further provides a first terminal, comprising:

[0037] The AR display module is used to display a first AR image on the video call interface when the first terminal and the second terminal conduct a video call based on the operator's call network, where the first AR image is an AR image determined based on information related to the second terminal.

[0038] Optionally, the first terminal further includes:

[0039] A receiving module, configured to receive information of the first AR image sent by an edge computing network device;

[0040] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0041] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0042] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0043] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0044] In a fifth aspect, the present invention further provides an edge computing network device, the edge computing network device is an edge computing network device currently accessed by a second terminal, the second terminal and the first terminal conduct a video call based on an operator call network, and the edge computing network device includes:

[0045] A determination module, configured to determine information of the first AR image according to information related to the second terminal;

[0046] The first sending module is configured to send information about the first AR image to the first terminal.

[0047] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0048] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0049] The determination module comprises:

[0050] The first planning unit is used to plan the AR navigation according to the current location information of the first terminal and the second terminal, map information and the current location information of the first terminal.

[0051] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0052] The determination module comprises:

[0053] A positioning unit, used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0054] The second planning unit is used to plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

[0055] Optionally, the edge computing network device further includes:

[0056] The third sending module is used to send information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0057] Optionally, the edge computing network device further includes:

[0058] A first video image acquisition module, used to acquire a first video image sent by the first terminal;

[0059] An extraction module, configured to extract a user portrait of the first terminal from the first video image;

[0060] The superposition module is used to superimpose the user portrait of the first terminal onto the real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0061] In a sixth aspect, the present invention further provides a second terminal, including:

[0062] An acquisition module, configured to acquire information related to the second terminal upon receiving an AR video call request based on an operator call network sent by the first terminal;

[0063] The second sending module is used to send information related to the second terminal.

[0064] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0065] Optionally, the second terminal further includes:

[0066] A second AR image receiving module, configured to receive information of a second AR image sent by an edge computing network device;

[0067] The second AR image display module is used to display the second AR image on the video call interface, where the second AR image includes the real-time moving position of the first terminal and the path navigation from the first terminal to the position of the second terminal.

[0068] In a seventh aspect, the present invention further provides a first terminal, comprising: a transceiver and a processor;

[0069] The processor is used to display a first AR image on a video call interface when the first terminal and the second terminal conduct a video call based on an operator call network, where the first AR image is an AR image determined based on information related to the second terminal.

[0070] Optionally, the transceiver is used to receive information of the first AR image sent by an edge computing network device;

[0071] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0072] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0073] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0074] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0075] In an eighth aspect, the present invention further provides an edge computing network device, the edge computing network device is an edge computing network device currently accessed by a second terminal, the second terminal and the first terminal conduct a video call based on an operator call network, the edge computing network device comprising: a transceiver and a processor;

[0076] The processor is configured to determine information about the first AR image according to information related to the second terminal;

[0077] The transceiver is configured to send information about the first AR image to the first terminal.

[0078] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0079] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0080] The processor is used to plan the AR navigation according to the current location information of the first terminal and the second terminal, the map information and the current location information of the first terminal.

[0081] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0082] The processor is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0083] The processor is further used to plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

[0084] Optionally, the transceiver is further used to send information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0085] Optionally, the transceiver is further used to obtain a first video image sent by the first terminal;

[0086] The processor is further configured to extract a user portrait of the first terminal from the first video image;

[0087] The processor is further configured to superimpose a user portrait of the first terminal onto a real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0088] In a ninth aspect, the present invention further provides a second terminal, comprising: a transceiver and a processor;

[0089] The processor is configured to obtain information related to the second terminal when receiving an AR video call request based on an operator call network sent by the first terminal;

[0090] The transceiver is used to send information related to the second terminal.

[0091] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0092] Optionally, the transceiver is further used to receive information of a second AR image sent by the edge computing network device;

[0093] The processor is further configured to display the second AR image on the video call interface, where the second AR image includes the real-time moving position of the first terminal and the path navigation from the first terminal to the position of the second terminal.

[0094] In the tenth aspect, the present invention also provides a first terminal, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the steps in any one of the above-mentioned communication methods applied to the first terminal are implemented.

[0095] In the eleventh aspect, the present invention also provides an edge computing network device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, it implements any step in the above-mentioned communication method applied to the edge computing network device.

[0096] In the twelfth aspect, the present invention also provides a second terminal, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the steps in any one of the above-mentioned communication methods applied to the second terminal are implemented.

[0097] In a thirteenth aspect, the present invention further provides a readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in any one of the above-mentioned communication methods.

[0098] The beneficial effects of the above technical solution of the present invention are as follows:

[0099] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 This is a flow chart of a communication method in Embodiment 1 of the present invention;

[0101] Figure 2 This is a flow chart of a communication method in Embodiment 2 of the present invention;

[0102] Figure 3 A schematic diagram of a network architecture for implementing an embodiment of the present invention;

[0103] Figure 4 This is a flow chart of a communication method in Embodiment 3 of the present invention;

[0104] Figure 5 This is a schematic diagram of the structure of a first terminal in Embodiment 4 of the present invention;

[0105] Figure 6 This is a schematic diagram of the structure of an edge computing network device in Embodiment 5 of the present invention;

[0106] Figure 7 This is a schematic diagram of the structure of a second terminal in Embodiment 6 of the present invention;

[0107] Figure 8 This is a schematic diagram of the structure of a first terminal in Embodiment 7 of the present invention;

[0108] Fig. 9 This is a schematic diagram of the structure of an edge computing network device in Embodiment 8 of the present invention;

[0109] Fig.10 This is a schematic diagram of the structure of a second terminal in Embodiment 9 of the present invention;

[0110] Fig.11 This is a schematic diagram of the structure of a first terminal in Embodiment 10 of the present invention;

[0111] Fig.12 This is a schematic diagram of the structure of an edge computing network device in Embodiment 11 of the present invention;

[0112] Fig.13 This is a schematic diagram of the structure of a second terminal in Embodiment 12 of the present invention. DETAILED DESCRIPTION

[0113] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0114] See also Figure 1 , Figure 1 A flow chart of a communication method provided in Embodiment 1 of the present invention is provided. The method is applied to a first terminal and comprises the following steps:

[0115] Step 11: When the first terminal and the second terminal conduct a video call based on the operator's call network, a first augmented reality (AR) image is displayed on the video call interface, where the first AR image is an AR image determined based on information related to the second terminal.

[0116] In the embodiment of the present invention, the video call can rely on the mobile operator call (network) architecture, and does not rely on the real-time communication (network) architecture of the Internet. The operator call network can specifically be, but is not limited to, Voice over Long-Term Evolution (VoLTE), such as Voice over New Radio (VoNR) in the fifth generation mobile communication technology (5G), or a new operator call network proposed in the sixth generation mobile communication technology.

[0117] The information related to the second terminal includes information input by a user of the second terminal, location information of the second terminal, and / or information collected by the second terminal.

[0118] With the commercial use of 5G networks, AR (video) calls have a natural pipeline advantage for operators. While providing data transmission channels, they can also provide additional AR functions. AR functions include but are not limited to AR special effects, AR navigation, etc. When implementing the AR navigation function, the first AR image displayed on the video call interface can be an AR navigation image.

[0119] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0120] The above communication method is described below with examples.

[0121] Optionally, before displaying the first AR image on the video call interface, the method further includes:

[0122] Receiving information of the first AR image sent by the edge computing network device;

[0123] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0124] In the embodiment of the present application, the first AR image can be determined by the edge side. That is to say, the AR video call function based on the operator call network in the embodiment of the present application can be implemented in combination with the edge side, moving the processing capability to the edge side, reducing the processing capability requirements of the terminal device, and achieving better video processing effects.

[0125] In an embodiment of the present invention, when deploying AR edge computing processing capabilities, the AR capabilities can be deployed in the IP Multimedia Subsystem (IMS) network element of VoLTE. When the video call service is turned on, the video stream and the AR sensor data stream are first processed by the edge computing platform (MEC Platform, MEP) in the IMS network element, the original video stream is combined with the AR data, the AR video stream is output, and then the video stream is sent to the other party through the VoLTE SBC node.

[0126] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0127] In an embodiment of the present invention, AR navigation is combined with video calls. For example, an AR video navigation function is added to an existing VoLTE video call architecture, which provides a new application scenario for video calls and embodies a new application of AR calls.

[0128] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0129] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0130] In some specific implementations, when the first terminal makes the above-mentioned video call with the second terminal, if the user of the first terminal wants to go to the location of the user of the second terminal, the first terminal can initiate an AR video call. For example, the user of the first terminal can click the AR video call option on the video call interface to initiate an AR video call to the second terminal. Then the edge side can obtain the current location information of the first terminal and the current location information of the second terminal through the VoLTE network, plan the path information from the current location of the first terminal to the current location of the second terminal for the user of the first terminal, combine the path information with the map information, and send it to the first terminal in the form of a video stream. Specifically, the edge side can send the video stream to the VoLTE session border controller (Session Border Control, SBC) network element, and send the navigation video stream to the first terminal through the VoLTESBC network element, which is updated in real time according to the location information. That is to say, in an embodiment of the present invention, when a first terminal conducts a video call with a second terminal, the video call interface of the first terminal can display path navigation from the current location of the first terminal to the current location of the second terminal, but the path navigation is not based on the navigation application or map application on the first terminal, but is planned in real time by the edge side according to the locations of the first terminal and the second terminal, and sent to the first terminal in the form of a video stream through the call network.

[0131] The current location information of the first terminal may be location information located by the first terminal through GPS, and the current location information of the second terminal is also the same. The map information may be similar to maps provided by some existing map software, such as a common two-dimensional map.

[0132] Of course, the user of the first terminal can also directly initiate an AR video call to the second terminal without having to first initiate an existing video call and then initiate an AR video call. For example, when the user of the first terminal initiates a video call to the second terminal, he can directly select the AR call mode.

[0133] The above-mentioned method of determining path navigation based on map information, the current location information of the second terminal and the current location information of the first terminal can be applied to the scenario where the first terminal is not within the coverage of the edge computing network device (the edge computing network device currently connected to the second terminal), that is, it can be applied to the scenario where the first terminal has not yet connected to the edge computing network device. For example, user A (the user of the first terminal) goes to the high-speed rail station to pick up user B (the user of the second terminal). At this time, user B is already in the high-speed rail station, and an edge computing platform is deployed on the VoLTE call network covering the high-speed rail station. User B accesses the edge computing platform by accessing the VoLTE call network of the high-speed rail station, but because user A has not yet arrived at the high-speed rail station, he is not currently connected to the edge computing platform. At this time, the edge computing platform will obtain the current location information of the first terminal and the second terminal, and then plan a path navigation for the first terminal to the current location of the second terminal in combination with the map information.

[0134] The surrounding environment information collected by the second terminal includes image information (such as a panoramic image) and / or depth information of the surrounding environment, and may also include other information, so that the edge side can obtain the precise location of the second terminal by matching this information with the three-dimensional model map stored on the edge side. The three-dimensional environment information between the current location of the first terminal and the current location of the second terminal is obtained based on the three-dimensional model map stored on the edge side.

[0135] Specifically, the edge side can pre-store a three-dimensional model map of its coverage area, which can be obtained through information collected on the spot (including image information and / or depth information). For example, if the edge computing network device covers a high-speed railway station, the three-dimensional information of the high-speed railway station is collected in advance, and then a three-dimensional model map is made and stored in the edge computing network device. Of course, the edge computing network device can also cover squares, shopping malls, airports, etc.

[0136] Among them, the current location information of the first terminal can be the location information of the first terminal located by GPS, and then sent to the edge side through the VoLTE network. The path navigation determined according to the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal and the current location information of the first terminal is three-dimensional map navigation. However, this three-dimensional map navigation is not based on the navigation application or map application on the first terminal, but is obtained by the edge side based on its own three-dimensional model map, and then sent to the first terminal in the form of a video stream.

[0137] In other optional specific implementations, the user of the first terminal can also switch the front camera to the rear camera, and then combine the path navigation information sent by the edge side with the real scene captured by the rear camera to form AR navigation. For example, the rear camera of the first terminal collects the image of the road in front of the first terminal in real time, and then superimposes the collected image of the road in front with the image of the AR navigation sent by the edge side, and displays it in the video call interface.

[0138] The above-mentioned method of determining path navigation based on the three-dimensional environmental information between the current location of the first terminal and the current location of the second terminal, the surrounding environmental information of the second terminal, and the current location information of the first terminal can be applied to the scenario where the first terminal and the second terminal are both in the coverage range of the edge side network device, or the scenario where the first terminal and the second terminal are both connected to the edge side network device. Applicable application scenarios: In crowded and complex environments such as stations, shopping malls, squares or airports where the user's exact geographic location cannot be located through GPS, user A needs to reach the location of user B. Then A and B can talk through AR, and the rear camera of user B scans the three-dimensional environmental information of B's ​​location, and then uploads it to the edge side. The edge side matches the three-dimensional environmental information uploaded by the user with the pre-stored three-dimensional map information to obtain the precise spatial geographic location of user B, and plans AR navigation between AB.

[0139] Optionally, the video call interface of the first terminal may also include a switch control, which is used to switch the display of the video call interface. If the user of the first terminal wants to see the AR video screen, the AR video screen, such as the first AR image, can be displayed on the video call interface through the switch control. If the user wants to see the normal video call interface, the normal video call interface can be displayed on the video call interface through the switch control, such as the video screen captured by the front camera of the second terminal.

[0140] When the user of the first terminal or the user of the second terminal hangs up the video call, the AR navigation service ends and the VoLTE network releases the transmission channel.

[0141] See also Figure 2 , Figure 2 : is a flow diagram of a communication method provided in Embodiment 2 of the present invention, the method is applied to an edge computing network device, the edge computing network device is an edge computing network device currently accessed by a second terminal, the second terminal and the first terminal conduct a video call based on an operator call network, the method comprises the following steps:

[0142] Step 21: Determine information of the first AR image according to information related to the second terminal;

[0143] Specifically, when the first terminal conducts a video call with the second terminal based on the operator's call network, the first terminal can initiate an AR video call. After the user of the second terminal passes the AR video call request, the second terminal will send a message to the edge side to instruct to turn on the AR video call function, or the second terminal will send an AR video call response to the first terminal. The edge side obtains the information that the AR video call is turned on between the first terminal and the second terminal based on the response, and then the edge side determines the information of the first AR image based on the relevant information of the second terminal.

[0144] Of course, the video call currently being conducted between the second terminal and the first terminal may be an AR video call.

[0145] The information related to the second terminal includes information input by a user of the second terminal, location information of the second terminal, and / or information collected by the second terminal.

[0146] Step 22: Send information of the first AR image to the first terminal.

[0147] Among them, an edge computing platform is deployed on the edge computing network device, and the edge computing network device can also be called an edge computing node, or simply referred to as the edge side.

[0148] For architectural design, see Figure 3 , an edge computing platform can be built in the complex environment of airports and stations. When a user makes a VoLTE video call, if the user does not select an AR video call, the video stream flows through the Multi-access Edge Computing (MEC) platform without any processing, and is transmitted to the VoLTE SBC node via the edge computing network and then sent to the other party; when the user selects an AR video call, the video stream is processed by the MEC platform, then goes to the VoLTE SBC node, and the processed AR video stream is sent to the other party through the VoLTE SBC node.

[0149] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0150] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0151] The embodiment of the present invention provides an AR video navigation solution based on VoLTE video calling.

[0152] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0153] The determining the information of the first AR image according to the information related to the second terminal includes:

[0154] The AR navigation is planned according to the current location information of the first terminal and the second terminal, the map information and the current location information of the first terminal.

[0155] Of course, in other optional specific implementations, when it is determined that the first terminal has not yet accessed the edge computing network device, the AR navigation may be planned according to the current location information of the first terminal and the second terminal, the map information, and the current location information of the first terminal. That is, it is necessary to determine whether the first terminal is connected to the edge computing network device.

[0156] For example, user A (user of the first terminal) goes to the high-speed railway station to pick up user B (user of the second terminal). At this time, user B is already in the high-speed railway station, and the edge computing platform is deployed on the VoLTE call network covering the high-speed railway station. User B accesses the edge computing platform by accessing the VoLTE call network of the high-speed railway station, but because user A has not yet arrived at the high-speed railway station, he is not currently connected to the edge computing platform. At this time, the edge computing platform will obtain the current location information of the first terminal and the second terminal, and then plan a path navigation for the first terminal to the current location of the second terminal in combination with the map information.

[0157] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0158] The determining the information of the first AR image according to the information related to the second terminal includes:

[0159] Matching the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0160] That is, the edge side analyzes the surrounding environment information uploaded by the second terminal, matches it with the three-dimensional model map of the three-dimensional environment of the entire edge side stored on the edge side, and determines the precise position of the second terminal in the dense space;

[0161] The AR navigation is planned according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the first terminal and the second terminal.

[0162] In an embodiment of the present invention, the edge side renders the precise location (or exact location) of the second terminal in a three-dimensional model map, plans a specific path from the location of the first terminal to the location of the second terminal, and sends it to the first terminal in the form of a video stream. At the same time, the video call interface of the second terminal is displayed as the real-time line of the first terminal.

[0163] In an embodiment of the present invention, an AR function is added to the existing video call solution based on the operator's call network, and an edge computing solution is adopted. Specifically, functions such as 3D modeling and AR rendering are implemented on the edge computing platform.

[0164] The surrounding environment information collected by the second terminal includes image information and / or depth information of the surrounding environment, and of course may also include other information, so that the edge side can obtain the precise location of the second terminal by matching this information with the three-dimensional model map stored on the edge side. The three-dimensional environment information between the current location of the first terminal and the current location of the second terminal is obtained based on the three-dimensional model map stored on the edge side.

[0165] Specifically, the edge side can pre-store a three-dimensional model map of its coverage area, which can be obtained through information collected on the spot (including image information and / or depth information). For example, if the edge computing network device covers a high-speed railway station, the three-dimensional information of the high-speed railway station is collected in advance, and then a three-dimensional model map is made and stored in the edge computing network device. Of course, the edge computing network device can also cover squares, shopping malls, airports, etc.

[0166] Among them, the current location information of the first terminal can be the location information of the first terminal located by GPS, and then sent to the edge side through the VoLTE network. The path navigation determined according to the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal and the current location information of the first terminal is three-dimensional map navigation. However, this three-dimensional map navigation is not based on the navigation application or map application on the first terminal, but is obtained by the edge side based on its own three-dimensional model map, and then sent to the first terminal in the form of a video stream.

[0167] In other optional specific implementations, when it is determined that the first terminal has been connected to the edge computing network device, the surrounding environment information collected by the second terminal is matched with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; the AR navigation is planned based on the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal. That is, it is necessary to determine whether the first terminal is connected to the edge computing network device.

[0168] For example, user A (user of the first terminal) goes to the high-speed railway station to pick up user B (user of the second terminal). At this time, user B is already in the high-speed railway station, and an edge computing platform is deployed on the VoLTE call network covering the high-speed railway station. User B accesses the edge computing platform by accessing the VoLTE call network of the high-speed railway station, and user A has arrived at the high-speed railway station, so he is currently connected to the edge computing platform. At this time, the edge computing platform will match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the first terminal and the second terminal, the AR navigation is planned.

[0169] Optionally, the edge side can determine whether the first terminal has been connected when receiving the spatial positioning and rendering tasks. Specifically, the first terminal initiates an AR video navigation request to the second terminal. After the second terminal accepts the request, the edge side will receive the spatial positioning task for locating the second terminal and the rendering task for rendering the position of the second terminal in the three-dimensional model map.

[0170] Optionally, the method further includes:

[0171] Sending information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0172] In the embodiment of the present invention, when the edge side sends the AR navigation video stream to the first terminal, it also sends the real-time moving map location information of the first terminal to the second terminal in the form of a video stream and updates it in real time.

[0173] Further optionally, before sending the information of the second AR image to the second terminal, the method further includes:

[0174] Acquire a first video image sent by the first terminal; the first video image may be a video image captured by a front camera of the first terminal;

[0175] extracting a user portrait of the first terminal from the first video image;

[0176] The user portrait of the first terminal is superimposed on the real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0177] Specifically, the edge computing platform obtains the video stream collected and uploaded by the front camera of the first terminal through the VoLTE SBC, extracts the portrait information of the user of the first terminal, superimposes the portrait information of the user of the first terminal on the real-time navigation map, and sends the real-time mobile map location information of the user of the first terminal to the user of the second terminal in the form of a video stream.

[0178] When the user of the first terminal or the user of the second terminal hangs up the video call, the AR navigation service ends and the VoLTE network releases the transmission channel.

[0179] The embodiments of the present invention upgrade the existing call scenarios and call technologies, upgrade the inherent call scenarios and VoLTE call technologies, and provide a new AR call solution based on the VoLTE call system (or VoLTE call network). When users use VoLTE for video calls, they can use the AR function, which shows users a new call method under the 5G network.

[0180] The embodiment of the present invention provides a technical solution corresponding to the above-mentioned embodiment 1, having the same inventive concept, and can achieve the same technical effect. For details, please refer to the above-mentioned embodiment 1, which will not be repeated here.

[0181] See also Figure 4 , Figure 4 : is a flow chart of a communication method provided in Embodiment 3 of the present invention, the method is applied to a second terminal, and the method comprises the following steps:

[0182] Step 41: upon receiving an AR video call request based on an operator's call network sent by a first terminal, obtaining information related to the second terminal;

[0183] The information related to the second terminal includes information input by a user of the second terminal, location information of the second terminal, and / or information collected by the second terminal, etc. The specific acquisition method may be acquisition by camera, GPS positioning, touch screen, microphone, etc., which is not limited here.

[0184] Specifically, the second terminal may receive the AR video call request when conducting a video call based on the operator's call network with the first terminal, or may directly receive the AR video call request based on the operator's call network initiated by the first terminal.

[0185] Step 42: Send the information related to the second terminal. Specifically, the information may be sent to the first terminal via the VoLTE network or to the edge side.

[0186] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0187] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0188] In an embodiment of the present invention, the surrounding environment information of the second terminal includes image information (such as a panoramic image) and / or depth information of the surrounding environment, and of course may also include other information, so that the edge side can obtain the precise location of the second terminal by matching this information with the three-dimensional model map stored on the edge side.

[0189] For example, after the second terminal takes a panoramic image of its surroundings, it uploads the panoramic image to the edge computing platform at its location (i.e., the edge computing network device to which the second terminal is connected), and the edge computing platform (i.e., the edge side) calculates the precise location of the second terminal.

[0190] That is to say, after obtaining the information related to the second terminal, the second terminal sends the information related to the second terminal to the edge side (ie, the edge computing network device).

[0191] One application scenario is: user A (user of the first terminal) and user B (user of the second terminal) are making a video call based on the operator's call network. User A uses the AR video navigation option on the video call interface to initiate AR video navigation. After user B accepts the AR video navigation request, the rear camera of the second terminal is turned on, and prompts user B to take a panoramic image of B's ​​location. In addition, after detecting that the panoramic image has been taken, the second terminal can prompt the user to switch back to the front camera, or automatically turn on the front camera and turn off the rear camera.

[0192] Optionally, the method further includes:

[0193] Receiving information of a second AR image sent by the edge computing network device;

[0194] The second AR image is displayed on the video call interface, and the second AR image includes the real-time moving position of the first terminal and the path navigation from the first terminal to the position of the second terminal.

[0195] Furthermore, the second AR image also includes a portrait of a user of the first terminal in a video image captured by a front camera of the first terminal, and the position of the portrait of the user of the first terminal in the second AR image represents a real-time position of the user of the first terminal.

[0196] Of course, the video call interface may also include a switch control, which is used to switch the display of the video call interface. If the user of the second terminal wants to see the AR video screen, the AR video screen can be displayed on the video call interface through the switch control, such as the second AR image. If the user wants to see the normal video call interface, the normal video call interface can be displayed on the video call interface through the switch control, such as the video screen captured by the front camera of the first terminal.

[0197] When the user of the first terminal or the user of the second terminal hangs up the video call, the AR navigation service ends and the VoLTE network releases the transmission channel.

[0198] The embodiments of the present invention provide a technical solution corresponding to the above-mentioned embodiments 1 and 2, having the same inventive concept, and can achieve the same technical effect. For details, please refer to the above-mentioned embodiments 1 and 2, which will not be repeated here.

[0199] The following example illustrates the application of the embodiment of the present invention.

[0200] User A (user of the first terminal) goes to the high-speed railway station to pick up user B (user of the second terminal). At this time, user B is already in the high-speed railway station, but is not sure about the specific location of the high-speed railway station. User A and user B are using VoLTE video call for video communication. The current video interface of user A and user B shows the video screen captured by the front camera. User A initiates an AR navigation request to user B, user B accepts the request, user B turns on the rear camera and scans the surrounding environment. When user A has not yet arrived at the high-speed railway station, user A switches the display of the video call interface to the AR navigation interface. The starting position of the navigation is the position of user A, and the end position is the high-speed railway station where user B is located. At this time, the video call interface of user B is also synchronously updated to the navigation image of user A. When user A arrives at the high-speed rail station, user A and user B belong to the same edge computing (network) platform (for example, the high-speed rail station edge computing platform). After the edge computing platform detects the access of user A, it renders the precise location (or exact location) of user B in the three-dimensional map model of the high-speed rail station, plans the specific path from the location of user A to the location of user B, and sends it to user A in the form of a video stream. At the same time, the call interface of user B displays the real-time route of user A.

[0201] See also Figure 5 , Figure 5 : is a schematic diagram of the structure of a first terminal provided by Embodiment 4 of the present invention. The first terminal 50 includes:

[0202] The AR display module 51 is used to display a first AR image on the video call interface when the first terminal and the second terminal conduct a video call based on the operator's call network. The first AR image is an AR image determined according to information related to the second terminal.

[0203] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0204] Optionally, the first terminal 50 further includes:

[0205] A receiving module, configured to receive information of the first AR image sent by an edge computing network device;

[0206] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0207] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0208] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0209] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0210] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 1, so it will not be described in detail here. Please refer to the above-mentioned embodiment 1 for details.

[0211] See also Figure 6 , Figure 6 : is a structural diagram of an edge computing network device provided in Embodiment 5 of the present invention, the edge computing network device is an edge computing network device currently accessed by the second terminal, the second terminal and the first terminal conduct a video call based on the operator call network, and the edge computing network device 60 includes:

[0212] A determination module 61, configured to determine information of the first AR image according to information related to the second terminal;

[0213] The first sending module 62 is configured to send information about the first AR image to the first terminal.

[0214] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0215] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0216] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0217] The determination module 61 includes:

[0218] The first planning unit is used to plan the AR navigation according to the current location information of the first terminal and the second terminal, map information and the current location information of the first terminal.

[0219] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0220] The determination module 61 includes:

[0221] A positioning unit, used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0222] The second planning unit is used to plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

[0223] Optionally, the edge computing network device 60 further includes:

[0224] The third sending module is used to send information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0225] Optionally, the edge computing network device 60 further includes:

[0226] A first video image acquisition module, used to acquire a first video image sent by the first terminal;

[0227] An extraction module, configured to extract a user portrait of the first terminal from the first video image;

[0228] The superposition module is used to superimpose the user portrait of the first terminal onto the real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0229] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 2, so it will not be described in detail here. Please refer to the above-mentioned embodiment 2 for details.

[0230] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of a second terminal provided in Embodiment 6 of the present invention. The second terminal 70 includes:

[0231] The acquisition module 71 is configured to acquire information related to the second terminal upon receiving an AR video call request based on an operator call network sent by the first terminal;

[0232] The second sending module 72 is used to send information related to the second terminal.

[0233] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0234] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0235] Optionally, the second terminal 70 further includes:

[0236] A second AR image receiving module, configured to receive information of a second AR image sent by an edge computing network device;

[0237] The second AR image display module is used to display the second AR image on the video call interface, where the second AR image includes the real-time moving position of the first terminal and the path navigation from the first terminal to the position of the second terminal.

[0238] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 3, so it will not be described here in detail. Please refer to the above-mentioned embodiment 3 for details.

[0239] See also Figure 8 , Figure 8 is a structural diagram of a first terminal provided by Embodiment 7 of the present invention, the first terminal 80 includes: a transceiver 81 and a processor 82;

[0240] The processor 82 is configured to display a first AR image on a video call interface when the first terminal and the second terminal conduct a video call based on an operator call network, where the first AR image is an AR image determined based on information related to the second terminal.

[0241] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0242] Optionally, the transceiver 81 is used to receive information of the first AR image sent by an edge computing network device;

[0243] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0244] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0245] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0246] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0247] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 1, so it will not be described in detail here. Please refer to the above-mentioned embodiment 1 for details.

[0248] See also Fig. 9 , Fig. 9 1 is a schematic diagram of the structure of an edge computing network device provided in Embodiment 8 of the present invention, wherein the edge computing network device is an edge computing network device currently accessed by a second terminal, and the second terminal and the first terminal conduct a video call based on an operator call network, and the edge computing network device 90 includes: a transceiver 91 and a processor 92;

[0249] The processor 92 is configured to determine information of the first AR image according to information related to the second terminal;

[0250] The transceiver 91 is configured to send information of the first AR image to the first terminal.

[0251] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0252] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0253] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0254] The processor 92 is used to plan the AR navigation according to the current location information of the first terminal and the second terminal, the map information and the current location information of the first terminal.

[0255] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0256] The processor 92 is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0257] The processor 92 is further configured to plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

[0258] Optionally, the transceiver 91 is further configured to send information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0259] Optionally, the transceiver 91 is further configured to obtain a first video image sent by the first terminal;

[0260] The processor 92 is further configured to extract a user portrait of the first terminal from the first video image;

[0261] The processor 92 is further configured to superimpose a user portrait of the first terminal onto the real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0262] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 2, so it will not be described in detail here. Please refer to the above-mentioned embodiment 2 for details.

[0263] See also Fig.10 , Fig.10 1 is a schematic diagram of a structure of a second terminal provided by Embodiment 9 of the present invention. The second terminal 100 includes: a transceiver 101 and a processor 102;

[0264] The processor 102 is configured to obtain information related to the second terminal when receiving an AR video call request based on an operator call network sent by the first terminal;

[0265] The transceiver 101 is used to send information related to the second terminal.

[0266] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0267] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0268] Optionally, the transceiver 101 is further configured to receive information of a second AR image sent by an edge computing network device;

[0269] The processor 102 is further configured to display the second AR image on the video call interface, where the second AR image includes the real-time moving position of the first terminal and path navigation from the first terminal to the position of the second terminal.

[0270] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 3, so it will not be described here in detail. Please refer to the above-mentioned embodiment 3 for details.

[0271] See also Fig.11 , Fig.11 1 is a schematic diagram of the structure of a first terminal provided in Embodiment 10 of the present invention. The first terminal 110 includes a processor 111, a memory 112, and a program stored in the memory 112 and executable on the processor 111; when the processor 111 executes the program, the following steps are implemented:

[0272] When the first terminal and the second terminal conduct a video call based on the operator call network, a first AR image is displayed on the video call interface, and the first AR image is an AR image determined according to information related to the second terminal.

[0273] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0274] Optionally, the processor 111 may further implement the following steps when executing the program:

[0275] Before displaying the first AR image on the video call interface, the method further includes:

[0276] Receiving information of the first AR image sent by the edge computing network device;

[0277] The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

[0278] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0279] Optionally, the information related to the second terminal includes current location information of the second terminal, and the AR navigation is a path navigation determined according to map information, the current location information of the second terminal, and the current location information of the first terminal; or,

[0280] The information related to the second terminal includes the surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined based on the three-dimensional environment information between the current position of the first terminal and the current position of the second terminal, the surrounding environment information of the second terminal and the current position information of the first terminal.

[0281] The specific working process of the embodiment of the present invention is consistent with that in the above method embodiment 1, so it will not be repeated here. Please refer to the description of the method steps in the above method embodiment 1 for details.

[0282] See also Fig.12 , Fig.12 1 is a schematic diagram of the structure of an edge computing network device provided in the eleventh embodiment of the present invention, the edge computing network device is an edge computing network device currently accessed by a second terminal, the second terminal and the first terminal conduct a video call based on an operator call network, the edge computing network device 120 includes a processor 121, a memory 122, and a program stored in the memory 122 and executable on the processor 121; when the processor 121 executes the program, the following steps are implemented:

[0283] Determine information of the first AR image according to information related to the second terminal;

[0284] The information of the first AR image is sent to the first terminal.

[0285] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0286] Optionally, the first AR image is an AR navigation image, and the AR navigation is a path navigation for moving the first terminal to a location where the second terminal is located.

[0287] Optionally, the information related to the second terminal includes current location information of the second terminal;

[0288] When the processor 121 executes the program, the following steps may also be implemented:

[0289] The determining the information of the first AR image according to the information related to the second terminal includes:

[0290] The AR navigation is planned according to the current location information of the first terminal and the second terminal, the map information and the current location information of the first terminal.

[0291] Optionally, the information related to the second terminal includes surrounding environment information collected by the second terminal;

[0292] When the processor 121 executes the program, the following steps may also be implemented:

[0293] The determining the information of the first AR image according to the information related to the second terminal includes:

[0294] Matching the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal;

[0295] The AR navigation is planned according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

[0296] Optionally, the processor 121 may further implement the following steps when executing the program:

[0297] Sending information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

[0298] Optionally, the processor 121 may further implement the following steps when executing the program:

[0299] Before sending the information of the second AR image to the second terminal, the method further includes:

[0300] Acquire a first video image sent by the first terminal;

[0301] extracting a user portrait of the first terminal from the first video image;

[0302] The user portrait of the first terminal is superimposed on the real-time position of the first terminal in the path navigation image to obtain the second AR image.

[0303] The specific working process of the embodiment of the present invention is consistent with that in the above method embodiment 2, so it will not be repeated here. Please refer to the description of the method steps in the above method embodiment 2 for details.

[0304] See also Fig.13 , Fig.131 is a schematic diagram of the structure of a second terminal provided in Embodiment 12 of the present invention. The second terminal 130 includes a processor 131, a memory 132, and a program stored in the memory 132 and executable on the processor 131; when the processor 131 executes the program, the following steps are implemented:

[0305] When receiving an AR video call request based on the operator's call network sent by the first terminal, obtaining information related to the second terminal;

[0306] Send information related to the second terminal.

[0307] In the embodiment of the present invention, an AR function is added on the basis of a video call based on an operator call architecture to implement an AR video call function, that is, an AR video call function is added to an existing video call, which can bring a brand new call experience to users.

[0308] Optionally, the information related to the second terminal includes surrounding environment information of the second terminal.

[0309] Optionally, the processor 131 may further implement the following steps when executing the program:

[0310] Receiving information of a second AR image sent by the edge computing network device;

[0311] The second AR image is displayed on the video call interface, and the second AR image includes the real-time moving position of the first terminal and the path navigation from the first terminal to the position of the second terminal.

[0312] The specific working process of the embodiment of the present invention is consistent with that in the above method embodiment 3, so it will not be repeated here. Please refer to the description of the method steps in the above embodiment 3 for details.

[0313] Embodiment 13 of the present invention provides a readable storage medium on which a program is stored, and when the program is executed by a processor, the steps in any one of the communication methods in Embodiments 1 to 3 above are implemented. For details, please refer to the description of the method steps in the corresponding embodiment above.

[0314] The terminal in the embodiment of the present invention can be a wireless terminal or a wired terminal. The wireless terminal can be a device that provides voice and / or other service data connectivity to users, a handheld device with wireless connection function, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (Radio Access Network, referred to as RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or "cellular" phone) and a computer with a mobile terminal, for example, a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device, which exchanges language and / or data with the radio access network. For example, personal communication service (Personal Communication Service, referred to as PCS) phone, cordless phone, Session Initiation Protocol (Session Initiation Protocol, referred to as SIP) phone, wireless local loop (Wireless Local Loop, referred to as WLL) station, personal digital assistant (Personal Digital Assistant, referred to as PDA) and other devices. A wireless terminal may also be referred to as a system, a subscriber unit (Subscriber Unit), a subscriber station (Subscriber Station), a mobile station (MobileStation), a mobile station (Mobile), a remote station (Remote Station), a remote terminal (Remote Terminal), an access terminal (Access Terminal), a user terminal (User Terminal), a user agent (User Agent), a terminal (User Device or User Equipment), without limitation herein.

[0315] The above-mentioned readable storage medium includes computer-readable storage medium. Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0316] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A communication method, applied to a first terminal, characterized in that: The method comprises: When the first terminal and the second terminal conduct a video call based on the operator call network, displaying a first AR image on the video call interface, where the first AR image is an AR image determined according to information related to the second terminal; The first AR image is an AR navigation image, and the AR navigation is a path navigation for the first terminal to move to a location where the second terminal is located; The information related to the second terminal includes surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined according to three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal, and the current location information of the first terminal; Among them, the edge computing network device is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; and plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

2. The method according to claim 1, characterized in that Before displaying the first AR image on the video call interface, the method further includes: Receiving information of the first AR image sent by the edge computing network device; The information of the first AR image is determined by an edge computing network device according to information related to the second terminal, and the edge computing network device is an edge computing network device to which the second terminal is currently connected.

3. A communication method, using edge computing network equipment, characterized in that: The edge computing network device is an edge computing network device currently accessed by the second terminal, and the second terminal and the first terminal conduct a video call based on the operator call network, and the method includes: Determine information of the first AR image according to information related to the second terminal; Sending information of the first AR image to the first terminal; The first AR image is an AR navigation image, and the AR navigation is a path navigation for the first terminal to move to a location where the second terminal is located; The information related to the second terminal includes surrounding environment information collected by the second terminal; The determining the information of the first AR image according to the information related to the second terminal includes: Matching the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; The AR navigation is planned according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

4. The method according to claim 3, characterized in that Also includes: Sending information of a second AR image to the second terminal, where the second AR image includes the real-time location of the first terminal and path navigation from the first terminal to the location of the second terminal.

5. The method according to claim 4, characterized in that Before sending the information of the second AR image to the second terminal, the method further includes: Acquire a first video image sent by the first terminal; extracting a user portrait of the first terminal from the first video image; The user portrait of the first terminal is superimposed on the real-time position of the first terminal in the path navigation image to obtain the second AR image.

6. A communication method, applied to a second terminal, characterized in that: The method comprises: When receiving an AR video call request based on the operator's call network sent by the first terminal, obtaining information related to the second terminal; Sending information related to the second terminal; The method further includes: receiving information of a second AR image sent by an edge computing network device; displaying the second AR image on a video call interface, wherein the second AR image includes a real-time moving position of the first terminal and a path navigation from the first terminal to the position of the second terminal; and the AR navigation is a path navigation from the first terminal to the position of the second terminal; The information related to the second terminal includes surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined according to the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal, and the current location information of the first terminal; Among them, the edge computing network device is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; and plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

7. A first terminal, characterized in that: include: An AR display module, configured to display a first AR image on a video call interface when the first terminal and the second terminal conduct a video call based on an operator call network, wherein the first AR image is an AR image determined according to information related to the second terminal; The first AR image is an AR navigation image, and the AR navigation is a path navigation for the first terminal to move to a location where the second terminal is located; The information related to the second terminal includes surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined according to three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal, and the current location information of the first terminal; Among them, the edge computing network device is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; and plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

8. An edge computing network device, characterized in that: The edge computing network device is an edge computing network device currently accessed by the second terminal, and the second terminal and the first terminal conduct a video call based on the operator call network. The edge computing network device includes: A determination module, configured to determine information of the first AR image according to information related to the second terminal; A first sending module, configured to send information of the first AR image to the first terminal; The first AR image is an AR navigation image, and the AR navigation is a path navigation for the first terminal to move to a location where the second terminal is located; The information related to the second terminal includes surrounding environment information collected by the second terminal; The determining the information of the first AR image according to the information related to the second terminal includes: Matching the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; The AR navigation is planned according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

9. A second terminal, characterized in that: include: An acquisition module, configured to acquire information related to the second terminal upon receiving an AR video call request based on an operator call network sent by the first terminal; A second sending module, used for sending information related to the second terminal; The second terminal is further used to receive information of a second AR image sent by the edge computing network device; display the second AR image on the video call interface, where the second AR image includes the real-time moving position of the first terminal and a path navigation from the first terminal to the position of the second terminal; the AR navigation is a path navigation from the first terminal to the position of the second terminal; The information related to the second terminal includes surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined according to the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal, and the current location information of the first terminal; Among them, the edge computing network device is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; and plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

10. A first terminal, characterized in that: include: transceivers and processors; The processor is configured to display a first AR image on a video call interface when the first terminal and the second terminal conduct a video call based on an operator call network, where the first AR image is an AR image determined according to information related to the second terminal; The first AR image is an AR navigation image, and the AR navigation is a path navigation for the first terminal to move to a location where the second terminal is located; The information related to the second terminal includes surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined according to three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal, and the current location information of the first terminal; Among them, the edge computing network device is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; and plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

11. An edge computing network device, characterized in that: The edge computing network device is an edge computing network device currently accessed by the second terminal, the second terminal and the first terminal conduct a video call based on the operator call network, and the edge computing network device includes: a transceiver and a processor; The processor is configured to determine information about the first AR image according to information related to the second terminal; The transceiver is used to send information of the first AR image to the first terminal; The first AR image is an AR navigation image, and the AR navigation is a path navigation for the first terminal to move to the location of the second terminal; the information related to the second terminal includes the surrounding environment information collected by the second terminal; The determining the information of the first AR image according to the information related to the second terminal includes: Matching the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; The AR navigation is planned according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

12. A second terminal, characterized in that: include: transceivers and processors; The processor is configured to obtain information related to the second terminal when receiving an AR video call request based on an operator call network sent by the first terminal; The transceiver is used to send information related to the second terminal; The second terminal is further used to receive information of a second AR image sent by the edge computing network device; display the second AR image on the video call interface, where the second AR image includes the real-time moving position of the first terminal and a path navigation from the first terminal to the position of the second terminal; the AR navigation is a path navigation from the first terminal to the position of the second terminal; The information related to the second terminal includes surrounding environment information collected by the second terminal, and the AR navigation is a path navigation determined according to the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal, the surrounding environment information of the second terminal, and the current location information of the first terminal; Among them, the edge computing network device is used to match the surrounding environment information collected by the second terminal with the three-dimensional environment information stored by the edge computing network device to determine the precise location information of the second terminal; and plan the AR navigation according to the precise location information of the second terminal, the current location information of the first terminal, and the three-dimensional environment information between the current location of the first terminal and the current location of the second terminal.

13. A first terminal, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the steps in the communication method according to any one of claims 1 to 2 are implemented.

14. An edge computing network device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: The edge computing network device is an edge computing network device to which the second terminal is currently connected, and the second terminal conducts a video call with the first terminal based on the operator's call network. When the processor executes the program, the steps in the communication method as described in any one of claims 3 to 5 are implemented.

15. A second terminal, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the steps in the communication method according to claim 6 are implemented.

16. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the program implements the steps of the communication method according to any one of claims 1 to 2, or implements the steps of the communication method according to any one of claims 3 to 5, or implements the steps of the communication method according to claim 6.

Images