Call interruption processing method, processing apparatus, ar device, and storage medium

Abstract

The present disclosure provides an audio and video call interruption processing method, a processing device, an AR device and a storage medium, and relates to the technical field of real-time calls. The audio and video call interruption processing method comprises the following steps: detecting that an audio and video call application for holding an engineer conference is closed, and reloading the audio and video call application; displaying an expert list in a window of the AR device, the expert list being used for recording pre-stored guide expert identifiers; determining a call target in response to a selection operation on the expert list; judging whether a background has an engineer conference thread in a conference state with the call target; and if it is judged that the background has the engineer conference thread, continuing to play the audio and video of the interrupted engineer conference in the window. Through the technical scheme of the present disclosure, the interrupted audio and video call can be directly recovered, the normal progress of the audio and video call is ensured, and the user experience of the audio and video call is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of real-time communication, and in particular, to an audio and video call interruption processing method, an audio and video call interruption processing device, an AR device and a computer readable storage medium. BACKGROUND

[0002] The rapid development of the Internet improves the convenience of communication. Currently, audio and video calls are mainly used in life scenarios. If a user accidentally exits during an audio and video call, the interrupted audio and video call cannot be directly resumed, and a new call connection is required, which affects the user experience of the audio and video call.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The purpose of the present disclosure is to provide an audio and video call interruption processing method, an audio and video call interruption processing device, an AR device and a computer readable storage medium, which at least partially solves the problem that the interrupted audio and video call cannot be directly resumed in the related art.

[0005] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0006] According to one aspect of the present disclosure, an audio and video call interruption processing method is provided, comprising: detecting that an audio and video call application for conducting an engineer meeting is closed, reloading the audio and video call application; displaying an expert list in a window of the AR device, the expert list being used to record pre-stored guidance expert identifiers; determining a call target in response to a selection operation on the expert list; determining whether the background has an engineer meeting thread in a conference state with the call target; if it is determined that the background has the engineer meeting thread, continuing to play the audio and video of the interrupted engineer meeting in the window.

[0007] In one embodiment, detecting that the audio and video call application for conducting the engineer meeting is closed further comprises: identifying a closing operation feature of the closing of the audio and video call application; if the closing operation feature is identified as a closing operation feature that has been confirmed, ending the engineer meeting thread; if the closing operation feature is identified as a closing operation feature that has not been confirmed, maintaining the engineer meeting thread.

[0008] In an embodiment, the playing the audio and video of the interrupted engineer conference in the window further includes: if the determination result of the response operation is to join the unfinished engineer conference, closing the prompt pop-up window; calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; and if the conference notification interface is called successfully, playing the audio and video of the interrupted engineer conference in the window based on a conference identifier of the unfinished engineer conference and a current timestamp.

[0009] In an embodiment, the playing the audio and video of the interrupted engineer conference in the window further includes: if the determination result of the response operation is to join the unfinished engineer conference, closing the prompt pop-up window; calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; and if the conference notification interface is called successfully, playing the audio and video of the interrupted engineer conference in the window based on a conference identifier of the unfinished engineer conference and a current timestamp.

[0010] In an embodiment, the playing the audio and video of the interrupted engineer conference in the window further includes: if the determination result of the response operation is to join the unfinished engineer conference, closing the prompt pop-up window; calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; and if the conference notification interface is called successfully, playing the audio and video of the interrupted engineer conference in the window based on a conference identifier of the unfinished engineer conference and a current timestamp.

[0011] In an embodiment, the playing the audio and video of the interrupted engineer conference in the window further includes: if the determination result of the response operation is to join the unfinished engineer conference, closing the prompt pop-up window; calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; and if the conference notification interface is called successfully, playing the audio and video of the interrupted engineer conference in the window based on a conference identifier of the unfinished engineer conference and a current timestamp.

[0012] In an embodiment, the playing the audio and video of the interrupted engineer conference in the window further includes: if the determination result of the response operation is to join the unfinished engineer conference, closing the prompt pop-up window; calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; and if the conference notification interface is called successfully, playing the audio and video of the interrupted engineer conference in the window based on a conference identifier of the unfinished engineer conference and a current timestamp.

[0013] In an embodiment, the playing the audio and video of the interrupted engineer conference in the window further includes: if the determination result of the response operation is to join the unfinished engineer conference, closing the prompt pop-up window; calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; and if the conference notification interface is called successfully, playing the audio and video of the interrupted engineer conference in the window based on a conference identifier of the unfinished engineer conference and a current timestamp.

[0014] In an embodiment, before the AR device generates the guidance information to guide the collection of the scene image of the object to be repaired, the method further comprises: after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the conference room of the new engineer conference has been entered; if the call target does not respond to the real-time communication call information, generating a reconnection prompt information; reloading the cloud conference toolkit to exit the conference room of the new engineer conference, and rejoining the ongoing engineer conference based on the call target.

[0015] In an embodiment, the AR device displays the expert list in a window, and the method further comprises: determining the matched expert list based on the collected scene image of the object to be repaired.

[0016] In an embodiment, the AR device detects that the audio and video call application for the engineer conference is closed, and reloads the audio and video call application, and the method further comprises: sending an abnormal offline reminder information to the call target to display an engineer entering an abnormal offline state in a conference room window of the call target, and notifying the call target to continue to stay in the engineer conference.

[0017] According to another aspect of the present disclosure, there is provided an audio and video call interruption processing apparatus, comprising: a loading module configured to reload an audio and video call application based on an abnormal interruption of the application; a display module configured to display an expert list in a window of the AR device, the expert list being used to record pre-stored guidance expert identifiers; a determination module configured to determine a call target in response to a selection operation on the expert list; a judgment module configured to judge whether a background has an engineer conference thread in a conference state with the call target; and a playing module configured to play audio and video of the engineer conference in the window if it is determined that the background has the engineer conference thread.

[0018] According to still another aspect of the present disclosure, there is provided an AR device, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the audio and video call interruption processing method of any one of the first aspect or the audio and video call interruption processing method of any one of the second aspect via execution of the executable instructions.

[0019] According to yet another aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the audio and video call interruption processing method of any one of the above aspects.

[0020] The audio and video call interruption processing scheme provided by the embodiment of the present disclosure, when applied to the audio and video call scene of remote assistance, if the audio and video call application of conducting the engineer conference is detected to be closed, that is, the application program of the audio and video call is detected to be abnormally exited, the application program is automatically restarted and logged in, after the restart, the expert list is directly displayed, and the operation of reinitiating the joining of the engineer conference is reinitiated based on the selection operation of the user, at this time, if the background is detected to have the engineer conference thread in the conference state with the call target, that is, the conference room of the user is in the online state, the joining of the conference is directly recovered, the audio and video of the interrupted engineer conference is displayed to continue playing, the interrupted audio and video call is directly recovered, the normal proceeding of the audio and video call is ensured, and the user experience of the audio and video call is improved.

[0021] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0022] The drawings incorporated into the specification and constituting a part of the specification show embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 A schematic diagram of an audio and video call interruption processing system in an embodiment of the present disclosure is shown;

[0024] Figure 2 A flowchart of an audio and video call interruption processing method in an embodiment of the present disclosure is shown;

[0025] Figure 3 A schematic diagram of an audio and video call interruption processing scheme in an embodiment of the present disclosure is shown;

[0026] Figure 4 A schematic diagram of another audio and video call interruption processing scheme in an embodiment of the present disclosure is shown;

[0027] Figure 5 A schematic diagram of still another audio and video call interruption processing scheme in an embodiment of the present disclosure is shown;

[0028] Figure 6 A schematic diagram of still another audio and video call interruption processing scheme in an embodiment of the present disclosure is shown;

[0029] Figure 7 A schematic diagram of still another audio and video call interruption processing scheme in an embodiment of the present disclosure is shown;

[0030] Figure 8Fig. 1 shows a schematic diagram of another method for processing interruption of an audio / video call according to an embodiment of the present disclosure;

[0031] Figure 9 Fig. 2 shows a schematic diagram of an apparatus for processing interruption of an audio / video call according to an embodiment of the present disclosure;

[0032] Figure 10 Fig. 3 shows a schematic diagram of an AR device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Example implementations will now be described with reference to the figures. Example implementations can take form in various components and arrangements of components, and are not limited to the examples described herein; rather, the example implementations are provided as example implementations to make the present disclosure more thorough and complete, and to fully convey the scope of the example implementations to those skilled in the art. The described features, structures or characteristics can be combined in one or more implementations in any suitable form.

[0034] Furthermore, the accompanying drawings are only schematic and are non-limiting detailed descriptions of the application. Identical components have been given the same reference numerals in the various drawings and a repeated description is omitted. Some of the blocks in the drawings are functional blocks, which can be implemented in software, hardware or a combination thereof. The functional blocks can be implemented in one or more hardware or software modules or integrated circuits.

[0035] The scheme provided in the present application, when applied to the scene of a remote assistance audio / video call, if it is detected that the audio / video call application for holding an engineer conference is closed, i.e., it is detected that the application program of the audio / video call is abnormally exited, the application program is automatically restarted and logged in, after the restart, the expert list is directly displayed, and based on the selection operation of the user, the operation of joining the engineer conference is reinitiated, at this time, if it is detected that the background has an engineer conference thread in a conference state with the call target, i.e., the conference room of the user is in an online state, the joining of the conference is directly resumed, the audio / video of the interrupted engineer conference is displayed to continue playing, the interrupted audio / video call is directly resumed, the normal progress of the audio / video call is ensured, and the user experience of the audio / video call is improved.

[0036] Figure 1 Fig. 1 is a structural schematic diagram of a computer system provided in an example embodiment of the present application. The system includes a plurality of AR devices 120, a plurality of terminals 140 and a server cluster 160.

[0037] The terminal 140 can be an AR (Augmented Reality) device, a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, a VR (Virtual Reality) device, or the like, or can be a PC (Personal Computer) such as a laptop computer or a desktop computer.

[0038] The AR device 120 and the terminal 140 can be installed with an application program for providing the audio / video call interruption processing.

[0039] The AR device 120 and the terminal 140 are connected to the server cluster 160 through a communication network to realize real-time communication between the AR device 120 and the terminal 140. The communication network can be a wired network or a wireless network.

[0040] The server cluster 160 can be a single server, a plurality of servers, a virtualization platform, or a cloud computing service center. The server cluster 160 is used to provide a background service for the application program for providing the service commodity statistics. Alternatively, the server cluster 160 undertakes the main computing work, and the AR device 120 and the terminal 140 undertake the secondary computing work; or the server cluster 160 undertakes the secondary computing work, and the AR device 120 and the terminal 140 undertake the main computing work; or the AR device 120 and the terminal 140 and the server cluster 160 adopt a distributed computing architecture for collaborative computing.

[0041] In some optional embodiments, the server cluster 160 is used to store the audio / video call interruption processing program information.

[0042] Alternatively, the application program clients installed in different AR devices 120 and terminals 140 are the same, or the application program clients installed on the two AR devices 120 and the terminal 140 are the same type of application program clients of different control system platforms. Based on the difference in the terminal platform, the specific form of the application program client can also be different, for example, the application program client can be a mobile phone client, a PC client, or a Web (World Wide Web) client, etc.

[0043] Those skilled in the art will understand that the number of AR devices 120 and terminals 140 described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of devices.

[0044] Optionally, the system may also include a management device ( Figure 1 (Not shown), the management device is connected to the server cluster 160 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0045] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0046] The following will describe in more detail the audio and video call interruption handling method based on the interaction between AR devices and terminals in this example embodiment, with reference to the accompanying drawings and embodiments.

[0047] like Figure 2 As shown, an audio / video call interruption handling method according to an embodiment of the present disclosure, applied to an AR device, includes:

[0048] Step S202: It is detected that the audio / video call application for the engineer meeting is closed, and the audio / video call application is reloaded.

[0049] The engineer is a wearer of the AR device, and the engineer meeting is a meeting established by the engineer to solve technical difficulties by calling an expert, i.e., a meeting scenario in which the engineer and the expert have a meeting.

[0050] In addition, when detecting that the audio and video call application for the engineer meeting is closed, it can be further detected whether the audio and video interruption is caused by abnormal exit of the application program, wherein the abnormal exit phenomenon includes but is not limited to automatic flashback of the application program, interruption of the application program by other programs, or direct closing of the application program without confirmation, etc.

[0051] Step S204: displaying an expert list in a window of the AR device, the expert list being used to record pre-stored guidance expert identifiers.

[0052] The expert list displayed in the window of the AR device can be based on a subject of a technical difficulty being executed before the interruption, display a matched expert list, or based on a selection of the user, display a matched expert list, or directly display all expert lists.

[0053] The guidance expert identifier can be a mobile phone number, a WeChat number, or other social account of the expert.

[0054] In an embodiment, displaying the expert list in the window of the AR device includes: determining a matched expert list based on a collected scene image of an object to be repaired.

[0055] The type of the object to be repaired is recognized through recognition of the scene image, so as to further call a pre-stored matched expert list.

[0056] Step S206: determining a call target in response to a selection operation on the expert list.

[0057] The call target can be understood as a target in a meeting call state before the interruption.

[0058] The call target can be a mobile phone or an AR device used by the expert, etc.

[0059] Step S208: determining whether the background has an engineer meeting thread in a meeting state with the call target.

[0060] The determination of whether the background has an engineer meeting thread in a meeting state with the call target is based on detection of the meeting thread to determine whether the engineer is already in a meeting room, i.e., whether the meeting state of the device is "meeting". If yes, the interrupted original meeting or a new meeting can be directly joined based on the current meeting thread.

[0061] Step S210, if it is determined that the background has the engineer conference thread, the audio and video of the interrupted engineer conference is continued to be played in the window.

[0062] In this embodiment, in the audio and video call scenario applied to remote assistance, if it is detected that the audio and video call application for conducting the engineer conference is closed, that is, the application program is detected to be abnormally exited, the application program is automatically restarted and logged in, after the restart, the expert list is directly displayed, and the operation of reinitiating the joining of the engineer conference is reinitiated based on the selection operation of the user, at this time, if it is detected that the background has the engineer conference thread in the conference state with the call target, that is, the conference room of the user is in the online state, the joining of the conference is directly resumed, the audio and video of the interrupted engineer conference is continued to be played, the interrupted audio and video call is directly resumed, the normal progress of the audio and video call is ensured, and the user experience of the audio and video call is improved.

[0063] As shown in FIG. 8, in one embodiment, the detection of the closing of the audio and video call application for conducting the engineer conference also includes: Figure 3

[0064] Step S302, a closing operation feature of the audio and video call application is identified.

[0065] The closing operation feature refers to a feature of how the call application is closed, including but not limited to a user normal closing, a user forced closing, a program crash closing, a closing by other threads, and the like.

[0066] Step S304, if the closing operation feature is identified as a closing confirmation operation feature, the engineer conference thread is ended.

[0067] The closing confirmation operation feature refers to a closing operation confirmed by the user. Illustratively, after the conference is ended, a feature corresponding to the closing operation confirmed by the user can be regarded as the closing confirmation operation feature.

[0068] Illustratively, the closing confirmation operation feature can refer to the user actively closing the application by clicking the closing button or using the shortcut key and the like.

[0069] The closing operation not confirmed by the user can include a system closing operation, that is, when the device has low power or insufficient memory, the operating system can automatically close the application to save resources, at this time, the background closing operation is prohibited.

[0070] The closing operation not confirmed by the user can also include a forced closing, that is, in some cases, the system or the application program can forcibly close the application, for example, a serious error occurs or the application program is unresponsive, at this time, the background closing operation is prohibited.

[0071] ​The operation without the closing confirmation can also include a background closing, that is, when the application is switched to run in the background, the operating system can automatically close the audio and video call function of the application, at which time the background closing operation is prohibited.

[0072] In step S306, if the closing operation feature is identified as the operation feature without the closing confirmation, the engineer conference thread is maintained.

[0073] The features correspond to operations such as forced closing by the user, program crash closing, application being occupied by other threads, and the like.

[0074] In this embodiment, when the audio and video call application of the engineer conference is closed, the features of the closing operation are identified to confirm whether the background synchronous closing of the engineer conference thread is needed, so that when the abnormal operation is identified, the engineer conference thread continues to run in the background, so that when the user needs to resume the interrupted conference, it is directly switched to run in the foreground, thereby improving the recovery efficiency of the interrupted audio and video call.

[0075] In one embodiment, if it is determined that the background has the engineer conference thread, the audio and video of the interrupted engineer conference is continued to be played in the window, including:

[0076] If it is determined that the background has the engineer conference thread, a prompt pop-up window is popped up, and the prompt pop-up window is used to prompt that there is an unfinished engineer conference.

[0077] The prompt pop-up window is used to prompt the user that there is an ongoing engineer conference.

[0078] The response operation to the prompt pop-up window is obtained.

[0079] The response operation to the prompt pop-up window includes voice operation of the user, touch operation of the AR device, or touch operation of the virtual interface displayed on the AR device.

[0080] If the identification result of the response operation is to join the unfinished engineer conference, the audio and video of the interrupted engineer conference is continued to be played in the window.

[0081] The way of playing the audio and video of the interrupted engineer conference includes displaying a play box at a specified position of a real environment image collected by the AR device in real time, and the play box plays the video picture of the expert and the synchronous audio information.

[0082] In this embodiment, the prompt pop-up window is generated to prompt the user that the background currently has an unfinished engineer conference, and further to provide the user with the option of whether to resume the unfinished engineer conference, so as to determine whether to resume the interrupted engineer conference based on the user's selection, and ensure the reliability of the recovery operation.

[0083] As Figure 4As shown, in one embodiment, if the identification result of the response operation is to join the ongoing engineer conference, the audio and video of the engineer conference are played in the window, including:

[0084] In step S402, if the identification result is to join the ongoing engineer conference, the prompt pop-up window is closed.

[0085] In step S404, a re-join interface is called to load a cloud conference toolkit to join the ongoing engineer conference.

[0086] The re-join interface (reJoinOneRoom interface) is used to perform the function of re-joining the engineer conference.

[0087] The cloud conference toolkit (Software Development Kit, SDK) is used to provide software support for audio and video conferences.

[0088] In step S406, after the cloud conference toolkit is loaded, a conference notification interface is called to notify the background to detect whether the ongoing engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference.

[0089] The conference notification interface is used to detect whether the ongoing conference is successfully entered, and if so, to pull the audio and video of the interrupted engineer conference to ensure the timeliness of audio and video transmission.

[0090] In step S408, if the conference notification interface is successfully called, based on the conference identifier of the ongoing engineer conference and the current timestamp, the audio and video of the interrupted engineer conference are continued to be played in the window.

[0091] In this embodiment, in the process of successfully joining the ongoing engineer conference, the re-join interface is called, the cloud conference toolkit is loaded, and the conference notification interface is called, to ensure the reliability of the user entering the conference room and continuing to play the audio and video of the interrupted engineer conference when the interrupted engineer conference is resumed.

[0092] In one embodiment, it also includes: if the conference notification interface fails to be called, an error prompt information is generated; the cloud conference toolkit is reloaded to exit the ongoing engineer conference, and the ongoing engineer conference is re-joined based on the call target.

[0093] In this embodiment, if the conference notification interface fails to be called, it indicates that the user has not successfully entered the conference room. After the error prompt information is generated, the window of the expert list is switched to, so that the user can re-enter the interrupted engineer conference based on the selection operation.

[0094] In one embodiment, if it is determined that the background has the engineer conference thread, the conference interface is displayed in the window, and further comprising: if the identification result of the response operation is to re-initiate the conference, closing the engineer conference thread of the background; initiating a new engineer conference to the call target.

[0095] In one embodiment, if it is determined that the background has the engineer conference thread, the conference interface is displayed in the window, and further comprising: if the identification result of the response operation is to re-initiate the conference, closing the engineer conference thread of the background; initiating a new engineer conference to the call target.

[0096] In one embodiment, further comprising: if it is determined that the background does not have the engineer conference thread, initiating a new engineer conference to the call target.

[0097] In one embodiment, further comprising: if it is determined that the background does not have the engineer conference thread, initiating a new engineer conference to the call target.

[0098] In addition, while initiating a new engineer conference to the call target, the expert information corresponding to the call target can be displayed in the window, and in addition, while initiating a new engineer conference to the call target, the technical problem to be solved can be sent to the call target.

[0099] In this embodiment, if the identification result of the response operation to the prompt pop-up window is to re-initiate the conference, or the thread of the engineer in the conference room is not detected, a new engineer conference can be directly initiated to the call target to ensure the timeliness of the conference initiation.

[0100] As shown in FIG. Figure 5 In one embodiment, a specific implementation of initiating a new engineer conference to the call target includes:

[0101] Step S502, calling a call interface to pop up a call pop-up window in the window.

[0102] Step S504, in response to the confirmation operation to the call pop-up window, loading a cloud conference toolkit to join a new engineer conference with the call target.

[0103] Step S506, determining the conference room for entering the new engineer conference, generating guidance information, and guiding the use of an AR device to collect a scene image of the object to be maintained.

[0104] The guidance information can be voice guidance or text guidance, etc.

[0105] Step S508, sharing the collected scene image in the conference room, and sending real-time communication call information to the call target.

[0106] In the conference room, the collected scene image is shared, that is, when the call target accepts the call invitation, the shared scene image is transmitted to the call target.

[0107] In this embodiment, by calling the call interface, the application can pop up a call popup window in the window to prompt the user to make a call operation. Further, in response to the confirmation operation of the call popup window, when the user confirms the call popup window, the application loads the cloud conference toolkit to realize a new engineer conference with the call target. The application guides the user to the conference room of the new engineer conference, generates guidance information to guide the user to use the AR device to collect the scene image of the object to be repaired, and shares the scene image collected by the AR device in the conference room. At the same time, the application sends real-time communication call information to the call target to realize real-time communication and cooperation. Through the call interface and the cloud conference toolkit, real-time communication and cooperation between engineers are realized. Combined with the use of the AR device, more intuitive and efficient scene image collection and sharing functions can be provided, thereby improving the efficiency and quality of the engineer conference.

[0108] As shown in Figure 6 In one embodiment, before determining to enter the conference room of the new engineer conference, generating the guidance information, and guiding to use the AR device to collect the scene image of the object to be repaired, the method further includes:

[0109] Step S602, after the cloud conference toolkit is loaded, a conference notification interface is called to notify the background to detect whether the conference room of the new engineer conference has been entered.

[0110] Step S604, if the call target does not respond to the real-time communication call information, reconnection prompt information is generated.

[0111] Step S606, the cloud conference toolkit is reloaded to exit the conference room of the new engineer conference, and the call target is re-joined in the ongoing engineer conference based on the call target.

[0112] In one embodiment, after detecting that the audio and video call application for the engineer conference is closed, the audio and video call application is reloaded, and the method further includes: sending abnormal offline reminder information to the call target to display that the engineer enters the abnormal offline state in the conference room window of the call target, and notifying the call target to continue to stay in the engineer conference.

[0113] In this embodiment, after detecting that the audio and video call application for the engineer conference is closed, the abnormal offline reminder information is sent to the call target in real time. At this time, in the conference room, the avatar of the user can be switched to the abnormal offline state,

[0114] As shown in Figure 7 The audio and video call interruption processing method according to another embodiment of the present disclosure includes:

[0115] In step S702, if the audio / video call application crashes or is accidentally exited due to user error, the user re-enters the AR device to make a call and attempts to rejoin the original meeting.

[0116] In step S704, the AR device's backend system receives an RTC call request from a user who unexpectedly exited the system. The system then checks the online status of the unexpectedly exited user through the online interface.

[0117] In step S706, the backend system detects that the meeting room status of a user who has unexpectedly left is "meeting" and issues a system pop-up window prompting the user to join the original meeting or start a new meeting.

[0118] Step S708: If the backend system receives a user's request to join the original meeting, it calls the reJoinOneRoom interface and uses the cloud audio and video SDK to join the original audio and video conference.

[0119] In step S710, after the system successfully calls the cloud audio and video SDK, it adds the user to the meeting room that was unexpectedly logged out and records the timestamp information of joining the room.

[0120] like Figure 8 As shown, a method for handling audio / video call interruptions according to another embodiment of this disclosure includes:

[0121] Step S802: The audio / video call application unexpectedly exits the audio / video call due to program crash or user misoperation.

[0122] Step S804: Reload the audio / video call application and display the expert list interface.

[0123] Step S806: Select the target expert and make a call.

[0124] Step S808: Use the online judgment interface to determine whether the engineer is in the room, that is, determine whether there is an engineer meeting thread in the background where the target expert is in a meeting state. If "yes", proceed to step S810; if "no", proceed to step S816.

[0125] Step S810: A pop-up notification window appears in the window to indicate that there is an unfinished engineer meeting.

[0126] Step S812: Obtain the response to the prompt pop-up to detect whether the user is joining the original meeting or initiating a new meeting. If the user "joins the original meeting", proceed to step S814; if the user "initiates a new meeting", proceed to step S816.

[0127] Step S814: Call the rejoin interface to load the cloud meeting toolkit in order to join the ongoing engineer meeting.

[0128] Step S816, call the calling interface, pop up the call pop-up window.

[0129] Step S818, load the cloud conference toolkit to join the initiated new engineer conference.

[0130] Step S820, send an IM call message to call the expert.

[0131] It should be noted that the above-described figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is readily understood that the processes shown in the above-described figures do not indicate or limit the time sequence of the processes. In addition, it is readily understood that the processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0132] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method or a program product. Therefore, various aspects of the present application can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" herein.

[0133] The audio / video call interruption processing apparatus 900 according to this embodiment of the present application will be described below with reference to Figure 9 Figure 9 The illustrated audio / video call interruption processing apparatus 900 is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0134] The audio / video call interruption processing apparatus 900 is in the form of a hardware module. The components of the audio / video call interruption processing apparatus 900 can include, but are not limited to: a loading module 902 for reloading the application based on the audio / video call application exception interruption; a display module 904 for displaying an expert list in a window of the AR device, the expert list being used to record the pre-stored guide expert identification; a determination module 906 for determining the call target in response to a selection operation on the expert list; a judgment module 908 for judging whether the background has an engineer conference thread in a conference state with the call target; and a playing module 910 for playing the audio / video of the engineer conference in the window if it is determined that the background has the engineer conference thread.

[0135] The AR device 1000 according to this embodiment of the present application will be described below with reference to Figure 10 Figure 10 The illustrated AR device 1000 is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0136] As Figure 10 ​​As shown, the AR device 1000 is in the form of a general computing device. The components of the AR device 1000 can include, but are not limited to, the at least one processing unit 1010 described above, the at least one storage unit 1020 described above, and a bus 1030 that connects the different system components including the storage unit 1020 and the processing unit 1010.

[0137] The storage unit stores program code that can be executed by the processing unit 1010 such that the processing unit 1010 performs the steps according to various exemplary embodiments of the present application described in the above "Exemplary Methods" section of the specification. For example, the processing unit 1010 can perform the steps S202 and S208 as shown in FIG. 2, and other steps defined in the audio / video call interrupt processing method of the present disclosure. Figure 1

[0138] The storage unit 1020 can include a readable medium in the form of volatile storage such as a random access memory (RAM) 10201 and / or cache memory 10202, and can further include a non-volatile storage such as read-only memory (ROM) 10203.

[0139] The storage unit 1020 can further include a program / utility 10204 having a set (at least one) of program modules 10205 such as an operating system, one or more application programs, other program modules, and program data, and can include implementation of a network environment, each or a combination of these examples.

[0140] The bus 1030 can be representative of one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics acceleration port, a processing bus, or a local bus using any of a variety of bus architectures.

[0141] ​The AR device 1000 can also communicate with one or more external devices 1060 such as a keyboard, a pointing device, a Bluetooth device, etc., and can also communicate with one or more devices that enable a user to interact with the AR device, and / or any devices (e.g., routers, modems, etc.) that enable the AR device 1000 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 1050. Still yet, the AR device 1000 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the public network, e.g., the Internet) via a network adapter 1050. As illustrated, the network adapter 1050 communicates with the other components of the AR device 1000 via the bus 1030. It should be appreciated that although not shown, other hardware and / or software modules could be used in connection with the AR device. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0142] From the above description of the embodiments, those skilled in the art will readily appreciate that the example embodiments described herein can be implemented by software and / or by hardware coupled with software. Accordingly, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (e.g., a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or a network, and includes a number of instructions for causing a computing device (e.g., a personal computer, a server, a terminal device, or a network device, etc.) to execute the methods according to the embodiments of the present disclosure.

[0143] In the example embodiments of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above-described methods of the present disclosure is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to execute the steps described in the above “Example Method” section according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0144] The program product for implementing the above-described methods according to the embodiments of the present disclosure can take the form of a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, a readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0145] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that can be involved in

[0146] The code can be transmitted in any form, including, but not limited to, radio frequency, optical, electrical, or the like, or any suitable combination thereof.

[0147] The program code can be implemented in any of a variety of programming languages, including, but not limited to, Java, C++, or the like, and can be executed by one or more of a variety of operating systems. The program code can be executed on a single computer, on a plurality of computers, or on a plurality of computing devices connected by a network. The computing devices can be connected by a network in any form, including, but not limited to, a local area network (LAN) or a wide area network (WAN), or the like, or any suitable combination thereof.

[0148] It should be noted that, although the above detailed description refers to several modules or units of the device for action execution, such a division is not mandatory. Indeed, according to an embodiment of the present disclosure, features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, features and functionalities of one module or unit described above can be further divided into several modules or units.

[0149] Moreover, although the various steps of the methods of the present disclosure are described in a particular order in the figures, this is not mandatory or implies that the steps must be performed in that particular order, or that all of the steps must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, several steps can be combined into one step, one step can be split into several steps, etc.

[0150] Those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware, through the above description of the embodiments. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0151] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure that come within the scope of the following claims and their equivalents. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

Claims

1. A method for handling an audio-video call interruption, the method comprising: Applied to an AR device, comprising: Detecting that an audio and video call application for holding an engineer conference is closed, identifying a closing operation feature of the audio and video call application being closed, if the closing operation feature is identified as a closing operation feature that has been confirmed, ending an engineer conference thread of the engineer conference, if the closing operation feature is identified as a closing operation feature that has not been confirmed, maintaining the engineer conference thread and reloading the audio and video call application, and sending abnormal offline prompt information to a call target; Displaying an expert list in a window of the AR device; Determining the call target in response to a selection operation on the expert list, wherein the call target is a target in a conference call state before interruption; Judging whether a background has an engineer conference thread in a conference state with the call target; If it is judged that the background has the engineer conference thread, continuing to play audio and video of the interrupted engineer conference in the window, comprising: if it is judged that the background has the engineer conference thread, popping up a prompt pop-up window in the window, the prompt pop-up window being used to prompt that there is an unfinished engineer conference; obtaining a response operation on the prompt pop-up window; if the identification result of the response operation is to join the unfinished engineer conference, continuing to play the audio and video of the interrupted engineer conference in the window.

2. The method of claim 1, wherein, The continuing to play the audio and video of the interrupted engineer conference in the window if the identification result of the response operation is to join the unfinished engineer conference, comprising: If the identification result is to join the unfinished engineer conference, closing the prompt pop-up window; Calling a re-joining interface to load a cloud conference toolkit to join the unfinished engineer conference; After the cloud conference toolkit is loaded, calling a conference notification interface to notify the background to detect whether the unfinished engineer conference is successfully joined, and to pull the audio and video of the interrupted engineer conference; If the conference notification interface is called successfully, based on a conference identifier of the unfinished engineer conference and a current timestamp, continuing to play the audio and video of the interrupted engineer conference in the window.

3. The method of claim 2, wherein, Further comprising: If the conference notification interface fails to be called, generating an error prompt information; Re-loading the cloud conference toolkit to exit the unfinished engineer conference, and re-joining the unfinished engineer conference based on the call target.

4. The method of claim 1, wherein, The displaying a conference interface in the window if it is judged that the background has the engineer conference thread, further comprising: If the identification result of the response operation is to re-initiate a conference, closing the engineer conference thread of the background; Initiating a new engineer conference to the call target.

5. The method of AV call interrupt processing according to claim 1, wherein, Further comprising: If it is judged that the background does not have the engineer conference thread, initiating a new engineer conference to the call target.

6. The method of claim 4, wherein, The initiating a new engineer conference to the call target, comprising: Calling a call interface to pop up a call pop-up window in the window; In response to a confirmation operation on the call pop-up window, loading a cloud conference toolkit to join the new engineer conference with the call target; determining a conference room for entering the new engineer conference, generating guiding information for guiding the AR device to collect a scene image of a to-be-repaired object; sharing the collected scene image in the conference room, and sending real-time communication call information to the call target.

7. The method of AV call interrupt processing according to claim 6, wherein, Before the determination of the conference room for entering the new engineer conference, the method further comprises: after the cloud conference toolkit is loaded, calling a conference notification interface to notify a background to detect whether the conference room for entering the new engineer conference has been entered; if the call target does not respond to the real-time communication call information, generating reconnection prompt information; reloading the cloud conference toolkit to exit the conference room for entering the new engineer conference, and rejoining an ongoing engineer conference based on the call target.

8. The method of AV call interrupt processing according to claim 6, wherein, The expert list displayed in the window of the AR device comprises: determining the matched expert list based on the collected scene image of the to-be-repaired object.

9. The audio and video call interruption processing method according to claim 1, wherein the abnormal offline prompt information is used to display that an engineer enters an abnormal offline state in a conference room window of the call target, and notify the call target to continue to stay in the engineer conference.

10. An audio / video call interruption handling device, characterized in that, The application is applied to an AR device, comprising: a loading module configured to detect that an audio and video call application for an engineer conference is closed, identify a closing operation feature of the audio and video call application, if the closing operation feature is identified as a closing operation feature that has been confirmed, end an engineer conference thread of the engineer conference, and if the closing operation feature is identified as a closing operation feature that has not been confirmed, maintain the engineer conference thread and reload the audio and video call application, and send abnormal offline prompt information to a call target; a display module configured to display an expert list in a window of the AR device; a determination module configured to determine the call target in response to a selection operation on the expert list, wherein the call target is a target in a conference call state before interruption; a judgment module configured to judge whether a background has an engineer conference thread in a conference state with the call target; a playing module configured to continue to play audio and video of the interrupted engineer conference in the window if it is determined that the background has the engineer conference thread, comprising: if it is determined that the background has the engineer conference thread, popping up a prompt pop-up window in the window, the prompt pop-up window is used to prompt that there is an ongoing engineer conference; obtaining a response operation on the prompt pop-up window; and if the identification result of the response operation is to join the ongoing engineer conference, continuing to play the audio and video of the interrupted engineer conference in the window.

11. An AR device, comprising: comprise: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the audio and video call interruption processing method according to any one of claims 1-9 by executing the executable instructions.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the audio and video call interruption processing method according to any one of claims 1-9.

Images