Cross-terminal call state detection method and device, electronic equipment and read storage medium

Abstract

The invention relates to a cross-terminal call state detection method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: respectively carrying out call detection based on at least one of foreground information and a sound output mode of a first terminal; each call detection corresponds to at least one call detection result; and if two or more call detection results are obtained, determining a call cooperation state between the first terminal and the second terminal according to the two or more call detection results, the call cooperation state being a call state of any one of the first terminal and the second terminal. By adopting the method, the accuracy of the call coordination state can be improved.

Description

Technical Field

[0001] The present application relates to call status processing, and in particular to a cross-end call status detection method, device, electronic device and computer-readable storage medium. Background Art

[0002] When the first terminal is making a VOIP call, if the first terminal maintains a wireless connection with the second terminal, the second terminal can receive call status information sent by the first terminal. By monitoring the call status information transmitted by wireless audio, the second terminal can infer the change of the VOIP call status of the first terminal and display the corresponding call interface on the second terminal, thereby realizing the collaboration between the first terminal and the second terminal.

[0003] Due to the limitation that the wireless protocol only contains limited call status information, it is impossible to obtain more detailed call status changes, resulting in inaccurate status recognition between the first terminal and the second terminal during the call collaboration process. Summary of the invention

[0004] The embodiments of the present application provide a cross-terminal call status detection method, device, electronic device, and computer-readable storage medium, which can improve the accuracy of call collaboration status detection.

[0005] In a first aspect, the present application provides a cross-terminal call status detection method. The method comprises:

[0006] Perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; each call detection corresponds to at least one call detection result;

[0007] If two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results, and the call coordination state is the call state of any one of the first terminal and the second terminal.

[0008] In a second aspect, the present application also provides a cross-terminal call status detection device. The device includes:

[0009] A call detection module, configured to perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; each call detection corresponds to at least one call detection result;

[0010] A status judgment module is used to determine the call coordination status between the first terminal and the second terminal according to the one or more call detection results if one or more call detection results are obtained, and the call coordination status is the call status of any one of the first terminal and the second terminal.

[0011] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0012] Perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; each call detection corresponds to at least one call detection result;

[0013] If two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results, and the call coordination state is the call state of any one of the first terminal and the second terminal.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0015] Perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; each call detection corresponds to at least one call detection result;

[0016] If two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results, and the call coordination state is the call state of any one of the first terminal and the second terminal.

[0017] The above-mentioned cross-terminal call status detection method performs call detection based on at least one of the foreground information and the sound output mode of the first terminal; the foreground information focuses on obtaining the current running information, is not limited to whether the network call application has been started, and is less affected by certain factors brought by the network call application itself, and can more accurately identify certain call states; the sound output mode is obtained based on the detection of the network phone application, and the sound output mode is one of the elements of the call process. When the call can be carried out, the corresponding call state can be determined, and its accuracy is high; and the call detection based on the foreground information and the sound output mode corresponds to at least one call detection result respectively; if two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results, and its accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is an application environment diagram of a cross-terminal call status detection method in one embodiment;

[0020] Figure 2 is a flow chart of a cross-terminal call status detection method in one embodiment;

[0021] Figure 3 A structural block diagram of a VOIP communication module in one embodiment;

[0022] Figure 4 A flowchart of generating a first foreground call detection result in one embodiment;

[0023] Figure 5 A flowchart of generating a second foreground call detection result in one embodiment;

[0024] Figure 6 is a structural block diagram of a cross-terminal call status detection device in one embodiment;

[0025] Figure 7 A computer device in one embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The present application embodiment provides a cross-terminal call status detection method, which can be applied to Figure 1In the application environment shown. The first terminal 102 communicates with the second terminal 104 through the network. The first terminal 102 and the second terminal 104 may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc.; head-mounted devices include augmented reality glasses. The first terminal 102 and the second terminal 104 may be electronic devices of the same type or different types, for example, the first terminal 102 is a mobile phone terminal, and the second terminal 104 is a car terminal.

[0029] Among them, the application scenarios of the cross-end call status detection method include a product that uses the phone call function on a device connected to a smartphone; the cross-end call is an Internet call based on an Internet phone application, and the Internet phone is a voice transmission technology based on IP (Voice over Internet Protocol, VOIP).

[0030] In one embodiment, Figure 2 As shown, a cross-terminal call status detection method is provided, and the method is applied to Figure 1 Taking the first terminal 102 in the example as an example, the following steps are included:

[0031] Step 202: performing call detection based on at least one of the foreground information and the sound output mode of the first terminal; each call detection corresponds to at least one call detection result.

[0032] The foreground information is the current operation information of the first terminal. The foreground information is not limited to a certain application, but focuses on obtaining the current operation information. In the process of call detection through the foreground information, it is not limited to whether the network call application has been started, and is less affected by the network call application itself, and can more accurately identify certain call states. For example: the foreground information includes but is not limited to the information of the foreground application when it is started, running, or closed, and regardless of whether the foreground information is a network call application, the foreground call detection result can be transmitted.

[0033] The foreground information can be refined for different angles of the operation process of the first terminal, so as to perform detections respectively through the refined foreground information to obtain at least one foreground call detection result after the foreground information is refined; the refined information includes but is not limited to: the information of the foreground application and the foreground screen content. The foreground application is at least one application running on the first terminal, and the foreground application information is obtained by detecting the operation process of the foreground application, which can be used to determine the first foreground call detection result; the foreground screen content information is the information displayed by the first terminal during the operation process. The foreground screen content is obtained by detecting the foreground screen, which can be used to determine the second foreground call detection result, and the foreground screen is the screen being displayed by the first terminal.

[0034] The sound output mode is a call detection method based on the Internet phone application, and the call detection method is not limited to whether the Internet phone application is a foreground application; the sound mode detection result corresponds to at least one call detection result, and at least one call detection result may include the result obtained by directly detecting the first terminal itself, or may include the result obtained by detecting and feeding back the corresponding result through the second terminal. Optionally, the sound output mode is obtained through the state monitor (voipstatemonitor) of the Internet phone application. Optionally, the sound output mode is used to determine the change behavior of the sound output and sound receiving device, and the sound output and sound receiving device can be the first terminal, the second terminal or other terminals. In the case where the mobile phone is the first terminal and the car computer is the second terminal, the car computer can be the sound output and sound receiving device, and the mobile phone can also be the sound output and sound receiving device. Optionally, the sound output mode is used to determine the change behavior of the sound output and sound receiving device, so as to switch the sound output and sound receiving device in time.

[0035] The sound output mode is detected based on the Internet phone application, and the sound output mode is one of the elements of the call process. When the call can be carried out, the corresponding call status can be determined with high accuracy. In the case where the Internet phone application is installed on the first terminal and the second terminal at the same time, the sound output mode is one of the elements of the call process. When the call can be carried out, the synchronization between the first terminal and the second terminal is high.

[0036] In an optional embodiment, call detection is performed based on the foreground information of the first terminal. Call detection is performed based on the foreground information and the sound output mode of the first terminal, respectively, including: performing call detection on the first terminal based on multiple call detection methods of the foreground information; if it is detected that the foreground information meets the preset conditions of a certain foreground information call detection method, then a foreground call detection result of this foreground information call detection method is generated; if it is detected that the foreground information does not meet the preset conditions of a certain foreground information call detection method, then the foreground call detection result of this foreground information call detection method is not generated. Thus, the foreground information is detected through different foreground information call detection methods to obtain as many foreground call detection results as possible; and there is a situation where an arbitrary number of foreground call detection results are not generated to reduce the amount of data processed by the first terminal.

[0037] In another optional embodiment, call detection is performed based on the sound output mode. Call detection is performed based on the foreground information and the sound output mode of the first terminal respectively, including: performing call detection on the first terminal based on at least one call detection method of the sound output mode; if it is detected that the sound output mode meets the preset conditions of a certain sound output mode call detection method, then generating a sound output mode call detection result of this sound output mode call detection method; if it is detected that the sound output mode does not meet the preset conditions of a certain call detection method, then not generating a sound output mode call detection result of this sound output mode call detection method. Thus, the sound output mode is detected through different sound output mode call detection methods to obtain as many sound output mode call detection results as possible; moreover, it can be applied to the situation where the foreground call detection result is not generated to reduce the amount of data processed by the first terminal.

[0038] Among them, one call detection method based on the sound output mode is to detect based on the sound output mode of the first terminal; another call detection method based on the sound output mode is to detect the sound output mode of the second terminal and feed back the corresponding result.

[0039] Step 204: If two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results. The call coordination state is the call state of any one of the first terminal and the second terminal.

[0040] The call detection result is the information transmitted by each call detection method when the call detection method is detected successfully, and each call detection method corresponds to at least one call detection result. Optionally, the call detection result can be used to indicate whether the first terminal and the second terminal are in the open and closed state of the Internet phone application, the answering state, the hanging state or other call states.

[0041] When two or more call detection results are the same, the accuracy of the call detection is relatively higher. Optionally, when one call detection result is obtained based on foreground information detection and the other call detection result is obtained based on sound output mode detection, the two call detection results are detected from the current operation information of the first terminal and the information of the Internet phone application respectively to more accurately determine the call coordination state. Among them, for two or more call detection results, the foreground information of the first terminal can also be refined, and the two or more call detection results can be determined based on the refined call detection method.

[0042] The call collaboration state is the call state of any one of the first terminal and the second terminal. The call collaboration state is the call state across the first terminal and the second terminal. When the first terminal and the second terminal are in a collaborative state, the first terminal performs logical processing such as judgment and selection on the relevant information of the call to obtain a processing result; and then feeds back the logical processing result to the second terminal. The relevant information of the call event may be information related to the call interface. The call control instruction issued by the second terminal instructs the first terminal to execute certain steps, which can move the data processing process to the first terminal and simplify the data processing requirements of the second terminal during the call.

[0043] Optionally, the call collaboration state is used to indicate the state of the first terminal and the second terminal being in a collaborative state and having a call with a third terminal. The third terminal may be at least one terminal that actually exists, or may be a terminal that does not exist. For example: when the first terminal and the second terminal are in a collaborative state, a call request sent by at least one third terminal is received, and the third terminal exists at this time; when the first terminal and the second terminal are in a collaborative state, a call request is sent to the third terminal, and the third terminal may not exist.

[0044] In the above-mentioned cross-terminal call status detection method, call detection is performed based on at least one of the foreground information and the sound output mode of the first terminal; the foreground information focuses on obtaining the current running information, is not limited to whether the network call application has been started, and is less affected by certain effects brought by the network call application itself, and can more accurately identify certain call states; the sound output mode is obtained based on the detection of the network phone application, and the sound output mode is one of the elements of the call process. When the call can be carried out, the corresponding call state can be determined, and its accuracy is high; and the call detection based on the foreground information and the sound output mode corresponds to at least one call detection result respectively; therefore, if two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results, and its accuracy is higher.

[0045] In one embodiment, call detection is performed based on at least one of the foreground information and the sound output mode of the first terminal, including: call detection based on the foreground application of the first terminal; call detection based on the foreground screen content of the first terminal; and call detection based on the sound output mode of the first terminal.

[0046] The foreground application is the application currently running on the first terminal, and the application functions of the foreground application and the events triggered by it are detected; while the call detection based on the foreground screen content is to detect the interface of the first terminal. These two call detection methods are based on the refined foreground information. The sound output mode of the first terminal is detected based on the Internet phone application installed on the first terminal.

[0047] When testing based on the foreground application, the test is performed from the perspective of the foreground application's operation; when testing based on the foreground screen content, the test is performed from the interface; when testing based on the sound output mode, the test is performed based on the Internet phone application. These three call detection methods are used to try to obtain their own call detection results from different angles to accurately determine the call coordination status.

[0048] In one embodiment, call detection is performed based on the foreground application of the first terminal, including: determining whether the foreground application running on the first terminal is an Internet phone application; if it is an Internet phone application, determining a target call event triggered by the Internet phone application; and determining a first foreground call detection result based on the target call event.

[0049] The foreground application running on the first terminal includes, but is not limited to, the foreground application when starting, running, or ending.

[0050] In one embodiment, determining whether the foreground application running on the first terminal is an Internet phone application includes: determining whether the foreground application running on the first terminal matches the Internet phone application based on an identifier, activity information, or event notification; if they match, the foreground application is the Internet phone application; if they do not match, the foreground application is not the Internet phone application.

[0051] In one embodiment, determining a target call event triggered by an Internet phone application includes: obtaining activity information when a foreground program is running; and determining the target call event triggered by the Internet phone application according to information indicated by the activity information. The activity information is the running of the foreground program (activity), and when at least one life cycle of the running of the foreground program ends, the activity judgment result of the running of the foreground program is output from the call operation involved in the execution of the code. The monitoring of the target call event triggered by the Internet phone application can reduce the time delay of monitoring the call status because the life cycle of the running of the foreground program changes quickly.

[0052] Optionally, call detection based on the foreground application of the first terminal also includes: if the current foreground application running on the first terminal is not an Internet phone application, then when the first terminal starts the next foreground application of the current foreground application, it is determined whether the next foreground application is an Internet phone application, until the current foreground application running on the first terminal is a Internet phone application.

[0053] Call detection based on the foreground application of the first terminal is based on the characteristic that the Internet phone application is always displayed in the foreground. The first foreground call detection result obtained in this way is not limited to the start and termination of the Internet phone application, and can accurately identify call states such as call establishment and cancellation.

[0054] In one embodiment, determining a target call event triggered by a call application includes: determining a first target call event according to a call event notification of an event triggered by an Internet phone application, wherein the first target call event is one of an incoming call event and an outgoing call event.

[0055] Correspondingly, determining a first foreground call detection result according to the target call event includes: determining the first foreground call detection result according to the call state indicated by the first target call event.

[0056] An incoming call event refers to an event triggered when a call request from a third terminal is received in a collaborative state between a first terminal and a second terminal; an outgoing call event refers to an event triggered when a call request is sent to a third terminal in a collaborative state between a first terminal and a second terminal. When a call event notification indicates an incoming call event, the first target call event is an incoming call event; when a call event notification indicates an outgoing call event, the first target call event is an outgoing call event.

[0057] The call state indicated by the first target call event is one of the main attributes in the first foreground call detection result; if the first target call event indicates an incoming call state, the first foreground call detection result includes the incoming call state; if the first target call event indicates an outgoing call state, the first foreground call detection result includes the outgoing call state. Optionally, the first foreground call detection result may also include other call states.

[0058] Through the first target call event, the first terminal and the second terminal can be accurately controlled to perform corresponding processing of the collaborative call according to the incoming call event or the outgoing call event, so as to ensure that when the first terminal or the second terminal receives the call operation instruction, the first terminal and the second terminal can perform the corresponding process according to the first target call event. For example: when the second terminal detects the call operation instruction and sends the call operation instruction to the first terminal, the first terminal generates a call interface based on the first target call event; and then feeds the call interface back to the second terminal for display.

[0059] In one embodiment, determining a target call event triggered by a call application further includes: determining a second target call event based on a display event notification of an event triggered by the Internet phone application, wherein the second target call event is one of a partial display event and a full screen display event.

[0060] Correspondingly, the first foreground call detection result is determined according to the call status indicated by the first target call event, including: determining the first foreground call detection result according to the call status indicated by the first target call event and the call status indicated by the second target call event.

[0061] The partial display event is an event that displays the call status by means of a banner or a pop-up window, and the full screen display event is an event that displays the call status. Optionally, the partial display event and the full screen display event correspond to respective call interfaces, and each call interface can be determined by the Internet phone application or by data extracted by the Internet phone application.

[0062] When the display event notification indicates a banner display event, the second target display event is a banner display event; when the display event notification indicates a full-screen display event, the second target display event is a full-screen display event.

[0063] The call status indicated by the second target call event is one of the attributes in the first foreground call detection result. There is no necessary connection between the first target call event and the second target call event, and the two indicate different types of call status respectively. The first target call event is used to indicate the incoming call status or the outgoing call status, while the second target call event is used to indicate the partial display call status or the full screen display call status. The call status indicated by the first target call event and the call status indicated by the second target call event need to be superimposed.

[0064] For example: if the first target call event indicates an incoming call status, and the second target call event indicates a banner display call status, then the first foreground call detection result refers to the banner display call status of the incoming call status; if the first target call event indicates an outgoing call status, and the second target call event indicates a banner display call status, then the first foreground call detection result refers to the banner display call status of the outgoing call status.

[0065] Through the second target call event, the first terminal and the second terminal can be accurately controlled to perform corresponding processing of the collaborative call according to the partial display event or the full-screen display event, so as to ensure that when the first terminal or the second terminal receives the call operation instruction, the first terminal and the second terminal can both perform the corresponding process according to the first target call event, and can ensure the consistency of data processing between the first terminal and the second terminal according to the second target call event, so as to more accurately control this execution process. For example: when the second terminal detects the call operation instruction and sends the call operation instruction to the first terminal, the first terminal generates a full-width call interface based on the first target call event; and then feeds back the full-width call interface to the second terminal for display.

[0066] In one embodiment, call detection is performed based on the foreground screen content of the first terminal, including: if the foreground screen of the first terminal displays a page of an Internet phone application, image recognition is performed on the page of the Internet phone application to obtain the foreground screen content; and a second foreground call detection result is determined based on the call status indicated by the foreground screen content.

[0067] Optionally, the foreground screen of the first terminal displays the page of the Internet phone application, which means that the foreground screen of the first terminal contains the page of the Internet phone application and the page is in a displayed state. The page of the Internet phone application is determined to be a local call interface or a full-screen call interface based on the second target call event. Optionally, the foreground screen of the first terminal is the foreground screen executed by the main screen of the first terminal, and the main screen is the physical screen of the first terminal.

[0068] Optionally, it can be determined by monitoring whether the web phone application page displays the web phone application page.

[0069] Optionally, it is also possible to determine that the foreground screen of the first terminal displays the page of the Internet phone application by means of feature matching; the process includes: extracting features of the foreground screen of the first terminal to obtain foreground screen features; judging whether the foreground screen features match the page of the Internet phone application, and if so, determining that the foreground screen of the first terminal displays the page of the Internet phone application; if not, the foreground screen of the first terminal does not display the page of the Internet phone application; wherein the foreground screen features can be determined based on the shape, color, feature points and area ratio displayed on the call page, so as to identify the foreground screen content without image recognition and using a relatively low amount of calculation. Optionally, if the foreground screen of the first terminal does not display the page of the Internet phone application, it is also possible to continue to judge whether the foreground screen of the first terminal displays the page of the Internet phone application.

[0070] In one embodiment, performing image recognition on the web phone application page to obtain the foreground screen content includes: performing image recognition on the web phone application page through a trained image recognition model to obtain the foreground screen content; the call status indicated by the foreground screen content includes the call status such as incoming call, outgoing call, two-way call or busy line to the third terminal. The foreground screen content includes the meaning conveyed by the text and the book.

[0071] By judging whether the page of the Internet phone application is displayed, the call status is detected from the perspective of changes in the foreground screen of the mobile phone. The monitoring mechanism of the foreground application does not limit the generation of the second foreground detection result. Since the second foreground detection result is based on image recognition, the Internet phone application that changes in the foreground screen can be accurately detected.

[0072] In one embodiment, performing image recognition on the web page of the Internet phone application to obtain the foreground screen content includes: performing image text recognition on the web page of the Internet phone application to obtain the page text.

[0073] Correspondingly, determining the second foreground call detection result according to the call status indicated by the foreground screen content includes: determining the second foreground call detection result according to the call status represented by the page text.

[0074] The page text is the text content of the web phone application page, which directly contains the call information of the current call status; therefore, the first foreground call detection result is compared with the second foreground call detection result, and the second foreground call detection result can identify the call information of the call status, so that the call information of the call coordination status is synchronized on the first terminal and the second terminal. The call information includes, but is not limited to, the information of the third terminal, certain buttons in the current call status, the identification information of the web phone application, etc.

[0075] In one embodiment, performing call detection based on the audio output mode of the first terminal includes: monitoring the call status of the Internet phone application; and determining a first audio output call detection result according to the call status of the Internet phone application.

[0076] Optionally, monitoring the call state of the Internet phone application refers to: monitoring the call state of the Internet phone application through a sound output mode monitor (such as voipstatemonitor) to accurately obtain the call state of the Internet phone application.

[0077] In one embodiment, a first audio call detection result is determined based on a call status of an Internet phone application, including: when the call status of the Internet phone application changes, determining the call status after the change by monitoring the Internet phone application; and determining the call status after the change as the first audio call detection result.

[0078] By directly monitoring the call status of the Internet phone application, the first audio call detection result can be accurately determined, and since the first terminal and the second terminal are both installed with the same Internet phone application, the synchronization of the first audio call detection result is high. In addition, the change event of the audio device and the audio receiving device can be accurately determined, and the second terminal can be intelligently notified to switch the audio device and the audio receiving device in time, providing a convenient user experience.

[0079] In one embodiment, the method further includes: receiving a sound output mode feedback result sent by the second terminal, and generating a second sound output call detection result according to the sound output mode feedback result; wherein the sound output mode feedback result is determined by the second terminal monitoring the call status of the Internet telephone application and according to the call status of the Internet telephone application.

[0080] The audio mode feedback result is generated by the second terminal, and can be fed back to the first terminal so that the two have synchronized audio mode call detection results. The generation method of the audio mode feedback result can be the same as the audio mode call detection result.

[0081] Since the second terminal generates the audio mode feedback result, the first terminal can directly use it as the second audio call detection result, which reduces the resource requirements of the first terminal for calculation and reduces the burden on the first terminal. For example: when the first terminal is a mobile phone and the second terminal is a car computer, this embodiment uses the computing power of the car computer to reduce the burden on the mobile phone, reduce the power consumption of the mobile phone, extend the battery life of the mobile phone, and improve the user experience.

[0082] In one embodiment, the method further includes: if during the process of performing call detection in the sound output mode of the first terminal, a sound output mode feedback result sent by the second terminal is received, then generating a sound output mode call detection result according to the sound output mode feedback result; wherein the sound output mode feedback result is determined by the second terminal monitoring the call status of the Internet phone application and according to the call status of the Internet phone application. Thus, the call status detection is performed synchronously by the first terminal and the second terminal to improve accuracy and synchronization.

[0083] In one embodiment, the method also includes: if the audio stream detection result sent by the second terminal is obtained, the audio stream detection result is one of the two or more call detection results; if the audio stream detection result sent by the second terminal is not obtained, the two or more detection results include at least two call detection results of a first foreground call detection result, a second foreground call detection result and a sound output mode call detection result, wherein the first foreground call detection result is a result based on the foreground application call detection, and the second foreground call detection result is a result based on the foreground screen content call detection; the sound output mode call detection result includes a first sound output call detection result detected by the first terminal, and a second sound output call detection result detected by the second terminal.

[0084] The audio stream detection result may be determined based on the call status information sent by the first terminal, or may be determined based on other methods.

[0085] Therefore, audio stream detection results alone are not the only way to detect call coordination status; when there are audio detection results, the detected call status is used as a reference factor to increase accuracy; at least when only one foreground call detection result or audio mode call detection result is obtained, accurate recognition can still be improved.

[0086] When there is no audio detection result, the call coordination state is accurately determined by at least two call detection results among the first foreground call detection result, the second foreground call detection result, and the sound output mode call detection result. In addition, the hardware of the second terminal can be used for data processing to reduce the power consumption of the first terminal.

[0087] The audio stream detection result is determined by the second terminal according to a template that the audio stream change sent by the first terminal complies with.

[0088] Specifically, the process of generating the audio stream detection result includes: the second terminal obtains the audio stream sent by the first terminal; the second terminal determines the audio template consistent with the change of the audio stream; and determines the call state indicated by the audio template as the audio stream detection result.

[0089] Thus, the second terminal determines the audio template that matches the change in the audio stream, and determines the audio stream detection result based on the information covered by the audio template; at least there is no need to monitor the communication content, and there is no need to perform deep packet inspection and analysis on the network traffic. In the case of wireless such as Bluetooth, the audio stream detection result can be fed back to accurately determine the call coordination status.

[0090] Optionally, the confidence levels of the voice-out mode call detection result, the first foreground call detection result, the second foreground call detection result and the audio stream detection result decrease in sequence.

[0091] Optionally, the first audio call detection result and the second audio call detection result are obtained through continuous monitoring, and the vehicle computer and the mobile phone are updated when the call status changes, which is almost synchronized, and there is no need to distinguish between the first audio call detection result and the second audio call detection result.

[0092] In an optional embodiment, the call coordination state between the first terminal and the second terminal is determined based on at least two call detection results, including: if at least two identical call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined based on the call state indicated by the identical call detection results; if at least two different call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined based on the call state indicated by the call detection result with higher confidence.

[0093] Therefore, various call detection results are classified by confidence, reducing the possibility of conflicts, thereby more accurately determining the call coordination status.

[0094] In an exemplary embodiment, the first terminal is a mobile phone, the second terminal is an electronic device such as a car computer, a television, a watch, a tablet, etc., and the Internet phone application is VOIP, which realizes the recognition and judgment of the call coordination state; its device is as follows Figure 3 As shown, through the combination and collaboration of these modules, we can achieve accurate identification and timely feedback of the VOIP call status on the mobile phone, improve the user experience, and provide users with convenient call control and device switching functions.

[0095] Specifically, the call coordination status simultaneously involves at least two call detection results in four dimensions: the first foreground call detection result, the second foreground call detection result, the audio output mode call detection result, and the audio stream detection result.

[0096] like Figure 4 As shown, in the process of monitoring the foreground application on the mobile phone, firstly, the foreground application is determined to be open by monitoring, and then, when the foreground application is VOIP, an incoming call event or an outgoing call event is determined; then one of the local display event and the full-screen display event is determined, and then the first foreground call detection result, the attribute of this call coordination state, is obtained. In this process, it is mainly through monitoring the events triggered by the foreground application of the mobile phone system (such as the user's opening behavior), and is executed according to the characteristics that VOIP is always in the foreground display, which can accurately identify the establishment and cancellation of the call status, ensure the synchronous display of the call status between the mobile phone and the smart terminal, and reduce the delay of the call status.

[0097] like Figure 5As shown, in the process of obtaining the second foreground call detection result, the mobile phone foreground screen change monitoring: monitor the changes in the mobile phone foreground screen, when the screen shows a VOIP incoming call banner, pop-up window or full-screen incoming call page, read the text on the foreground screen to analyze the call status, and determine whether the current call status is incoming call, outgoing call, two-way call or busy line. In this way, detailed information on the current call status is obtained.

[0098] As for the audio mode call detection result, it uses the audio mode monitor to monitor the audio mode of the mobile phone and the car computer to determine whether a VOIP call has been established and obtain detailed call status, such as calling, dialing, hanging up, etc. This module has high accuracy and synchronization, and can accurately determine the user's audio / receiving device change behavior, and intelligently notify the other terminal to switch the corresponding device in time, providing a convenient user experience.

[0099] The audio stream detection result can be obtained based on Bluetooth audio stream monitoring. When the user connects the mobile phone to the car computer via Bluetooth, the Bluetooth module of the car computer can monitor the changes in the Bluetooth audio stream to assist in inferring the change behavior of the VOIP call status on the mobile phone. At the same time, the hardware capabilities of the car computer can be used to reduce the power consumption of the mobile phone.

[0100] Based on this, this embodiment provides a cross-end VOIP call status identification method, which realizes high-accuracy and high-synchronization call status identification under various complex network and connection conditions while ensuring user privacy and security. This solution provides reliable call status identification capabilities through the collaborative work of multiple modules, greatly improving the user experience of call-related services on the car / mobile phone side. The advantage of this solution lies in its wide applicability. Regardless of the mobile phone platform system or terminal device, this solution is almost applicable. The collaborative work of multiple modules ensures that even if some modules fail on certain specific mobile phone / car platforms, normal identification work can still be maintained with high accuracy.

[0101] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0102] Based on the same inventive concept, the embodiment of the present application also provides a cross-end call status detection device for implementing the cross-end call status detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more cross-end call status detection devices provided below can refer to the limitations of the cross-end call status detection method above, and will not be repeated here.

[0103] In one embodiment, Figure 6 As shown, a cross-terminal call status detection device is provided, comprising:

[0104] A call detection module 602, configured to perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; each call detection corresponds to at least one call detection result;

[0105] The status judgment module 604 is used to determine the call coordination status between the first terminal and the second terminal according to the two or more call detection results if two or more call detection results are obtained. The call coordination status is the call status of any one of the first terminal and the second terminal.

[0106] In one embodiment, the call detection module 602 is used to:

[0107] Performing call detection based on the foreground application of the first terminal;

[0108] Performing call detection based on the foreground screen content of the first terminal; and

[0109] Call detection is performed based on the sound output mode of the first terminal.

[0110] In one embodiment, the call detection module 602 is used to:

[0111] Determining whether the foreground application running on the first terminal is an Internet phone application;

[0112] If it is the Internet phone application, determining the target call event triggered by the Internet phone application;

[0113] A first foreground call detection result is determined according to the target call event.

[0114] In one embodiment, the call detection module 602 is used to:

[0115] Determine a first target call event according to a call event notification of an event triggered by the Internet phone application, wherein the first target call event is one of an incoming call event and an outgoing call event;

[0116] A first foreground call detection result is determined according to the call state indicated by the first target call event.

[0117] In one embodiment, the call detection module 602 is used to:

[0118] Determining a second target call event according to a display event notification of an event triggered by the Internet phone application, wherein the second target call event is one of a partial display event and a full-screen display event;

[0119] The first foreground call detection result is determined according to the call status indicated by the first target call event and the call status indicated by the second target call event.

[0120] In one embodiment, the call detection module 602 is used to:

[0121] If the foreground screen of the first terminal displays a web phone application page, image recognition is performed on the web phone application page to obtain the foreground screen content;

[0122] A second foreground call detection result is determined according to the call status indicated by the foreground screen content.

[0123] In one embodiment, the call detection module 602 is used to:

[0124] Performing image and text recognition on the web page of the Internet phone application to obtain the page text;

[0125] A second foreground call detection result is determined according to the call status represented by the page text.

[0126] In one embodiment, the call detection module 602 is used to:

[0127] Monitor the call status of the Internet phone application;

[0128] A first outgoing audio call detection result is determined according to the call status of the Internet phone application.

[0129] In one embodiment, the call detection module 602 is used to:

[0130] When the audio mode feedback result sent by the second terminal is received, a second audio call detection result is generated according to the audio mode feedback result; wherein the audio mode feedback result is determined by the second terminal monitoring the call status of the Internet phone application and according to the call status of the Internet phone application.

[0131] In one embodiment, the state determination module 604 is used to:

[0132] If the audio stream detection result sent by the second terminal is obtained, the audio stream detection result is one call detection result among the two or more call detection results;

[0133] If the audio stream detection result sent by the second terminal is not obtained, the two or more call detection results include at least two call detection results of a first foreground call detection result, a second foreground call detection result and a sound output mode call detection result, wherein the first foreground call detection result is a result based on the foreground application call detection, and the second foreground call detection result is a result based on the foreground screen content call detection; the sound output mode call detection result includes a first sound output call detection result detected by the first terminal, and a second sound output call detection result detected by the second terminal.

[0134] In one of the embodiments, the audio stream detection result is determined by the second terminal according to a template that the audio stream change sent by the first terminal complies with.

[0135] In one of the embodiments, the confidence levels of the voice-out mode call detection result, the first foreground call detection result, the second foreground call detection result, and the audio stream detection result decrease in sequence.

[0136] Each module in the above-mentioned cross-terminal call status detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0137] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a cross-terminal call status detection method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0138] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] The embodiment of the present application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, the processors execute the steps of the cross-end call status detection method.

[0140] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute a cross-end call status detection method.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A cross-terminal call status detection method, characterized in that: include: Perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; Each call detection corresponds to at least one call detection result; If two or more call detection results are obtained, the call coordination state between the first terminal and the second terminal is determined according to the two or more call detection results, and the call coordination state is the call state of any one of the first terminal and the second terminal.

2. The method according to claim 1, characterized in that The performing call detection based on at least one of the foreground information and the sound output mode of the first terminal includes: Performing call detection based on the foreground application of the first terminal; Performing call detection based on the foreground screen content of the first terminal; and Call detection is performed based on the sound output mode of the first terminal.

3. The method according to claim 2, characterized in that The performing call detection based on the foreground application of the first terminal includes: Determining whether the foreground application running on the first terminal is an Internet phone application; If it is the Internet phone application, determining the target call event triggered by the Internet phone application; A first foreground call detection result is determined according to the target call event.

4. The method according to claim 3, characterized in that The determining the target call event triggered by the call application includes: Determine a first target call event according to a call event notification of an event triggered by the Internet phone application, wherein the first target call event is one of an incoming call event and an outgoing call event; The determining, according to the target call event, a first foreground call detection result includes: A first foreground call detection result is determined according to the call state indicated by the first target call event.

5. The method according to claim 4, characterized in that The determining of the target call event triggered by the call application further includes: Determining a second target call event according to a display event notification of an event triggered by the Internet phone application, wherein the second target call event is one of a partial display event and a full-screen display event; The determining a first foreground call detection result according to the call state indicated by the first target call event includes: The first foreground call detection result is determined according to the call status indicated by the first target call event and the call status indicated by the second target call event.

6. The method according to claim 2, characterized in that The performing call detection based on the foreground screen content of the first terminal includes: If the foreground screen of the first terminal displays a web phone application page, image recognition is performed on the web phone application page to obtain the foreground screen content; A second foreground call detection result is determined according to the call status indicated by the foreground screen content.

7. The method according to claim 6, characterized in that Perform image recognition on the web page of the Internet phone application to obtain the foreground screen content, including: Performing image and text recognition on the web page of the Internet phone application to obtain the page text; The step of determining a second foreground call detection result according to the call status indicated by the foreground screen content includes: A second foreground call detection result is determined according to the call status represented by the page text.

8. The method according to claim 2, characterized in that: The performing call detection based on the sound output mode of the first terminal includes: Monitor the call status of the Internet phone application; A first outgoing audio call detection result is determined according to the call status of the Internet phone application.

9. The method according to claim 1, characterized in that: The method further comprises: When the audio mode feedback result sent by the second terminal is received, a second audio call detection result is generated according to the audio mode feedback result; wherein the audio mode feedback result is determined by the second terminal monitoring the call status of the Internet phone application and according to the call status of the Internet phone application.

10. The method according to claim 1, characterized in that The method further comprises: If the audio stream detection result sent by the second terminal is obtained, the audio stream detection result is one call detection result among the two or more call detection results; If the audio stream detection result sent by the second terminal is not obtained, the two or more call detection results include at least two call detection results of a first foreground call detection result, a second foreground call detection result and a sound output mode call detection result, wherein the first foreground call detection result is a result based on the foreground application call detection, and the second foreground call detection result is a result based on the foreground screen content call detection; the sound output mode call detection result includes a first sound output call detection result detected by the first terminal, and a second sound output call detection result detected by the second terminal.

11. The method according to claim 10, characterized in that The audio stream detection result is determined by the second terminal according to a template that the audio stream change sent by the first terminal complies with.

12. The method according to claim 11, characterized in that The confidence levels of the audio mode call detection result, the first foreground call detection result, the second foreground call detection result, and the audio stream detection result decrease in sequence.

13. A cross-terminal call status detection device, characterized in that: include: A call detection module, configured to perform call detection based on at least one of the foreground information and the sound output mode of the first terminal; Each call detection corresponds to at least one call detection result; A status judgment module is used to determine the call coordination status between the first terminal and the second terminal according to the one or more call detection results if one or more call detection results are obtained, and the call coordination status is the call status of any one of the first terminal and the second terminal.

14. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor executes the steps of the cross-end call status detection method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images