Network state detection method, terminal equipment and computer readable storage medium

Abstract

The invention provides a network state detection method, terminal equipment and a computer readable storage medium, relates to the technical field of terminal equipment, and can realize network state detection based on a baseband processor and an application processor of the terminal equipment. The baseband processor can report the cache duration of the data message in the cache queue to a network detection module in the application processor, and after the network detection module receives the cache duration of the data message reported by the baseband processor, the data message can be cached according to the cache duration of the data message reported by the baseband processor. According to the uplink network state determining method and device, the network state is dynamically determined according to the cache condition of the data message in the baseband processor, the network quality can be obtained in time, and the accuracy of the network state can be ensured.

Description

Technical Field

[0001] The present application relates to the technical field of terminal equipment, and in particular to a network status detection method, a terminal equipment and a computer-readable storage medium. Background Art

[0002] With the rapid development of mobile Internet technology and the popularization of smart terminals, more and more users use terminal devices to access video services in cellular mobile networks, such as video calls, outdoor live broadcasts, online conferences, etc. The quality of audio and video data transmission depends largely on the network status of the cellular mobile network. When the network status is poor, network congestion may occur between the terminal device and the access network device, making it difficult to transmit audio and video data in the terminal device in a timely manner, which may easily cause problems such as video freezes. Therefore, the accuracy and timeliness of network status detection are the key points to ensure the quality of video transmission. Summary of the invention

[0003] The embodiments of the present application provide a network status detection method, a terminal device, and a computer-readable storage medium, which can improve the timeliness and accuracy of network status detection through information interaction between an application processor and a baseband processor.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, a network status detection method is provided, which is applied to a terminal device, wherein the terminal device includes a baseband processor and an application processor; the method includes: the baseband processor sends a cache time of a data message in a cache queue to the application processor; the cache queue is established for the baseband processor and is used to cache data messages to be sent; the application processor obtains the network status of the terminal device according to the cache time.

[0006] Exemplarily, when the cache time length is greater than a first preset time length threshold, the network state of the terminal device is a congested state, and when the cache time length is less than the first preset time length threshold, the network state of the terminal device is a normal state.

[0007] The solution provided in the first aspect above implements network status detection based on the information interaction between the baseband processor and the application processor of the terminal device. Among them, the baseband processor can report the cache duration of the data message in the cache queue to the network detection module in the application processor. After receiving the cache duration of the data message reported by the baseband processor, the network detection module can determine the uplink network status of the terminal device according to the cache duration of the data message reported by the baseband processor. Compared with realizing network status detection by using the message interaction between the sender and the receiver, the embodiment of the present application uses the cache status of the data message in the baseband processor to dynamically determine the network status, which can not only obtain the network quality in time, but also ensure the accuracy of the network status.

[0008] As a possible implementation, when the cache duration of the data message in the cache queue is greater than the first preset duration threshold, the baseband processor sends the cache duration of the data message in the cache queue to the application processor; or, after receiving the query instruction sent by the application processor, the baseband processor sends the cache duration of the data message in the cache queue to the application processor. In this way, the baseband processor can avoid frequently sending the cache duration to the application processor, and the power consumption caused by the transmission of the cache duration can be effectively reduced.

[0009] As a possible implementation method, the query instruction is an instruction sent when the network state is a congested state, and the application processor sends the query instruction to the baseband processor at a set time interval. In this way, when the network state is a congested state, the query instruction is sent to the baseband processor at a set time interval, and the change of the network state can be obtained in time, and then the sending code rate of the data message can be adjusted according to the change of the network state to improve the transmission quality.

[0010] As a possible implementation, after the application processor obtains the network status of the terminal device according to the cache duration, the method also includes: when the network status is a congested state, reducing the sending code rate of the data message; or, when the network status is a normal state, increasing the sending code rate of the data message; wherein the congested state is a network state in which the cache duration is greater than a first preset duration threshold, and the normal state is a network state in which the cache duration is less than the first preset duration threshold. In this way, when the network status is a congested state, by reducing the sending code rate of the data message, the amount of data sent can be reduced to prevent excessive data from further aggravating network congestion. When the network status is a normal state, by increasing the sending code rate of the data message, the transmission efficiency of the data message can be improved to improve the user experience.

[0011] As a possible implementation, the method further includes: the application processor obtains an adjustment range for reducing or increasing the transmission bit rate according to the duration interval in which the cache duration is located; wherein different duration intervals correspond to different adjustment ranges. In this way, the transmission bit rate is adjusted according to the adjustment range corresponding to the duration interval in which the cache duration is located, reducing the complexity of the bit rate adjustment, being able to adapt to the network environment in a timely manner, and improving the user experience.

[0012] As a possible implementation, the method further includes: the baseband processor sends a bandwidth threshold to the application processor; wherein the transmission code rate of the data message is less than the bandwidth threshold. Thus, by sending the bandwidth threshold to the application processor, it is possible to avoid the application processor exceeding the bandwidth threshold when adjusting the transmission code rate.

[0013] As a possible implementation, the method further includes: when the cache duration is greater than a second preset duration threshold, the application processor sends a cleanup instruction to the baseband processor; wherein the cleanup instruction is used to instruct the baseband processor to clean up the data packets in the cache queue; and the second preset duration threshold is greater than the first preset duration threshold. In this way, by cleaning up the data packets in the cache queue, when the network is congested, the bandwidth occupied by invalid data can be reduced, the transmission delay of valid data can be reduced, and the newly generated data packets can be sent in time.

[0014] As a possible implementation, the method further includes: when receiving a key frame request, the application processor sends a cleanup instruction to the baseband processor; wherein the cleanup instruction is used to instruct the baseband processor to clean up the data messages in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device. In this way, by cleaning up the data messages in the cache queue, the bandwidth occupied by invalid data can be reduced when the network is congested, so that the latest key frame can be sent to the receiving party in a timely manner.

[0015] As a possible implementation, the method further includes: when a key frame request is received and the cache duration is greater than a third preset duration threshold, the application processor sends a cleanup instruction to the baseband processor; wherein the cleanup instruction is used to instruct the baseband processor to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device. In this way, by cleaning up the data packets in the cache queue through the cache duration and the key frame request, it is possible to avoid the problem that when the data receiving device frequently initiates key frame requests, the re-encoded key frame is deleted before it is sent out from the cache queue, and it can be ensured that the latest key frame can be successfully sent to the recipient.

[0016] As a possible implementation, the cleaning instruction includes message marking information, and the cleaning instruction is used to instruct the baseband processor to clean up the data messages with the same message identification information in the cache queue. In this way, by cleaning up the messages in the cache queue through the message marking information, the data messages can be accurately cleaned up and the waste of resources can be reduced.

[0017] As a possible implementation method, the data message is an audio and video data message generated by the instant messaging software. In this way, the video freeze during instant messaging can be effectively reduced.

[0018] In a second aspect, a terminal device is provided, which includes a baseband processor and an application processor; the baseband processor includes a perception and reporting module, and the application processor includes an interactive service module and a network detection module; the interactive service module is used to exchange information between the perception and reporting module and the network detection module; the perception and reporting module is used to send a cache time of data packets in a cache queue to the network detection module through the interactive service module, and the cache queue is established for the baseband processor and is used to cache data packets to be sent; the network detection module is used to obtain the network status of the terminal device according to the cache time.

[0019] The interactive service module encapsulates an interactive API interface between the baseband processor and the application layer. Exemplarily, the interactive API interface may include an information upload API interface for the perception reporting module to report information to the network detection module, and an information sending API interface for the network detection module to send information to the perception reporting module.

[0020] Exemplarily, when the cache time length is greater than a first preset time length threshold, the network state of the terminal device is a congested state, and when the cache time length is less than the first preset time length threshold, the network state of the terminal device is a normal state.

[0021] The solution provided in the second aspect above implements network status detection based on the information interaction between the baseband processor and the application processor of the terminal device. Among them, the baseband processor can report the cache duration of the data message in the cache queue to the network detection module in the application processor. After receiving the cache duration of the data message reported by the baseband processor, the network detection module can determine the uplink network status of the terminal device according to the cache duration of the data message reported by the baseband processor. Compared with realizing network status detection by using the message interaction between the sender and the receiver, the embodiment of the present application uses the cache status of the data message in the baseband processor to dynamically determine the network status, which can not only obtain the network quality in time, but also ensure the accuracy of the network status.

[0022] As a possible implementation method, the sensing and reporting module is used to send the cache duration of the data message in the cache queue to the network detection module through the interactive service module when the cache duration of the data message in the cache queue is greater than the first preset duration threshold; or, after receiving the query instruction sent by the network detection module through the interactive service module, send the cache duration of the data message in the cache queue to the network detection module through the interactive service module. In this way, the baseband processor can be prevented from frequently sending the cache duration to the application processor, and the power consumption caused by the transmission of the cache duration can be effectively reduced.

[0023] As a possible implementation method, the query instruction is an instruction sent when the network state is a congested state, and the application processor sends the query instruction to the baseband processor at a set time interval. In this way, when the network state is a congested state, the query instruction is sent to the baseband processor at a set time interval, and the change of the network state can be obtained in time, and then the sending code rate of the data message can be adjusted according to the change of the network state to improve the transmission quality.

[0024] As a possible implementation method, the network detection module is also used to: when the network state is a congested state, reduce the sending code rate of the data message; or, when the network state is a normal state, increase the sending code rate of the data message; wherein the congested state is a network state in which the cache duration is greater than a first preset duration threshold, and the normal state is a network state in which the cache duration is less than the first preset duration threshold. In this way, when the network state is a congested state, by reducing the sending code rate of the data message, the amount of data sent can be reduced to prevent excessive data from further aggravating network congestion. When the network state is a normal state, by increasing the sending code rate of the data message, the transmission efficiency of the data message can be improved to improve the user experience.

[0025] As a possible implementation method, the network detection module is specifically used to: obtain an adjustment range for reducing or increasing the sending bit rate according to the time interval in which the cache duration is located; wherein different time intervals correspond to different adjustment ranges. In this way, the sending bit rate is adjusted according to the adjustment range corresponding to the time interval in which the cache duration is located, reducing the complexity of the bit rate adjustment, being able to adapt to the network environment in a timely manner, and improving the user experience.

[0026] As a possible implementation, the perception reporting module is further used to: send a bandwidth threshold to the network detection module through the interactive service module; wherein the transmission code rate of the data message is less than the bandwidth threshold. In this way, by sending the bandwidth threshold to the application processor, it is possible to avoid the application processor exceeding the bandwidth threshold when adjusting the transmission code rate.

[0027] As a possible implementation, when the cache duration is greater than the second preset duration threshold, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; wherein the cleanup instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue; the second preset duration threshold is greater than the first preset duration threshold. In this way, by cleaning up the data packets in the cache queue, the bandwidth occupied by invalid data can be reduced when the network is congested, and the transmission delay of valid data can be reduced, so that the latest generated data packets can be sent in time.

[0028] As a possible implementation, when receiving a key frame request, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; wherein the cleanup instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device. In this way, by cleaning up the data packets in the cache queue, the bandwidth occupied by invalid data can be reduced when the network is congested, so that the latest key frame can be sent to the receiver in time.

[0029] As a possible implementation method, when a key frame request is received and the cache duration is greater than the third preset duration threshold, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; wherein the cleanup instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device. In this way, by cleaning up the data packets in the cache queue through the cache duration and the key frame request, it is possible to avoid the problem that when the data receiving device frequently initiates key frame requests, the re-encoded key frame is deleted before it is sent out from the cache queue, and it can ensure that the latest key frame can be successfully sent to the recipient.

[0030] As a possible implementation method, the cleaning instruction includes message marking information, and the cleaning instruction is used to instruct the perception reporting module to clean up the data messages with the same message identification information in the cache queue. In this way, by cleaning up the messages in the cache queue through the message marking information, accurate cleaning of data messages can be achieved, reducing the waste of resources.

[0031] As a possible implementation method, the data message is an audio and video data message generated by the instant messaging software. In this way, the video freeze during instant messaging can be effectively reduced.

[0032] According to a third aspect, an electronic device is provided, comprising: a processor coupled to a memory, the processor being configured to execute a computer program stored in the memory, so that the electronic device executes the method described in any possible implementation manner in the first aspect.

[0033] According to a fourth aspect, a processor is provided, wherein the processor is used to execute the method described in any possible implementation manner of the first aspect.

[0034] In a fifth aspect, a communication system is provided, which includes one or more terminal devices and one or more network devices.

[0035] In a sixth aspect, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are executed on a computer, the computer executes the method described in any possible implementation manner in the first aspect.

[0036] In a seventh aspect, a computer program product is provided, comprising a computer program or instructions, which, when executed on a computer, causes the computer to execute the method described in any possible implementation of the first aspect.

[0037] In an eighth aspect, a chip system is provided, the chip system comprising a processing circuit and a storage medium, wherein the storage medium stores computer program instructions; when the computer program instructions are executed by the processing circuit, the method in any possible implementation of the first aspect is implemented. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0038] In a ninth aspect, a terminal device is provided, comprising a memory and one or more processors; the memory is coupled to the one or more processors, and the one or more processors are used to execute a computer program stored in the memory, so that the terminal device executes a method as in any possible implementation of the first aspect; wherein the one or more processors include a baseband processor and an application processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the principle of network status detection in conventional technology;

[0040] Figure 2 A schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0041] Figure 3 One of the application scenario schematic diagrams of the communication system provided in the embodiment of the present application;

[0042] Figure 4 A second schematic diagram of an application scenario of a communication system provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0044] Figure 6 One of the principle schematic diagrams of the network status detection method provided in the embodiment of the present application;

[0045] Figure 7 The second schematic diagram of the principle of the network status detection method provided in the embodiment of the present application;

[0046] Figure 8 The third principle diagram of the network status detection method provided in the embodiment of the present application;

[0047] Fig. 9 One of the flow charts of the network status detection method provided in the embodiment of the present application;

[0048] Fig.10 One of the interactive timing diagrams of the network status detection method provided in the embodiment of the present application;

[0049] Fig.11 One of the schematic diagrams of the receiving frame rate effect provided in the embodiment of the present application;

[0050] Fig.12 A fourth schematic diagram of the principle of the network status detection method provided in an embodiment of the present application;

[0051] Fig.13 The second interactive sequence diagram of the network status detection method provided in the embodiment of the present application;

[0052] Fig.14 A fifth schematic diagram of the principle of the network status detection method provided in the embodiment of the present application;

[0053] Fig.15 The third interactive sequence diagram of the network status detection method provided in the embodiment of the present application;

[0054] Fig.16 The second schematic diagram of the receiving frame rate effect provided in the embodiment of the present application;

[0055] Fig.17 A second flowchart of a network status detection method provided in an embodiment of the present application;

[0056] Fig.18 A third flowchart of the network status detection method provided in an embodiment of the present application;

[0057] Fig.19 A fourth flowchart of a network status detection method provided in an embodiment of the present application;

[0058] Fig. 20 The fifth flowchart of the network status detection method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0060] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0061] In addition, in the embodiments of the present application, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word example is intended to present concepts in a concrete way.

[0062] As described in the background technology, with the rapid development of mobile Internet technology and the popularization of smart terminals, more and more users use terminal devices to perform video services in cellular mobile networks, such as video calls, outdoor live broadcasts, online conferences, etc. The quality of audio and video data transmission depends to a large extent on the network status of the cellular mobile network. When the network status is poor, network congestion may occur between the terminal device and the access network device, resulting in difficulty in timely transmission of audio and video data in the terminal device, which may easily cause problems such as video freeze. Therefore, the accuracy and timeliness of network status detection are the key points to ensure the quality of video transmission.

[0063] Currently, the detection of network quality during audio and video data transmission is mostly completed through real-time transport protocol (RTP) and real-time transport control protocol (RTCP).

[0064] RTP / RTCP is a protocol standard proposed by the Internet Engineering Task Force (IETF), and the corresponding RFC document is RFC3550. The RTP protocol details the standard data message format for transmitting audio and video on the Internet. RTP is used to provide end-to-end real-time transmission services for various multimedia data that require real-time transmission, such as voice, images, and fax. RTP provides time information and stream synchronization for end-to-end real-time transmission on the Internet, but does not guarantee the quality of service. The quality of service is provided by RTCP.

[0065] The types of RTCP messages include sender report (SR) and receiver report (RR), where the sender refers to the application or terminal device that sends the RTP data message, and the receiver refers to the application or terminal device that only receives the RTP data message.

[0066] The sender report SR consists of three parts. The first part is the RTCP header, which includes the version, padding, length, synchronization source identifier (SSRC) and other information. The second part is the sender information, including the network time protocol (NTP) timestamp, RTP timestamp, number of sent packets and number of sent bytes. The third part is the receiving report block, which carries statistical information about the continuous code stream reception, including the number of lost packets, the maximum sequence number of received data packets, the arrival time jitter, the time of receiving the last SR, the delay of receiving the last SR, etc. The receiver report RR is similar to the sender report SR, but does not include the sender information, which will not be repeated here.

[0067] like Figure 1 As shown, Figure 1 This is a schematic diagram of the principle of network status detection in conventional technology. The sender sends RTP messages (audio and video streams) and SR messages to the receiver. The receiver receives the RTP messages and sends RR messages to the sender. The sender can calculate the packet loss rate, jitter, delay and other information through the contents of the SR messages and RR messages, so as to infer the network transmission quality and adjust the sending rate based on the inferred network quality to improve network congestion.

[0068] However, this type of network status detection relies on the message reply from the receiving end, which has a lag and the detection result is not accurate. Moreover, when the network is interrupted and the RTCP message cannot be transmitted normally, the detection mechanism fails and the sending end cannot perceive the network abnormality in time.

[0069] Based on the above problems, an embodiment of the present application provides a network status detection method, which implements network status detection based on the baseband processor and application processor of the terminal device. Among them, the baseband processor can report the cache duration of the data message in the cache queue to the network detection module in the application processor. After receiving the cache duration of the data message reported by the baseband processor, the network detection module can determine the uplink network status of the terminal device according to the cache duration of the data message reported by the baseband processor. Compared with realizing network status detection by using the message interaction between the sender and the receiver, the embodiment of the present application uses the cache status of the data message in the baseband processor to dynamically determine the network status, which can not only obtain the network quality in time, but also ensure the accuracy of the network status.

[0070] To facilitate understanding, an explanation of relevant terms is given first.

[0071] Socket: Also known as "socket", applications usually send requests to the network or respond to network requests through "sockets".

[0072] Quintuple: usually refers to source IP address, source port, destination IP address, destination port and transport layer protocol.

[0073] Key frame: refers to the frame that plays a decisive role among the frames that make up an animation.

[0074] The network status detection method provided in the embodiment of the present application can be applied in the communication system of the mobile communication network, such as the 4th generation (4G) mobile communication system, such as the long term evolution (LTE) system, the world-wide interoperability for microwave access (WiMAX) communication system, the fifth generation (5G) mobile communication system, such as the new radio (NR) system, and the future communication system, such as the sixth generation (6G) mobile communication system. The communication system may include at least one terminal device and at least one access network device. The terminal device and the access network device can exchange information based on air interface resources, such as signaling, audio stream, or video stream.

[0075] See also Figure 2 , Figure 2 A schematic diagram of the architecture of a communication system applicable to the network status detection method provided in the embodiment of the present application. Figure 2As shown, the communication system may include at least one terminal device, at least one access network device and at least one server. A communication connection may be established between the terminal device and the access network device, information may be exchanged between the terminal device and the access network device that have established the communication connection, and a communication connection may be established between the access network device and the server.

[0076] Taking at least one terminal device including terminal device 100a and terminal device 100b, at least one access network device including access network device 200a and access network device 200b, and at least one server including server 300a and server 300b as an example, a communication connection can be established between terminal device 100a and access network device 200a to realize information interaction between terminal device 100a and access network device 200a. A communication connection can be established between terminal device 100b and access network device 200b to realize information interaction between terminal device 100b and access network device 200b. Access network device 200a can establish a communication connection with server 300a to obtain service information from server 300a and send it to terminal device 100a, or receive service information from terminal device 100a and send it to server 300a. The access network device 200b may establish a communication connection with the server 300a to obtain information from the server 300a and send it to the terminal device 100b, or receive information from the terminal device 100b and send it to the server 300a. The terminal device 100a or the terminal device 100b may also establish a connection with the server 300b to send signaling to the server 300b, or receive signaling from the server 300b. For example, the signaling may be call signaling.

[0077] In addition, at least one server may further include a server 300c. The terminal device 100a or the terminal device 100b may further establish a connection with the server 300c to obtain from the server 300c a destination address of a server (such as the server 300a or the server 300b) corresponding to the information to be sent, such as a destination Internet protocol (IP) address, and a destination port number.

[0078] Among them, the above-mentioned access network equipment is a device located on the network side of the above-mentioned communication system and has a wireless transceiver function or a chip or chip system that can be set in the device, which is used to provide network (such as cellular mobile network) services for terminal devices. The above-mentioned access network equipment includes but is not limited to various types of base stations, node B (Node B, NB), base station controller (basestation controller, BSC), base transceiver station (base transceiver station, BTS), home base station (for example, home evolved NodeB, or home Node B, HNB), baseband unit (baseband unit, BBU), wireless relay node, wireless backhaul node, transmission point (transmission and reception point, TRP or transmission point, TP), etc., and can also be 5G, such as, gNB in ​​the new radio (new radio, NR) system, or, transmission point (TRP or TP), one or a group of (including multiple antenna panels) antenna panels of a base station in the 5G system, or, it can also be a network node constituting a gNB or a transmission point, such as a baseband unit (BBU), or, a distributed unit (distributed unit, DU), a road side unit (road side unit, RSU) with base station function, etc.

[0079] The above-mentioned server is a device with data processing and data receiving and sending functions or a chip or chip system that can be set in the device. For example, the server can be a signaling server, a server for interactive business information (such as a media server), a domain name system (DNS) server, or a terminal routing system (TRS) server. Exemplarily, the signaling server may include an account management server, which can be used to perform registration, information maintenance, and information query functions of the call device; in addition, the signaling server may also include a signaling forwarding server for transmitting call signaling such as calling, answering, and rejecting between devices. The media server can be used to establish a route between the media server and the end-side device, and is responsible for forwarding audio, video, and media control data between call devices. Under the framework of the RTP or RTCP protocol, the media server is responsible for forwarding RTP or RTCP messages. Both the DNS server and the TRS server can be used to provide the IP address and port number of the protocol message.

[0080] Exemplarily, the server 300a may be a server for exchanging service information, such as a media server; the server 300b may be a signaling server; and the server 300c may be a DNS server or a TRS server.

[0081] The terminal device is a terminal that accesses the communication system and has a wireless transceiver function or a chip or chip system that can be set in the terminal. The terminal device can also be called an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device. The terminal device in the embodiment of the present application can be a mobile phone, a tablet computer, a computer with a wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a vehicle-mounted terminal, an RSU with terminal function, etc. The terminal device of the present application may also be a vehicle-mounted module, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip or vehicle-mounted unit that is built into the vehicle as one or more components or units. The vehicle can implement the uplink transmission method provided by the present application through the built-in vehicle-mounted module, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip or vehicle-mounted unit.

[0082] based on Figure 2 The system architecture shown in the figure, the communication system provided by the embodiment of the present application can be applied to outdoor live broadcast, video call and online conference and other scenarios. Figure 3 As shown, Figure 3 This is a schematic diagram of an outdoor live broadcast scenario. After the terminal device corresponding to the live broadcast end generates an audio and video stream, it transmits the audio and video stream to the access network device. After receiving the audio and video stream, the access network device sends the audio and video stream to the media server, so that the media server forwards the audio and video stream to the terminal device corresponding to the audience end. In this way, the audience can watch the audio and video of the live broadcast end. Among them, the media server can forward the audio and video stream to the terminal device corresponding to the audience end through the access network device in the area where the audience end is located.

[0083] like Figure 4 As shown, Figure 4The diagram is a scene diagram of a video call and an online conference. After the terminal device of participant 1 generates an audio and video stream, it transmits the audio and video stream to the access network device. After receiving the audio and video stream, the access network device sends the audio and video stream to the media server, so that the audio and video stream is forwarded to the terminal device corresponding to participant 2 through the media server. In this way, participant 2 can watch the audio and video of participant 1. Correspondingly, when the terminal device corresponding to participant 2 generates an audio and video stream, it can also transmit the audio and video stream to the access network device through the media server. After receiving the audio and video stream, the access network device sends the audio and video stream to participant 1. In this way, participant 1 can watch the audio and video of participant 2. Among them, the audio and video generated by the terminal device corresponding to participant 2 can also be transmitted to the media server through the access network device in the area.

[0084] For ease of understanding, the structure of the terminal device provided in the embodiment of the present application is described below. Figure 5 , Figure 5 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application, such as Figure 5 As shown, the terminal device 100 includes a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 140, a wireless communication module 150, an audio module 160, a speaker 160A, a receiver 160B, a microphone 160C, an earphone interface 160D, a display screen 170, a sensor module 180, a camera 190, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0085] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0086] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0087] The controller can be the nerve center and command center of the terminal device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0088] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0089] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0090] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 190, etc. through different I2C bus interfaces.

[0091] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 160 via the I2S bus to achieve communication between the processor 110 and the audio module 160.

[0092] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 160 and the wireless communication module 150 can be coupled via a PCM bus interface. In some embodiments, the audio module 160 can also transmit audio signals to the wireless communication module 150 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0093] The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.

[0094] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 170 and the camera 190. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like.

[0095] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 190, the display 170, the wireless communication module 150, the audio module 160, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0096] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0097] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0098] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 140, the wireless communication module 150, the modem processor and the baseband processor.

[0099] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0100] The mobile communication module 140 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the terminal device 100. The mobile communication module 140 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0101] The mobile communication module 140 can receive electromagnetic waves through the antenna 1, and filter, amplify and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 140 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 140 can be set in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 140 can be set in the same device as at least some modules of the processor 110.

[0102] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through the display screen 170. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 140 or other functional modules.

[0103] The wireless communication module 150 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 150 can be one or more devices integrating at least one communication processing module. The wireless communication module 150 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0104] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 140, and antenna 2 is coupled to wireless communication module 150, so that terminal device 100 can communicate with the network and other devices through wireless communication technology.

[0105] The terminal device 100 implements the display function through a GPU, a display screen 170, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 170 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0106] The terminal device 100 can realize the shooting function through the ISP, the camera 190, the video codec, the GPU, the display screen 170 and the application processor.

[0107] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0108] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0109] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0110] The terminal device 100 can implement audio functions such as music playing and recording through the audio module 160, the speaker 160A, the receiver 160B, the microphone 160C, the headphone jack 160D, and the application processor.

[0111] The audio module 160 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 160 can also be used to encode and decode audio signals. In some embodiments, the audio module 160 can be arranged in the processor 110, or some functional modules of the audio module 160 can be arranged in the processor 110.

[0112] based on Figure 5 The structure shown in the embodiment of the present application takes the application processor and the baseband processor as an example to illustrate the network status detection method provided by the embodiment of the present application. Figure 6 , Figure 6 A schematic diagram of the principle of the network status detection method provided in the embodiment of the present application, such as Figure 6 As shown, the application processor is connected to the baseband processor (also known as a modem, a modem core or a baseband chip). The application processor may include functions for implementing the following functional modules: an application layer, an application framework layer, a kernel layer, a driver layer, and a network interface controller (NIC) layer, wherein the application layer may include a series of application packages, such as applications such as calls, music, and videos, and the application layer may also include coding modules, such as audio coding and video coding. The application framework layer provides an application programming interface (API) and a programming framework for the application of the application layer. A kernel protocol stack is built in the kernel layer, and the kernel protocol stack includes a user datagram protocol (UDP) layer, an Internet Protocol (IP) layer, and an Ethernet protocol (Ethernet) layer. For the specific implementation of the kernel protocol stack, reference may be made to the implementation principle of the kernel protocol stack in the prior art, which will not be repeated here. The driver layer includes a device driver. The Network Interface Controller (NIC) layer includes a forwarding module for forwarding data between the application processor and the baseband processor.

[0113] The above-mentioned baseband processor may include functions for implementing the following functional modules: packet data convergence protocol (PDCP) layer, radio link control layer protocol (RLC) layer, medium access control (MAC) layer and physical layer (PHY).

[0114] In the instant messaging scenario, when the application generates an audio and video stream, the generated audio and video stream is encoded by the encoding module, encapsulated by the RTP protocol and the RTCP protocol, and then encapsulated in turn by the UDP layer, IP layer, and Ethernet layer in the kernel protocol stack to obtain a data message, and the data message is sent to the device driver, which is sent to the forwarding module by the device driver. After receiving the data message, the forwarding module sends the data message to the baseband processor, and the PDCP layer in the baseband processor sends the data message to the RLC layer. After the data message is processed by the RLC layer, the MAC layer, and the PHY layer in turn, it is sent to the access network device through the air interface, and the data message is forwarded to the server through the access network device, so that the data message is forwarded to the terminal device corresponding to the recipient through the server. It can be understood that the data message is the audio and video data message generated by the instant messaging software.

[0115] See also Figure 7 In an embodiment of the present application, the baseband processor can establish a cache queue based on the PDCP layer, and the data message sent by the forwarding module to the baseband processor can be placed in the cache queue for caching. After that, the PDCP layer can send the data messages in the cache queue in sequence according to the order in which the data messages are placed. When the data message is successfully sent to the access network device, the data message will be cleared from the cache queue accordingly.

[0116] In the embodiment of the present application, for each data message, after entering the cache queue, an entry time will be generated accordingly, and by monitoring the entry time of each data message, the cache duration of each data message in the cache queue can be obtained. Among them, the cache duration of each data message in the cache queue is the difference between the current time and the entry time of the data message entering the cache queue.

[0117] Considering that when network congestion occurs, the data transmission rate is slow, which will cause the cache time of data packets in the cache queue to become longer. Therefore, the embodiment of the present application can detect whether the network is congested by the cache time of data packets in the cache queue.

[0118] In order to obtain the cache duration of the data message in the cache queue and detect the network status based on the cache duration, in the embodiment of the present application, the application layer also includes a network detection module and an interactive toolkit (interaction SDK), and the application framework layer also includes an interactive service module. Among them, the network detection module is used to detect the network status and adjust the network status, and the interactive service module is used to realize cross-core information interaction between the application layer and the baseband processor in the application processor.

[0119] In the embodiment of the present application, the interactive service module encapsulates an interactive API interface between the baseband processor and the application layer. Exemplarily, the interactive API interface may include an information upload API interface for the baseband processor to report information to the application layer, and an information delivery API interface for the application layer to deliver information to the baseband processor.

[0120] The interactive SDK is used to expose the interactive API interface encapsulated by the interactive service module to the application layer, so that the network detection module in the application layer can interact with the interactive service module through the interactive SDK, and then realize cross-core information interaction between the interactive service module and the baseband processor, wherein data interaction is carried out between the interactive service module and the baseband processor through the forwarding module.

[0121] based on Figure 7 In the terminal device architecture diagram shown in the embodiment of the present application, the baseband processor can monitor the cache duration of the data message in the cache queue, determine the cache duration of the data message in the cache queue, and then report the cache duration of the data message in the cache queue to the interactive service module through the forwarding module, and the interactive service module transmits the cache duration to the network detection module in the application layer through the interactive SDK. After obtaining the cache duration, the network detection module can determine the network status of the current terminal device based on the cache duration.

[0122] Exemplarily, the cache time of data packets in the cache queue can be monitored through the PDCP in the baseband processor, and the cache time of data packets in the cache queue can be reported to the information upload API interface in the interactive service module through the forwarding module, and the cache time can be transmitted to the network detection module in the application layer through the interactive SDK.

[0123] It should be noted that in the embodiment of the present application, the PDCP in the baseband processor is used as a perception and reporting module to monitor the cache duration of the data message in the cache queue and report the cache duration. The monitoring of the cache duration of the data message in the cache queue and the reporting of the cache duration can also be achieved through other modules with perception and reporting functions in the baseband processor, which can be set according to actual conditions, and the embodiment of the present application does not make specific limitations.

[0124] In some embodiments of the present application, after obtaining the cache time, the network detection module can compare the cache time with a first preset time threshold. If the cache time is greater than the first preset time threshold, it can be determined that the current network state is a congested state. If the cache time is less than or greater than the first preset time threshold, it can be determined that the current network state is a normal state, that is, no congestion occurs.

[0125] The first preset duration threshold can be set according to the cache duration of the data message in the cache queue under normal network conditions. Exemplarily, the first preset duration threshold is greater than the cache duration of the data message in the cache queue under normal network conditions. For example, in a real-time fourth generation mobile communication technology (4G) network, when the network state is normal, that is, when the network is not congested, the cache duration of the data message in the cache queue is less than 50ms, then the first preset duration threshold can be greater than 50ms, and preferably, the first preset duration threshold can be set to 200ms.

[0126] The specific value of the first preset duration threshold can be set according to business requirements, and the embodiment of the present application does not impose any specific limitation.

[0127] Considering that there are multiple data packets stored in the cache queue, and the time when each data packet enters the cache queue may be different, the cache duration of each data packet in the cache queue is different. In order to reduce complexity and improve processing efficiency, in an embodiment of the present application, the baseband processor can only report the cache duration of the data packet that first enters the current cache queue, that is, the baseband processor can only report the cache duration of the data packet that enters the current cache queue the earliest. After the network detection module obtains the cache duration of the data packet that enters the current cache queue the earliest reported by the baseband processor, the cache duration is compared with the first preset duration threshold. If the cache duration is greater than the first preset time threshold, it is determined that the current network state is a congested state. If the cache duration is less than or equal to the first preset duration threshold, it can be determined that the current network state is a normal state.

[0128] In an optional implementation, the baseband processor may report the cache time of each data message in the cache queue to the network detection module, and after obtaining the cache time of each data message in the cache queue, the network detection module may calculate the average cache time of each data message to obtain the average cache time, and then compare the average cache time with a first preset time threshold. If the average cache time is greater than the first preset time threshold, it is determined that the current network state is a congested state. If the average cache time is less than or equal to the first preset time threshold, it can be determined that the current network state is a normal state.

[0129] Taking into account that the data packet at the head of the cache queue is usually the data packet to be sent out first in the cache queue, in order to reduce complexity, obtain the current network status in time, and improve processing efficiency, in an embodiment of the present application, the baseband processor may also only report the cache time of the data packet at the head of the cache queue. After the network detection module obtains the cache time of the data packet at the head of the cache queue reported by the baseband processor, the cache time of the data packet at the head of the queue is compared with the first preset time threshold. When the cache time of the data packet at the head of the queue is greater than the first preset time threshold, it can be determined that the current network is in a congested state. When the cache time of the data packet at the head of the queue is not greater than the first preset time threshold, it can be determined that the current network state is normal.

[0130] Taking into account that the baseband processor will consume power when reporting the cache time of the data message in the cache queue to the network detection module, in order to reduce power consumption, in an embodiment of the present application, the baseband processor can transmit the cache time of the data message in the cache queue to the network detection module when the cache time of the data message in the cache queue is greater than the first preset time threshold. In this way, frequent transmission of the cache time can be avoided, which can effectively reduce power consumption.

[0131] Exemplarily, the baseband processor may report the cache duration of the data message at the head of the cache queue when the cache duration of the data message at the head of the cache queue is greater than a first preset duration threshold.

[0132] It can be understood that when the resources and data processing capabilities of devices such as the baseband processor and application processor are sufficient, the baseband processor can also report the cache duration of the data packets in the cache queue in real time, or report the cache duration of the data packets in the cache queue according to a preset period, which can be set specifically according to actual needs.

[0133] In an embodiment of the present application, the network detection module can send a first preset duration threshold to the baseband processor through the information sending API interface in the interactive service module, so that the baseband processor can determine the transmission time point of the cache duration of the data packet in the cache queue based on the first preset duration threshold, that is, when the monitoring obtains that the cache duration is greater than the first preset duration threshold, the cache duration of the data packet in the cache queue is reported to the network detection module.

[0134] In order to improve the quality of data transmission, in the embodiment of the present application, after the network detection module obtains the network status, the sending code rate of the data message can also be adjusted according to the network status, so that the data volume of the data message to be sent can meet the current network throughput, thereby improving the data transmission quality. Among them, the sending code rate of the data message refers to the amount of data transmitted by the data message per unit time.

[0135] For example, if the current network state is congested, the sending code rate of the data message can be reduced to reduce the amount of data sent and prevent excessive data from further aggravating network congestion. If the current network state is normal, the sending code rate of the data message can be increased to improve the transmission efficiency of the data message and improve the user experience.

[0136] In order to quickly alleviate network congestion, when reducing the data message sending rate, the longer the cache time is, the greater the reduction in the sending rate is. In order to improve data transmission efficiency and user experience, when increasing the data message sending rate, the shorter the cache time is, the greater the increase in the sending rate is.

[0137] In order to promptly obtain the effect of the network status after the sending code rate is adjusted, in an embodiment of the present application, after obtaining that the network status is a congested state, the network detection module can also start a polling thread and send a query instruction to the baseband processor at a set time interval to instruct the baseband processor to report the cache time length of the data packet in the cache queue.

[0138] Based on this, in an embodiment of the present application, the baseband processor may also send the cache duration of the data message in the cache queue to the network detection module after receiving the query instruction sent by the network detection module.

[0139] During the polling process, for each query, after the network detection module obtains the cache time length sent by the baseband processor, if the cache time length is still greater than the first preset time length threshold, that is, when the network status obtained by polling is still in a congested state, the sending bit rate of the data message can continue to be reduced. If the cache time length is less than the first preset time length threshold, that is, when the network congestion status obtained by polling is released, the polling thread can be terminated and the sending bit rate of the data message can be increased.

[0140] In the embodiment of the present application, the terminal device has a bandwidth threshold, that is, the maximum amount of data that can be transmitted per unit time. When the baseband processor sends a data message to the access network device, it needs to first obtain the bandwidth authorized to the terminal device from the access network device, and then send it over the air interface according to the authorized bandwidth, so as to avoid the transmission rate of the data message exceeding the authorized bandwidth, wherein the bandwidth authorized to the terminal device by the access network device is the bandwidth threshold. Therefore, when the transmission rate of the data message is increased, the increased data rate cannot exceed the bandwidth threshold.

[0141] In order to facilitate the network detection module to adjust the transmission code rate of the data message, in the embodiment of the present application, when the baseband processor transmits the cache duration to the network detection module, it can also transmit the bandwidth threshold to the network detection module. Accordingly, the query instruction from the network detection module to the baseband processor is used to instruct the baseband processor to report the cache duration and bandwidth threshold of the data message in the current cache queue.

[0142] See also Figure 8 The PDCP in the baseband processor monitors the cache time and bandwidth threshold of the data packets in the cache queue, and reports the cache time and bandwidth threshold of the data packets in the cache queue to the interactive service module through the forwarding module. The interactive service module transmits the cache time and bandwidth threshold to the network detection module in the application layer through the interactive SDK. After obtaining the cache time, if the cache time is greater than the first preset time threshold, the network detection module can reduce the sending code rate of the data message, and can start the polling thread, and periodically send query instructions to the interactive service module through the interactive SDK, and the interactive service module sends the query instructions to the PDCP in the baseband processor through the forwarding module. According to the query instruction, the PDCP reports the cache time and bandwidth threshold of the data message in the current cache queue to the forwarding module, and the forwarding module reports the cache time and bandwidth threshold of the data message in the current cache queue to the interactive service module. The interactive service module uploads the cache time and bandwidth threshold of the data message in the current cache queue to the network detection module through the interactive SDK. The network detection module detects the cache time of the data message in the current cache queue. If the cache time of the data message in the current cache queue is still greater than the first preset time threshold, that is, the network is continuously congested, the sending code rate of the data message is reduced. If the cache duration of the data message in the current cache metric is less than the first preset duration threshold, that is, the network congestion is relieved, the polling thread is terminated, and the transmission code rate of the data message is increased with the bandwidth threshold as the upper limit.

[0143] In an embodiment of the present application, the network detection module can transmit the query instruction to the baseband processor through the information delivery API interface in the interactive service module.

[0144] In order to facilitate the network detection module to adjust the sending code rate of the data message, so as to adapt to the network environment in time and improve the user experience, in the embodiment of the present application, after obtaining the cache time of the data message, the adjustment range of the sending code rate can be obtained according to the time interval in which the cache time is located. Different time intervals correspond to different adjustment ranges.

[0145] Taking the first preset duration threshold of 200ms as an example, when the first preset duration threshold is 200ms, the duration intervals can be set to (0, 200ms), (200ms, 500ms), [500ms, 1000ms), [1000ms, +∞). When the cache duration is in the duration interval of (0, 200ms), it indicates that the network is not congested. Therefore, the adjustment range of the sending bit rate can be set to increase by 20%. When the cache duration is in the duration interval of (200ms, 500ms), the adjustment range of the sending bit rate can be set to increase by 20%. s), it indicates that the network congestion is relatively mild. Therefore, the adjustment range of the sending bit rate can be set to decrease by 20%. When the cache time is in the time interval of [500ms, 1000ms), it indicates that the network congestion is relatively serious. Therefore, the adjustment range of the sending bit rate can be set to decrease by 40%. When the cache time is in the time interval of [1000ms, +∞), it indicates that the network congestion is very serious. Therefore, the adjustment range of the sending bit rate can be set to decrease by 60%.

[0146] like Fig. 9 As shown, Fig. 9 A flow chart of a network status detection method provided by an embodiment of the present application. First, the baseband processor monitors the cache duration in the cache queue. When the cache duration is greater than 200ms, the baseband processor reports the cache duration to the interactive service module. The interactive service module uploads the cache duration to the application layer. The application layer determines the cache duration. When the cache duration exceeds 200ms, the polling thread is started. Through the timer, a query instruction is sent to the baseband processor at every set time interval (such as 500ms) to determine the duration interval in which the cache duration is located. If the cache duration is within the duration interval of (200ms, 500ms), the sending code rate of the data message is reduced by 20%. If the cache duration is within the duration interval of [500ms, 1000ms), the sending code rate of the data message is reduced by 40%. If the cache duration is within the duration interval of [1000ms, +∞), the sending code rate of the data message is reduced by 60%.

[0147] If the cache time obtained by polling is less than 200ms, the congestion is determined to be relieved, the polling thread is stopped, and it is determined whether the sending code rate of the data message is less than the bandwidth threshold after increasing by 20%. If it is less than the bandwidth threshold, the sending code rate of the data message is increased by 20%. If it is greater than the bandwidth threshold, the sending code rate of the data message is adjusted to the bandwidth threshold.

[0148] In some implementations, when it is determined that the sending code rate is still less than the bandwidth threshold after increasing by 20%, the sending code rate may be increased again until the sending code rate of the data message is adjusted to the bandwidth threshold or to a set code rate threshold.

[0149] It can be understood that after the application layer determines that the congestion is relieved and increases the sending bit rate, it can wait for the next report from the baseband processor. When the baseband processor monitors again and finds that the cache time of the data message in the cache queue is greater than 200ms, it will trigger the reporting process again, that is, report the cache time of the data message in the cache queue.

[0150] It should be noted that in the embodiment of the present application, a minimum sending code rate is set. In the process of polling to adjust the sending code rate, the reduced sending code rate cannot be lower than the minimum sending code rate threshold to avoid service interruption.

[0151] It can be understood that after adjusting the sending bit rate of the data message, the network detection module can send the adjusted sending bit rate to the encoding module, so that the encoding module can encode and send the generated audio and video stream according to the adjusted sending bit rate.

[0152] The embodiment of the present application sets the correspondence between the duration interval and the adjustment amplitude. After obtaining the cache duration of the data message, the sending bit rate of the data message can be adjusted according to the adjustment amplitude corresponding to the duration interval where the cache duration is located, so that the data transmission can better and faster adapt to the current network status, improve the quality of data transmission, and provide users with a better experience.

[0153] For ease of understanding, take the audio and video service as an example. Fig.10 The step flow shown illustrates the network status detection method provided in the embodiment of the present application.

[0154] S101: The audio and video application is started and the baseband processor information is registered for monitoring.

[0155] Among them, when the audio and video application of the application layer is started and the audio and video service (such as video call service) begins, the network detection module in the application layer can register the baseband processor information monitoring through the interactive service module, so that the baseband processor can monitor the cache duration of the data message.

[0156] In an optional implementation, in order to reduce power consumption, in step S101, the network detection module can also pass a first preset duration threshold to the baseband processor through the interactive service module when registering for baseband processor information monitoring. In this way, during the process of monitoring the cache duration of the data message, the baseband processor can report when the cache duration is greater than the first preset duration threshold, thereby reducing the power consumption occupied by data reporting.

[0157] S102: Transmit the data message to the baseband processor.

[0158] Among them, after the audio and video application in the application layer generates audio and video streams, the generated audio and video streams are encoded and encapsulated by the RTP protocol and the RTCP protocol, and then enter the kernel protocol stack, encapsulated into data packets through the kernel protocol stack, and the data packets are sent to the device driver, which is sent to the forwarding module by the device driver, and sent to the perception reporting module (such as PDCP) in the baseband processor through the forwarding module.

[0159] S103: The baseband processor adds the data message to a cache queue.

[0160] S104: The baseband processor authorizes the access network device for the bandwidth threshold.

[0161] Among them, the perception reporting module in the baseband processor can initiate a bandwidth authorization request to the access network device. After receiving the authorization request, the access network device can send the authorized bandwidth to the perception reporting module in the baseband processor, where the authorized bandwidth is the bandwidth threshold.

[0162] Taking into account that the bandwidth threshold may change with the network environment, in an embodiment of the present application, the baseband processor can complete the authorization of the bandwidth threshold with the access network device at a set time interval.

[0163] S105: Send the data packets in the buffer queue through the air interface according to the bandwidth threshold.

[0164] S106: Monitor the cache duration of the data message in the cache queue to determine the cache duration of the data message in the cache queue.

[0165] S107: Send the cache duration to the application processor.

[0166] In order to facilitate the application processor to adjust the sending of data messages, the perception reporting module in the baseband processor can also send a bandwidth threshold when sending the cache duration to the application processor.

[0167] Exemplarily, the perception reporting module in the baseband processor can transmit the cache time and bandwidth threshold to the interactive service module, and the interactive service module transmits the cache time and bandwidth threshold to the network detection module in the application processor.

[0168] In order to reduce power consumption during transmission of cache duration and bandwidth threshold, in some embodiments, the perception reporting module in the baseband processor can send the cache duration and bandwidth threshold in the cache queue to the application processor when the cache duration is greater than a first preset duration threshold.

[0169] S108: If the cache duration is greater than a first preset duration threshold, determine that the network state is a congested state, and reduce the sending code rate of the data message.

[0170] Among them, after obtaining the cache time and bandwidth threshold, the network detection module compares the cache time with the first preset time threshold. If the cache time is greater than the first preset time threshold, it means that the network is congested, and then the sending bit rate of the data message is reduced to reduce the amount of data sent.

[0171] In an optional implementation, when the cache duration is less than the first preset duration threshold, it indicates that the network state is normal, and the data message transmission rate can be increased to improve data transmission efficiency. It should be noted that the increased transmission rate cannot be greater than the bandwidth threshold.

[0172] S109: Start the polling thread and send a query instruction to the baseband processor.

[0173] In an embodiment of the present application, after adjusting the transmission code rate of the data message, a polling thread can be started to send a query instruction to the baseband processor at a set time interval to timely obtain the impact of the adjustment of the transmission code rate on the network status.

[0174] After starting the polling thread, the network detection module can send a query instruction to the interactive service module, which then sends the query instruction to the baseband processor. The query instruction is used to instruct the baseband processor to feedback the cache duration and bandwidth threshold of the data message in the current cache queue.

[0175] S110: The baseband processor sends the cache duration and bandwidth threshold of the data message in the current cache queue to the application processor.

[0176] Among them, after receiving the query instruction, the perception reporting in the baseband processor executes the query instruction and reports the cache duration and bandwidth threshold of the data message in the current cache queue to the network detection module through the interactive service module.

[0177] S111: If the buffering time of the data message in the current buffer queue is greater than the first preset time threshold, it is determined that the network state is a congestion state, and the sending code rate of the data message continues to be adjusted.

[0178] S112: If the cache duration of the data message in the current cache queue is not greater than the first preset duration threshold, determine that the network state is decongested, stop polling, and increase the sending bit rate of the data message with the bandwidth threshold as the upper limit.

[0179] Among them, after obtaining the cache time feedback from the baseband processor according to the query instruction, the network detection module compares the cache time with the first preset time threshold. If the cache time of the data message in the current cache queue is greater than the first preset time threshold, it is determined that the network state is a congestion state, and the sending bit rate of the data message continues to be adjusted. If the cache time of the data message in the current cache queue is not greater than the first preset time threshold, it is determined that the network state is decongested, polling is stopped, and the bandwidth threshold is used as the upper limit to increase the sending bit rate of the data message.

[0180] The embodiment of the present application adds a network detection module and an interactive SDK in the application layer, and adds an interactive service module in the application framework layer, thereby realizing cross-core information interaction between the application layer in the application processor and the baseband processor. In this way, the baseband processor can directly report the cache time of the data message in the cache queue to the network detection module in the application processor. After receiving the cache time of the data message reported by the baseband processor, the network detection module can realize network status detection according to the cache time reported by the baseband processor, thereby improving the timeliness and accuracy of network detection.

[0181] At the same time, the embodiment of the present application adjusts the transmission bit rate of the data message according to the cache duration of the data message reported by the baseband processor, so that the transmission of the data message can adapt to the network status in time, thereby improving the quality of data transmission. The technical solution provided by the embodiment of the present application is used for video call services in high-speed scenarios, such as video calls on high-speed trains, which can improve the clarity of video calls by 30%, increase the receiving frame rate by 70%, and reduce video freezes by 10%.

[0182] As shown in Table 1, Table 1 is the index value obtained by testing the call quality of the video call service in a high-speed scenario. It can be seen from Table 1 that, compared with the conventional technical solution, the embodiment of the present application has an improvement in the proportion of video clarity resolution above 720 and the proportion of receiving frame rate above 11, a decrease in the sending delay, and a more obvious improvement in the receiving frame rate, such as Fig.11 As shown, Fig.11 (a) is a receiving frame rate diagram of an embodiment of the present application, Fig.11 (b) in the figure is the receiving frame rate diagram of the conventional technical solution.

[0183] Table 1

[0184]

[0185] The embodiment of the present application adjusts the sending bit rate of the data message according to the network status, so that the data volume of the data message to be sent can fluctuate within a small range of the current network throughput. In scenarios such as video calls, it can ensure that the video receiving end can watch the video smoothly and uninterruptedly, thereby improving the call quality and the viewing experience.

[0186] In order to further improve the data transmission quality in the state of network congestion, the embodiment of the present application can also clean up the data packets in the cache queue when the network is congested, thereby reducing the bandwidth occupied by invalid data and reducing the transmission delay of valid data.

[0187] In some embodiments of the present application, after obtaining the cache time of the data message in the cache queue reported by the baseband processor, the network detection module of the application layer detects the cache time. If the cache time is greater than the second preset time threshold, a clearing instruction can be sent to the baseband processor to instruct the baseband processor to clear the data message in the cache queue.

[0188] Among them, the second preset time threshold is greater than the first preset time threshold. The second preset time threshold can be set based on business needs. For example, in a video call scenario, the second preset time threshold can be set between 2s-3s. For example, if the second preset time threshold is 3s, when it exceeds 3s, the data packets in the cache queue still cannot be transmitted normally, then the data packets in the cache queue can be cleared so that the latest audio and video data packets can be sent in time.

[0189] like Fig.12 As shown, when the network detection module detects that the cache time of the data message reported by the PDCP is greater than the second preset time threshold after obtaining the cache time, the interactive SDK can send a cleanup instruction to the interactive service module, and the interactive service module transmits the cleanup instruction to the PDCP in the baseband processor. After receiving the cleanup instruction, the PDCP can execute the cleanup instruction to clean up the data messages in the cache queue.

[0190] For ease of understanding, take the audio and video service as an example. Fig.13 The step flow shown illustrates the network status detection method provided in the embodiment of the present application.

[0191] S201: The audio and video application is started and the baseband processor information is registered for monitoring.

[0192] S202: Transmit the data message to the baseband processor.

[0193] S203: The baseband processor adds the data message to a cache queue.

[0194] S204: The baseband processor authorizes the access network device for the bandwidth threshold.

[0195] S205: Send the data packets in the buffer queue through the air interface according to the bandwidth threshold.

[0196] S206: Monitor the cache duration of the data message in the cache queue to determine the cache duration of the data message in the cache queue.

[0197] S207: Send the cache duration to the application processor.

[0198] Among them, steps S201 to S207 can refer to steps S101 to S107, and are not described in detail here.

[0199] S208: If the cache duration is greater than a second preset duration threshold, a clearing instruction is sent to the baseband processor.

[0200] Among them, after obtaining the cache time, the network detection module in the application processor compares the cache time with the second preset time threshold. If the cache time is greater than the second preset threshold, a cleanup instruction is sent to the interactive service module, and the interactive service module transmits the cleanup instruction to the perception upload module in the baseband processor.

[0201] S209: The baseband processor clears the data messages in the cache queue according to the clearing instruction.

[0202] Among them, after receiving the cleaning instruction, the perception reporting module in the baseband processor can execute the cleaning and then clean up the data messages in the cache queue.

[0203] In order to reduce data loss, in an optional implementation, after the network detection module obtains the cache duration, the cache duration is compared with the first preset duration threshold and the second preset duration threshold. If the cache duration is greater than the second preset duration threshold, a cleanup instruction is sent to the baseband processor to instruct the baseband processor to clean up the data message in the cache queue, and if the cache duration is less than or equal to the second preset duration threshold and greater than the first preset duration threshold, the sending code rate of the data message can be adjusted first, and the polling thread is started. In the polling process, for each query, if the cache duration is greater than the second preset duration threshold, a cleanup instruction is sent to the baseband processor to instruct the baseband processor to clean up the data message in the cache queue, if the cache duration is less than or equal to the second preset duration threshold and greater than the first preset duration threshold, the sending code rate is adjusted, and if the cache duration is less than or equal to the first preset duration threshold, the polling is terminated, and the sending code rate of the data message is increased, and the increased sending code rate cannot be greater than the bandwidth threshold. In this way, when network congestion occurs, it can be adjusted by adjusting the bit rate and cleaning up the packets to improve the quality of data transmission and user experience.

[0204] In audio and video business scenarios, when network congestion occurs, key frames may be lost or incompletely transmitted during transmission, resulting in the data receiving device being unable to successfully decode the key frames. After the data receiving device fails to decode the key frames, it will send a key frame request to the sender to indicate that the key frames need to be resent. When the sender receives the key frame request, it will re-encode the key frame and send it to the data receiving device.

[0205] In order to ensure that the latest key frame can be sent to the data receiving device in a timely manner, in some embodiments of the present application, the network detection module may also send a cleanup instruction to the baseband processor after receiving the key frame request reported by the data receiving device, so as to instruct the baseband processor to clean up the data packets in the cache queue.

[0206] like Fig.14 As shown, after the network detection module receives the key frame request sent by the data receiving device, it can send a cleanup instruction to the interactive service module through the interactive SDK. The interactive service module transmits the cleanup instruction to the PDCP in the baseband processor. After receiving the cleanup instruction, the PDCP can execute the cleanup instruction to clean up the data packets in the cache queue.

[0207] Considering that when the network state is jittery or congested, the data receiving device may frequently initiate key frame requests. If the data packets in the cache queue are cleared only according to the key frame requests, the re-encoded key frame may be deleted before it is sent out from the cache queue, resulting in the data never being sent. Based on this, in some embodiments of the present application, the data packets in the cache queue can also be cleared in combination with the key frame request and the cache duration of the data packets in the current cache queue.

[0208] Exemplarily, after receiving a key frame request reported by a data receiving device, the network detection module detects whether the cache duration of the data message in the current cache queue is greater than a third preset duration threshold. If it is greater than the third preset duration threshold, a cleanup instruction is sent to the baseband processor to instruct the baseband processor to clean up the data message in the cache queue. If it is not greater than the third preset duration threshold, there is no need to send a cleanup instruction to the baseband processor.

[0209] In an embodiment of the present application, after the network detection module receives a key frame request reported by a data receiving device, if it detects that the cache duration of the data message in the current cache queue is greater than the third preset duration threshold, it indicates that the network is congested and the latest key frame may not be sent to the data receiving device in time. Therefore, the data message in the cache queue can be cleared to avoid invalid data messages occupying bandwidth and ensure that the latest key frame can be sent to the data receiving device in time.

[0210] If it is detected that the cache duration of the data message in the current cache queue is not greater than the third preset duration threshold, it means that the latest generated key frame can be sent to the data receiving device in a short time. Therefore, when the cache duration of the data message is not greater than the third preset duration threshold, there is no need to clean up the data message in the cache queue. The latest generated key frame can be sent to the baseband processor and put into the cache queue to be sent to the data receiving device.

[0211] In an embodiment of the present application, the third preset duration threshold may be greater than the first preset duration threshold, or may be less than or equal to the first preset duration threshold. It may be specifically set according to business requirements and is not specifically limited here.

[0212] For ease of understanding, take the audio and video service as an example. Fig.15 The step flow shown illustrates the network status detection method provided in the embodiment of the present application.

[0213] S301: The audio and video application is started and the baseband processor information is registered for monitoring.

[0214] S302: Transmit the data message to the baseband processor.

[0215] S303: The baseband processor adds the data message to a cache queue.

[0216] S304: The baseband processor authorizes the access network device for the bandwidth threshold.

[0217] S305: Send the data packets in the buffer queue through the air interface according to the bandwidth threshold.

[0218] S306: Monitor the cache duration of the data message in the cache queue to determine the cache duration of the data message in the cache queue.

[0219] S307: Send the cache duration to the application processor.

[0220] Among them, steps S301 to S307 can refer to steps S101 to S107, and are not described in detail here.

[0221] S308: The application processor receives a key frame request, and the cache duration is greater than a third preset duration threshold, and sends a clearing instruction to the baseband processor.

[0222] S309: The baseband processor clears the data messages in the cache queue according to the clearing instruction.

[0223] When the network detection module in the application processor receives the key frame request sent by the data receiving device, it compares the cache duration in the current cache queue with the third preset duration threshold. If the cache duration in the current cache queue is greater than the third preset duration threshold, a cleanup instruction can be sent to the interactive service module, and the interactive service module transmits the cleanup instruction to the perception upload module in the baseband processor. After receiving the cleanup instruction, the perception reporting module in the baseband processor can clean up the data packets in the cache queue.

[0224] It can be understood that when the network detection module sends a cleanup instruction to the baseband processor, the network detection module can trigger the encoding module to re-encode the key frame, and after protocol encapsulation, send it to the cache queue in the baseband processor through the forwarding module to resend it to the data receiving device.

[0225] Accordingly, if the cache duration in the current cache queue is not greater than the third preset duration threshold, the encoding module can be triggered to re-encode the key frame, and after protocol encapsulation, it is sent to the cache queue in the baseband processor through the forwarding module to be resent to the data receiving device.

[0226] In an optional implementation, in order to improve the quality of data transmission, when the network is congested, the transmission code rate adjustment can also be combined with the key frame request processing. Exemplarily, in the process of adjusting the transmission code rate, if a key frame request is received, as long as it is detected that the current cache duration is greater than the third preset duration threshold, a cleanup instruction can be sent to the baseband processor to instruct the baseband processor to clean up the data message in the cache queue, and if the current cache duration is not greater than the third preset duration threshold, the transmission code rate of the data message continues to be adjusted without sending a cleanup instruction to the baseband processor. At the same time, in the process of adjusting the transmission code rate, when the cache duration of the data message in the cache queue is greater than the second preset duration threshold, even if the key frame request is not received, a cleanup instruction can be sent to the baseband processor to instruct the baseband processor to clean up the data message in the cache queue. Among them, the second preset duration threshold is greater than the third preset duration threshold. In this way, when the network is congested, the occupation of bandwidth by invalid data can be avoided, the quality of data transmission can be improved, the video jam can be avoided, and the user experience can be improved.

[0227] In an embodiment of the present application, the network detection module can transmit the cleanup instruction to the baseband processor through the information sending API interface in the interactive service module. That is, in an embodiment of the present application, the information sending API interface may include a threshold information sending API interface, a query instruction sending API interface, and a cleanup instruction sending API interface. Among them, the threshold information sending API interface is used to send a first preset time threshold to the baseband processor, so that the baseband processor reports the cache time of the data message in the cache queue when the cache time of the data message in the cache queue is greater than the first preset time threshold. The query instruction sending API interface is used to send query instructions to the baseband processor to instruct the baseband processor to report the cache time and bandwidth threshold of the data message in the current cache queue. The cleanup instruction sending API interface is used to send cleanup instructions to the baseband processor to instruct the baseband processor to clean up the data messages in the cache queue.

[0228] It can be understood that after receiving the cleaning instruction, the baseband processor can execute the cleaning instruction to clean up the data messages in the cache queue.

[0229] In order to reduce the waste of resources, in an embodiment of the present application, some invalid data messages in the cache queue may also be cleared.

[0230] Exemplarily, the cleaning instruction sent by the network detection module to the baseband processor may include the message marking information of the data message to be cleaned. After receiving the cleaning instruction, the baseband processor can clean the data messages with the same message marking information in the cache queue according to the message marking information in the cleaning instruction. In this way, the data messages in the cache queue can be accurately cleaned, thereby avoiding waste of resources.

[0231] Optionally, in an embodiment of the present application, the message marking information may be five-tuple information of the message, or may be a message feature. The message feature may be, but is not limited to, an offset, a frame length, etc., and may be specifically set according to business requirements. The embodiment of the present application does not impose any specific restrictions.

[0232] It should be noted that in some other application scenarios, a cleanup instruction may be sent to the baseband processor when a socket exception or service interruption is detected. For example, in a scenario where text and / or images are uploaded, the text and / or images are encapsulated into data packets and added to the cache queue for transmission. If the business logic of the corresponding application detects that the socket is abnormal, a cleanup instruction may be sent to the baseband processor to instruct the baseband processor to clean up the data packets in the cache queue.

[0233] For example, in a scanning and identification scenario, the object to be scanned is set as an identification code. When the terminal device scans the identification code, the data generated by the scanning will be encapsulated into a data message and added to the cache queue for transmission. If the business logic of the corresponding application detects that the scan is interrupted, a clearing instruction can also be sent to the baseband processor to instruct the baseband processor to clear the data messages in the cache queue.

[0234] For example, in a file transfer scenario, the file to be transmitted will be encapsulated into a data message and added to the cache queue for transmission. During the transmission process, when the business logic of the corresponding application detects that the file transmission is abandoned, a cleanup instruction can be sent to the baseband processor to instruct the baseband processor to clean up the data messages in the cache queue.

[0235] It can be understood that the scenarios listed above are only examples of clearing data packets in the cache queue in the embodiments of the present application, and do not constitute specific limitations. The specific application scenarios can be set according to business needs, and the embodiments of the present application do not impose specific limitations.

[0236] In the embodiment of the present application, when the network status is blocked, the cleaning instruction is sent to the baseband processor to realize the cleaning of invalid data in the cache queue, which can avoid the occupation of bandwidth by invalid data, improve the actual effective utilization rate of bandwidth, and reduce the transmission delay of valid data. The technical solution provided by the embodiment of the present application is used in a poor network environment, such as an area with poor cellular mobile network coverage, which can reduce 40% of key frame requests and avoid long-term freezing of video images.

[0237] As shown in Table 2, Table 2 is the index value obtained by testing the call quality of the video call service in a poor network environment. From Table 2, it can be seen that the technical solution provided by the embodiment of the present application has reduced the number of key frame requests, transmission delay, packet loss rate, etc. compared with the conventional technical solution, and has improved the receiving frame rate. Fig.16 As shown, Fig.16 (a) is a receiving frame rate diagram of an embodiment of the present application, Fig.16 (b) in the figure is the receiving frame rate diagram of the conventional technical solution.

[0238] Table 2

[0239]

[0240] The embodiment of the present application can timely and accurately perceive the network status through information interaction between the application processor and the baseband processor, and can be used as a supplement to RTP / RTCP network detection in scenarios such as real-time audio and video calls to improve call quality. Through information interaction between the application processor and the baseband processor, the data in the cache queue of the baseband processor is cleaned up, which can improve the actual effective utilization rate of the bandwidth and reduce the transmission delay of effective data.

[0241] For easier understanding, please refer to Fig.17 The network status detection method provided in the embodiment of the present application is described. Fig.17 A flow chart of a network status detection method provided in an embodiment of the present application is as follows: Fig.17 As shown, the network status detection method provided in the embodiment of the present application may include the following steps:

[0242] S401: The baseband processor sends the cache duration of the data message in the cache queue to the application processor.

[0243] The baseband processor may monitor the cache duration of the data message in the cache queue to obtain the cache duration of the data message in the cache queue.

[0244] In order to reduce power consumption, in an embodiment of the present application, the baseband processor may transmit the cache duration of the data packet in the cache queue to the network detection module when the cache duration of the data packet in the cache queue is greater than a first preset duration threshold, or, after receiving a query instruction sent by the application processor, send the cache duration of the data packet in the cache queue to the application processor, thereby avoiding frequent transmission of the cache duration and effectively reducing power consumption.

[0245] S402: The application processor obtains the network status of the terminal device according to the cache duration.

[0246] Among them, after obtaining the cache time, the application processor can compare the cache time with the first preset time threshold. If the cache time is greater than the first preset time threshold, it can be determined that the current network state is a congested state. If the cache time is less than or greater than the first preset time threshold, it can be determined that the current network state is a normal state, that is, no congestion occurs.

[0247] In order to improve the quality of data transmission, in the embodiment of the present application, after obtaining the network status, the sending code rate of the data message can also be adjusted according to the network status, so that the data volume of the data message to be sent can meet the current network throughput, thereby improving the data transmission quality. Fig.18 The network status detection method provided in the embodiment of the present application may further include step S4031:

[0248] S4031: When the network state is a congested state, reduce the sending code rate of the data message, or when the network state is a normal state, increase the sending code rate of the data message.

[0249] The congestion state is a network state in which the cache time is longer than a first preset time threshold, and the normal state is a network state in which the cache time is shorter than the first preset time threshold.

[0250] In order to timely obtain changes in network status after the sending code rate is adjusted, in an embodiment of the present application, after obtaining that the network status is congested, the application processor can also start a polling thread and send a query instruction to the baseband processor at a set time interval to instruct the baseband processor to report the cache length of the data message in the cache queue.

[0251] During the polling process, for each query, after the application processor obtains the cache time length sent by the baseband processor, if the cache time length is still greater than the first preset time length threshold, that is, when the network status obtained by polling is still in a congested state, the sending bit rate of the data message can continue to be reduced. If the cache time length is less than the first preset time length threshold, that is, when the network congestion status obtained by polling is released, the polling thread can be terminated and the sending bit rate of the data message can be increased.

[0252] In order to facilitate the application processor to adjust the transmission code rate of the data message, so as to adapt to the network environment in time and improve the user experience, in the embodiment of the present application, after obtaining the cache time of the data message, the application processor can obtain the adjustment range of the transmission code rate according to the time interval in which the cache time is located. Different time intervals correspond to different adjustment ranges.

[0253] In order to avoid the situation where the adjusted sending code rate exceeds the bandwidth threshold when adjusting the sending code rate, in an embodiment of the present application, the baseband processor may transmit the bandwidth threshold to the application processor when transmitting the cache time to the application processor. In this way, the application processor may adjust the sending code rate of the data message based on the broadband threshold as the upper limit to avoid the situation where the adjusted sending code rate of the data message exceeds the bandwidth threshold.

[0254] To further improve the quality of data transmission in a congested network, please refer to Fig.19 After the application processor obtains the cache duration of the data message in the cache queue, the network status detection method may further include step S4032:

[0255] S4032: If the cache duration is greater than a second preset duration threshold, the application processor sends a cleanup instruction to the baseband processor.

[0256] The second preset time threshold is greater than the first preset time threshold, and the cleaning instruction is used to instruct the baseband processor to clean up the data messages in the cache queue.

[0257] In this way, when the network is congested, by clearing the data packets in the cache queue, the bandwidth occupied by invalid data can be reduced and the transmission delay of valid data can be reduced.

[0258] In some embodiments, a cleanup instruction may be sent to the baseband processor according to the key frame request. Therefore, after receiving the key frame request, the application processor may send a cleanup instruction to the baseband processor to instruct the baseband processor to clean up the data packets in the cache queue.

[0259] It can be understood that the key frame request is sent by the data receiving device corresponding to the terminal device.

[0260] Considering that when the network status is jittery or congested, the data receiving device may frequently initiate key frame requests. If the data packets in the cache queue are cleared only according to the key frame requests, the re-encoded key frames may be deleted before they are sent out from the cache queue, resulting in the data never being sent. Based on this, in some embodiments of the present application, the data packets in the cache queue can also be cleared in combination with the key frame requests and the cache duration of the data packets in the current cache queue. Please refer to Fig. 20 After the application processor obtains the cache duration of the data message in the cache queue, the network status detection method may further include step S4033:

[0261] S4033: When a key frame request is received and the cache duration is greater than a third preset duration threshold, the application processor sends a cleanup instruction to the baseband processor.

[0262] Among them, when the application processor receives the key frame request reported by the data receiving device, if it detects that the cache time of the data message in the current cache queue is greater than the third preset time threshold, it means that the network is congested and the latest key frame may not be sent to the data receiving device in time. Therefore, the data message in the cache queue can be cleared to avoid invalid data messages occupying bandwidth and ensure that the latest key frame can be sent to the data receiving device in time.

[0263] If it is detected that the cache duration of the data message in the current cache queue is not greater than the third preset duration threshold, it means that the latest generated key frame can be sent to the data receiving device in a short time. Therefore, when the cache duration of the data message is not greater than the third preset duration threshold, there is no need to clean up the data message in the cache queue. The latest generated key frame can be sent to the baseband processor and put into the cache queue to be sent to the data receiving device.

[0264] In order to reduce the waste of resources, in the embodiment of the present application, the cleaning instruction sent by the application processor to the baseband processor may also include message marking information, so that when the baseband processor receives the cleaning instruction, it can clean up the data messages with the same message marking information in the cache queue according to the message marking information in the cleaning instruction. In this way, the data messages in the cache queue can be accurately cleaned up, thereby avoiding the waste of resources.

[0265] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above steps S401, S402, S4031, S4032, and S4033 can refer to the corresponding processes in the aforementioned embodiments and will not be repeated here.

[0266] Based on the same inventive concept, an embodiment of the present application further provides a terminal device, the terminal device includes a baseband processor and an application processor; the baseband processor includes a perception reporting module, and the application processor includes an interactive service module and a network detection module;

[0267] The interactive service module is used to exchange information between the perception reporting module and the network detection module;

[0268] The perception reporting module is used to send the cache duration of the data message in the cache queue to the network detection module through the interactive service module; wherein the cache queue is established by the baseband processor and is used to cache the data message to be sent;

[0269] The network detection module is used to obtain the network status of the terminal device according to the cache duration.

[0270] In an optional implementation, the perception reporting module is used to send the cache duration of the data message in the cache queue to the network detection module through the interactive service module when the cache duration of the data message in the cache queue is greater than the first preset duration threshold; or,

[0271] After receiving the query instruction sent by the network detection module through the interactive service module, the buffering time length of the data message in the buffer queue is sent to the network detection module through the interactive service module.

[0272] In an optional implementation, the query instruction is an instruction sent when the network state is a congested state, and the application processor sends the query instruction to the baseband processor at a set time interval.

[0273] In an optional implementation manner, the network detection module is further used to:

[0274] When the network is in a congested state, the sending rate of the data message is reduced; or when the network is in a normal state, the sending rate of the data message is increased;

[0275] The congestion state is a network state in which the cache time is longer than a first preset time threshold, and the normal state is a network state in which the cache time is shorter than the first preset time threshold.

[0276] In an optional implementation manner, the network detection module is specifically used to:

[0277] According to the duration interval in which the cache duration is located, an adjustment range for reducing or increasing the sending bit rate is obtained; wherein different duration intervals correspond to different adjustment ranges.

[0278] In an optional implementation manner, the perception reporting module is further used to:

[0279] Sending bandwidth threshold to network detection module through interactive service module;

[0280] The sending code rate of the data message is less than the bandwidth threshold.

[0281] In an optional embodiment, when the cache duration is greater than a second preset duration threshold, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; wherein the cleanup instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue; and the second preset duration threshold is greater than the first preset duration threshold.

[0282] In an optional implementation, when a key frame request is received, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; wherein the cleanup instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device.

[0283] In an optional embodiment, when a key frame request is received and the cache duration is greater than a third preset duration threshold, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; wherein the cleanup instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device.

[0284] In an optional implementation, the cleaning instruction includes message marking information, and the cleaning instruction is used to instruct the perception reporting module to clean up data messages with the same message identification information in the cache queue.

[0285] In an optional implementation, the data message is an audio or video data message generated by instant messaging software.

[0286] The embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processing circuit, the functions or steps in the above-mentioned materialized view update method are implemented.

[0287] In addition, an embodiment of the present application may also provide a chip system, which includes a processing circuit and a storage medium, wherein the storage medium stores computer program instructions that implement the functions or steps in the above-mentioned materialized view update method when the computer program instructions are executed by the processing circuit.

[0288] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and module described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0289] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0290] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0291] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A network status detection method, characterized in that: Applied to a terminal device, the terminal device includes a baseband processor and an application processor; the method includes: The baseband processor sends a cache duration of a data message in a cache queue to the application processor; the cache queue is established by the baseband processor and is used to cache data messages to be sent; The application processor obtains the network status of the terminal device according to the cache duration.

2. The method according to claim 1, characterized in that The baseband processor sends the cache duration of the data message in the cache queue to the application processor when the cache duration of the data message in the cache queue is greater than a first preset duration threshold; or, After receiving the query instruction sent by the application processor, the baseband processor sends the cache duration of the data message in the cache queue to the application processor.

3. The method according to claim 2, characterized in that The query instruction is an instruction sent when the network state is a congested state, and the application processor sends the query instruction to the baseband processor at a set time interval; the congested state is a network state in which the cache time length is greater than the first preset time length threshold.

4. The method according to claim 1, characterized in that: After the application processor obtains the network status of the terminal device according to the cache duration, the method further includes: When the network state is a congested state, reducing the sending code rate of the data message; or, When the network state is normal, increasing the sending code rate of the data message; The congestion state is a network state in which the cache time is greater than a first preset time threshold, and the normal state is a network state in which the cache time is less than the first preset time threshold.

5. The method according to claim 4, characterized in that The method further comprises: The application processor obtains an adjustment range for reducing or increasing the sending bit rate according to the duration interval in which the cache duration is located; wherein different duration intervals correspond to different adjustment ranges.

6. The method according to claim 4, characterized in that The method further comprises: The baseband processor sends a bandwidth threshold to the application processor; The sending code rate of the data message is less than the bandwidth threshold.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the cache duration is greater than a second preset duration threshold, the application processor sends a cleanup instruction to the baseband processor; The cleaning instruction is used to instruct the baseband processor to clean up the data messages in the cache queue; and the second preset time threshold is greater than the first preset time threshold.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When receiving a key frame request, the application processor sends a cleanup instruction to the baseband processor; The cleaning instruction is used to instruct the baseband processor to clean up the data messages in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device.

9. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When a key frame request is received and the cache duration is greater than a third preset duration threshold, the application processor sends a cleanup instruction to the baseband processor; The cleaning instruction is used to instruct the baseband processor to clean up the data messages in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device.

10. The method according to any one of claims 7 to 9, characterized in that: The cleaning instruction includes message marking information, and the cleaning instruction is used to instruct the baseband processor to clean up data messages with the same message identification information in the cache queue.

11. The method according to any one of claims 1 to 10, characterized in that: The data message is an audio and video data message generated by instant messaging software.

12. A terminal device, characterized in that: The terminal device includes a baseband processor and an application processor; the baseband processor includes a perception reporting module, and the application processor includes an interactive service module and a network detection module; The interactive service module is used to perform information interaction between the perception reporting module and the network detection module; The perception reporting module is used to send the cache duration of the data message in the cache queue to the network detection module through the interactive service module, and the cache queue is established by the baseband processor and is used to cache the data message to be sent; The network detection module is used to obtain the network status of the terminal device according to the cache duration.

13. The terminal device according to claim 12, characterized in that: The perception reporting module is used to send the cache time of the data message in the cache queue to the network detection module through the interactive service module when the cache time of the data message in the cache queue is greater than the first preset time threshold; or, After receiving the query instruction sent by the network detection module through the interactive service module, the cache duration of the data message in the cache queue is sent to the network detection module through the interactive service module.

14. The terminal device according to claim 13, characterized in that: The query instruction is an instruction sent when the network state is a congested state, and the application processor sends the query instruction to the baseband processor at a set time interval; the congested state is a network state in which the cache time length is greater than the first preset time length threshold.

15. The terminal device according to claim 12, characterized in that: The network detection module is also used for: When the network state is a congested state, reducing the sending code rate of the data message; or, When the network state is normal, increasing the sending code rate of the data message; The congestion state is a network state in which the cache time is greater than a first preset time threshold, and the normal state is a network state in which the cache time is less than the first preset time threshold.

16. The terminal device according to claim 15, characterized in that: The network detection module is specifically used for: According to the duration interval in which the cache duration is located, an adjustment range for reducing or increasing the sending code rate is obtained; wherein different duration intervals correspond to different adjustment ranges.

17. The terminal device according to claim 15, characterized in that: The perception reporting module is also used for: Sending a bandwidth threshold to the network detection module through the interactive service module; The sending code rate of the data message is less than the bandwidth threshold.

18. The terminal device according to any one of claims 12 to 17, characterized in that: When the cache duration is greater than a second preset duration threshold, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; Among them, the cleaning instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue; the second preset time threshold is greater than the first preset time threshold.

19. The terminal device according to any one of claims 12 to 17, characterized in that: When receiving a key frame request, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; Among them, the cleaning instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device.

20. The terminal device according to any one of claims 12 to 17, characterized in that: When a key frame request is received and the cache duration is greater than a third preset duration threshold, the network detection module sends a cleanup instruction to the perception reporting module through the interactive service module; Among them, the cleaning instruction is used to instruct the perception reporting module to clean up the data packets in the cache queue, and the key frame request is sent by the data receiving device corresponding to the terminal device.

21. The terminal device according to any one of claims 17 to 19, characterized in that: The cleaning instruction includes message marking information, and the cleaning instruction is used to instruct the perception reporting module to clean up data messages with the same message identification information in the cache queue.

22. The terminal device according to any one of claims 12 to 19, characterized in that: The data message is an audio and video data message generated by instant messaging software.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program or instructions. When the computer program or instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 11.

24. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to perform the method according to any one of claims 1 to 11.

25. A chip system, characterized in that: include: At least one processor and a communication interface, wherein the at least one processor is coupled to a memory via the communication interface, and when the at least one processor executes a computer program or instruction in the memory, the method according to any one of claims 1 to 11 is executed.

26. A terminal device, characterized in that: comprising a memory, one or more processors; The memory is coupled to the one or more processors, and the one or more processors are used to execute the computer program stored in the memory, so that the terminal device performs the method according to any one of claims 1 to 11; The one or more processors include a baseband processor and an application processor.

Images