Audio and video playback method, apparatus and device, and readable storage medium

By using the audio and video playback methods of monitoring equipment, obtaining the playback address, determining the cause of failure and reconnecting, and receiving and decoding network packets, the problem of unstable video playback on monitoring equipment was solved, and a continuous and stable video playback experience was achieved.

WO2024230134A1PCT designated stage expired Publication Date: 2024-11-14E SURFING VISION TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/134822
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-10
Filing Date
2023-11-28
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing surveillance equipment is prone to stuttering or malfunctions when playing videos for extended periods, making it difficult to meet the video playback needs of different scenarios.

Method used

By obtaining the playback address of the audio and video data, the system determines whether the acquisition was successful, identifies the reason for failure, and executes a preset reconnection strategy until a successful connection is established. Then, it receives and decodes network packets, renders, and plays the audio and video data.

Benefits of technology

It effectively solves the problem of unstable video caused by frequent reconnection or failure to reconnect in time due to malfunction of monitoring equipment, improves user experience, and ensures continuous and stable video playback.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023134822_14112024_PF_FP_ABST
    Figure CN2023134822_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present application an audio and video playback method, apparatus and device, and a readable storage medium. When a fault occurs in playback data of a monitoring device, the present application can detect abnormal video streams which have not been successively received in a continuous and complete way, or actively detect whether an anomaly occurs in data streams sent by a server end, so as to determine fault causes and perform troubleshooting on the fault causes. Therefore, the method provided in the embodiments of the present application can not only monitor the network of audio and video data but also monitor the correctness of the audio and video data itself, detects different playback errors occurring in monitoring devices, and uses different reconnection policies for troubleshooting, thereby effectively improving user experience, solving the problem of users being unable to see a stable and continuous picture due to too frequent device reconnection, and further solving the problem of long-time inability to see video images due to device failures without timely reconnection.
Need to check novelty before this filing date? Find Prior Art

Description

Audio and video playback method, device, equipment and readable storage medium Technical Field

[0001] The present application relates to the field of monitoring data processing technology, and in particular to an audio and video playback method, device, equipment and readable storage medium. Background Art

[0002] In actual applications, security and defense considerations often require that surveillance equipment's video feeds be played continuously for extended periods. Currently, commonly used surveillance equipment generates varying audio and video data formats. Some surveillance equipment may generate abnormal data during long-term testing, causing issues like playback freezes and data loss. Some surveillance equipment also suffers from unstable graphics card performance, sometimes leading to abnormalities in audio and video decoding. Consequently, conventional surveillance equipment struggles to meet the video playback requirements of diverse scenarios.

[0003] Summary of the Invention

[0004] The present application aims to solve at least one of the above-mentioned technical defects. In view of this, the present application provides an audio and video playback method, device, equipment and readable storage medium, which are used to solve the technical defects in the prior art that monitoring equipment is prone to freeze or abnormality when playing monitoring videos.

[0005] An audio and video playback method, comprising:

[0006] Get the playback address of the audio and video data to be played;

[0007] Determine whether the playback address is successfully obtained;

[0008] If the playback address is not successfully obtained, determining the reason for the failure to obtain the playback address;

[0009] Executing a preset reconnection strategy according to the reason for the failure to obtain the playback address until the playback address is successfully obtained;

[0010] If the playback address has been successfully obtained, a connection relationship is established with the playback address;

[0011] Determine whether a connection relationship is successfully established with the playback address;

[0012] If a connection relationship has been successfully established with the playback address, then waiting for and receiving a network packet corresponding to the audio and video data to be played;

[0013] Determining whether the network packet is successfully received;

[0014] If the network packet has been successfully received, decoding the network packet;

[0015] Determining whether the network packet is successfully decoded;

[0016] If the network packet is decoded successfully, the audio and video data to be played corresponding to the network packet is obtained, and the audio and video data to be played corresponding to the network packet is rendered and played.

[0017] Preferably, the method further comprises:

[0018] If a failure occurs in playing the audio and video data to be played corresponding to the network packet, determining the type of the failure;

[0019] Based on the fault type, the system returns to executing the preset reconnection strategy until the audio and video data to be played corresponding to the network packet is successfully replayed.

[0020] Preferably, executing a preset reconnection strategy until the playback address is successfully obtained includes:

[0021] If it is determined that the reason for the failure to obtain the playback address is an interface call error, and the playback address fails to be obtained for the first time, reconnecting to the interface corresponding to the playback address for the first time within a preset first time;

[0022] If the interface corresponding to the playback address fails to reconnect for the first time within the preset first time, determining whether the reason for the failure to reconnect for the interface corresponding to the playback address for the first time within the preset first time is an interface call error;

[0023] If it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time period is still an interface call error, then reconnecting to the interface corresponding to the playback address within a preset second time period starting from the moment when the first reconnection failure is determined;

[0024] If the interface corresponding to the playback address fails to reconnect again within the preset second time, determining whether the reason for the failure to reconnect again with the interface corresponding to the playback address within the preset second time is an interface call error;

[0025] If it is determined that the reason for the failure to reconnect to the interface corresponding to the playback address within the preset second time is still an interface call error, then from the moment it is determined that the reconnection has failed, the interface corresponding to the playback address will be reconnected within a preset third time period until the playback address is successfully obtained.

[0026] Preferably, the method further comprises:

[0027] If it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time period is non-interface call error, then reconnecting to the interface corresponding to the playback address within a preset fourth time period starting from the moment when the first reconnection failure is determined;

[0028] Determining whether reconnection to the interface corresponding to the playback address is successful within a preset fourth time period starting from the moment when the first reconnection failure is determined;

[0029] If the interface corresponding to the playback address fails to reconnect again within the preset fourth time, determining whether the reason for the failure to reconnect again with the interface corresponding to the playback address within the preset fourth time is an interface call error;

[0030] If it is determined that the reason for the failure to reconnect to the interface corresponding to the playback address within the preset second time period is an interface call error, reconnecting to the interface corresponding to the playback address within the preset first time period from the moment when the reconnection failure is determined;

[0031] Determining whether reconnection to the interface corresponding to the playback address is successful within a preset first time period from the moment when reconnection failure is determined;

[0032] If, from the moment when the reconnection failure is determined, reconnection to the interface corresponding to the playback address fails within a preset first time period, determining whether a reason for the reconnection failure to the interface corresponding to the playback address within the preset first time period from the moment when the reconnection failure is determined is an interface call error;

[0033] If, from the moment when it is determined that the reconnection has failed, the reason for the failure to reconnect to the interface corresponding to the playback address within a preset first time period is an interface call error, then returning to executing the operation of reconnecting to the interface corresponding to the playback address within a preset second time period until the playback address is successfully obtained;

[0034] If, from the moment when it is determined that the reconnection has failed, the reason for the failure to reconnect to the interface corresponding to the playback address within the preset first time period is a non-interface call error, then the operation of reconnecting to the interface corresponding to the playback address within the preset fourth time period is returned to until the playback address is successfully obtained.

[0035] Preferably, executing a preset reconnection strategy until the playback address is successfully obtained includes:

[0036] If it is determined that the reason for the failure to obtain the playback address is not an interface call error, the interface corresponding to the playback address is reconnected for the first time within a preset fourth time until the playback address is successfully obtained.

[0037] Preferably, determining the reason for failure in obtaining the playback address includes:

[0038] If the target monitoring device is offline or a network connection failure occurs, determining that the reason for the failure to obtain the playback address is an interface error;

[0039] If the target monitoring device fails to play the audio and video data to be played, it is determined that the reason for the failure to obtain the playback address is that there is an error in the playback address.

[0040] Preferably, determining the fault type includes:

[0041] Determine whether the playback address of the current connection is the playback address corresponding to the audio and video data to be played, and if the playback address of the current connection is not the playback address corresponding to the audio and video data to be played, determine the fault type as a playback address fault;

[0042] or,

[0043] Detecting whether a target monitoring device has not displayed the data stream to be played for more than a preset first time period, and if the target monitoring device has not displayed the data stream to be played for more than the preset first time period, determining that the fault type is that the target monitoring device is offline or that the streaming media server corresponding to the data stream to be played has a performance fault;

[0044] or,

[0045] Detecting whether the decoder of the target monitoring device has been successfully opened, if the decoder of the target monitoring device fails to be opened, determining that the fault type is that the audio and video data format of the target monitoring device is abnormal or the memory consumption of the target monitoring device exceeds the maximum value that the memory of the target monitoring device can bear;

[0046] or,

[0047] Comparing the video timestamp of the target monitoring device with the timestamp of the control system corresponding to the target monitoring device, if the video timestamp of the target monitoring device is later than the timestamp of the control system corresponding to the target monitoring device, determining that the fault type is a video delay data delay fault;

[0048] or,

[0049] Determining whether the decoding function of the decoder of the target monitoring device is normal; if the decoding function of the decoder of the target monitoring device is abnormal, determining that the fault type is an abnormal video data format of the target monitoring device or a fault exists in the local player of the control system corresponding to the target monitoring device;

[0050] or,

[0051] determining whether the target monitoring device receives a preset number of complete data frames within a preset second time period; and determining that the fault type is a data loss fault if the target monitoring device does not receive the preset number of complete data frames within the preset second time period.

[0052] or,

[0053] Determine whether the renderer of the target monitoring device fails to open due to a problem with the control system corresponding to the target monitoring device. If the renderer of the target monitoring device fails to open due to a problem with the control system corresponding to the target monitoring device, determine that the fault type is a system fault of the control system corresponding to the target monitoring device.

[0054] An audio and video playback device, comprising:

[0055] A playback address acquisition unit, used to acquire the playback address of the audio and video data to be played;

[0056] A first judging unit, configured to judge whether the playback address is successfully obtained;

[0057] a first determining unit configured to determine a reason for failure in obtaining the playback address if the execution result of the first judging unit determines that the playback address is not successfully obtained;

[0058] a reconnection unit, configured to execute a preset reconnection strategy according to a reason for failing to obtain the playback address, until the playback address is successfully obtained;

[0059] a construction unit, configured to establish a connection relationship with the playback address when the execution result of the first judgment unit determines that the playback address has been successfully obtained;

[0060] A second judgment unit is used to judge whether a connection relationship is successfully established with the playback address;

[0061] a receiving unit, configured to wait for and receive a network packet corresponding to the audio and video data to be played when the execution result of the second determining unit determines that a connection relationship has been successfully established with the playback address;

[0062] A third judging unit, configured to judge whether the network packet is successfully received;

[0063] a decoding unit, configured to decode the network packet when the execution result of the third judgment unit is that the network packet has been successfully received;

[0064] a fourth determining unit, configured to determine whether the network packet is successfully decoded;

[0065] The playback unit is configured to obtain the audio and video data to be played corresponding to the network packet when the execution result of the fourth judgment unit determines that the network packet is decoded successfully, and render and play the audio and video data to be played corresponding to the network packet.

[0066] An audio and video playback device, comprising: one or more processors, and a memory;

[0067] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the audio and video playback method described in any one of the above introductions are implemented.

[0068] A readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors implement the steps of the audio and video playback method as described in any of the above descriptions.

[0069] It can be seen from the technical solution introduced above that when a fault occurs in the playback screen of the monitoring device and it is necessary to detect it for troubleshooting, the method provided in the embodiment of the present application can obtain the playback address of the audio and video data to be played, and determine whether the playback address is successfully obtained; if the playback address is not successfully obtained, it means that the acquisition of the playback address may have failed, and the reason for the failure to obtain the playback address can be further determined; and based on the reason for the failure to obtain the playback address, the preset reconnection strategy is executed until the playback address is successfully obtained, so that the audio and video data to be played can be obtained through the playback address. If the playback address has been successfully obtained, it means that the audio and video data to be played can be obtained through the playback address, and a connection relationship can be established with the playback address; and it can be further determined whether a connection relationship is successfully established with the playback address; if a connection relationship is successfully established with the playback address, it means that the audio and video data to be played can be obtained through the established network connection relationship. Therefore, after establishing a connection relationship with the playback address, you can wait for and receive the network packet corresponding to the audio and video data to be played; and determine whether the network packet is successfully received; if the network packet has been successfully received, you can decode the network packet; so that the audio and video data to be played can be obtained. Therefore, after decoding the network packet, you can further determine whether the network packet is successfully decoded; if the network packet is successfully decoded, it means that the audio and video data to be played corresponding to the network packet can be obtained. After obtaining the audio and video data to be played, the audio and video data to be played corresponding to the network packet can be rendered and played.

[0070] From the above analysis, it can be seen that the method provided by the embodiment of the present application can detect the abnormal video stream that has not been successfully received continuously and completely when the playback data of the monitoring device fails, or actively detect whether there is an abnormality in the data stream sent by the server, determine the cause of the failure, and promptly troubleshoot the cause of the failure. Therefore, the method provided by the implementation of the present application can not only monitor the network of audio and video data, but also monitor the correctness of the audio and video data itself, detect different playback errors that occur in the monitoring device, and use different reconnection strategies to troubleshoot, effectively improving the user experience, solving the problem of users not being able to see a stable and continuous picture due to too frequent device reconnection, and solving the problem of not being able to see the video picture for a long time due to a device failure and not reconnecting in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0072] FIG1 is a flow chart of a method for implementing audio and video playback provided by an embodiment of the present application;

[0073] FIG2 is a schematic diagram of the structure of an audio and video playback device according to an embodiment of the present application;

[0074] FIG3 is a block diagram of the hardware structure of an audio and video playback device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0076] In view of the fact that most of the current audio and video playback solutions are difficult to adapt to the complex and changing business needs, the applicant has studied an audio and video playback solution. This audio and video playback method can detect the abnormal video stream that has not been successfully received continuously and completely when the playback data of the monitoring device fails, or actively detect whether there is an abnormality in the data stream sent by the server, determine the cause of the failure, and promptly troubleshoot the cause of the failure. Therefore, the method provided by the implementation of this application can not only monitor the network of audio and video data, but also monitor the correctness of the audio and video data itself, detect different playback errors that occur in the monitoring device, and use different reconnection strategies to troubleshoot, effectively improving the user experience, solving the problem that users cannot see a stable and continuous picture due to too frequent device reconnection, and solving the problem that the device fails and fails to reconnect in time, resulting in a long-term inability to see the video picture.

[0077] The methods provided in the embodiments of the present application can be used in a variety of general-purpose or specialized computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, and distributed computing environments including any of the above.

[0078] An embodiment of the present application provides an audio and video playback method, which can be applied to various monitoring control systems or security management systems, and can also be applied to various computer terminals or smart terminals. Its execution subject can be the processor or server of the computer terminal or smart terminal.

[0079] The following describes the process of the audio and video playback method provided in an embodiment of the present application in conjunction with FIG1 . As shown in FIG1 , the process may include the following steps:

[0080] Step S101: Obtain the playback address of the audio and video data to be played.

[0081] Specifically, in actual applications, monitoring devices generally have limited storage space. If the monitoring data of the monitoring device is stored in real time on the monitoring device, the monitoring device may not be able to save so much data, which is not conducive to data traceability. Therefore, in actual applications, the monitoring data of the monitoring device is generally stored on a server so that the monitoring data can be viewed at any time.

[0082] Therefore, when a monitoring device has a playback failure, it may be because the data to be played is not obtained in time or the device connection fails. In order to determine the cause of the failure of the monitoring device and troubleshoot it in time, the playback address of the audio and video data to be played can be obtained so that it can be confirmed whether it can be successfully connected to the playback address of the audio and video data to be played to obtain the audio and video data to be played.

[0083] in,

[0084] The playback address of the audio and video data to be played is a storage path pointing to the storage of the audio and video data to be played. The audio and video data to be played can be obtained through the playback address of the audio and video data to be played.

[0085] Step S102: determine whether the playback address is successfully obtained.

[0086] Specifically, as can be seen from the above introduction, the audio and video data to be played can be obtained through the playback address of the audio and video data to be played.

[0087] If the playback address is not successfully obtained, it may mean that the connection with the playback address of the audio and video data to be played has not been successfully established, or it may be that a device failure has caused the inability to obtain the playback address. Therefore, when the playback address of the audio and video data to be played is not successfully obtained, step S103 can be executed.

[0088] Step S103: determining the reason for failure in obtaining the playback address.

[0089] Specifically, as can be seen from the above introduction, if it is necessary to obtain the audio and video data to be played, it is necessary to obtain the playback address corresponding to the audio and video data to be played.

[0090] There may be several reasons why the playback address corresponding to the audio or video data to be played is not obtained.

[0091] For example, if the playback address corresponding to the audio or video data to be played is not obtained, it may be due to an interface call error, the decoder of the monitoring device is not turned on, or the device fails to connect.

[0092] Therefore, after determining that the playback address of the audio and video data to be played has not been successfully obtained, the reason for the failure to obtain the playback address can be further determined, so that reconnection can be performed according to the specific reason for the failure to obtain the playback address.

[0093] Step S104: executing a preset reconnection strategy according to the reason for the failure to obtain the playback address until the playback address is successfully obtained.

[0094] Specifically, as can be seen from the above introduction, if the playback address is not successfully obtained, there may be various reasons.

[0095] The method provided in the embodiment of the present application can further determine the reason for the failure to obtain the playback address after determining that the playback address has not been successfully obtained.

[0096] After determining the reason for the failure in obtaining the playback address, a preset reconnection strategy may be further executed according to the reason for the failure in obtaining the playback address until the playback address is successfully obtained.

[0097] If the playback address has been successfully obtained, step S105 may be executed.

[0098] Step S105: establishing a connection relationship with the playback address.

[0099] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can reconstruct the connection according to the reason for the failure to obtain the playback data until the playback address corresponding to the audio and video data to be played is successfully obtained.

[0100] If the playback address corresponding to the audio and video data to be played has been successfully obtained, it means that the audio and video data to be played can be obtained through the playback address.

[0101] Therefore, after determining that the playback address corresponding to the audio and video data to be played has been successfully obtained, a connection relationship can be further established with the obtained playback address so that the obtained playback address can be accessed through the network connection relationship to obtain the audio and video data to be played.

[0102] Step S106: Determine whether a connection relationship is successfully established with the playback address.

[0103] Specifically, from the above introduction, it can be seen that after determining that the audio and video data to be played has been successfully obtained, the method provided by the implementation of this application can further establish a connection relationship with the obtained playback address, so that the obtained playback address can be accessed through a network connection to obtain the audio and video data to be played.

[0104] In order to ensure that the obtained playback address can be successfully accessed to obtain the video data to be played, after establishing a connection relationship with the obtained playback address, it can be further determined whether a connection relationship has been successfully established with the playback address, so as to confirm whether the playback address can be successfully accessed to obtain the audio and video data to be played.

[0105] If a connection relationship with the obtained playback address is not successfully established, it may be necessary to analyze the reason for the failure in obtaining the playback address so that the connection relationship can be re-established.

[0106] If a connection relationship has been successfully established with the acquired playback address, step S107 may be executed.

[0107] Step S107: Wait and receive a network packet corresponding to the audio and video data to be played.

[0108] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can establish a connection relationship with the obtained playback address after obtaining the playback address corresponding to the audio and video data to be played.

[0109] If a connection relationship has been established with the obtained playback location, it means that the audio and video data to be played can be obtained through the obtained playback address.

[0110] Therefore, after determining that a connection relationship has been successfully established with the acquired playback address, the playback address can be further accessed through the network connection relationship to obtain the audio and video data to be played.

[0111] In actual application, the audio and video data to be played are generally stored in the server corresponding to the monitoring control system or the security system. Through the playback address, the storage path corresponding to the playback address can be accessed to obtain the audio and video data to be played.

[0112] Therefore, after determining that a connection relationship has been successfully established with the obtained playback address, you can access the server corresponding to the playback address, send the request instruction for the required data to be played to the server corresponding to the playback address, and finally wait for and receive the network packet sent by the server corresponding to the audio and video data to be played.

[0113] With the rapid development and widespread use of the internet, network transmission has become an integral part of our daily lives. Network transmission enables the transmission of data packets from one location to another, and it does so with high speed and efficiency. Data packets, the fundamental unit of network transmission, are composed of many different components. In this article, we will detail the composition of data packets and their importance in network transmission.

[0114] A data packet is the fundamental unit of network transmission. It is a sequence of binary data consisting of electrical pulses. A data packet consists of two main parts: a header and data. The header contains important information such as the source and destination IP addresses, protocol type, and packet size. This information in the header determines the packet's routing and ensures its proper transmission across the network. The data portion contains the actual information being transmitted, such as text, audio, or images.

[0115] Each layer appends a header to the data being sent. This header contains essential information for that layer, such as the destination address and protocol-related information. Typically, the information provided to the protocol is the packet header, while the content to be sent is the data. From the perspective of the next layer, all packets received from the previous layer are considered data for that layer.

[0116] Therefore, the network packet corresponding to the audio and video data to be played may include relevant information and data of the audio and video data to be played.

[0117] After receiving the network packet corresponding to the audio and video data to be played, the audio and video data to be played can be obtained by parsing the network packet corresponding to the audio and video data to be played so that the network packet corresponding to the audio and video data to be played can be played smoothly.

[0118] Step S108: Determine whether the network packet is successfully received.

[0119] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can wait for and receive the network packet corresponding to the audio and video data to be played after determining that a connection relationship has been successfully established with the acquired playback address.

[0120] During the process of receiving network packets, it is possible to further determine whether the network packets corresponding to the audio and video data to be played are successfully received, so that after determining that the network packets corresponding to the audio and video data to be played have not been received in full, timely feedback can be provided and the network packets corresponding to the audio and video data to be played can be re-requested.

[0121] If the network packet has been successfully received, step S109 may be executed.

[0122] Step S109: Decode the network packet.

[0123] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can wait for and receive the network packet corresponding to the audio and video data to be played after determining that a connection relationship has been successfully established with the acquired playback address.

[0124] If it is determined that the network packet corresponding to the audio and video data to be played has been successfully received, it means that the audio and video data to be played can be obtained through the received network packet.

[0125] In actual application, monitoring equipment can generally support audio and video data in different formats. For different audio and video data, the monitoring equipment can use its own decoder to decode audio and video data packets in different formats to obtain data that the monitoring equipment can play.

[0126] Therefore, after determining that the complete network packets corresponding to the audio and video to be played have been successfully received, the received complete network packets corresponding to the audio and video to be played can be further decoded to obtain the audio and video data to be played.

[0127] Step S110, determining whether the network packet is successfully decoded;

[0128] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can decode the received complete network packets corresponding to the audio and video to be played.

[0129] Furthermore, if the decoding is successful, the audio and video data to be played can be successfully obtained and played smoothly. If the decoding is unsuccessful, it means that the audio and video data to be played cannot be obtained and played smoothly.

[0130] Therefore, in order to ensure whether the audio and video data that can be played can be successfully obtained, after decoding the network packets corresponding to the complete audio and video to be played, it is possible to further determine whether the decoding of the complete network packets corresponding to the received audio and video to be played has been successfully completed, so that when it is determined that the decoding has failed successfully, the network packets can be decoded again or another decoding method can be used to decode the network packets again so that the audio and video data to be played can be successfully obtained.

[0131] If the received complete network packets corresponding to the audio and video to be played have been successfully decoded, step S111 may be executed.

[0132] Step S111 , obtaining the audio and video data to be played corresponding to the network packet, and rendering and playing the audio and video data to be played corresponding to the network packet.

[0133] Specifically, it can be seen from the technical solution introduced above that the method provided in the embodiment of the present application can further determine whether the decoding of the network packets corresponding to the received complete audio and video to be played has been successfully completed after decoding the network packets corresponding to the received complete audio and video to be played.

[0134] If the network packet corresponding to the complete audio and video to be played has been successfully decoded, it means that the audio and video data that can be played smoothly can be obtained, and the audio and video data to be played corresponding to the network packet can be obtained, and the audio and video data to be played corresponding to the network packet can be rendered and played.

[0135] Step S112: If a failure occurs in playing the audio and video data to be played corresponding to the network packet, the type of the failure is determined.

[0136] Specifically, in actual application, various fault conditions may occur after playing audio and video data. When a fault occurs in playing the audio and video to be played, the method provided in the embodiment of the present application can further determine the type of fault that occurs based on the fault condition, and perform troubleshooting for different fault types. Based on the type of fault that occurs, return to execute according to the preset reconnection strategy until the audio and video data to be played corresponding to the network packet is successfully replayed, so as to ensure that the monitoring equipment can play the monitoring data stably and smoothly.

[0137] in,

[0138] In the method provided in the embodiment of the present application, determining the fault type may include the following judgment methods:

[0139] The first one,

[0140] Determine whether the playback address of the current connection is the playback address corresponding to the audio and video data to be played. If the playback address of the current connection is not the playback address corresponding to the audio and video data to be played, it may cause data acquisition failure and inability to play data. It may be caused by a network failure or an address access error. The fault type can be determined as a playback address failure.

[0141] The second type,

[0142] It is possible to detect whether the monitoring device has not displayed the data stream to be played for more than a preset first time period. If the monitoring device has not displayed the data stream to be played for more than the preset first time period, it may be because the monitoring device suddenly goes offline or there is a performance failure of the streaming media server corresponding to the monitoring device. Therefore, it can be determined that the fault type is that the target monitoring device is offline or there is a performance failure of the streaming media server corresponding to the data stream to be played.

[0143] in,

[0144] The data stream to be played may include one or more data of an audio and video data stream, a video data stream or a subtitle data stream;

[0145] The preset first duration can be set to 30 seconds.

[0146] The third type,

[0147] It is possible to detect whether the decoder of the monitoring device has been successfully opened. If the decoder of the monitoring device fails to open, the audio and video data stream may not be played because the network packets corresponding to the audio and video data stream to be played cannot be decoded normally. It may also be because the video data format of the monitoring device is abnormal or the high resolution and high bit rate cause the monitoring device to consume too much memory, which has reached the maximum value that the software can use, causing playback failure. Therefore, it can be determined that the fault type is an abnormal audio and video data format of the monitoring device or the memory consumption of the monitoring device exceeds the maximum value that the monitoring device's memory can bear.

[0148] The fourth type,

[0149] You can compare the video timestamp of the monitoring device with the timestamp of the control system corresponding to the monitoring device. If the video timestamp of the monitoring device is later than the timestamp of the control system corresponding to the monitoring device, it is possible that the monitoring data playback has failed or been stuck due to video delay. Therefore, you can determine that the fault type is a video delay data delay fault.

[0150] The fifth type,

[0151] It is possible to determine whether the decoding function of the decoder of the monitoring device is normal. If the decoding function of the decoder of the monitoring device is abnormal, the audio and video data to be played may not be decoded. The playback failure may be caused by abnormal video data format on the monitoring device side or internal problems of the local player. The fault type can be determined as abnormal video data format of the monitoring device or a fault in the local player of the control system corresponding to the monitoring device.

[0152] The sixth type,

[0153] Determine whether the monitoring device receives a preset number of complete data frames within the preset second time period. If the monitoring device does not receive the preset number of complete data frames within the preset second time period, it means that some data may be lost, and the fault type can be determined to be a data loss fault.

[0154] Specifically, during network transmission, stream data may be easily lost due to network jitter and other reasons.

[0155] For example, some streaming media servers use TCP to send data, which includes an internal mechanism for actively discarding network packets. If the packet buffer size is set too small, it can result in only sending the first half of the fragmented packets that make up an i-frame to the monitoring client, while the second half remains unsent. This can cause the monitoring device's player to be unable to parse the image and remain stuck at the initialization screen.

[0156] Therefore, when data loss occurs, reconnection can prevent excessive data loss when network jitter is severe during certain periods, causing the player to be unable to play the image for a long time.

[0157] in,

[0158] The preset second duration can be set to 5 minutes.

[0159] The seventh type,

[0160] Determine whether the renderer of the monitoring device fails to open due to a problem with the control system corresponding to the monitoring device. If the renderer of the monitoring device fails to open due to a problem with the control system corresponding to the target monitoring device, it may be impossible to complete the rendering of the audio and video data to be played, resulting in the inability to play the monitoring data normally. It can be determined that the fault type is a system failure of the control system corresponding to the monitoring device.

[0161] In actual application, problems with the operating system itself may cause occasional inability to open videos.

[0162] Furthermore, the method provided in the embodiment of the present application can output local logs and prompt information based on different errors in the data played by the monitoring device.

[0163] For example,

[0164] When an interface error occurs on the monitoring platform, the code value and the content of msg can be output, and a prompt message can be displayed, wherein the prompt output content can be the content of msg.

[0165] When an error occurs during the playback of monitoring data, if it is determined that there is a problem with the playback address, the code value and error cause can be printed and a prompt message can be displayed. For other errors, the error cause can be printed, where the prompt output content can be the error cause.

[0166] It can be seen from the technical solutions introduced above that when the audio and video playback of the monitoring device fails, the method provided by the embodiment of the present application can detect the abnormal video stream that has not been successfully received continuously and completely, or actively detect whether there is an abnormality in the data stream sent by the server, determine the cause of the failure and give an error prompt message, and timely troubleshoot the cause of the failure according to the error prompt message. Therefore, the method provided by the implementation of the present application can not only monitor the network of audio and video data, but also monitor the correctness of the audio and video data itself, detect and use different reconnection strategies to troubleshoot different playback errors that occur in the monitoring device, effectively improving the user experience, solving the problem that the user cannot see a stable and continuous picture due to too frequent device reconnection, and solving the problem that the device fails and does not reconnect in time, resulting in a long-term inability to see the video picture. Furthermore, the method provided by the embodiment of the present application can be applied to single-grid or multi-grid devices that need to monitor video pictures for a long time. There is no need for external triggering of reconnection. The reconnection mechanism will be automatically triggered in the event of an error, which can effectively improve the troubleshooting efficiency.

[0167] As can be seen from the above description, when it is determined that the playback address corresponding to the audio and video data to be played cannot be successfully obtained, the method provided in the embodiment of the present application can determine the reason for the failure to successfully obtain the playback address, and execute a preset reconnection strategy according to the specific reason for the failure to successfully obtain the playback address until the playback address is successfully obtained. The following describes the process, which may include the following steps:

[0168] Step S201: If it is determined that the reason for the failure to obtain the playback address is an interface call error, and the playback address fails to be obtained for the first time, a first reconnection is performed with the interface corresponding to the playback address within a preset first time.

[0169] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can analyze and determine the specific reason for the failure to obtain the playback address.

[0170] There are many reasons for the failure to obtain the playback address. When it is determined that the reason for the failure to obtain the playback address is an interface call error, if the first acquisition of the playback address corresponding to the audio and video data to be played fails, the first reconnection can be performed with the interface corresponding to the playback address corresponding to the audio and video data to be played within the preset first time.

[0171] in,

[0172] In order to quickly reconnect and obtain the playback address corresponding to the audio and video data to be played again, thereby reducing the waiting time, the preset first time can be set to 1 minute.

[0173] Therefore, when it is determined that the reason for the failure to obtain the playback address is an interface call error, if the first acquisition of the playback address corresponding to the audio and video data to be played fails, the first reconnection can be made to the interface corresponding to the playback address corresponding to the audio and video data to be played within one minute, so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible.

[0174] Step S202: If the interface corresponding to the playback address fails to reconnect for the first time within the preset first time, it is determined whether the reason for the failure to reconnect for the interface corresponding to the playback address for the first time within the preset first time is an interface call error.

[0175] Specifically, from the above introduction, it can be seen that the method provided in the embodiment of the present application can be used to determine that the reason for the failure to obtain the playback address is an interface call error. If the playback address corresponding to the audio and video data to be played fails to be obtained for the first time, then the interface corresponding to the playback address corresponding to the audio and video data to be played can be reconnected for the first time as soon as possible within the preset first time, so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible.

[0176] In actual application, if the first reconnection is performed as soon as possible within the preset first time with the interface corresponding to the playback address corresponding to the audio and video data to be played, the connection may fail again.

[0177] Therefore, if the interface corresponding to the playback address corresponding to the audio and video data to be played fails to reconnect for the first time within the preset first time, it can be further determined whether the reason for the failure to reconnect for the interface corresponding to the playback address within the preset first time is an interface call error.

[0178] Analyzing the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time can help adopt different reconnection strategies. If the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is still an interface call error, step S203 can be executed.

[0179] Step S203: When it is determined that the first reconnection fails, reconnecting to the interface corresponding to the playback address within a preset second time period.

[0180] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can analyze the reason for the first reconnection failure of the interface corresponding to the playback address within a preset first time and execute different reconnection strategies according to the reason for the first reconnection failure.

[0181] If it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is still an interface call error, then the interface corresponding to the playback address can be reconnected again within the preset second time period from the moment the first reconnection failure is determined.

[0182] in,

[0183] The preset second time can be set to 2 minutes.

[0184] Therefore, when it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within one minute is still an interface call error, the interface corresponding to the playback address can be reconnected within 2 minutes from the moment the first reconnection failure is determined, so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible.

[0185] In actual application, when it is determined that the first reconnection fails, the interface corresponding to the playback address is reconnected within a preset second time period, and a connection failure may occur again.

[0186] Therefore, if it is determined that the interface corresponding to the playback address corresponding to the audio and video data to be played fails to be reconnected again within the preset second time, step S204 can be executed.

[0187] Step S204: determining whether the reason for the failure of the interface corresponding to the playback address to reconnect again within a preset second time period is an interface call error.

[0188] Specifically, in actual application, when it is determined that the first reconnection fails, the interface corresponding to the playback address is reconnected within the preset second time period, and a connection failure may occur again.

[0189] Therefore, if it is determined that the interface corresponding to the playback address corresponding to the audio and video data to be played fails to reconnect again within the preset second time, it can be further determined whether the reason for the first failure to reconnect the interface corresponding to the playback address within the preset second time is still an interface call error.

[0190] If the reason why the interface corresponding to the playback address fails to reconnect again within the preset second time is still an interface call error, it means that each previous connection has called an interface error, and step S205 can be executed.

[0191] Step S205: from the moment when it is determined that the reconnection fails, reconnecting to the interface corresponding to the playback address within a preset third time period until the playback address is successfully obtained.

[0192] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can analyze the reason why the interface corresponding to the playback address fails to reconnect within the preset second time and execute different reconnection strategies according to the reason for the reconnection failure.

[0193] In actual application, when it is determined that the first reconnection fails, the interface corresponding to the playback address is reconnected within a preset second time period, and a connection failure may occur again.

[0194] Therefore, if it is determined that the interface corresponding to the playback address corresponding to the audio and video data to be played has failed to reconnect within the preset second time, and if it is determined that the reason for the failure to reconnect with the interface corresponding to the playback address within the preset second time is still an interface call error, then the interface corresponding to the playback address can be reconnected within the preset third time period from the moment when the reconnection failure is determined.

[0195] in,

[0196] The preset third time can be set to 5 minutes.

[0197] Therefore, if it is determined that the reason for the failure to reconnect to the interface corresponding to the playback address within 2 minutes is still an interface call error, the interface corresponding to the playback address can be reconnected within 5 minutes from the moment the reconnection failure is determined, so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible. Furthermore, in actual application, if the interface corresponding to the playback address is reconnected within a preset third time period from the moment the reconnection failure is determined, it is possible that the connection failure will occur again.

[0198] If the interface corresponding to the playback address fails to be reconnected within the preset third time period, the connection can be continued according to the interface corresponding to the playback address within the preset third time period from the moment the connection failure is determined until the playback address corresponding to the audio and video data to be played is successfully obtained.

[0199] It can be seen from the technical solution introduced above that when the audio and video playback of the monitoring device fails, and when it is determined that the playback address corresponding to the audio and video data to be played has not been successfully obtained, the method provided in the embodiment of the present application can determine the reason for the failure to successfully obtain the playback address, and execute the preset reconnection strategy according to the specific reason for the failure to successfully obtain the playback address, until the playback address corresponding to the audio and video data to be played is successfully obtained, so as to obtain the audio and video data to be played through the playback address corresponding to the audio and video data to be played.

[0200] In actual application, if it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is not an interface call error, the reconnection strategy implemented by the method provided in the embodiment of the present application is different. The following describes the reconnection strategy implemented by the method provided in the embodiment of the present application when it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is not an interface call error. Specifically, it may include the following steps:

[0201] Step S301: When it is determined that the first reconnection fails, reconnect to the interface corresponding to the playback address within a preset fourth time period.

[0202] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can analyze and determine the specific reason for the failure to obtain the playback address.

[0203] In actual applications, there are many reasons for failure to obtain the playback address, and there are many reasons for failure to initiate a reconnection.

[0204] For example, the reason for the reconnection failure may be an interface call error or a non-interface call error.

[0205] If it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is a non-interface call error, it can be determined that from the moment of the first reconnection failure, the interface corresponding to the playback address will be reconnected again within the preset fourth time period so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible.

[0206] in,

[0207] In order to obtain the playback address corresponding to the audio and video data to be played as quickly as possible, the preset fourth time period can be set to 3 seconds.

[0208] For example, if it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within 1 minute is non-interface call error, it can be determined that the interface corresponding to the playback address should be reconnected within 3 seconds from the moment of the first reconnection failure, so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible.

[0209] Step S302 , determining whether reconnection to the interface corresponding to the playback address is successful within a preset fourth time period starting from the moment when the first reconnection failure is determined.

[0210] Specifically, from the above introduction, it can be seen that the method provided in the embodiment of the present application can determine that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is a non-interface call error, and then it can be determined that from the moment of the first reconnection failure, the interface corresponding to the playback address will be reconnected again within the preset fourth time period.

[0211] In actual application, from the moment when the first reconnection failure is determined, reconnection is performed again with the interface corresponding to the playback address within the preset fourth time period, and a connection failure may occur again.

[0212] Therefore, in order to confirm whether the interface corresponding to the playback address is successfully reconnected within the preset fourth time period from the moment when the first reconnection failure is determined, it can be further determined whether the interface corresponding to the playback address is successfully reconnected within the preset fourth time period from the moment when the first reconnection failure is determined.

[0213] If the interface corresponding to the playback address fails to reconnect again within the preset fourth time, step S303 can be executed so that different reconnection strategies can be implemented according to the cause of the connection failure so that the audio and video playback data to be played can be obtained as soon as possible.

[0214] Step S303: Determine whether the reason for the failure to reconnect the interface corresponding to the playback address within a preset fourth time period is an interface call error.

[0215] Specifically, in actual application, from the moment when the first reconnection failure is determined, reconnection is performed again with the interface corresponding to the playback address within the preset fourth time period, and a connection failure may occur again.

[0216] If the interface corresponding to the playback address fails to reconnect within the preset fourth time period, in order to confirm the specific reason for the reconnection of the interface corresponding to the playback address within the preset fourth time period, it can be further determined whether the reason for the failure to reconnect with the interface corresponding to the playback address within the preset fourth time period is an interface call error.

[0217] If it is determined that the reason why the interface corresponding to the playback address fails to reconnect again within the preset second time is an interface call error, step S304 may be executed.

[0218] Step S304: reconnecting to the interface corresponding to the playback address within a preset first time period from the moment when it is determined that the reconnection fails.

[0219] Specifically, it can be seen from the technical solution introduced above that the method provided in the embodiment of the present application can determine whether the reason for the failure to reconnect the interface corresponding to the playback address within the preset fourth time is an interface call error.

[0220] If the connection failure is determined to be caused by an interface call error, the reconnection strategy implemented will be different.

[0221] Therefore, if it is determined that the reason for the failure to reconnect to the interface corresponding to the playback address within the preset second time period is an interface call error, then reconnection can be performed to the interface corresponding to the playback address within the preset first time period starting from the moment when the reconnection failure is determined. Step S305 determines whether the reconnection to the interface corresponding to the playback address within the preset first time period is successful starting from the moment when the reconnection failure is determined.

[0222] Specifically, from the above introduction, it can be seen that the method provided in the embodiment of the present application can determine that the reason for the failure to reconnect to the interface corresponding to the playback address within the preset second time is an interface call error. Then, from the moment it is determined that the reconnection has failed, the interface corresponding to the playback address can be reconnected within the preset first time period.

[0223] In actual application, when it is determined that the reconnection fails, the interface corresponding to the playback address is reconnected within a preset first time period, and a connection failure may occur again.

[0224] Therefore, in order to confirm whether the reconnection to the interface corresponding to the playback address is successful within the preset first time period from the moment when the reconnection failure is determined, it can be further determined whether the reconnection to the interface corresponding to the playback address is successful within the preset first time period from the moment when the reconnection failure is determined.

[0225] If reconnection to the interface corresponding to the playback address fails within a preset first time period starting from the moment when reconnection failure is determined, it indicates that the currently executed reconnection strategy cannot resolve the fault, and step S306 may be executed.

[0226] Step S306 , determining whether the reason for the failure to reconnect to the interface corresponding to the playback address within a preset first time period from the moment when the reconnection failure is determined is an interface call error.

[0227] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can determine whether the reconnection to the interface corresponding to the playback address is successful within a preset first time period from the moment when the reconnection failure is determined.

[0228] If the interface corresponding to the playback address fails to reconnect again within a preset first time period from the moment it is determined that the reconnection has failed, it means that the currently executed reconnection strategy cannot solve the fault, and the specific cause of the connection failure can be further analyzed so that reconnection can be performed as soon as possible.

[0229] Therefore, when the interface corresponding to the playback address fails to reconnect again within the preset first time period from the moment when the reconnection failure is determined, it can be determined whether the reason for the failure to reconnect again with the interface corresponding to the playback address within the preset first time period from the moment when the reconnection failure is determined is an interface call error.

[0230] If, from the moment when it is determined that the reconnection has failed, the reason for the failure to reconnect to the interface corresponding to the playback address within the preset first time period is an interface call error, it means that you can continue to reconnect according to the reconnection strategy for the interface call error introduced above, and you can return to execute the operation of reconnecting to the interface corresponding to the playback address within the preset second time period until you successfully obtain the playback address, so that you can obtain the audio and video data to be played through the playback address corresponding to the audio and video data to be played.

[0231] If, from the moment when it is determined that the reconnection has failed, the reason for the failure to reconnect to the interface corresponding to the playback address within the preset first time period is a non-interface call error, it means that the reconnection can be continued according to the reconnection strategy for non-interface call errors introduced above, and the operation of reconnecting to the interface corresponding to the playback address within the preset fourth time period can be returned to until the playback address is successfully obtained, so that the audio and video data to be played can be obtained through the playback address corresponding to the audio and video data to be played.

[0232] It can be seen from the technical solution introduced above that when the audio and video playback of the monitoring device fails, and when it is determined that the playback address corresponding to the audio and video data to be played has not been successfully obtained, the method provided in the embodiment of the present application can determine the reason for the failure to successfully obtain the playback address, and execute the preset reconnection strategy according to the specific reason for the failure to successfully obtain the playback address, until the playback address corresponding to the audio and video data to be played is successfully obtained, so as to obtain the audio and video data to be played through the playback address corresponding to the audio and video data to be played.

[0233] In actual application, when it is determined that the playback address corresponding to the audio and video data to be played has not been successfully obtained, the method provided in the embodiment of the present application can determine the reason for the failure to successfully obtain the playback address, and execute a preset reconnection strategy according to the specific reason for the failure to successfully obtain the playback address until the playback address is successfully obtained. Another implementation method of this process is described below, which is as follows:

[0234] If it is determined that the reason for the failure to obtain the playback address corresponding to the audio and video data to be played is a non-interface call error, it means that the currently called interface is accurate, and the failure to obtain the playback address corresponding to the audio and video data to be played may be due to other fault reasons. In this case, the first reconnection with the interface corresponding to the playback address can be performed within the preset fourth time, so that the playback address corresponding to the audio and video data to be played can be obtained as soon as possible to obtain the audio and video data to be played.

[0235] It can be seen from the technical solution introduced above that when the audio and video playback of the monitoring device fails, and when it is determined that the playback address corresponding to the audio and video data to be played has not been successfully obtained, the method provided in the embodiment of the present application can determine the reason for the failure to successfully obtain the playback address, and execute the preset reconnection strategy according to the specific reason for the failure to successfully obtain the playback address, until the playback address corresponding to the audio and video data to be played is successfully obtained, so as to obtain the audio and video data to be played through the playback address corresponding to the audio and video data to be played.

[0236] As can be seen from the technical solution described above, when the playback address corresponding to the audio and video data to be played fails to be obtained, the method provided in the embodiment of the present application can determine the reason for the failure to obtain the playback address. The following describes several judgment methods, which specifically include the following:

[0237] The first one,

[0238] If the target monitoring device is offline or a network connection failure occurs, it can be determined that the reason for the failure to obtain the playback address is an interface error.

[0239] The second type,

[0240] If the target monitoring device fails to play the audio and video data to be played, it is determined that the reason for the failure to obtain the playback address is that there is an error in the playback address.

[0241] It can be seen from the technical solution introduced above that when the audio and video playback of the monitoring device fails, the method provided in the embodiment of the present application can further determine the reason for the failure to obtain the playback address corresponding to the audio and video data to be played after failing to obtain the playback address, so that different reconnection strategies can be executed according to the specific reason.

[0242] The following describes an audio and video playback device provided in an embodiment of the present application. The audio and video playback device described below and the audio and video playback method described above can refer to each other.

[0243] Refer to FIG2 , which is a schematic structural diagram of an audio and video playback device disclosed in an embodiment of the present application.

[0244] As shown in FIG2 , the audio and video playback device may include:

[0245] The playback address acquisition unit 101 is used to acquire the playback address of the audio and video data to be played;

[0246] The first judging unit 102 is used to judge whether the playback address is successfully obtained;

[0247] A first determining unit 103 is configured to determine a reason for failure in obtaining the playback address if the execution result of the first determining unit determines that the playback address has not been successfully obtained;

[0248] The reconnection unit 104 is configured to execute a preset reconnection strategy according to the reason for the failure to obtain the playback address until the playback address is successfully obtained;

[0249] A construction unit 105 is configured to establish a connection relationship with the playback address when the execution result of the first judgment unit determines that the playback address has been successfully obtained;

[0250] The second judgment unit 106 is used to judge whether a connection relationship is successfully established with the playback address;

[0251] The receiving unit 107 is configured to wait for and receive a network packet corresponding to the audio and video data to be played when the execution result of the second determining unit determines that a connection relationship has been successfully established with the playback address;

[0252] The third judgment unit 108 is used to judge whether the network packet is successfully received;

[0253] a decoding unit 109 configured to decode the network packet when the execution result of the third judgment unit is that the network packet has been successfully received;

[0254] The fourth determining unit 110 is configured to determine whether the network packet is successfully decoded;

[0255] The playing unit 111 is configured to obtain the audio and video data to be played corresponding to the network packet when the execution result of the fourth judgment unit determines that the network packet is decoded successfully, and render and play the audio and video data to be played corresponding to the network packet.

[0256] It can be seen from the technical solution introduced above that when a fault occurs in the playback screen of the monitoring device and it is necessary to detect it for troubleshooting, the device provided in the embodiment of the present application can obtain the playback address of the audio and video data to be played, and determine whether the playback address is successfully obtained; if the playback address is not successfully obtained, it means that the acquisition of the playback address may have failed, and the reason for the failure to obtain the playback address can be further determined; and based on the reason for the failure to obtain the playback address, the preset reconnection strategy is executed until the playback address is successfully obtained, so that the audio and video data to be played can be obtained through the playback address. If the playback address has been successfully obtained, it means that the audio and video data to be played can be obtained through the playback address, and a connection relationship can be established with the playback address; and it can be further determined whether a connection relationship is successfully established with the playback address; if a connection relationship is successfully established with the playback address, it means that the audio and video data to be played can be obtained through the established network connection relationship. Therefore, after establishing a connection relationship with the playback address, you can wait for and receive the network packet corresponding to the audio and video data to be played; and determine whether the network packet is successfully received; if the network packet has been successfully received, you can decode the network packet; so that the audio and video data to be played can be obtained. Therefore, after decoding the network packet, you can further determine whether the network packet is successfully decoded; if the network packet is successfully decoded, it means that the audio and video data to be played corresponding to the network packet can be obtained. After obtaining the audio and video data to be played, the audio and video data to be played corresponding to the network packet can be rendered and played.

[0257] From the above analysis, it can be seen that the device provided by the embodiment of the present application can detect the abnormal video stream that has not been successfully received continuously and completely when the playback data of the monitoring device fails, or actively detect whether there is an abnormality in the data stream sent by the server, determine the cause of the failure, and promptly troubleshoot the cause of the failure. Therefore, the method provided by the implementation of this application can not only monitor the network of audio and video data, but also monitor the correctness of the audio and video data itself, detect different playback errors that occur in the monitoring device, and use different reconnection strategies to troubleshoot, effectively improving the user experience, solving the problem of users not being able to see a stable and continuous picture due to too frequent device reconnection, and solving the problem of not being able to see the video picture for a long time due to a device failure and not reconnecting in time.

[0258] Among them, the specific processing flow of each unit included in the above-mentioned audio and video playback device can refer to the relevant introduction of the audio and video playback method part above, and will not be repeated here.

[0259] The audio and video playback device provided in the embodiments of the present application can be applied to audio and video playback devices, such as terminals such as mobile phones and computers. Optionally, FIG3 shows a block diagram of the hardware structure of the audio and video playback device. Referring to FIG3 , the hardware structure of the audio and video playback device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0260] In an embodiment of the present application, the number of processor 1 , communication interface 2 , memory 3 , and communication bus 4 is at least one, and the processor 1 , communication interface 2 , and memory 3 communicate with each other through the communication bus 4 .

[0261] The processor 1 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0262] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0263] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement various processing flows in the aforementioned terminal audio and video playback solution.

[0264] An embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to implement various processing flows of the aforementioned terminal in the audio and video playback solution.

[0265] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0266] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0267] The above description of the disclosed embodiments is intended to enable those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. The various embodiments may be combined with one another. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio and video playback method, characterized in that: include: Get the playback address of the audio and video data to be played; Determine whether the playback address is successfully obtained; If the playback address is not successfully obtained, determining the reason for the failure to obtain the playback address; According to the reason for the failure to obtain the playback address, a preset reconnection strategy is executed until the playback address is successfully obtained; If the playback address has been successfully obtained, a connection relationship is established with the playback address; Determine whether a connection relationship is successfully established with the playback address; If a connection relationship has been successfully established with the playback address, then waiting for and receiving a network packet corresponding to the audio and video data to be played; Determining whether the network packet is successfully received; If the network packet has been successfully received, decoding the network packet; Determining whether the network packet is successfully decoded; If the network packet is decoded successfully, the audio and video data to be played corresponding to the network packet are obtained, and the audio and video data to be played corresponding to the network packet are rendered and played.

2. The method according to claim 1, characterized in that The method further includes: If a failure occurs in playing the audio and video data to be played corresponding to the network packet, determining the type of the failure; According to the fault type, return to execute according to the preset reconnection strategy until the audio and video data to be played corresponding to the network packet is successfully played again.

3. The method according to claim 1, characterized in that The executing a preset reconnection strategy until the playback address is successfully acquired includes: If it is determined that the reason for the failure to obtain the playback address is an interface call error, and the playback address fails to be obtained for the first time, reconnecting to the interface corresponding to the playback address for the first time within a preset first time; If the first reconnection of the interface corresponding to the playback address fails within the preset first time, determining whether the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time is an interface call error; If it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time period is still an interface call error, then from the moment when the first reconnection failure is determined, the playback address is reconnected within the preset second time period. Reconnecting the interface corresponding to the playback address; If the interface corresponding to the playback address fails to reconnect again within the preset second time, determining whether the reason for the failure to reconnect again with the interface corresponding to the playback address within the preset second time is an interface call error; If it is determined that the reason for the failure to reconnect to the interface corresponding to the playback address within the preset second time is still an interface call error, then from the moment when the reconnection failure is determined, the interface corresponding to the playback address is reconnected within a preset third time period until the playback address is successfully obtained.

4. The method according to claim 3, characterized in that The method further includes: If it is determined that the reason for the first reconnection failure of the interface corresponding to the playback address within the preset first time period is non-interface call error, then reconnecting to the interface corresponding to the playback address within a preset fourth time period from the moment when the first reconnection failure is determined; Determining whether reconnection is successful again at the interface corresponding to the playback address within a preset fourth time period from the moment when the first reconnection failure is determined; If the interface corresponding to the playback address fails to reconnect again within the preset fourth time, determining whether the reason for the failure to reconnect again with the interface corresponding to the playback address within the preset fourth time is an interface call error; If it is determined that the reason why the interface corresponding to the playback address fails to reconnect again within the preset second time period is an interface call error, reconnecting again with the interface corresponding to the playback address within the preset first time period from the moment when the reconnection failure is determined; Determine whether reconnection with the interface corresponding to the playback address is successful within a preset first time period from the moment when the reconnection failure is determined; If the interface corresponding to the playback address fails to reconnect again within a preset first time period from the moment when the reconnection failure is determined, determining whether the reason for the failure to reconnect again with the interface corresponding to the playback address within the preset first time period from the moment when the reconnection failure is determined is an interface call error; If, from the moment when it is determined that the reconnection fails, the reason for the failure of reconnecting the interface corresponding to the playback address within the preset first time period is an interface call error, then returning to execute the operation of reconnecting the interface corresponding to the playback address within the preset second time period until the playback address is successfully obtained; If the reason for the failure of reconnection in the interface corresponding to the playback address to reconnect again within the preset first time period is not an interface call error, then the execution of the The interface corresponding to the playback address is reconnected within a preset fourth time period until the playback address is successfully acquired.

5. The method according to claim 1, characterized in that The executing a preset reconnection strategy until the playback address is successfully obtained includes: If it is determined that the reason for the failure to obtain the playback address is not an interface call error, the interface corresponding to the playback address is reconnected for the first time within a preset fourth time until the playback address is successfully obtained.

6. The method according to claim 1, characterized in that The determining of the reason for failure in obtaining the playback address includes: If the target monitoring device is offline or a network connection failure occurs, determining that the reason for the failure to obtain the playback address is an interface error; If the target monitoring device fails to play the audio and video data to be played, it is determined that the reason for the failure to obtain the playback address is that there is an error in the playback address.

7. The method according to claim 2, characterized in that The determining of the fault type comprises: Determine whether the playback address of the current connection is the playback address corresponding to the audio and video data to be played, and if the playback address of the current connection is not the playback address corresponding to the audio and video data to be played, determine the fault type as a playback address fault; or, Detecting whether the target monitoring device has not displayed the data stream to be played for more than a preset first time period, if the target monitoring device has not displayed the data stream to be played for more than the preset first time period, determining that the fault type is that the target monitoring device is offline or that the streaming media server corresponding to the data stream to be played has a performance fault; or, Detecting whether the decoder of the target monitoring device has been successfully opened, if the decoder of the target monitoring device fails to be opened, determining the fault type as an abnormal audio and video data format of the target monitoring device or a memory consumption of the target monitoring device exceeding a maximum value that the memory of the target monitoring device can bear; or, Comparing the video timestamp of the target monitoring device with the timestamp of the control system corresponding to the target monitoring device, if the video timestamp of the target monitoring device is later than the timestamp of the control system corresponding to the target monitoring device, determining that the fault type is a video delay data delay fault; or, Determine whether the decoding function of the decoder of the target monitoring device is normal. If the decoding function of the decoder of the target monitoring device is abnormal, determine that the fault type is that the video data format of the target monitoring device is abnormal or the local player of the control system corresponding to the target monitoring device is faulty; or, Determining whether the target monitoring device receives a preset number of complete data frames within a preset second time period, and if the target monitoring device does not receive a preset number of complete data frames within the preset second time period, determining that the fault type is a data loss fault; or, Determine whether the renderer of the target monitoring device fails to open due to a problem with the control system corresponding to the target monitoring device. If the renderer of the target monitoring device fails to open due to a problem with the control system corresponding to the target monitoring device, determine that the fault type is a system fault of the control system corresponding to the target monitoring device.

8. An audio and video playback device, characterized in that: include: A playback address acquisition unit, used to acquire the playback address of the audio and video data to be played; A first judging unit, used to judge whether the playback address is successfully obtained; a first determining unit, configured to determine a reason for failure in obtaining the playback address if the execution result of the first determining unit determines that the playback address is not successfully obtained; A reconnection unit, configured to execute a preset reconnection strategy according to a reason for failure to obtain the playback address, until the playback address is successfully obtained; a construction unit, configured to establish a connection relationship with the playback address when the execution result of the first judgment unit determines that the playback address has been successfully acquired; A second judgment unit, used to judge whether a connection relationship is successfully established with the playback address; a receiving unit, configured to wait for and receive a network packet corresponding to the audio and video data to be played when the execution result of the second judging unit determines that a connection relationship has been successfully established with the playback address; A third judging unit, used to judge whether the network packet is successfully received; a decoding unit, configured to decode the network packet when the execution result of the third judgment unit is to determine that the network packet has been successfully received; A fourth determination unit, used to determine whether the network packet is successfully decoded; a playback unit, configured to obtain the audio and video data to be played corresponding to the network packet when the execution result of the fourth judgment unit determines that the network packet is decoded successfully, and to play the audio and video data to be played corresponding to the network packet; The audio and video data to be played is rendered and played.

9. An audio and video playback device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the audio and video playback method as described in any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors implement the steps of the audio and video playback method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Server access monitoring method and device, proxy server and storage medium

    CN112235138A

  • Audio and video playing method and system, storage medium and computer program product

    CN114040251A

  • Video playing abnormity diagnosis method and related device

    CN114827636A

  • Audio and video playing method and device, equipment and readable storage medium

    CN116506666A

  • Video decoding method and device, storage medium, and program product

    WO2023024801A1