Audio and video data processing method and system, electronic equipment and storage medium

By detecting the audio and video data frame rate in the monitoring system and discarding the incomplete picture group, the problems of low efficiency of human eye judgment and frequent restarts are solved, and automated detection and processing are realized, improving system stability and user experience.

CN118984401APending Publication Date: 2024-11-19VISIONVERA INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411015973.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The normality of audio and video data in existing monitoring systems mainly depends on human eye judgment, which is inefficient and can easily lead to fatigue and negligence. The common solution is to simply restart, which cannot fundamentally troubleshoot, which may lead to data loss or delay. Impacts the real-time and continuity of the system, and frequent restarts may damage the device.

Method used

By detecting the audio and video data frame rate information of the monitoring terminal on the server side, if the frame rate conditions are not met, the incomplete picture group will be discarded, and the device detection signaling will be sent to determine the camera status if necessary. If it is abnormal, it will be marked as an abnormal status. , if the video stream does not exist, it will be resent, and the audio and video data is processed in parallel by using the detection thread to improve detection efficiency.

Benefits of technology

It realizes automated detection and processing, improves the efficiency and accuracy of audio and video data judgment, reduces manual intervention, reduces labor costs, extends the service life of the equipment, ensures the timeliness and continuity of monitoring data, and improves user experience .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984401A_ABST
    Figure CN118984401A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an audio and video data processing method and system applied to a server, and the method comprises the steps: detecting audio and video data of a monitoring terminal; acquiring frame rate information of the audio and video data; and if the frame rate information does not meet the frame rate condition, discarding at least one incomplete picture group of the audio and video data. According to the embodiment of the invention, by automatically detecting and processing the frame rate information, the audio and video data judgment efficiency and accuracy are improved, the labor cost is reduced, and the stability of the system and the user experience are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of audio and video technology, and in particular to a method for processing audio and video data, a system for processing audio and video data, an electronic device, and a computer-readable storage medium. Background Art

[0002] In today's surveillance systems, the normality of audio and video data is crucial to the overall performance of the system and user experience. However, the normality of audio and video data in most surveillance terminals still mainly depends on the judgment of the human eye. This traditional method has many disadvantages and limitations.

[0003] First, the efficiency of human judgment is low. The audio and video data in the monitoring system is usually real-time and continuous, requiring all-weather monitoring and judgment. In this case, relying on manual observation and judgment is not only time-consuming and labor-intensive, but also prone to fatigue and negligence, thus affecting the accuracy of judgment.

[0004] Secondly, when there is a problem with audio and video data, a common solution is to simply restart the monitoring terminal. However, this direct restart method often only solves the problem temporarily and cannot fundamentally eliminate the source of the fault. Although the restart operation is simple, its effectiveness is not high, and it may cause temporary loss or delay of monitoring data, affecting the real-time and continuity of the system. In addition, frequent restart operations may also cause certain damage to the hardware of the device and shorten the service life of the device. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a method for processing audio and video data, a system for processing audio and video data, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In order to solve the above problems, an embodiment of the present invention discloses a method for processing audio and video data, which is applied to a server, and the method includes: detecting audio and video data of a monitoring terminal; obtaining frame rate information of the audio and video data; if the frame rate information does not meet the frame rate condition, discarding at least one incomplete picture group of the audio and video data.

[0007] Optionally, if the frame rate information does not meet the frame rate condition, at least one incomplete picture group of the audio and video data is discarded, including: if the frame rate information meets the condition that no key frame is received within a preset period, the incomplete picture group before the next received key frame is discarded.

[0008] Optionally, the method also includes: if the audio and video data is not detected, sending a packet sending instruction to the monitoring terminal; if the audio and video data is still not detected, sending a device detection signaling to the monitoring terminal to determine whether the camera of the monitoring terminal is normal; if the camera is abnormal, marking the camera working status of the monitoring terminal as an abnormal state in the monitoring list, and marking the camera working status as an abnormal state on the display screen.

[0009] Optionally, after sending the device detection signaling to the monitoring terminal, the method also includes: if no response information is received from the monitoring terminal within a first time period, or if response information is received from the monitoring terminal after a second time period, then marking the signal state of the monitoring terminal as a weak state in the monitoring list, and marking the response state of the monitoring terminal as a slow state on the display screen.

[0010] Optionally, the method also includes: obtaining fault signaling from a streaming media server, wherein the fault signaling is uploaded to the streaming media server by a client, and the fault signaling indicates that a fault has occurred in the monitoring terminal; extracting a target monitoring terminal from a terminal array list according to the fault signaling; detecting whether a video stream exists at the target monitoring terminal; if the video stream exists at the target monitoring terminal, sending video stream notification information to the streaming media server, so that the streaming media server detects whether the video stream is transmitted to the client according to the video stream notification information; if the video stream does not exist at the target monitoring terminal, forwarding the fault signaling to the target monitoring terminal, so that the target monitoring terminal resends the video stream according to the fault signaling.

[0011] Optionally, before the audio and video data of the monitoring terminal is detected, the method further includes: obtaining the number of the monitoring terminals; starting at least one detection thread according to the number; and detecting the audio and video data using the detection thread.

[0012] Optionally, after detecting the audio and video data using the detection thread, the method further includes: if the audio and video data of the monitoring terminal is normal, moving the monitoring terminal to the end of the terminal array list.

[0013] An embodiment of the present invention also discloses a system for processing audio and video data, which is applied to a server. The system includes: an audio and video data detection module, which is used to detect the audio and video data of a monitoring terminal; a frame rate information acquisition module, which is used to obtain the frame rate information of the audio and video data; and a picture group discarding module, which is used to discard at least one incomplete picture group of the audio and video data if the frame rate information does not meet the frame rate condition.

[0014] Optionally, the picture group discarding module is configured to discard the incomplete picture group before the next received key frame if the frame rate information satisfies that no key frame is received within a preset period.

[0015] Optionally, the system also includes: a packet instruction sending module, which is used to send a packet instruction to the monitoring terminal if the audio and video data is not detected; a detection signaling sending module, which is used to send a device detection signaling to the monitoring terminal if the audio and video data is still not detected, so as to determine whether the camera of the monitoring terminal is normal; and a status marking module, which is used to mark the camera working status of the monitoring terminal as an abnormal state in the monitoring list if the camera is abnormal, and mark the camera working status as an abnormal state on the display screen.

[0016] Optionally, the status marking module is also used to mark the signal status of the monitoring terminal as a weak state in the monitoring list and mark the response status of the monitoring terminal as a slow state on the display screen after the detection signaling sending module sends the device detection signaling to the monitoring terminal, if no response information is received from the monitoring terminal within a first time period, or if a response information is received from the monitoring terminal after a second time period.

[0017] Optionally, the system also includes: a fault signaling acquisition module, which is used to acquire fault signaling from a streaming media server, wherein the fault signaling is uploaded to the streaming media server by a client, and the fault signaling indicates that a fault has occurred in the monitoring terminal; a monitoring terminal extraction module, which is used to extract a target monitoring terminal from a terminal array list according to the fault signaling; a video stream detection module, which is used to detect whether a video stream exists at the target monitoring terminal; a notification information sending module, which is used to send video stream notification information to the streaming media server if the video stream exists at the target monitoring terminal, so that the streaming media server detects whether the video stream is transmitted to the client according to the video stream notification information; and a fault signaling forwarding module, which is used to forward the fault signaling to the target monitoring terminal if the video stream does not exist at the target monitoring terminal, so that the target monitoring terminal resends the video stream according to the fault signaling.

[0018] Optionally, the system also includes: a terminal quantity acquisition module, used to obtain the number of monitoring terminals before the audio and video data detection module detects the audio and video data of the monitoring terminals; a detection thread starting module, used to start at least one detection thread according to the number; and an audio and video data detection module, used to use the detection thread to detect the audio and video data.

[0019] Optionally, the system further includes: a monitoring terminal moving module, configured to move the monitoring terminal to the end of the terminal array list after the audio and video data detection module detects the audio and video data using the detection thread, if the audio and video data of the monitoring terminal is normal.

[0020] An embodiment of the present invention further discloses an electronic device, comprising: one or more processors; and one or more machine-readable media storing instructions thereon, which, when executed by the one or more processors, enables the electronic device to execute the method for processing audio and video data as described above.

[0021] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the above-mentioned method for processing audio and video data.

[0022] The embodiments of the present invention include the following advantages:

[0023] The audio and video data processing solution provided by the embodiment of the present invention is applied to a server, which detects the audio and video data of the monitoring terminal and then obtains the frame rate information of the audio and video data. If the frame rate information does not meet the frame rate condition, at least one incomplete picture group of the audio and video data is discarded.

[0024] The embodiment of the present invention automatically detects the frame rate information of audio and video data through a server, thereby avoiding the inefficiency and subjectivity of manual judgment. Automated detection is not only fast, but also highly accurate, and can detect anomalies in audio and video data in real time. The detection and judgment of frame rate information are carried out in real time. Once it is found that the frame rate does not meet the conditions, it can immediately respond and discard the incomplete picture group to ensure the timeliness and continuity of monitoring data. Through automated detection and processing, the need for manual intervention is reduced, and the labor cost is greatly reduced. When it is detected that the frame rate information does not meet the frame rate conditions, the incomplete picture group is automatically discarded, avoiding the transmission and storage of invalid or problematic data. This self-repair capability can quickly take measures when a problem occurs to prevent the problem from expanding and ensure the quality of monitoring data. Compared with frequent restart operations, through intelligent detection and processing, the number of physical restarts of the monitoring terminal is reduced, the loss of the equipment is reduced, and the service life of the equipment is extended. By discarding the incomplete picture group, the user is prevented from seeing the situation of freezing or flower screen, the overall quality of the monitoring picture is improved, and a better user experience is provided.

[0025] In summary, the audio and video data processing solution applied to the server improves the efficiency and accuracy of audio and video data judgment by automatically detecting and processing frame rate information, reduces labor costs, and improves system stability and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a method for processing audio and video data according to an embodiment of the present invention;

[0027] Figure 2 It is a schematic diagram of the principle of a patrol inspection solution for a large number of monitoring videos according to an embodiment of the present invention;

[0028] Figure 3 It is a structural block diagram of a system for processing audio and video data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] The audio and video data processing solution proposed in the embodiment of the present invention is applied to the server, and the quality of the audio and video data is judged in real time by automatically detecting the frame rate information of the audio and video data of the monitoring terminal. When the frame rate information does not meet the preset frame rate condition, at least one incomplete picture group will be automatically discarded to ensure the continuity and quality of the monitoring data. This solution not only improves the judgment efficiency and accuracy, reduces the dependence on manual intervention, but also improves the stability and reliability of the monitoring system through self-repair capabilities. Compared with traditional manual judgment and simple restart methods, this solution significantly reduces labor costs, reduces workload, and improves user experience.

[0031] Reference Figure 1 , shows a flowchart of the steps of a method for processing audio and video data according to an embodiment of the present invention. The method for processing audio and video data can be applied to a server, and the server can be specifically a co-conversion server. The co-conversion server can be understood as a server connected to a monitoring terminal, a client, a streaming media server and other devices. The method for processing audio and video data can specifically include the following steps:

[0032] Step 101: Detect audio and video data of a monitoring terminal.

[0033] In an embodiment of the present invention, the co-conversion server needs to establish a stable connection with the monitoring terminal. Through a network protocol (such as a real-time streaming protocol), the monitoring terminal sends audio and video data or data streams to the co-conversion server. This process needs to ensure the stability of the network bandwidth and connection to prevent data loss or delay. When the audio and video data arrives at the co-conversion server, the co-conversion server needs to parse the received audio and video data. The audio and video data are usually encapsulated in a specific format, and the co-conversion server needs to decode to obtain the original audio and video signal. During the decoding process, the audio signal and the video signal need to be synchronized. This is crucial to ensure the integrity and consistency of the data. For example, the audio clip corresponding to a video frame needs to be processed and played at the same time to avoid the problem of audio and video being out of sync. The co-conversion server also needs to monitor the received audio and video data in real time to detect the quality and integrity of the audio and video data. By calculating the arrival rate and delay of the data packet, the quality of the network transmission can be judged. If packet loss or excessive delay is detected, the co-conversion server can take corresponding measures to adjust.

[0034] For example: suppose a monitoring terminal sends an audio and video data stream to a co-transmission server via a real-time streaming protocol. The co-transmission server establishes a connection and starts receiving data packets. By parsing each data packet, the co-transmission server obtains the video frame and audio clip. Then, the co-transmission server calculates the number of data packets received per second in real time and compares it with the expected value, finding that the network transmission quality is good and there is no packet loss.

[0035] Step 102, obtaining frame rate information of audio and video data.

[0036] In an embodiment of the present invention, the co-conversion server needs to calculate the frame rate of the received audio and video data. The audio and video data consists of a series of continuous frames, each frame representing a static image. By calculating the number of frames received per second, the co-conversion server can obtain the actual frame rate of the audio and video stream. Each video frame is usually accompanied by timestamp information for recording the generation and sending time of the frame. By parsing these timestamp information, the co-conversion server can accurately calculate the time interval between frames, thereby obtaining the frame rate information. The co-conversion server needs to monitor the changes in the frame rate in real time and detect whether the frame rate remains within a preset range. Too low a frame rate or too large a fluctuation can cause the video to freeze or not be smooth, affecting the user's viewing experience. During the processing, the co-conversion server will set a frame rate condition, including the minimum acceptable frame rate value. This minimum acceptable frame rate value is usually determined according to the needs of the specific application scenario. For example, high-definition video usually requires 30 frames per second (FPS), while ordinary surveillance video may only require 15 frames per second.

[0037] For example: After receiving the audio and video data, the co-conversion server starts counting the video frames received per second. Assuming that the co-conversion server receives 25 frames per second in a certain period of time, through simple calculation, it can be concluded that the current frame rate is 25FPS. The co-conversion server also parses the timestamp information of each frame to verify whether the time interval between frames is consistent to ensure the accuracy of the frame rate calculation.

[0038] During real-time monitoring, the co-transmission server sets a frame rate condition of 20FPS. By monitoring the current frame rate, the co-transmission server finds that the frame rate drops to 15FPS during certain periods, which is lower than the set frame rate condition. This may be caused by insufficient network bandwidth or terminal device performance issues. The co-transmission server records the information of these frame rate changes and is ready to take corresponding measures to deal with the low frame rate problem in the next step.

[0039] Step 103: If the frame rate information does not meet the frame rate condition, at least one incomplete picture group of the audio and video data is discarded.

[0040] In an embodiment of the present invention, the co-convergence server compares the current frame rate information with the preset frame rate condition. If the current frame rate information is lower than the set minimum frame rate value, it is determined that the frame rate condition is not met. Audio and video data are usually sent in groups, and each group contains multiple frames. The co-convergence server needs to identify which picture groups are incomplete or of poor quality. These picture groups may not be displayed normally due to frame loss or low frame rate. After identifying the incomplete picture groups, the co-convergence server decides to discard these data. The discarding strategy needs to be designed according to the actual situation to ensure that it does not affect the continuity of the overall audio and video data. Normally, the co-convergence server will discard those frames that obviously affect the viewing experience and retain frames with higher quality. While discarding the incomplete picture group, the co-convergence server will record the detailed information of the discarding operation, including the time, reason and amount of discarded data. These records can be used for subsequent analysis and optimization of system performance.

[0041] For example, the co-conversion server detects that the frame rate of the audio and video data has dropped to 15FPS, which is lower than the preset frame rate condition of 20FPS. Through further analysis, the co-conversion server finds that some picture groups have frame loss due to network bandwidth problems, resulting in incomplete picture groups. To ensure the overall quality of the audio and video data, the co-conversion server decides to discard these incomplete picture groups.

[0042] The co-transmission server identifies which picture groups are incomplete. For example, a picture group should contain 30 frames, but only 20 frames are currently received. The missing 10 frames cause the picture group to be unable to display normally. The co-transmission server marks this picture group as incomplete and discards it in the next processing cycle. At the same time, the co-transmission server records this operation, noting that the reason for the discard is that the frame rate is too low, and retains the relevant timestamp information.

[0043] The audio and video data processing solution provided by the embodiment of the present invention is applied to a server, which detects the audio and video data of the monitoring terminal and then obtains the frame rate information of the audio and video data. If the frame rate information does not meet the frame rate condition, at least one incomplete picture group of the audio and video data is discarded.

[0044] The embodiment of the present invention automatically detects the frame rate information of audio and video data through a server, thereby avoiding the inefficiency and subjectivity of manual judgment. Automated detection is not only fast, but also highly accurate, and can detect anomalies in audio and video data in real time. The detection and judgment of frame rate information are carried out in real time. Once it is found that the frame rate does not meet the conditions, it can immediately respond and discard the incomplete picture group to ensure the timeliness and continuity of monitoring data. Through automated detection and processing, the need for manual intervention is reduced, and the labor cost is greatly reduced. When it is detected that the frame rate information does not meet the frame rate conditions, the incomplete picture group is automatically discarded, avoiding the transmission and storage of invalid or problematic data. This self-repair capability can quickly take measures when a problem occurs to prevent the problem from expanding and ensure the quality of monitoring data. Compared with frequent restart operations, through intelligent detection and processing, the number of physical restarts of the monitoring terminal is reduced, the loss of the equipment is reduced, and the service life of the equipment is extended. By discarding the incomplete picture group, the user is prevented from seeing the situation of freezing or flower screen, the overall quality of the monitoring picture is improved, and a better user experience is provided.

[0045] In summary, the audio and video data processing solution applied to the server improves the efficiency and accuracy of audio and video data judgment by automatically detecting and processing frame rate information, reduces labor costs, and improves system stability and user experience.

[0046] In an exemplary embodiment of the present invention, if the frame rate information does not meet the frame rate condition, at least one incomplete picture group of the audio and video data is discarded. An implementation method is that if the frame rate information meets the condition that the key frame is not received within a preset period, the incomplete picture group before the next received key frame is discarded. It is a processing for key frames, which helps to ensure the quality and continuity of audio and video data. The key frame is an important frame in the audio and video data. It contains complete image data, and subsequent frames rely on it for differential encoding. If the key frame is not received within the preset period, the subsequent frames may not be correctly decoded and displayed, so these incomplete picture groups need to be discarded.

[0047] In actual application, the cooperative conversion server monitors the frame rate information of the audio and video data to determine whether it meets the preset frame rate conditions. Within the preset period, the cooperative conversion server detects whether the key frame is received. If the key frame is not received, the incomplete picture group before the next key frame received is discarded. Specifically, the cooperative conversion server monitors the frame rate information of the audio and video data to determine in real time whether the current frame rate meets the preset frame rate conditions. The frame rate condition is usually set according to the specific application scenario. For example, the frame rate condition of high-definition video may be set to 30 frames per second (FPS). The cooperative conversion server calculates the number of frames received per second and parses the timestamp information of the frame to obtain the actual frame rate information. If the actual frame rate information is lower than the preset value, it is necessary to further check the reception of the key frame. The key frame is a frame that contains complete image data in the video encoding. Unlike ordinary frames, the key frame does not rely on the previous frame for decoding. Therefore, the key frame plays a vital role in the continuity and quality of the video stream. The cooperative conversion server detects the reception of the key frame within a preset period. If the key frame is not received within this period, the subsequent normal frames may not be decoded correctly, resulting in incomplete and stuck video streams. If the key frame is not received within the preset period, the co-transmission server will initiate a discarding strategy. Specifically, the co-transmission server will discard the incomplete picture groups before the next received key frame. These incomplete picture groups cannot be displayed normally due to the lack of key frame support. Discarding them can prevent users from seeing stuck or erroneous pictures. When discarding incomplete picture groups, the co-transmission server will record detailed information about the discarding operation, including the reason, time, and specific amount of data for the discarding. These records can be used for subsequent system analysis and optimization.

[0048] For example: Assume that a monitoring system processes audio and video data through a co-transmission server. Due to insufficient network bandwidth, the co-transmission server detects that the actual frame rate of the video stream has dropped to 15FPS, which is lower than the preset frame rate condition of 20FPS. Further detection found that no key frames were received in the past 2 seconds. At this time, the co-transmission server decides to discard the incomplete picture group before the next key frame received. For example, at the 3rd second, the co-transmission server receives a key frame, so the co-transmission server discards the ordinary frames received at the 1st and 2nd seconds, because these ordinary frames lack the support of key frames and cannot be displayed normally. Through this processing, the co-transmission server can ensure that the video stream starting from the 3rd second is complete and continuous, and the user will not see the incomplete pictures at the 1st and 2nd seconds, thereby improving the overall quality of the video stream and user experience.

[0049] The embodiment of the present invention avoids video freeze and erroneous display caused by missing key frames by discarding incomplete picture groups, thereby ensuring the continuity and clarity of audio and video data. Ordinary frames that cannot be decoded correctly are discarded, reducing the transmission and processing of invalid data and optimizing the use of network bandwidth. Users will not see freeze or erroneous pictures, which improves the reliability of the monitoring system and user satisfaction. It automatically adapts to network conditions and data flow changes, improving the system's adaptability and stability.

[0050] In an exemplary embodiment of the present invention, if no audio and video data is detected, a packet sending instruction is sent to the monitoring terminal; if no audio and video data is still detected, a device detection signaling is sent to the monitoring terminal to determine whether the camera of the monitoring terminal is normal; if the camera is abnormal, the camera working state of the monitoring terminal is marked as abnormal in the monitoring list, and the camera working state is marked as abnormal on the display screen. In the actual application process, the cooperative conversion server monitors the audio and video data received from the monitoring terminal in real time. If no audio and video data is detected within a preset time, further measures need to be taken for diagnosis. In the case where no audio and video data is detected, the cooperative conversion server sends a packet sending instruction to the monitoring terminal. The purpose of sending the packet sending instruction is to notify the monitoring terminal to resend the audio and video data. If no audio and video data is still detected after sending the packet sending instruction, the cooperative conversion server will further send a device detection signaling to the monitoring terminal. The device detection signaling is a specific instruction used to detect the various components of the monitoring terminal, especially the working state of the camera. The device detection signaling triggers the monitoring terminal to execute a self-test program, check the status of key components such as the camera, network interface, and processor, and feed back the detection results to the cooperative conversion server. Through the information returned by the device detection signaling, the coordination server can determine whether the camera of the monitoring terminal is working properly. If the detection result shows that the camera is faulty or cannot work properly, the coordination server will record this abnormal state. The specific causes of the failure may include damage to the camera hardware, loose connections, power failure, etc. The coordination server will take corresponding measures based on the detection results, such as notifying maintenance personnel to perform repairs. If the camera is judged to be abnormal, the coordination server will mark the working status of the camera of the monitoring terminal as abnormal in the monitoring list. At the same time, the abnormal mark of the camera working status will also be displayed on the display screen of the monitoring system, so that the monitoring personnel can find and deal with the problem in time.

[0051] For example: Assume that in a certain monitoring system, the coordination server does not detect the audio and video data of a certain monitoring terminal in a certain period of time. The coordination server sends a packet instruction to the monitoring terminal, but still does not receive the response audio and video data. Subsequently, the coordination server sends a device detection signal to the monitoring terminal, and the detection result shows that the camera is faulty. According to the detection result, the coordination server marks the camera status of the monitoring terminal as abnormal in the monitoring list, and displays the abnormal mark on the display screen of the monitoring system. After the monitoring personnel discovered the abnormality, they immediately notified the technicians to go to the site for inspection and repair, and found that the camera could not work due to a power failure. After the technicians replaced the power supply, the camera returned to normal, the audio and video data transmission was restored, and the monitoring system operated normally again.

[0052] The embodiments of the present invention can quickly diagnose network problems and equipment failures by sending packet instructions and device detection signaling, thereby reducing the time for fault location. By marking abnormal status in the monitoring list and display screen, monitoring personnel can promptly discover problems, respond and handle them quickly, and reduce the time of monitoring blind spots. Timely discovery and resolution of failures of key equipment such as cameras ensures the stable operation of the monitoring system and improves overall security. Automated fault detection and status marking reduce reliance on manual monitoring and judgment, and improve system management efficiency and maintenance effects.

[0053] In an exemplary embodiment of the present invention, after sending a device detection signaling to a monitoring terminal, an implementation method is that if no response information is received from the monitoring terminal within a first time period, or if a response information is received from the monitoring terminal after a second time period, the signal state of the monitoring terminal is marked as a weak state in the monitoring list, and the response state of the monitoring terminal is marked as a slow state on the display screen. In actual application, the coordination server sends a device detection signaling to the monitoring terminal, and the device detection signaling is a specific instruction for the monitoring terminal to perform a self-check and return status information. This process is intended to confirm whether the various components of the monitoring terminal are working properly, especially the network connection and the status of the camera. Two time periods are set, the first time period is used to monitor the initial response time of the device detection signaling, and the second time period is used to monitor the delayed response time. The first time period is usually short, such as 1-2 seconds, so as to quickly determine whether there is a network or device failure. The second time period is longer, such as 5-10 seconds, and is used to detect delayed response situations. After sending the device detection signaling, the coordination server starts timing and waits for the response of the monitoring terminal. If no response information is received within the first time period, it means that the monitoring terminal may have a weak signal or a network problem. At this time, the coordination server needs to further observe the response situation within the second time period. If the response information is received after the second time period, it means that the response speed of the monitoring terminal is slow, which may be caused by network delays, slow terminal processing speed, etc. According to the reception of the response information, the coordination server marks the signal status of the monitoring terminal as weak in the monitoring list, and marks the response status of the monitoring terminal as slow on the display screen. The signal status marked as weak means that the network connection quality of the monitoring terminal is poor, which may cause unstable audio and video data transmission. The response status marked as slow means that the response speed of the monitoring terminal is slow, which may affect the real-time monitoring effect.

[0054] For example: Assume that in a monitoring system, the coordination server sends a device detection signaling to a monitoring terminal. The coordination server sets the first time period to 2 seconds and the second time period to 10 seconds. After the device detection signaling is sent, the coordination server does not receive a response from the monitoring terminal within 2 seconds. Then, at the 8th second, the monitoring terminal returns a response. Based on the response, the coordination server determines that the signal state of the monitoring terminal is weak, and marks the signal state of the monitoring terminal as weak in the monitoring list. At the same time, on the display screen of the monitoring system, the response state of the monitoring terminal is marked as slow, so that the monitoring personnel can understand the working status of the terminal in a timely manner. After discovering this situation, the monitoring personnel can take corresponding measures, such as adjusting the network settings and checking the terminal equipment, to ensure the normal operation of the monitoring system.

[0055] By setting the first time period and the second time period, the embodiment of the present invention can timely discover the problem of weak signal of the monitoring terminal, which is convenient for rapid processing and repair. By detecting the response time, the response delay of the monitoring terminal can be identified to ensure the real-time and effectiveness of the monitoring system. The signal status and response status are marked on the monitoring list and display screen, which is convenient for monitoring personnel to quickly understand the working conditions of each terminal and improve the system management efficiency. Timely discovery and processing of weak signal and slow response problems help prevent potential equipment failures and ensure the stable operation of the monitoring system.

[0056] In an exemplary embodiment of the present invention, a fault signaling is obtained from a streaming media server, and the fault signaling is uploaded to the streaming media server by a client, and the fault signaling indicates that a monitoring terminal has a fault. According to the fault signaling, a target monitoring terminal is extracted from a terminal array list; whether a video stream exists in the target monitoring terminal; if a video stream exists in the target monitoring terminal, video stream notification information is sent to the streaming media server, so that the streaming media server detects whether the video stream is transmitted to the client according to the video stream notification information; if the target monitoring terminal does not have a video stream, the fault signaling is forwarded to the target monitoring terminal, so that the target monitoring terminal resends the video stream according to the fault signaling. In actual application, the streaming media server monitors various signaling information uploaded by the client in real time, including fault signaling. When a client detects that the monitoring terminal to which it is connected has a fault, it uploads this information to the streaming media server. Fault signaling is a specific signal indicating that the client detects that there is a problem with the monitoring terminal, such as video stream interruption, unstable network connection, etc. After receiving the fault signaling, the streaming media server forwards the fault signaling to the coordination server, and the coordination server extracts the target monitoring terminal identified in the fault signaling from the terminal array list according to the fault signaling. The terminal array list is a collection of information about all monitoring terminals in the monitoring system, including the identification, status, and other information of each monitoring terminal. By extracting the target monitoring terminal, fault handling can be carried out in a targeted manner to avoid affecting other monitoring terminals that are operating normally. After extracting the target monitoring terminal, the co-transmission server will detect whether there is a video stream for the target monitoring terminal. The video stream is the real-time video data transmitted by the target monitoring terminal and is an important part of the monitoring system. The existence of the video stream can be detected through the status information of the monitoring terminal and the transmission record of the streaming media server. If the monitoring terminal is transmitting the video stream, it means that the video stream exists, otherwise the video stream does not exist. If the target monitoring terminal is detected to have a video stream, the co-transmission server will send video stream notification information to the streaming media server. The video stream notification information is information used to notify the streaming media server to check and process a specific video stream. The streaming media server detects whether the video stream is transmitted normally to the client based on the video stream notification information. If a transmission problem is found in the video stream, the streaming media server will make corresponding adjustments and processing to ensure smooth transmission of the video stream. If it is detected that the target monitoring terminal does not have a video stream, the co-transmission server will forward the fault signaling to the target monitoring terminal. The purpose of forwarding fault signaling is to notify the target monitoring terminal that there is a problem with its video stream and ask it to resend the video stream. After receiving the fault signaling, the target monitoring terminal will restart the video stream, including re-establishing the connection, restarting video acquisition and transmission, etc., to ensure that the video stream can be retransmitted to the streaming media server.

[0057] For example: Assume that in a monitoring system, the client of a monitoring terminal detects that the video stream is interrupted, and the client uploads a fault signaling to the streaming media server. After receiving the fault signaling, the streaming media server forwards the fault signaling to the coordination server. The coordination server extracts the monitoring terminal from the terminal array list, detects that the monitoring terminal has a video stream, and then sends a video stream notification message to the streaming media server. The streaming media server detects the transmission status of the video stream based on the notification information, and finds that the video stream is not transmitted to the client, so it makes adjustments to restore the normal transmission of the video stream. If it is detected that the monitoring terminal does not have a video stream, the coordination server will forward the fault signaling to the monitoring terminal. After receiving the fault signaling, the monitoring terminal restarts the transmission of the video stream to ensure that the video stream can be transmitted to the streaming media server and the client.

[0058] The embodiments of the present invention can quickly identify and solve the faults of the monitoring terminal by acquiring fault signaling and processing it in a targeted manner, thereby improving the efficiency of fault processing. By detecting and adjusting the video stream transmission situation, the consistency and stability of the video stream are ensured, and the interruption of the monitoring screen is avoided. By processing only the faulty monitoring terminal, the impact on other normally operating terminals is avoided, and the utilization of system resources is optimized. By automatically detecting and processing faults, the intelligence level of the monitoring system is improved, manual intervention is reduced, and the reliability and stability of the system are improved.

[0059] In an exemplary embodiment of the present invention, before the audio and video data of the monitoring terminal is detected, an implementation method is to obtain the number of monitoring terminals; start at least one detection thread according to the number; and use the detection thread to detect the audio and video data. In the actual application process, before the audio and video data detection is performed, the co-transfer server needs to count the number of all current monitoring terminals. This can be achieved by accessing the database of the monitoring system or obtaining the registration information of the monitoring terminal through the system interface. The purpose of obtaining the number of monitoring terminals is to reasonably allocate system resources to ensure that the audio and video data of each monitoring terminal can be detected in time, thereby improving the coverage and efficiency of the detection. According to the number of monitoring terminals obtained, the co-transfer server will start a corresponding number of detection threads according to a certain ratio. The detection thread is an independent execution unit for parallel processing of audio and video data detection tasks. The number of detection threads started can be adjusted according to actual conditions. For example, if there are 100 monitoring terminals in the monitoring system, the co-transfer server can start 10 detection threads, each detection thread is responsible for detecting the audio and video data of 10 monitoring terminals. This can effectively disperse the detection tasks and avoid excessive load on a single detection thread. The process of starting the detection thread involves creating a thread object, assigning tasks, and starting thread execution. Once the detection thread is started, it will start to detect the audio and video data of the assigned monitoring terminal. After each detection thread is started, it will detect the audio and video data of the assigned monitoring terminal according to the predetermined detection algorithm and strategy. The detection content includes video frame rate, audio quality, data transmission stability, etc. Through the parallel execution of detection threads, the coordination server can complete the audio and video data detection tasks of a large number of monitoring terminals in a short time, ensuring the real-time and accuracy of the detection results.

[0060] For example: suppose that in a monitoring system, there are 200 monitoring terminals that need to perform audio and video data detection. The coordination server obtains the number of monitoring terminals, which is 200, and then decides to start 20 detection threads, each of which is responsible for detecting the audio and video data of 10 monitoring terminals. After starting the detection thread, each detection thread is responsible for the audio and video data detection tasks of the 10 assigned monitoring terminals. For example, detection thread 1 is responsible for detecting the audio and video data from monitoring terminal 1 to monitoring terminal 10, detection thread 2 is responsible for detecting the audio and video data from monitoring terminal 11 to monitoring terminal 20, and so on. Each detection thread runs independently and processes detection tasks in parallel to ensure that data detection of all monitoring terminals is completed in a short time.

[0061] The embodiment of the present invention can process detection tasks in parallel and improve the efficiency of audio and video data detection by pre-acquiring the number of monitoring terminals and starting a corresponding number of detection threads. Reasonable allocation of detection tasks ensures that the audio and video data of each monitoring terminal can be detected in time to avoid missed detection and false detection. Through parallel detection and real-time processing, problems in audio and video data can be quickly discovered and processed, and the accuracy and reliability of detection results can be improved. System resources are reasonably allocated according to the number of monitoring terminals to avoid overloading of a single detection thread and waste of system resources, and optimize system performance. By dynamically adjusting the number of detection threads, it can adapt to the number of monitoring terminals of different sizes, and has high flexibility and scalability.

[0062] In an exemplary embodiment of the present invention, after using the detection thread to detect audio and video data, an implementation method is that if the audio and video data of the monitoring terminal is normal, the monitoring terminal is moved to the end of the terminal array list. In the actual application process, after starting the detection thread, each detection thread will perform a detailed detection of the audio and video data of the assigned monitoring terminal. The detection content includes video frame rate, audio quality, data transmission stability, etc. The detection thread improves the detection efficiency through parallel processing to ensure that the audio and video data of each monitoring terminal can be detected in time. After the detection is completed, the co-transfer server will judge the audio and video data status of each monitoring terminal. Normal audio and video data should meet the preset frame rate, sound quality and other standards. By analyzing the detection results, the co-transfer server can identify which monitoring terminals have normal audio and video data and which have problems. For monitoring terminals with normal audio and video data, the co-transfer server believes that its state is stable and frequent detection can be reduced. If the detection result shows that the audio and video data of the monitoring terminal is normal, the co-transfer server will move the monitoring terminal to the end of the terminal array list. The terminal array list is a collection of information of all monitoring terminals in the monitoring system, usually sorted by detection priority. Moving normal monitoring terminals to the end can reduce their priority in the next round of detection, allowing more system resources to be allocated to monitoring terminals that need to be detected and processed in detail.

[0063] For example: suppose there are 100 monitoring terminals in a monitoring system. The cooperative server starts the detection thread to detect the audio and video data of these 100 monitoring terminals. After the detection is completed, the cooperative server analyzes the detection results and finds that the data of 80 monitoring terminals are normal, and the data of 20 monitoring terminals have problems. For these 80 monitoring terminals with normal data, the cooperative server will move them to the end of the terminal array list. In this way, in the next round of detection, the cooperative server can give priority to the 20 monitoring terminals with problems to ensure that their problems are solved in time. For those monitoring terminals with normal data, since their status is stable, the detection frequency can be reduced, thereby optimizing the utilization of system resources.

[0064] By moving normal monitoring terminals to the end of the list, the embodiments of the present invention can allocate more resources to monitoring terminals that need to be detected and processed, thereby optimizing resource utilization. The frequent detection of monitoring terminals in normal status is reduced, the system load is reduced, and the overall detection efficiency is improved. Monitoring terminals with problems are processed first to ensure that their problems are solved in a timely manner, thereby enhancing the stability and reliability of the system. Monitoring terminals with stable status are detected less frequently to avoid redundant operations and improve system performance. Through efficient detection and problem handling, the normal operation of the monitoring system and data quality are ensured, and the user experience is improved.

[0065] Based on the above description of an embodiment of a method for processing audio and video data, a patrol scheme for a large number of surveillance videos is introduced below. Figure 2 , shows a schematic diagram of the principle of a large number of monitoring video inspection solutions of an embodiment of the present invention. The large number of monitoring video inspection solutions involve a coordination server, a streaming media server, a monitoring terminal, a mobile terminal and a visual networking terminal.

[0066] If this coordination server is responsible for 100 monitoring terminals, the coordination server will open ten detection threads. One detection thread is responsible for detecting one monitoring terminal at a time, checking whether the video stream of the monitoring terminal is normal, and analyzing the video packets in real time.

[0067] If no video packet is received, it means that the monitoring terminal has not sent the video stream to the coordination server. When the video packet cannot be parsed, the coordination server sends an instruction to the monitoring terminal to reconnect the real-time communication (RTC) and resend the packet. If there is still no video stream, the device detection signal is sent to the monitoring terminal to check whether the camera of the monitoring terminal is available. If the camera is unavailable, the coordination server marks the monitoring camera as unavailable in the monitoring list and displays it on the large screen. It issues a warning for manual participation to check the hardware equipment of the monitoring terminal, thereby solving the black screen problem at the source.

[0068] If a video packet is received, check whether the video packet is normal. The frame rate is the number of frames received in one second. If a key frame is not received within one second, all frames before the key frame is received again are discarded, which means an incomplete picture group is discarded. If a picture group lacks a key frame, the forward predictive coding image frame (i.e., P frame) and the bidirectional predictive coding image frame (i.e., B frame) cannot be decoded, resulting in a distorted screen or a green screen, and the next normal picture group is sent.

[0069] If the monitoring terminal responds slowly, for example, if the monitoring terminal does not respond to the signal sent by the coordination server to the monitoring terminal for more than 60 seconds or responds after more than 90 seconds, the coordination server will mark the monitoring terminal as having a weak signal and display the monitoring terminal responding slowly on the large screen. A warning will be issued to let personnel check the network signal of the monitoring terminal.

[0070] If everything is normal for this monitoring terminal or it is normal after processing, put this monitoring terminal at the end of the remaining monitoring set that has not been inspected, and take the first monitoring terminal from the remaining monitoring set for video detection. The operations of putting the monitoring terminal at the end of the set and taking the first monitoring terminal are performed in one detection thread to avoid errors caused by other detection threads performing the same operation at the same time. Each detection thread performs repeated operations.

[0071] At the same time, when viewing the monitoring terminal on the mobile terminal and the visual network terminal, personnel are allowed to manually report the current monitoring screen, black screen, mosaic, etc. This report sends the corresponding signaling to the streaming server through the client. For example, a signaling is sent, and the signaling content is a black screen. The streaming server then sends this signaling to the coordination server. The coordination server receives the black screen signaling, finds a currently idle detection thread, and takes out this monitoring terminal from the undetected array list for detection. If a video stream is detected, it sends a signaling to the streaming server to tell the streaming server that the video source of this monitoring terminal is fine, and asks the streaming server to check whether the video stream of this monitoring terminal is returned to the client normally. If the coordination server detects that there is no video stream, it sends this signaling to the corresponding monitoring terminal. The monitoring terminal receives this signaling and reconnects to the RTC to resend the packet. If there is still no video stream, it sends a device detection signaling to the monitoring terminal to check whether the camera of the monitoring terminal is available. If the camera is unavailable, the coordination server marks this monitoring camera as unavailable in the monitoring list and displays this monitoring camera as unavailable on the large screen. Issue a warning to allow manual participation in checking the hardware equipment of this monitoring terminal. Manual reporting of distorted screen, green screen and mosaic is done by sending signaling to the streaming media server through the client, and the streaming media server forwards it to the coordination server. When the coordination server receives the signaling of distorted screen, green screen and mosaic, it finds a currently idle detection thread to take out the monitoring terminal from the array list that has not been detected for video stream detection. If a key frame is not received within one second, all frames before the key frame is received again are discarded, that is, an incomplete picture group is discarded. If a picture group lacks a key frame, the forward predictive coded image frame (ie, P frame) and the bidirectional predictive coded image frame (ie, B frame) cannot be decoded, resulting in distorted screen and green screen, and the next normal picture group is sent.

[0072] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0073] Reference Figure 3 , shows a structural block diagram of an audio and video data processing system according to an embodiment of the present invention. The audio and video data processing system can be applied to a server, and the audio and video data processing system can specifically include the following modules.

[0074] The audio and video data detection module 31 is used to detect the audio and video data of the monitoring terminal;

[0075] A frame rate information acquisition module 32, used to acquire the frame rate information of the audio and video data;

[0076] The picture group discarding module 33 is used to discard at least one incomplete picture group of the audio and video data if the frame rate information does not meet the frame rate condition.

[0077] In an exemplary embodiment of the present invention, the picture group discarding module 33 is configured to discard the incomplete picture group before the next received key frame if the frame rate information satisfies that no key frame is received within a preset period.

[0078] In an exemplary embodiment of the present invention, the system further comprises:

[0079] A packet sending instruction sending module, used for sending a packet sending instruction to the monitoring terminal if the audio and video data is not detected;

[0080] A detection signaling sending module, used for sending a device detection signaling to the monitoring terminal if the audio and video data is still not detected, so as to determine whether the camera of the monitoring terminal is normal;

[0081] The status marking module is used to mark the camera working status of the monitoring terminal as an abnormal state in the monitoring list if the camera is abnormal, and mark the camera working status as an abnormal state on the display screen.

[0082] In an exemplary embodiment of the present invention, the status marking module is also used to, after the detection signaling sending module sends a device detection signaling to the monitoring terminal, if no response information is received from the monitoring terminal within a first time period, or if a response information is received from the monitoring terminal after a second time period, mark the signal status of the monitoring terminal as a weak state in the monitoring list, and mark the response status of the monitoring terminal as a slow state on the display screen.

[0083] In an exemplary embodiment of the present invention, the system further comprises:

[0084] A fault signaling acquisition module is used to acquire a fault signaling from a streaming media server, wherein the fault signaling is uploaded by a client to the streaming media server, and the fault signaling indicates that a fault occurs in the monitoring terminal;

[0085] A monitoring terminal extraction module, used to extract a target monitoring terminal from a terminal array list according to the fault signaling;

[0086] A video stream detection module, used to detect whether there is a video stream on the target monitoring terminal;

[0087] A notification information sending module, configured to send video stream notification information to the streaming media server if the target monitoring terminal has the video stream, so that the streaming media server detects whether the video stream is transmitted to the client according to the video stream notification information;

[0088] The fault signaling forwarding module is used to forward the fault signaling to the target monitoring terminal if the target monitoring terminal does not have the video stream, so that the target monitoring terminal resends the video stream according to the fault signaling.

[0089] In an exemplary embodiment of the present invention, the system further comprises:

[0090] A terminal quantity acquisition module, used to acquire the number of monitoring terminals before the audio and video data detection module 31 detects the audio and video data of the monitoring terminals;

[0091] A detection thread starting module, used to start at least one detection thread according to the number;

[0092] An audio and video data detection module is used to detect the audio and video data using the detection thread.

[0093] In an exemplary embodiment of the present invention, the system further comprises:

[0094] The monitoring terminal moving module is used to move the monitoring terminal to the end of the terminal array list if the audio and video data of the monitoring terminal is normal after the audio and video data detection module detects the audio and video data using the detection thread.

[0095] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0096] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0097] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0098] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0101] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0102] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0103] The above is a detailed introduction to a method for processing audio and video data and a system for processing audio and video data provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing audio and video data, characterized in that: Applied to a server, the method comprises: The audio and video data of the monitoring terminal is detected; Obtaining frame rate information of the audio and video data; If the frame rate information does not satisfy the frame rate condition, at least one incomplete picture group of the audio and video data is discarded.

2. The method according to claim 1, characterized in that If the frame rate information does not meet the frame rate condition, discarding at least one incomplete picture group of the audio and video data includes: If the frame rate information satisfies that no key frame is received within a preset period, the incomplete picture group before the next received key frame is discarded.

3. The method according to claim 1, characterized in that The method further comprises: If the audio and video data is not detected, a packet sending instruction is sent to the monitoring terminal; If the audio and video data is still not detected, sending a device detection signaling to the monitoring terminal to determine whether the camera of the monitoring terminal is normal; If the camera is abnormal, the camera working state of the monitoring terminal is marked as an abnormal state in the monitoring list, and the camera working state is marked as an abnormal state on the display screen.

4. The method according to claim 3, characterized in that After sending the device detection signaling to the monitoring terminal, the method further includes: If no response information is received from the monitoring terminal within the first time period, or if a response information is received from the monitoring terminal after the second time period, the signal state of the monitoring terminal is marked as a weak state in the monitoring list, and the response state of the monitoring terminal is marked as a slow state on the display screen.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a fault signaling from a streaming media server, wherein the fault signaling is uploaded by a client to the streaming media server, and the fault signaling indicates that a fault occurs in the monitoring terminal; Extracting a target monitoring terminal from a terminal array list according to the fault signaling; Detecting whether a video stream exists at the target monitoring terminal; If the target monitoring terminal has the video stream, sending video stream notification information to the streaming media server, so that the streaming media server detects whether the video stream is transmitted to the client according to the video stream notification information; If the target monitoring terminal does not have the video stream, the fault signaling is forwarded to the target monitoring terminal, so that the target monitoring terminal resends the video stream according to the fault signaling.

6. The method according to claim 5, characterized in that Before the audio and video data of the monitoring terminal is detected, the method further includes: Obtaining the number of monitoring terminals; Starting at least one detection thread according to the number; The audio and video data is detected using the detection thread.

7. The method according to claim 6, characterized in that After detecting the audio and video data using the detection thread, the method further includes: If the audio and video data of the monitoring terminal is normal, the monitoring terminal is moved to the end of the terminal array list.

8. A system for processing audio and video data, characterized in that: Applied to a server, the system comprises: An audio and video data detection module is used to detect the audio and video data of the monitoring terminal; A frame rate information acquisition module, used to acquire the frame rate information of the audio and video data; The picture group discarding module is used to discard at least one incomplete picture group of the audio and video data if the frame rate information does not meet the frame rate condition.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the method for processing audio and video data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the method for processing audio and video data as described in any one of claims 1 to 7.