Video lag detection method and system, electronic equipment and readable storage medium

Abstract

The invention provides a video lag detection method and system, electronic equipment and a readable storage medium, and relates to the technical field of coal mining, and the method comprises the steps: obtaining a video; whether the video can be played is determined, if yes, the video is analyzed and converted into an image sequence, and if not, fault information is sent; determining whether two adjacent frames of images in the image sequence are continuous images, if yes, determining whether the video is lagged according to the time difference value and a set threshold value, and if not, determining that the video is lagged; if the time difference value is greater than a set threshold value, determining that the video is lagged; if the time difference value is smaller than or equal to a set threshold value, determining that the video is not lagged; and under the condition that the video is determined to be lagged, adjusting video parameters, and sending fault information. According to the technical scheme of the invention, the video parameters are adaptively adjusted to reduce or improve the jamming phenomenon, and the fault information is sent in time, so that the normal work of monitoring personnel is ensured, the potential safety hazard is reduced, and the production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mining, and in particular to a video freeze detection method, system, electronic device and readable storage medium. Background Art

[0002] With the continuous improvement of coal mining technology and intelligent level, fully mechanized working faces have achieved high mechanization, automation and informationization. On fully mechanized working faces, the video monitoring system realizes real-time monitoring of fully mechanized working faces by installing cameras. These cameras can capture image information of fully mechanized working faces and transmit this information to the monitoring center. Monitoring personnel can understand the production situation of fully mechanized working faces by observing these image information, and timely discover and deal with abnormal situations.

[0003] However, due to the complex environment of the fully mechanized mining face, the large number of equipment, and the influence of network transmission, the video monitoring system may experience a jamming phenomenon during the transmission process. This jamming phenomenon not only affects the normal work of the monitoring personnel, but also poses a threat to production safety. Therefore, it is of great significance to develop a video jamming detection method based on the fully mechanized mining face. Summary of the invention

[0004] In order to solve or improve the technical problem that a jamming phenomenon may occur during video transmission in a fully-mechanized mining face, an object of the present invention is to provide a video jamming detection method based on a fully-mechanized mining face.

[0005] Another object of the present invention is to provide a video freeze detection system based on a fully mechanized mining working face.

[0006] Another object of the present invention is to provide an electronic device.

[0007] Another object of the present invention is to provide a computer-readable storage medium.

[0008] To achieve the above object, the first aspect of the present invention provides a video freeze detection method based on a fully mechanized mining working face, and the specific steps of the video freeze detection method include:

[0009] The first step is to obtain a video about the comprehensive mining working face.

[0010] The second step is to determine whether the video is playable. If the video is playable, the video is parsed and converted into an image sequence consisting of multiple frames. If the video is not playable, a fault message is sent.

[0011] The third step is to determine whether two adjacent frames in the image sequence are continuous images. If the two adjacent frames are continuous images, determine whether the video is stuck based on the time difference between the two adjacent frames and the set threshold; if the two adjacent frames are not continuous images, determine that the video is stuck.

[0012] In the fourth step, when the time difference is greater than the set threshold, it is determined that the video is stuck; when the time difference is less than or equal to the set threshold, it is determined that the video is not stuck.

[0013] Step 5: If the video is stuck, adjust the video parameters and send a fault message.

[0014] The present invention aims to provide a video freeze detection method based on a comprehensive mining face, which is used to monitor the video playback quality in real time under the complex environment of the comprehensive mining face, and adaptively adjust the video parameters when the video freezes, so as to reduce or improve the freeze phenomenon. When the video cannot be played, or the video still freezes after adjusting the video parameters, a fault message is sent to remind the on-site operation and maintenance personnel to deal with it in time. By adaptively adjusting the video parameters to reduce or improve the freeze phenomenon, and sending fault information in time, this method can ensure the normal work of the monitoring personnel and reduce safety hazards.

[0015] In addition, the above technical solution provided by the present invention may also have the following additional technical features:

[0016] In some technical solutions, optionally, determining whether a video is playable, if the video is playable, parsing the video and converting it into an image sequence consisting of multiple frames of images; if the video is not playable, sending fault information, the steps include:

[0017] Determine whether the video is playable and generate a first determination result.

[0018] If the first judgment result is yes, the video is parsed and converted into an image sequence consisting of multiple frames of images.

[0019] If the first judgment result is no, a fault message is sent.

[0020] In this technical solution, the purpose of this step is to verify whether the video about the fully mechanized mining face can be played. If the video can be played, the video detection process will be entered; if the video cannot be played, a fault message will be sent to remind the maintenance personnel to handle it in time.

[0021] Optionally, the video reading module is used to determine whether the video is playable and generate a first judgment result. If the first judgment result is yes (the video is playable), the video stream processing module is used to parse the video and convert it into an image sequence consisting of multiple frames of images. If the first judgment result is no (the video is not playable), the fault information sending module is used to send fault information.

[0022] In some technical solutions, optionally, the fault information includes fault location information and fault time information.

[0023] In this technical solution, maintenance personnel can determine the location of the fault and the time when the fault occurred based on the fault location information and fault time information, which is conducive to finding and repairing the fault in time, ensuring production safety and improving production efficiency.

[0024] In some technical solutions, optionally, determining whether two adjacent frames of images in an image sequence are continuous images, and if the two adjacent frames of images are continuous images, determining whether the video is stuck according to a time difference between the two adjacent frames of images and a set threshold; and determining that the video is stuck if the two adjacent frames of images are not continuous images, the steps include:

[0025] It is determined whether two adjacent frames of images in the image sequence are continuous images, and a second determination result is generated.

[0026] If the second judgment result is yes, whether the video freezes is determined based on the time difference between two adjacent frames of images and a set threshold.

[0027] If the second judgment result is no, it is determined that the video is stuck.

[0028] In this technical solution, a difference comparison is performed through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames of images in an image sequence are continuous images. The two adjacent frames of images are the current frame image and the previous frame image.

[0029] If the second judgment result is yes (two adjacent frame images are continuous images), the time difference between the current frame image and the previous frame image is calculated to determine whether the time difference is greater than a set threshold.

[0030] If the second judgment result is no (the two adjacent frames of images are not continuous images), there is no need to further judge whether the time difference is greater than the set threshold, and it can be directly determined that the video is stuck.

[0031] In some technical solutions, optionally, determining whether a video freeze occurs according to a time difference between two adjacent frames of images and a set threshold comprises:

[0032] It is determined whether the time difference between two adjacent frames of images is greater than a set threshold, and a third determination result is generated.

[0033] In this technical solution, the time difference between the current frame image and the previous frame image is calculated to determine whether the time difference is greater than a set threshold.

[0034] In some technical solutions, optionally, determining whether the video is stuck based on the time difference between two adjacent frames of images and a set threshold, the step further includes:

[0035] Before determining whether the time difference between two adjacent frames of images is greater than a set threshold and generating a third determination result, grayscale processing is performed on the image sequence.

[0036] In this technical solution, grayscale processing is a technique for converting a color image into a grayscale image. A grayscale image is an image in which each pixel has only one brightness value, and 8 bits (0-255) are usually used to represent different grayscale levels. This method can simplify image data while retaining basic information of the image, such as shape information and edge information, and is often used to simplify image processing, analysis, and display.

[0037] By performing grayscale processing on the image sequence, it is convenient to determine whether two adjacent frames of images in the image sequence are continuous images in the subsequent steps.

[0038] In some technical solutions, optionally, when the time difference is greater than a set threshold, it is determined that the video is stuck; when the time difference is less than or equal to the set threshold, it is determined that the video is not stuck, and the steps include:

[0039] If the third judgment result is yes, it is determined that the video is stuck.

[0040] If the third judgment result is no, it is determined that the video is not stuck.

[0041] In this technical solution, when two adjacent frames of images are continuous images, the time difference between the current frame image and the previous frame image is calculated to determine whether the time difference is greater than a set threshold. If so, it is determined that the video is stuck; otherwise, it is determined that the video is not stuck.

[0042] In some technical solutions, optionally, when it is determined that the video is stuck, adjusting video parameters and sending fault information include:

[0043] If the video is stuck, adjust the video parameters.

[0044] If the video is still stuck after adjusting the video parameters, a fault message is sent.

[0045] In this technical solution, when it is determined that the video is stuck (the video is stuck because two adjacent frames are not continuous images or the time difference is greater than a set threshold), the video parameters are automatically or adaptively adjusted to improve the stuck phenomenon. The video parameters include but are not limited to video encoding parameters and network transmission setting parameters.

[0046] It should be noted that the number of times the video parameters are adjusted does not exceed the expected configured number to reduce the buffer pressure of data processing.

[0047] If the video still freezes after the video parameters are adaptively adjusted, the fault information and corresponding troubleshooting experience will be sent to the maintenance personnel.

[0048] Optionally, the adaptive adjustment module is used to adjust video parameters when it is determined that the video is stuck. The fault information sending module is used to send fault information when it is determined that the video is stuck.

[0049] A second aspect of the present invention provides a video freeze detection system based on a comprehensive mining working face, comprising a video reading module, a video stream processing module, a video detection module, an adaptive adjustment module and a fault information sending module.

[0050] Among them, the video reading module is used to obtain the video about the comprehensive mining working face and determine whether the video can be played.

[0051] The video stream processing module is used to parse the video when the video is playable, convert it into an image sequence consisting of multiple frames of images, and determine whether two adjacent frames of images in the image sequence are continuous images.

[0052] The video detection module is used to determine whether the video is stuck based on the time difference between the two adjacent frames and a set threshold when the two adjacent frames are continuous images; determine that the video is stuck when the two adjacent frames are not continuous images; determine that the video is stuck when the time difference is greater than the set threshold; and determine that the video is not stuck when the time difference is less than or equal to the set threshold.

[0053] The adaptive adjustment module is used to adjust video parameters when it is determined that the video is stuck.

[0054] The fault information sending module is used to send fault information when the video cannot be played, and is used to send fault information when it is determined that the video is stuck.

[0055] The present invention aims to provide a video freeze detection system based on a comprehensive mining face, which is used to monitor the video playback quality in real time in the complex environment of the comprehensive mining face, and adaptively adjust the video parameters when the video freezes, so as to reduce or improve the freeze phenomenon. When the video cannot be played, or the video still freezes after adjusting the video parameters, a fault message is sent to remind the on-site operation and maintenance personnel to deal with it in time. By adaptively adjusting the video parameters to reduce or improve the freeze phenomenon, and sending fault information in time, this method can ensure the normal work of the monitoring personnel and reduce safety hazards.

[0056] The third aspect of the present invention provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the video freeze detection method based on the comprehensive mining working face in any of the above-mentioned technical solutions are implemented.

[0057] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the video freeze detection method based on the comprehensive mining working face in any of the above-mentioned technical solutions are implemented.

[0058] Additional aspects and advantages of the technical solutions of the present invention will become apparent in the following description or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flow chart of a method for detecting video freeze based on a fully mechanized mining face according to an embodiment of the present invention is shown;

[0060] Figure 2 A flow chart of a method for detecting video freeze based on a fully mechanized mining face according to another embodiment of the present invention is shown;

[0061] Figure 3 A flow chart of a method for detecting video freeze based on a fully mechanized mining face according to another embodiment of the present invention is shown;

[0062] Figure 4 A flow chart of a method for detecting video freeze based on a fully mechanized mining face according to another embodiment of the present invention is shown;

[0063] Figure 5 A flow chart of a method for detecting video freeze based on a fully mechanized mining face according to another embodiment of the present invention is shown;

[0064] Figure 6 A flow chart of a method for detecting video freeze based on a fully mechanized mining face according to another embodiment of the present invention is shown;

[0065] Figure 7 A structural block diagram of a video freeze detection system based on a fully mechanized mining working face according to an embodiment of the present invention is shown;

[0066] Figure 8 A structural block diagram of an electronic device according to an embodiment of the present invention is shown.

[0067] in, Figure 7 and Figure 8 The corresponding relationship between the reference numerals and the component names is as follows:

[0068] 200: Video freeze detection system based on comprehensive mining working face; 210: Video reading module; 220: Video stream processing module; 230: Video detection module; 240: Adaptive adjustment module; 250: Fault information sending module; 300: Electronic device; 310: Memory; 320: Processor. DETAILED DESCRIPTION

[0069] In order to more clearly understand the above-mentioned purposes, features and advantages of the embodiments of the present invention, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0070] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the embodiments of the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0071] Refer to the following Figures 1 to 8 A video freeze detection method based on a fully mechanized mining working face, a video freeze detection system based on a fully mechanized mining working face, an electronic device and a computer-readable storage medium are described according to some embodiments of the present invention.

[0072] In one embodiment according to the present invention, Figure 1 As shown, the specific steps of the video jam detection method based on the fully mechanized mining working face include:

[0073] S102, obtaining a video of the fully mechanized mining face.

[0074] The video stream is read in real time through the Emgu CV library (a class library for video stream reading). The video stream is a type of media data synchronously transmitted in a streaming media manner on the Internet.

[0075] Optionally, the video reading module is used to read the video stream in real time through the Emgu CV library.

[0076] S104, determining whether the video is playable, if the video is playable, parsing the video and converting it into an image sequence consisting of multiple frames; if the video is not playable, sending a fault message.

[0077] The purpose of this step is to verify whether the video about the fully mechanized mining face can be played. If the video can be played, the video detection process will be entered; if the video cannot be played, a fault message will be sent to remind the maintenance personnel to handle it in time.

[0078] The specific steps of the video detection process include: parsing the video and converting it into an image sequence consisting of multiple frames; performing grayscale processing on the image sequence; and performing difference comparison through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames in the image sequence are continuous images.

[0079] Optionally, the video reading module is used to determine whether the video is playable. If the video is playable, the video stream processing module is used to perform video detection on the playable video (entering the video detection process). If the video is not playable, the fault information sending module is used to send fault information.

[0080] S106, determining whether two adjacent frames of images in the image sequence are continuous images. If the two adjacent frames of images are continuous images, determining whether the video is stuck based on the time difference between the two adjacent frames of images and a set threshold; if the two adjacent frames of images are not continuous images, determining that the video is stuck.

[0081] The difference comparison is performed through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames of images in the image sequence are continuous images. The two adjacent frames of images are the current frame image and the previous frame image.

[0082] When two adjacent frames are continuous images, the time difference between the current frame and the previous frame is calculated to determine whether the time difference is greater than a set threshold. If so, it is determined that the video is stuck; otherwise, it is determined that the video is not stuck.

[0083] In the case where two adjacent frames of images are not continuous images, it is not necessary to further determine whether the time difference is greater than a set threshold, and it can be directly determined that the video is stuck.

[0084] Optionally, the video detection module is used to determine whether the video is stuck based on the time difference between the two adjacent frames and a set threshold when the two adjacent frames are continuous images; and to determine whether the video is stuck when the two adjacent frames are not continuous images.

[0085] S108, when the time difference is greater than a set threshold, it is determined that the video is stuck; when the time difference is less than or equal to the set threshold, it is determined that the video is not stuck.

[0086] When two adjacent frames are continuous images, the time difference between the current frame and the previous frame is calculated to determine whether the time difference is greater than a set threshold. If so, it is determined that the video is stuck; otherwise, it is determined that the video is not stuck.

[0087] Optionally, the video detection module is used to determine that the video is stuck when the time difference is greater than a set threshold; and to determine that the video is not stuck when the time difference is less than or equal to the set threshold.

[0088] S110, when it is determined that the video is stuck, adjust video parameters and send fault information.

[0089] When it is determined that the video is stuck (the video is stuck because two adjacent frames are not continuous or the time difference is greater than a set threshold), the video parameters are automatically or adaptively adjusted to improve the stuck phenomenon. The video parameters include but are not limited to video encoding parameters and network transmission setting parameters.

[0090] It should be noted that the number of times the video parameters are adjusted does not exceed the expected configured number to reduce the buffer pressure of data processing.

[0091] If the video still freezes after the video parameters are adaptively adjusted, the fault information and corresponding troubleshooting experience will be sent to the maintenance personnel.

[0092] Optionally, the adaptive adjustment module is used to adjust video parameters when it is determined that the video is stuck. The fault information sending module is used to send fault information when it is determined that the video is stuck.

[0093] The present invention aims to provide a video freeze detection method based on a comprehensive mining face, which is used to monitor the video playback quality in real time under the complex environment of the comprehensive mining face, and adaptively adjust the video parameters when the video freezes, so as to reduce or improve the freeze phenomenon. When the video cannot be played, or the video still freezes after adjusting the video parameters, a fault message is sent to remind the on-site operation and maintenance personnel to deal with it in time. By adaptively adjusting the video parameters to reduce or improve the freeze phenomenon, and sending fault information in time, this method can ensure the normal work of the monitoring personnel and reduce safety hazards.

[0094] It should be emphasized that the method provided by the present invention (the method for detecting video freezes based on the comprehensive mining working face) determines whether the video freezes by whether the time difference between two adjacent frames of images (the current frame image and the previous frame image) is greater than a set threshold. This judgment method can be applied to the complex environment of the comprehensive mining working face and has higher accuracy. The method provided by the present invention has an adaptive adjustment function, and can automatically or adaptively adjust the video parameters to reduce or improve the freeze phenomenon when it is determined that the video freezes. In addition, the method provided by the present invention has a feedback mechanism, which promptly feeds back the detected freeze situation and troubleshooting experience to the maintenance personnel, which is conducive to timely finding and repairing faults, ensuring production safety, and improving production efficiency.

[0095] In some embodiments, optionally, Figure 2 As shown, S104 (determine whether the video is playable, if the video is playable, parse the video and convert it into an image sequence consisting of multiple frames; if the video is not playable, send fault information), the specific steps include:

[0096] S1042, determine whether the video is playable, and generate a first determination result.

[0097] The purpose of this step is to verify whether the video about the fully mechanized mining face can be played.

[0098] S1044: If the first judgment result is yes, the video is parsed and converted into an image sequence consisting of multiple frames of images.

[0099] If the first judgment result is yes (the video can be played), the video detection process is entered.

[0100] S1046: If the first judgment result is no, send fault information.

[0101] If the first judgment result is no (the video cannot be played), a fault message is sent to remind maintenance personnel to handle it in time.

[0102] Optionally, the video reading module is used to determine whether the video is playable and generate a first judgment result. If the first judgment result is yes (the video is playable), the video stream processing module is used to parse the video and convert it into an image sequence consisting of multiple frames of images. If the first judgment result is no (the video is not playable), the fault information sending module is used to send fault information.

[0103] In some embodiments, optionally, the fault information includes fault location information and fault time information.

[0104] Maintenance personnel can determine the location of the fault and the time when the fault occurred based on the fault location information and fault time information, which is conducive to finding and repairing the fault in time, ensuring production safety and improving production efficiency.

[0105] In some embodiments, optionally, Figure 3 As shown, S106 (determining whether two adjacent frames of images in an image sequence are continuous images, and if the two adjacent frames of images are continuous images, determining whether the video is stuck according to the time difference between the two adjacent frames of images and a set threshold; if the two adjacent frames of images are not continuous images, determining that the video is stuck), the steps include:

[0106] S1062, determining whether two adjacent frames of images in the image sequence are continuous images, and generating a second determination result.

[0107] The difference comparison is performed through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames of images in the image sequence are continuous images. The two adjacent frames of images are the current frame image and the previous frame image.

[0108] S1064: If the second judgment result is yes, determine whether the video is stuck based on the time difference between two adjacent frames of images and a set threshold.

[0109] If the second judgment result is yes (two adjacent frame images are continuous images), the time difference between the current frame image and the previous frame image is calculated to determine whether the time difference is greater than a set threshold.

[0110] S1066: If the second judgment result is no, it is determined that the video is stuck.

[0111] If the second judgment result is no (the two adjacent frames of images are not continuous images), there is no need to further judge whether the time difference is greater than the set threshold, and it can be directly determined that the video is stuck.

[0112] In some embodiments, optionally, Figure 4 As shown, determining whether a video freeze occurs according to the time difference between two adjacent frames of images and a set threshold comprises the following steps:

[0113] S1071, grayscale processing is performed on the image sequence.

[0114] Grayscale processing is a technique for converting color images into grayscale images. A grayscale image is an image in which each pixel has only one brightness value, and 8 bits (0-255) are usually used to represent different grayscale levels. This method can simplify image data while retaining basic information of the image, such as shape information and edge information, and is often used to simplify image processing, analysis, and display.

[0115] By performing grayscale processing on the image sequence, it is convenient to determine whether two adjacent frames of images in the image sequence are continuous images in the subsequent steps.

[0116] S1072, determine whether the time difference between two adjacent frames of images is greater than a set threshold, and generate a third determination result.

[0117] Calculate the time difference between the current frame image and the previous frame image, and determine whether the time difference is greater than a set threshold.

[0118] In some embodiments, optionally, Figure 4 As shown, S108 (when the time difference is greater than the set threshold, it is determined that the video is stuck; when the time difference is less than or equal to the set threshold, it is determined that the video is not stuck), the steps include:

[0119] S1082: If the third judgment result is yes, it is determined that the video is stuck.

[0120] S1084: If the third judgment result is no, it is determined that the video is not stuck.

[0121] When two adjacent frames are continuous images, the time difference between the current frame and the previous frame is calculated to determine whether the time difference is greater than a set threshold. If so, it is determined that the video is stuck; otherwise, it is determined that the video is not stuck.

[0122] In some embodiments, optionally, Figure 5 As shown, S110 (adjusting video parameters and sending fault information when it is determined that the video is stuck) includes the following steps:

[0123] S1102: When it is determined that the video is stuck, adjust video parameters.

[0124] S1104: If the video is still stuck after adjusting the video parameters, send a fault message.

[0125] When it is determined that the video is stuck (the video is stuck because two adjacent frames are not continuous or the time difference is greater than a set threshold), the video parameters are automatically or adaptively adjusted to improve the stuck phenomenon. The video parameters include but are not limited to video encoding parameters and network transmission setting parameters.

[0126] It should be noted that the number of times the video parameters are adjusted does not exceed the expected configured number to reduce the buffer pressure of data processing.

[0127] If the video still freezes after the video parameters are adaptively adjusted, the fault information and corresponding troubleshooting experience will be sent to the maintenance personnel.

[0128] Optionally, the adaptive adjustment module is used to adjust video parameters when it is determined that the video is stuck. The fault information sending module is used to send fault information when it is determined that the video is stuck.

[0129] In some embodiments, optionally, Figure 6 As shown, the specific steps of the video jam detection method based on the fully mechanized mining working face include:

[0130] S121, obtaining a camera video stream.

[0131] On the fully mechanized mining face, the video monitoring system realizes real-time monitoring of the fully mechanized mining face by installing cameras. The purpose of this step is to obtain video about the fully mechanized mining face.

[0132] The video stream is read in real time through the Emgu CV library (a class library for video stream reading). The video stream is a type of media data synchronously transmitted in a streaming media manner on the Internet.

[0133] S122, determine whether the video is playable, and generate a fourth determination result.

[0134] The purpose of this step is to verify whether the video about the fully mechanized mining face can be played.

[0135] S123: If the fourth judgment result is yes, process each frame of the image.

[0136] When the video is playable, the video is parsed and converted into an image sequence consisting of multiple frames.

[0137] S124: If the fourth judgment result is no, a fault message is sent to prompt maintenance personnel.

[0138] In the event that the video cannot be played, a fault message is sent to alert maintenance personnel to handle it in a timely manner.

[0139] S125, comparing the current frame image with the previous frame image.

[0140] The difference comparison is performed through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames of images in the image sequence are continuous images. The two adjacent frames of images are the current frame image and the previous frame image.

[0141] S126, determining whether two adjacent frame images are continuous frame images, and generating a fifth determination result.

[0142] It should be noted that the continuous frame images here are the continuous images in any of the above embodiments.

[0143] S127: If the fifth judgment result is yes, obtain the timestamp of the image.

[0144] It should be noted that the timestamp can be understood as a time node. When two adjacent frames of images are continuous frames of images, the time node of each frame of image is obtained.

[0145] S128: If the fifth judgment result is no, adaptively adjust the video parameters.

[0146] When two adjacent frames are not continuous frames, video parameters are automatically or adaptively adjusted to improve the stuttering phenomenon, wherein the video parameters include but are not limited to video encoding parameters and network transmission setting parameters.

[0147] It should be noted that the number of times the video parameters are adjusted does not exceed the expected configured number to reduce the buffer pressure of data processing.

[0148] S129, dynamically calculating the time difference between two adjacent image frames.

[0149] Determine the time difference based on the time node.

[0150] S130, determining whether the time difference exceeds a set threshold, and generating a sixth determination result.

[0151] When two adjacent frames of images are continuous images, the time difference between the current frame of image and the previous frame of image is calculated to determine whether the time difference is greater than a set threshold.

[0152] S131: If the sixth judgment result is yes, adaptively adjust the video parameters.

[0153] When it is determined that the video is stuck (the video is stuck because two adjacent frames are not continuous images or the time difference is greater than a set threshold), the video parameters are automatically or adaptively adjusted to improve the stuck phenomenon.

[0154] It should be noted that if the sixth judgment result is no, the process returns to S123 (processing each frame of image).

[0155] S132, sending fault information to prompt maintenance personnel.

[0156] If the video still freezes after the video parameters are adaptively adjusted, the fault information and corresponding troubleshooting experience will be sent to the maintenance personnel.

[0157] In one embodiment according to the present invention, Figure 7As shown, the video freeze detection system 200 based on the comprehensive mining working face includes a video reading module 210, a video stream processing module 220, a video detection module 230, an adaptive adjustment module 240 and a fault information sending module 250.

[0158] Among them, the video reading module 210 is used to obtain the video about the comprehensive mining working face and determine whether the video can be played.

[0159] The video reading module 210 is used to read the video stream in real time through the Emgu CV library (a class library for video stream reading). The video stream is a kind of media data synchronously transmitted in the form of streaming media on the Internet.

[0160] After acquiring the video, the video reading module 210 is used to determine whether the video is playable.

[0161] The video stream processing module 220 is used to parse the video and convert it into an image sequence consisting of multiple frames of images when the video is playable, and to determine whether two adjacent frames of images in the image sequence are continuous images.

[0162] If the video is playable, the video stream processing module 220 is used to perform video detection on the playable video (entering the video detection process). If the video is not playable, the fault information sending module 250 is used to send fault information.

[0163] The specific steps of the video detection process include: parsing the video and converting it into an image sequence consisting of multiple frames; performing grayscale processing on the image sequence; and performing difference comparison through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames in the image sequence are continuous images.

[0164] The video stream processing module 220 is used to perform a difference comparison through the "CVInvoke.AbsDiff" method (a function calculation method) to determine whether two adjacent frames of images in the image sequence are continuous images. The two adjacent frames of images are the current frame image and the previous frame image.

[0165] The video detection module 230 is used to determine whether the video is stuck based on the time difference between the two adjacent frames and a set threshold when the two adjacent frames are continuous images; to determine that the video is stuck when the two adjacent frames are not continuous images; to determine that the video is stuck when the time difference is greater than the set threshold; and to determine that the video is not stuck when the time difference is less than or equal to the set threshold.

[0166] When two adjacent frames are continuous images, the time difference between the current frame and the previous frame is calculated to determine whether the time difference is greater than a set threshold. If so, it is determined that the video is stuck; otherwise, it is determined that the video is not stuck.

[0167] In the case where two adjacent frames of images are not continuous images, it is not necessary to further determine whether the time difference is greater than a set threshold, and it can be directly determined that the video is stuck.

[0168] When two adjacent frames are continuous images, the time difference between the current frame and the previous frame is calculated to determine whether the time difference is greater than a set threshold. If so, it is determined that the video is stuck; otherwise, it is determined that the video is not stuck.

[0169] The adaptive adjustment module 240 is used to adjust video parameters when it is determined that the video is stuck.

[0170] When it is determined that the video is stuck (the video is stuck because two adjacent frames are not continuous images or the time difference is greater than a set threshold), the adaptive adjustment module 240 is used to automatically or adaptively adjust the video parameters to improve the stuck phenomenon. The video parameters include but are not limited to video encoding parameters and network transmission setting parameters.

[0171] It should be noted that the number of times the video parameters are adjusted does not exceed the expected configured number to reduce the buffer pressure of data processing.

[0172] The fault information sending module 250 is used to send fault information when the video cannot be played, and is used to send fault information when it is determined that the video is stuck.

[0173] By establishing a feedback mechanism, detected jams and troubleshooting experience can be promptly fed back to maintenance personnel, which helps to find and repair faults in a timely manner, ensure production safety, and improve production efficiency.

[0174] The present invention aims to provide a video freeze detection system 200 based on a comprehensive mining face, which is used to monitor the video playback quality in real time in the complex environment of the comprehensive mining face, and adaptively adjust the video parameters when the video freezes, so as to reduce or improve the freeze phenomenon. When the video cannot be played, or the video still freezes after adjusting the video parameters, a fault message is sent to remind the on-site operation and maintenance personnel to deal with it in time. By adaptively adjusting the video parameters to reduce or improve the freeze phenomenon, and sending fault information in time, this method can ensure the normal work of the monitoring personnel and reduce safety hazards.

[0175] In one embodiment according to the present invention, Figure 8As shown, the electronic device 300 includes a processor 320, a memory 310, and a program or instruction stored in the memory 310 and executable on the processor 320. When the program or instruction is executed by the processor 320, the steps of the video freeze detection method based on the comprehensive mining working face in any of the above-mentioned embodiments are implemented.

[0176] In one embodiment of the present invention, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video freeze detection method based on the comprehensive mining working face in any of the above embodiments are implemented.

[0177] It should be noted that the computer-readable storage medium may be a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, etc.

[0178] In the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance; the term "plurality" refers to two or more, unless otherwise clearly defined. The terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0179] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by terms such as “upper”, “lower”, “left”, “right”, “front” and “back” are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or unit referred to must have a specific direction, be constructed and operated in a specific orientation, and therefore, cannot be understood as a limitation on the present invention.

[0180] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0181] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video freeze detection method based on fully mechanized mining working face, characterized in that: The video freeze detection method comprises: Get a video about the fully mechanized mining face; Determine whether the video is playable, and if the video is playable, parse the video and convert it into an image sequence consisting of multiple frames of images; if the video is not playable, send a fault message; Determine whether two adjacent frames of images in the image sequence are continuous images, and if the two adjacent frames of images are continuous images, determine whether the video is stuck according to a time difference between the two adjacent frames of images and a set threshold; if the two adjacent frames of images are not continuous images, determine that the video is stuck; When the time difference is greater than the set threshold, it is determined that the video is stuck; when the time difference is less than or equal to the set threshold, it is determined that the video is not stuck; When it is determined that the video is stuck, video parameters are adjusted and the fault information is sent.

2. The video freeze detection method based on fully mechanized mining working face according to claim 1 is characterized in that: determining whether the video is playable, and if the video is playable, parsing the video and converting it into an image sequence consisting of multiple frames of images; In the event that the video cannot be played, a fault message is sent, including: Determine whether the video is playable and generate a first determination result; If the first judgment result is yes, parsing the video and converting it into the image sequence composed of multiple frames of images; If the first judgment result is no, the fault information is sent.

3. The video freeze detection method based on fully mechanized mining working face according to claim 1 is characterized in that: The fault information includes fault location information and fault time information.

4. The video freeze detection method based on a fully mechanized mining face according to any one of claims 1 to 3, characterized in that: determining whether two adjacent frames of images in the image sequence are continuous images, and if the two adjacent frames of images are continuous images, determining whether the video is stuck according to a time difference between the two adjacent frames of images and a set threshold; When the two adjacent frames of images are not the continuous images, determining that the video freezes, includes: Determining whether the two adjacent frames of images in the image sequence are the continuous images, and generating a second determination result; If the second judgment result is yes, determining whether the video is stuck according to the time difference between the two adjacent frames of images and the set threshold; If the second judgment result is no, it is determined that the video is stuck.

5. The video freeze detection method based on fully mechanized mining working face according to claim 4 is characterized in that: The determining whether the video is stuck according to the time difference between the two adjacent frames of images and the set threshold value includes: It is determined whether the time difference between the two adjacent frames of images is greater than the set threshold, and a third determination result is generated.

6. The video freeze detection method based on fully mechanized mining working face according to claim 5 is characterized in that: The determining whether the video is stuck according to the time difference between the two adjacent frames of images and the set threshold value further includes: Before determining whether the time difference between the two adjacent frames of images is greater than the set threshold and generating a third determination result, grayscale processing is performed on the image sequence.

7. The video freeze detection method based on fully mechanized mining working face according to claim 5 is characterized in that: When the time difference is greater than the set threshold, determining that the video is stuck; When the time difference is less than or equal to the set threshold, determining that the video is not stuck includes: If the third judgment result is yes, it is determined that the video is stuck; If the third judgment result is no, it is determined that the video is not stuck.

8. The video freeze detection method based on fully mechanized mining working face according to any one of claims 1 to 3, characterized in that: When determining that the video freezes, adjusting video parameters and sending the fault information include: When it is determined that the video is stuck, adjusting the video parameters; If the video is still stuck after adjusting the video parameters, the fault information is sent.

9. A video freeze detection system based on fully mechanized mining working face, characterized in that: include: A video reading module (210) is used to obtain a video about the fully mechanized mining working face and determine whether the video can be played; A video stream processing module (220), used for parsing the video, converting it into an image sequence consisting of multiple frames of images, and determining whether two adjacent frames of images in the image sequence are continuous images, when the video is playable; A video detection module (230) is used to determine whether the video is stuck according to the time difference between the two adjacent frames of images and a set threshold when the two adjacent frames of images are the continuous images; determine whether the video is stuck when the two adjacent frames of images are not the continuous images; and determine whether the video is stuck when the time difference is greater than the set threshold; When the time difference is less than or equal to the set threshold, determining that no freeze occurs in the video; An adaptive adjustment module (240) is used to adjust video parameters when it is determined that the video is stuck; The fault information sending module (250) is used to send fault information when the video cannot be played, and is used to send the fault information when it is determined that the video is stuck.

10. An electronic device, characterized in that: It comprises a processor (320), a memory (310), and a program or instruction stored in the memory (310) and executable on the processor (320); when the program or instruction is executed by the processor (320), the steps of the video jamming detection method based on the fully mechanized mining face as described in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video jamming detection method based on a fully mechanized mining working face as described in any one of claims 1 to 8 are implemented.

Images