Overexposure detection method, device and equipment based on video code stream characteristics and medium

Abstract

The invention discloses an overexposure detection method and device based on video code stream features, equipment and a medium, and belongs to the technical field of computers. The method comprises the following steps: receiving video code stream data; analyzing the video code stream data to obtain prediction type information, brightness coefficient information and picture detail information of each macro block of the current frame; constructing a multi-dimensional feature vector based on the information change of each macro block among multiple frames; and inputting the multi-dimensional feature vector into a pre-constructed classifier model so as to determine an overexposure detection result of the video code stream data based on an output result of the classifier model. According to the technical scheme, the video stream does not need to be completely decoded, the computing resource consumption is reduced, the method is suitable for complex scenes such as a mine, and the accuracy and robustness of overexposure detection can be improved by integrating multi-dimensional features.

Description

Technical Field

[0001] In complex lighting monitoring scenarios such as underground coal mines, cameras often experience glare and overexposure due to strong light sources such as direct and reflected light from miners' lamps. This leads to loss of video details, reduced monitoring quality, and interference with subsequent video-based intelligent algorithm analysis, resulting in false alarms. Therefore, reliable real-time glare and overexposure detection technology is urgently needed.

[0002] Existing technologies mostly rely on decoded pixel-domain images for detection. For example, some solutions completely decode the video stream into YUV or RGB format images and identify overexposed areas by calculating texture features such as mean brightness, variance, and gray-level co-occurrence matrix. Other solutions decode the video, perform Gaussian filtering and brightness gradient calculation to obtain the light source outline, and combine the light source and background brightness to calculate the UGR glare value to determine overexposure.

[0003] However, these existing technologies have obvious drawbacks. On the one hand, they require complete decoding of the video stream to restore the pixel matrix, which consumes a lot of CPU or GPU computing power. In scenarios where multiple cameras are deployed underground, the computing pressure on edge devices or central servers increases exponentially, making it difficult to achieve real-time detection. On the other hand, detection based on pixel domain images is a post-analysis, and the real-time performance of the analysis cannot be guaranteed. Moreover, underground coal dust and water mist can cause the image to become blurry and the contrast to decrease, which can easily interfere with detection indicators such as brightness gradient and texture clarity, and can easily lead to misjudgment. Background Technology

[0004] In complex lighting monitoring scenarios such as underground coal mines, cameras often experience glare and overexposure due to strong light sources such as direct and reflected light from miners' lamps. This leads to loss of video details, reduced monitoring quality, and interference with subsequent video-based intelligent algorithm analysis, resulting in false alarms. Therefore, reliable real-time glare and overexposure detection technology is urgently needed.

[0005] Existing technologies mostly rely on decoded pixel-domain images for detection. For example, some solutions completely decode the video stream into YUV or RGB format images and identify overexposed areas by calculating texture features such as mean brightness, variance, and gray-level co-occurrence matrix. Other solutions decode the video, perform Gaussian filtering and brightness gradient calculation to obtain the light source outline, and combine the light source and background brightness to calculate the UGR glare value to determine overexposure.

[0006] However, these existing technologies have obvious drawbacks. On the one hand, they require complete decoding of the video stream to restore the pixel matrix, which consumes a lot of CPU or GPU computing power. In scenarios where multiple cameras are deployed underground, the computing pressure on edge devices or central servers increases exponentially, making it difficult to achieve real-time detection. On the other hand, detection based on pixel domain images is a post-analysis, and the real-time performance of the analysis cannot be guaranteed. Moreover, underground coal dust and water mist can cause the image to become blurry and the contrast to decrease, which can easily interfere with detection indicators such as brightness gradient and texture clarity, and can easily lead to misjudgment. Summary of the Invention

[0007] The purpose of this application is to provide an overexposure detection method, apparatus, device, and medium based on video stream features. This addresses the technical problems of traditional mine camera glare and overexposure detection, which requires complete video stream decoding, consumes significant computational resources, has poor real-time performance, and is susceptible to interference from coal dust and water mist, leading to a high false alarm rate. The method constructs a multi-dimensional feature vector by directly parsing the information obtained from the bitstream to determine the overexposure detection result. This eliminates the need for complete video stream decoding, reduces computational resource consumption to adapt to underground edge devices, and improves detection accuracy and anti-interference capabilities by integrating multi-dimensional features and multi-frame changes. It accurately identifies glare and overexposure caused by direct exposure to mine lamps, providing a reliable basis for subsequent alarm and video enhancement decisions.

[0008] In a first aspect, embodiments of this application provide an overexposure detection method based on video bitstream features, the method comprising: Receive video stream data; The video stream data is parsed to obtain the prediction type information, brightness coefficient information and image detail information of each macroblock in the current frame; Based on the information changes of macroblocks across multiple frames, a multidimensional feature vector is constructed. The multidimensional feature vector is input into a pre-built classifier model to determine the overexposure detection result of the video bitstream data based on the output of the classifier model.

[0009] Furthermore, the video stream data is parsed to obtain the prediction type information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; The residual coefficient array of each macroblock is reconstructed using the entropy decoder; Based on the information in the NAL unit, the proportion of macroblocks in the current frame whose prediction type is intra-frame prediction is determined.

[0010] Furthermore, the video stream data is parsed to obtain the luminance coefficient information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; Based on the information in the NAL unit and the residual coefficient array, the luminance coefficient information of each macroblock is identified.

[0011] Furthermore, after identifying the luminance coefficient information of each macroblock based on the information in the NAL unit and the residual coefficient array, the method further includes: The brightness coefficient information is subjected to mapping transformation processing so that the brightness coefficient information is within a preset range.

[0012] Furthermore, the video stream data is parsed to obtain the image detail information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; Based on the information in the NAL unit and the residual coefficient array, the image detail information of each macroblock is identified.

[0013] Furthermore, based on the information in the NAL unit and the residual coefficient array, the image detail information of each macroblock is identified, including: The DC and AC components are extracted based on the residual coefficient array of each macroblock. The DC component in the macroblock is the first element of the residual coefficient array, which represents the average brightness of the current macroblock. The other components are AC components. Calculate the sum of all AC component coefficients in each macroblock and the sum of the high-frequency AC component coefficients, and calculate the high-frequency energy percentage.

[0014] Furthermore, based on the information changes of each macroblock across multiple frames, a multi-dimensional feature vector is constructed, including: Based on the information changes of each macroblock between multiple frames, as well as the prediction type information, brightness coefficient information and image detail information of the current frame, a six-dimensional feature vector composed of static and dynamic features is constructed.

[0015] Secondly, embodiments of this application provide an overexposure detection device based on video bitstream features, the device comprising: The data receiving module is used to receive video stream data; The information extraction module is used to parse the video bitstream data to obtain the prediction type information, brightness coefficient information and image detail information of each macroblock in the current frame; The feature construction module is used to construct multi-dimensional feature vectors based on the information changes of macroblocks across multiple frames; An overexposure detection module is used to input the multidimensional feature vector into a pre-built classifier model to determine the overexposure detection result of the video bitstream data based on the output of the classifier model.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0017] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0018] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0019] The technical solution provided in this application embodiment receives video stream data; parses the video stream data to obtain prediction type information, brightness coefficient information, and image detail information of each macroblock in the current frame; constructs a multi-dimensional feature vector based on the information changes of each macroblock across multiple frames; inputs the multi-dimensional feature vector into a pre-built classifier model, and determines the overexposure detection result of the video stream data based on the output of the classifier model. This technical solution can eliminate the need for complete decoding of the video stream, reducing computational resource consumption, adapting to complex scenarios such as underground mines, and comprehensively improving the accuracy and robustness of overexposure detection by integrating multi-dimensional features. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the overexposure detection method based on video bitstream features provided in Embodiment 1 of this application; Figure 2 This is a system architecture diagram provided in Embodiment 2 of this application; Figure 3 This is a schematic diagram of the overexposure detection device based on video bitstream features provided in Embodiment 3 of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0024] The overexposure detection method, apparatus, equipment, and medium based on video bitstream features provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0025] Example 1 Figure 1 This is a flowchart illustrating the overexposure detection method based on video bitstream features provided in Embodiment 1 of this application. Figure 1 As shown, the specific steps include the following: S101, receives video stream data; Video stream data refers to a continuous video sequence that has been encoded and compressed. In this embodiment, it is specifically the H.264 (H.264 Advanced Video Coding) format stream output by the underground camera. It contains image encoding information, such as prediction mode and transform coefficients, rather than the original pixel data.

[0026] This solution can receive real-time streams from the IP address of cameras via industrial Ethernet in mines. For example, explosion-proof network cameras can establish a connection using RTSP (Real Time Streaming Protocol) and write the stream data to a local buffer.

[0027] S102, the video stream data is parsed to obtain the prediction type information, brightness coefficient information and image detail information of each macroblock in the current frame; The current frame refers to the single frame image being processed in the video sequence, which is organized in units of slices in H.264.

[0028] A macroblock is the basic unit of encoding, which in this embodiment is 16×16 pixels in size and contains luminance (Y) and chrominance (Cb, Cr) components.

[0029] Prediction type information refers to the coding dependency of macroblocks, which is divided into intra-frame prediction, which depends only on adjacent pixels in the current frame, and inter-frame prediction, which depends on the preceding reference frame.

[0030] The luminance coefficient information is a quantized value that reflects the overall luminance of a macroblock, derived from the DC component of the DCT (Discrete Cosine Transform).

[0031] Image detail information is a parameter that reflects the richness of macroblock texture and is derived from the AC component of DCT.

[0032] This approach first separates the NAL (Network Abstraction Layer) units, then parses the macroblock data in the slices to extract information such as prediction type identifiers and DCT coefficients. The specific parsing process can be combined with subsequent weighted constraints.

[0033] S103, constructs a multi-dimensional feature vector based on the information changes of each macroblock between multiple frames; Among them, multiple frames refer to video data of the current frame and several consecutive frames before it, such as 30 frames. Specifically, the frame rate can be set according to the underground mining scene. For example, in the case of 30 frames / second, multiple frames can be determined to be 30 frames, or only a few frames of data can be selected.

[0034] Information change refers to the difference between the average level of the same feature in the current frame and in historical frames, such as the difference between the brightness of the current frame and the average brightness of the previous 30 frames.

[0035] A multidimensional feature vector is an array containing multiple feature parameters, used to comprehensively describe the overexposure-related characteristics of the current frame.

[0036] This scheme can first calculate the static features of the current frame, such as the proportion of intra-frame predicted macroblocks and the average luminance coefficient, then calculate the average feature value of the previous 30 frames as the baseline, calculate the difference between the current frame and the baseline as the dynamic feature, and finally combine the static and dynamic features in order into a vector, such as [F1,F2,F3,ΔF1,ΔF2,ΔF3].

[0037] S104, the multidimensional feature vector is input into a pre-built classifier model to determine the overexposure detection result of the video bitstream data based on the output of the classifier model.

[0038] The classifier model refers to the machine learning model obtained through training. In this embodiment, it is an SVM (Support Vector Machine) model, which uses RBF (Radial Basis Function) as the kernel function.

[0039] Overexposure detection results refer to the determination of whether there is glare overexposure in the current frame, including categories such as normal, slightly overexposed, and severely overexposed.

[0040] In this scheme, the constructed multidimensional feature vector can be input into the SVM model. The model calculates the distance between the feature vector and the classification hyperplane and outputs the probability of belonging to the overexposure category, such as a value between 0 and 1. When the probability is greater than 0.8, it is determined that overexposure has been detected. For example, the bright areas in the image caused by the direct light from the lens of the mine lamp in the mine will be identified as overexposure.

[0041] The technical solution provided in this embodiment extracts features directly from the video bitstream, avoiding the high computational cost of fully decoding and reconstructing the image, and adapting to the hardware limitations of edge devices in underground mines. Simultaneously, it combines multi-frame information to construct multi-dimensional features, comprehensively considering static features and dynamic changes, effectively distinguishing between genuine overexposure and interference such as dust and water mist, thus improving the accuracy and robustness of detection.

[0042] In this embodiment, optionally, the video bitstream data is parsed to obtain the prediction type information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; The residual coefficient array of each macroblock is reconstructed using the entropy decoder; Based on the information in the NAL unit, the proportion of macroblocks in the current frame whose prediction type is intra-frame prediction is determined.

[0043] The internal buffer refers to the memory area used to store the bitstream to be parsed and the encoding parameters (such as SPS and PPS), which is divided into the bitstream buffer (2MB in size) and the parameter buffer (128KB in size). An entropy decoder is a module used to entropy decode compressed macroblock data, supporting both CABAC (Context-based Adaptive Binary Arithmetic Coding) and CAVLC (Context-based Adaptive Variable Length Coding) modes.

[0044] In this scheme, when the system starts, memory space can be allocated to initialize the buffer area and load the initial parameters of the entropy decoder, such as the probability table of CABAC and the code table of CAVLC, to ensure that the decoder is in a ready state.

[0045] The start code is a sequence of bytes that identifies the boundary of a NAL unit, specifically 0x000001 (3 bytes) or 0x00000001 (4 bytes), used to distinguish different NAL units from a continuous bitstream.

[0046] NAL units are the basic building blocks of H.264 bitstreams. Each NAL unit contains a 1-byte NAL header for identifying the type and a variable-length NAL body for storing encoded data.

[0047] In this scheme, the bitstream data can be read from the internal buffer and matched byte by byte. When 0x00, 0x00, 0x01 (or 0x00, 0x00, 0x00, 0x01) appear consecutively, it is determined to be a start code. The bitstream data between the end position of the previous start code and the start position of the current start code is marked as a NAL unit and its type is recorded, which is determined by the nal_unit_type field of the NAL header.

[0048] The residual coefficient array refers to the coefficient matrix obtained after macroblocks are predicted and transformed by DCT. In this embodiment, it is a two-dimensional array of size 4×4, corresponding to 4×4 luma sub-blocks, storing the result of the transformation of the difference between the actual pixel value and the predicted value.

[0049] Reconstruction refers to the process of restoring the entropy-encoded and compressed coefficient data to the original residual coefficient matrix.

[0050] For the selected P-frames, NAL units with frame type 1, the corresponding entropy decoding algorithm is called according to the entropy coding mode recorded in the PPS (Picture Parameter Set) to decode the encoded data of the macroblock, resulting in a 4×4 residual coefficient array, such as [-3,2,0,...,1].

[0051] The information in the NAL unit includes the video width and height, macroblock size, and prediction mode identifier in the SPS (Sequence Parameter Set), as well as the prediction mode identifier in the macroblock coding data.

[0052] The proportion of intra-frame prediction refers to the ratio of the number of macroblocks in the current frame that use intra-frame prediction to the total number of macroblocks. For example, if the total number of macroblocks is 1000 and 200 of them are intra-frame predictions, then the proportion is 20%.

[0053] In this scheme, the video width and height can be read from SPS to calculate the total number of macroblocks. Then, all macroblocks in the current frame are traversed, and the number of intra-frame predicted macroblocks is counted by using the prediction mode identifier corresponding to the residual coefficient array (0-34 for intra-frame prediction mode and other values ​​for inter-frame prediction mode). Finally, the proportion is obtained by dividing the number of intra-frame predicted macroblocks by the total number of macroblocks, which is the F1 feature value mentioned above.

[0054] This technical solution accurately extracts the proportion of intra-frame predicted macroblocks by analyzing NAL units and macroblock residual coefficients. This proportion increases sharply during overexposure because the encoder cannot use inter-frame prediction, becoming a key feature for overexposure detection. Simultaneously, the total number of macroblocks is calculated based on SPS parameters to ensure the accuracy of the proportion calculation, providing a data foundation for subsequent feature construction.

[0055] In this embodiment, optionally, the video stream data is parsed to obtain the luminance coefficient information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; Based on the information in the NAL unit and the residual coefficient array, the luminance coefficient information of each macroblock is identified.

[0056] The information in the NAL unit includes the quantization parameters in the PPS, which are used to adjust the coefficient amplitude, and the residual coefficient array is a 4×4 coefficient matrix reconstructed as described above.

[0057] The luminance coefficient information refers to the DC (Direct Current) component of the macroblock's luminance, which is the element at position (0,0) in the residual coefficient array, representing the average luminance level of the macroblock.

[0058] This scheme can extract the first element of the 4×4 residual coefficient array of each macroblock as the luminance coefficient of that macroblock; and adjust the coefficient in combination with the quantization parameters in PPS to obtain the final luminance coefficient information. For example, a DC component of 128 indicates that the macroblock has a high luminance.

[0059] This technical solution directly extracts the DC component as the luminance coefficient from the residual coefficient array, eliminating the need to reconstruct pixel images and reducing computational complexity. Simultaneously, adjustments are made to the quantization parameters in PPS to ensure that the luminance coefficient accurately reflects the actual luminance level of the macroblock, providing core luminance characteristics for overexposure detection.

[0060] In this embodiment, optionally, after identifying the luminance coefficient information of each macroblock based on the information in the NAL unit and the residual coefficient array, the method further includes: The brightness coefficient information is subjected to mapping transformation processing so that the brightness coefficient information is within a preset range.

[0061] The mapping transformation process refers to the process of converting the brightness coefficient information to a specified range through mathematical operations; in this embodiment, it is a normalization process. The preset range refers to the range of feature values ​​set according to the actual scene, such as [0,1] or [0,100], to unify the dimensions of different features.

[0062] In this scheme, a linear scaling method can be used to map the brightness coefficient information, such as the original range of 0-5000, to the interval [0,1]. For example, if the original brightness coefficient is 2500, it will be 0.5 after mapping.

[0063] This technical solution normalizes the brightness coefficient to a preset range through mapping transformation, eliminating the dimensional differences between different features. For example, the intra-frame prediction ratio is a percentage, while the brightness coefficient is an absolute value. This makes the elements in the subsequently constructed multi-dimensional feature vector comparable, improving the training and inference effects of the classifier model.

[0064] In this embodiment, optionally, the video stream data is parsed to obtain the image detail information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; Based on the information in the NAL unit and the residual coefficient array, the image detail information of each macroblock is identified.

[0065] Among them, the image detail information refers to the parameters that reflect the richness of details such as macroblock texture and edges. In this embodiment, it is characterized by the energy distribution of the AC (Alternating Current) component. This technical solution can extract 15 elements other than the DC component, namely the AC component, from the residual coefficient array. By calculating the amplitude and distribution characteristics of the AC component, such as the proportion of high-frequency components, the image detail information of the macroblock can be identified.

[0066] This technical solution obtains image detail information by analyzing the AC component in the residual coefficient array. This information is significantly reduced during overexposure due to the loss of detail caused by the brightness of the image. It complements the brightness information and together improves the accuracy of overexposure detection.

[0067] In this embodiment, optionally, based on the information in the NAL unit and the residual coefficient array, the image detail information of each macroblock is identified, including: The DC and AC components are extracted based on the residual coefficient array of each macroblock. The DC component in the macroblock is the first element of the residual coefficient array, which represents the average brightness of the current macroblock. The other components are AC components. Calculate the sum of all AC component coefficients and high-frequency AC component coefficients in each macroblock, and calculate the high-frequency energy percentage.

[0068] The DC component is the element at position (0,0) in the residual coefficient array, reflecting the overall brightness level of the macroblock. The AC components are the other 15 elements in the residual coefficient array besides the DC components, reflecting the detailed changes in the macroblock. Among them, the high-frequency AC components, such as elements other than (0,1) and (1,0), correspond to fine textures.

[0069] For each macroblock's 4×4 residual coefficient array, such as [10,2,-1,3,1,...,0], the first element 10 is determined as the DC component, and the remaining 15 elements (2,-1,3,1,...,0) are determined as the AC component.

[0070] The sum of all AC component coefficients refers to the sum of the absolute values ​​of the 15 AC components, such as |2|+|-1|+|3|+...+|0|=15.

[0071] The sum of high-frequency AC component coefficients refers to the sum of the absolute values ​​of the AC components representing fine textures, such as the elements at (1,1) and other positions in a 4×4 matrix, such as |-1|+|3|+...+|0|=8.

[0072] The high-frequency energy ratio refers to the ratio of the sum of the high-frequency AC component coefficients to the sum of all AC component coefficients, such as 8 / 15≈53.3%.

[0073] In this scheme, the sum of the AC components and the sum of the high-frequency components are calculated separately for each macroblock, and then the high-frequency energy percentage, i.e., the F3 characteristic value mentioned above, is obtained by division. For example, if the sum of the AC components of a macroblock is 20 and the sum of the high-frequency components is 5, then the high-frequency energy percentage is 25%.

[0074] This technical solution accurately characterizes the detail richness of macroblocks by quantifying the high-frequency energy ratio of the AC component. When overexposed, this ratio will decrease significantly due to the loss of high-frequency details, such as from 50% to 10%, providing key detail features for overexposure detection. It works in conjunction with brightness and prediction type features to improve detection accuracy.

[0075] In this embodiment, optionally, a multi-dimensional feature vector is constructed based on the information changes of each macroblock across multiple frames, including: Based on the information changes of each macroblock between multiple frames, as well as the prediction type information, brightness coefficient information and image detail information of the current frame, a six-dimensional feature vector composed of static and dynamic features is constructed.

[0076] Among them, static features refer to the feature parameters of the current frame, including prediction type information, namely the proportion of intra-frame predicted macroblocks F1, luminance coefficient information, namely normalized luminance F2, and image detail information, namely high-frequency energy proportion F3.

[0077] Dynamic features refer to the changes in information between multiple frames, that is, the difference between the static features of the current frame and the average features of the previous 30 frames, such as ΔF1=F1-mean F1, ΔF2=F2-mean F2, ΔF3=F3-mean F3.

[0078] Therefore, a six-dimensional feature vector is an array composed of the above six features in sequence, such as [F1,F2,F3,ΔF1,ΔF2,ΔF3].

[0079] Specifically, we can first calculate F1, F2, and F3 of the current frame as static features, then calculate the mean of F1, F2, and F3 from the features of the previous 30 frames in the cache, then calculate the difference between the current frame and the baseline to obtain ΔF1, ΔF2, and ΔF3 as dynamic features, and finally combine these 6 features in order into a six-dimensional vector, for example [0.3, 0.8, 0.2, 0.2, 0.5, -0.3].

[0080] This technical solution constructs a six-dimensional vector by combining static and dynamic features, which reflects both the absolute state of the current frame and captures the abrupt change trend relative to historical frames. For example, when overexposed, the dynamic features will deviate significantly from the baseline, greatly improving the discriminative power of the features. This enables the classifier model to more accurately identify overexposed scenes and reduce misjudgments caused by slow changes in illumination.

[0081] Example 2 In order to enable those skilled in the art to better understand the technical solution of this application, this application also provides a preferred embodiment. It should be understood that this preferred embodiment is intended to illustrate a specific implementation method and is not intended to limit the technical solution of this application.

[0082] The purpose of this invention is to provide a high-efficiency, low-power, high-real-time, and anti-interference-capable camera glare and overexposure detection technology. This technology is specifically designed for complex industrial monitoring scenarios such as underground coal mines. By innovatively bypassing the traditional pixel-domain analysis path, it directly utilizes the bitstream characteristics generated during video encoding to diagnose image anomalies, thereby achieving accurate and rapid identification of camera glare and overexposure phenomena. This provides crucial decision-making support for subsequent alarm functions, video enhancement, or system maintenance.

[0083] Video decoding is a computationally intensive task with extremely high power consumption. This is an unbearable burden for underground monitoring equipment that requires intrinsically safe design, battery power, or has limited power resources, restricting the widespread deployment of traditional methods that require video decoding and image analysis. This invention avoids the complete video decoding process, directly extracting key features through lightweight bitstream parsing. This reduces the processing time and computational resource consumption of a single video stream by an order of magnitude, supporting real-time glare and overexposure detection of multiple high-definition videos on low-performance edge computing devices. Furthermore, due to its low computational resource consumption, it is also easy to integrate into existing video surveillance or video intelligent analysis platforms.

[0084] This invention utilizes video stream features, such as abrupt changes in macroblock prediction modes and abnormal distribution of DCT coefficients, and their weak correlation with image content, such as dust and water mist, to establish a more specific diagnostic model for glare overexposure. This improves the robustness and accuracy of glare overexposure detection in the harsh environment of underground coal mines, and reduces false alarm and false negative rates.

[0085] The system provided by this invention fully utilizes the principles of video coding technology. Video coding is a technique that compresses captured video frames to preserve complete video frame information with a small amount of data. This invention focuses on extracting and analyzing macroblock feature information in the video bitstream. In normal video sequences, the content of adjacent frames has extremely high spatiotemporal correlation; that is, most areas of the frame are static or in continuous motion. For these areas, the video encoder employs inter-prediction technology, not encoding complete pixel values, but only encoding the differences and motion vectors between the current frame and the reference frame, using a very small number of bits to represent a large amount of content. Therefore, in typical video bitstreams, inter-prediction macroblocks constitute the vast majority, while intra-prediction macroblocks (Intra-MB_Ration) constitute a minority, with the proportion of intra-prediction macroblocks remaining at a low level. When a camera is suddenly shone directly into the lens by a strong light source such as a miner's lamp, an extremely bright, detail-destroyed, and completely different overexposed area appears in the image, unlike the previous frame. This overexposed area undergoes a radical and discontinuous change compared to the corresponding area in the reference frame, with almost no temporal correlation between the two. At this point, the overexposed area of ​​the current frame cannot find any similar reference blocks in the reference frame. The encoder's strategy is to abandon inter-frame prediction and switch to intra-frame prediction for macroblocks that have undergone drastic changes. Intra-frame prediction does not rely on other frames; it only uses the already encoded adjacent pixels within the current frame to predict the current block. Specifically, when exposed to strong light, a large number of macroblocks simultaneously undergo a sudden change from inter-frame prediction to intra-frame prediction at the coding level, with a sharp and significant peak in the number and proportion of intra-frame predicted macroblocks. The ingenious aspect of this scheme's method for detecting intra-frame predicted macroblock abrupt changes compared to traditional image processing is that it detects the "cause" rather than the "effect"—the abrupt change in coding strategy caused by an "abnormal change" in the image. This is a higher-level and more stable signal. Furthermore, this solution exhibits strong resistance to interference. While dust and fog underground can cause persistent blurring of the image, it does not result in sudden, drastic changes in the encoding strategy, as seen with direct sunlight. Therefore, adopting this solution can improve detection accuracy.

[0086] In this invention's system, by using the abrupt change in the proportion of intra-predicted macroblocks in the video bitstream as a key feature, it is possible to detect overexposure caused by "glare," such as direct light from a mine lamp. However, extreme scenarios such as a sudden high-speed entry of a rubber-wheeled vehicle or a large coal object suddenly obscuring the view can also cause a sharp increase in the proportion of intra-predicted macroblocks. Therefore, to reduce false alarms, this invention also utilizes the DC and AC coefficients from the Integer Discrete Cosine Transform (DCT) process performed on the residual data during video encoding as supplementary judgment criteria. In video coding standards such as H.264, the video encoder performs Integer Discrete Cosine Transform (DCT) on the residual data. Its basic principle is to transform an image block, such as a 4x4 or 8x8 block, from the spatial domain to the frequency domain. The DC coefficients, located at the upper left corner of the transformation matrix (e.g., (0,0), represent the average brightness or DC component of the image block. This can be understood as the background color of the entire block. The AC coefficients are all other coefficients in the transformation matrix besides the DC coefficients, representing high-frequency information such as details, textures, and edges in the image block. The further away from the DC coefficient, the higher the frequency. When a camera in a mine is exposed to direct, strong light, the physical properties of overexposed areas in the image cause the DCT coefficient to exhibit the following unique variation pattern: 1) The pixel values ​​in the overexposed area become saturated due to reaching the sensor's limit, and the entire area becomes a bright white or light spot with extremely high and very uniform brightness. This directly leads to an abnormally large amplitude (absolute value) of the DC coefficient, which represents the average brightness. When the brightness DC coefficient of all blocks in a whole frame is counted, a significant peak will appear when overexposure occurs. 2) Due to pixel saturation, a large amount of texture, detail, and edge information in the overexposed area is lost. The original coal wall texture, equipment outline, etc. are all submerged in a uniform bright light. The high-frequency AC coefficients that represent details and textures will become very small in amplitude due to the lack of coded information, or even become zero in many cases. The high-frequency AC energy ratio of all blocks in a whole frame will be significantly reduced.

[0087] The characteristics of this type of DCT coefficients are similar to those of traditional video-based analysis. The difference lies in that it does not perform complete video decoding of the entire image. Instead, it directly extracts the DC and AC coefficients from the transformation matrix in the bitstream, omitting the process of reconstructing the video frame from the video bitstream, which can greatly reduce the amount of computation. Furthermore, it complements the aforementioned characteristic of abrupt changes in the proportion of macroblocks in intra-frame prediction.

[0088] In this invention system, the final decision is to fuse three video stream features that are only triggered simultaneously when the camera experiences "glare" overexposure: a sudden increase in the intra-predictive macroblock ratio (intra_mb_ratio, denoted as F1), a sudden increase in the luminance DC coefficient (avg_dc_energy, denoted as F2), and a sudden decrease in the high-frequency AC energy ratio (avg_ac_high_ratio, denoted as F3). This is achieved by collecting raw video clips from underground coal mine cameras. The data includes positive samples, "glare" overexposure, and deliberately recorded "glare" overexposure video clips from different positions and angles by shining a miner's lamp directly into the camera lens. Negative samples (normal): A large number of normal operation videos are collected, including but not limited to normal vehicle and personnel passage in the tunnel, equipment operation, and scenarios that are highly likely to cause false alarms, such as mine cars quickly passing by the camera or switching ventilation doors on and off. The collected video clips are segmented by frame and manually labeled, with each frame marked as either "glare" or "normal". Using the bitstream parsing method proposed in this invention, each video bitstream segment is processed to calculate its three core feature values: intra-frame prediction macroblock ratio (F1); luminance DC coefficient (F2); and F3: high-frequency AC energy ratio (F3). Then, combining the feature values ​​of the preceding N video bitstreams, its temporal difference features are calculated, constructing a set of feature vectors [F1, ∆F1, F2, ∆F2, F3, ∆F3, Label] containing the current frame's feature information and feature change trends. Finally, an SVM support vector machine classification model containing the current frame's "static features" and "dynamic trends" is trained to detect in real time whether the video bitstream from the underground camera experiences "glare" overexposure.

[0089] Since the technical solution of the present invention does not require complete decoding of the video stream, it can accurately detect "glare" and overexposure of the camera by simply extracting the original feature information of the video stream in the encoding stage and combining it with the pre-trained model. It occupies very low computing resources and can therefore be widely deployed on edge devices with limited resources in underground coal mines.

[0090] Furthermore, this invention, as a glare detection algorithm and system for underground coal mine cameras, when it detects glare overexposure caused by factors such as "miner's lamp" illumination, publishes the detection event results externally by broadcasting MQTT messages or calling the business system's HTTP interface. In addition to recording glare alarm events generated by the camera, it can also notify business systems such as video surveillance and intelligent video analysis, triggering them to perform business linkages such as video glare alerts, video algorithm enhancement, and alarm event filtering.

[0091] Figure 2 This is a system architecture diagram provided in Embodiment 2 of this application. For example... Figure 2 As shown, this solution relies on four core functional modules: The bitstream parsing module directly extracts macroblock type and encoded DCT coefficient information for "glare" overexposure judgment by performing NAL unit identification, entropy decoding and other operations on the original video stream. The feature processing module performs unified dimensional processing and temporal change rate statistics on the extracted macroblock type, DC coefficient, AC coefficient and other feature data to construct a set of feature vectors containing the current frame and change rate. The model detection module comprises two phases: offline model training and real-time detection. The offline training phase utilizes a large collection of videos of normal underground coal mine operations and glare / overexposure footage to extract feature vectors from video frames and train a classification detection model. The real-time detection phase performs real-time bitstream parsing and feature processing on each frame of the camera's video stream, ultimately using the classification detection model to obtain the "glare" detection result.

[0092] The output and control module, when it detects a "glare" event from the camera in real time, constructs the detection results into structured event data and distributes the detection results externally via MQTT or HTTP. On the one hand, it serves as an alarm event for use by business systems, and on the other hand, it can trigger subsequent business systems such as video surveillance and intelligent video analysis to perform linkage processing.

[0093] The bitstream parsing module performs incomplete decoding on the received H.264 compressed bitstream from the camera. It obtains syntax elements such as prediction type (MB_TYPE) and discrete cosine transform coefficients (DCT) of all macroblocks (MBs) in the current frame through operations such as decapsulating the Network Abstraction Unit (NALU), parsing the slice header, and entropy decoding. It then calculates the number of intra-predicted macroblocks and the total number of macroblocks in the current frame to obtain the intra-predicted macroblock ratio (intra_mb_ratio); calculates the average absolute value of the luminance DC coefficient of each macroblock in the current frame (avg_dc_energy); and calculates the ratio of the high-frequency AC energy of the macroblocks in the current frame to the total component energy to obtain the high-frequency AC energy ratio (avg_ac_high_ratio). This module abandons the energy-intensive path of full decoding, image reconstruction, and image reprocessing in traditional schemes, achieving a direct and efficient mapping from the bitstream to feature values, laying a solid foundation for the system's real-time performance.

[0094] The business process of the stream parsing module is as follows: Initialization: Initializes internal buffers, entropy decoder, and other context information. Video stream input: Start a thread based on the configured camera address information to collect the H264 video stream bytes of the camera in real time.

[0095] NAL Unit Decapsulation: The module parses the bitstream byte by byte, identifies the start code 0x000001 or 0x00000001, and thus divides the continuous bitstream into individual Network Abstraction Layer (NAL) units.

[0096] Slice extraction: SPS frames (frame type 7) are selected, and the sequence parameter set contained in these frames is cached. This sequence parameter set includes parameters such as video height, video width, and frame rate. PPS frames (frame type 8) are selected, and the image parameter set is cached. This image parameter set includes encoding parameters such as entropy coding mode, slice segmentation type, and initial quantization parameters. P frames (frame type 1) are selected. P frames are forward prediction frames, which store the difference and motion vector between the current frame and the previous I or P frame. The feature information to be extracted in this invention is mainly contained in P frames. I frames (frame type 5) are filtered out. I frames are keyframes, which store complete information of the current image but lack inter-frame prediction information, and therefore are not used as frames for feature extraction.

[0097] Entropy Decoding and Syntax Element Parsing: The H.264 standard supports two entropy coding schemes: Context-Based Adaptive Binary Arithmetic Coding (CABAC) and Context-Based Adaptive Variable-Length Coding (CAVLC). The module first determines the current entropy coding mode based on the `entropy_coding_model_flag` syntax element parsed from the buffered PPS frame. Then, it iterates through each macroblock (MB) in the parsed slice according to the decoding order and reconstructs a 4x4 luma block residual coefficient array using CAVLC or CABAC entropy decoding.

[0098] Calculate the intra-frame predicted macroblock proportion feature value Before parsing the image slices in step 5 above, it is necessary to first access the SPS series parameter set cached in step 4 above and parse it. and The syntax element, module, calculates the macroblock grid size, i.e., the total number of macroblocks, for each frame of image using the following formula: ; ; ; Where `total_mb` represents the total number of macroblocks contained in the current frame. In step 5 above, all macroblocks are iterated over, and when the prediction type of a macroblock is intra-prediction, the intra-prediction macroblock count `count_intra_mb` is updated. Finally, the proportion of intra-prediction macroblocks in the current frame is calculated: ; When a camera experiences glare or overexposure, the video encoder will use intra-frame prediction extensively for video encoding due to the sudden changes in the video image and the loss of a large area of ​​detail. This will cause the coefficient to increase sharply.

[0099] Extract the DC and AC components from the residual coefficient array of the 4x4 luminance block obtained in step 5 above: For better understanding, a 4x4 DCT coefficient block is represented by the following diagram:

[0100] The DC component in the macroblock is the first element of the residual coefficient array, which is the coefficient value at position (0,0) in the transformation matrix. dc_energy represents the average brightness of the current macroblock.

[0101] All other components in the macroblock transform matrix are AC components. Calculate the sum of the coefficients of all AC components in the macroblock:

[0102] The lower right corner of the macroblock transform matrix represents the high-frequency region. Calculate the sum of the high-frequency AC component coefficients in the macroblock:

[0103] This yields the high-frequency energy percentage of the current macroblock:

[0104] The high-frequency AC energy percentage represents the texture detail of the current macroblock. When a large amount of detail is lost in the video frame, the high-frequency AC energy percentage will drop sharply.

[0105] Calculate the overall average luminance DC coefficient characteristic value of the current frame. Following step 7 above, the DC coefficients of a single macroblock can be obtained. Following step 5 above, all macroblocks are traversed to calculate the average energy value of the current frame. Since H.264 video encoding uses a 4x4 transform matrix by default for macroblocks, the total number of blocks equals the number of macroblocks multiplied by 16. ; When a camera experiences glare and overexposure, the video image will appear overly bright over a large area, and this coefficient value will increase sharply.

[0106] Similarly, the overall average high-frequency AC component energy ratio characteristic value AC_High_Freq_Ratio of the current frame is calculated: Following step 7 above, the AC high-frequency coefficient ratio of a single macroblock can be obtained. Following step 5 above, all macroblocks are traversed to calculate the overall high-frequency energy ratio of the current frame. ; When a camera experiences glare and overexposure, a large area of ​​detail will be lost in the video image, and this coefficient value will drop sharply.

[0107] It should be noted that the above-mentioned bitstream parsing module extracts feature values ​​according to the H.264 video encoding method. The above-mentioned bitstream parsing module can also be extended to other video encoding methods such as H.265 (HEVC), only the method of extracting video encoding feature values ​​is slightly different. Therefore, the encoding compatibility of this invention with camera terminal devices can be extended.

[0108] The feature processing module receives the raw feature sequence output by the code stream parsing module and completes high-level feature construction through the following steps to train the SVG support vector machine classification and detection model. This mainly includes: 1) Normalization of feature values: For the mean absolute value of the brightness DC coefficient (… 1) A scaling factor needs to be set for simple normalization to make it numerically comparable to the other two feature values. 2) Temporal feature enhancement: To overcome the potential misjudgment caused by a single frame feature value, this module not only focuses on the instantaneous value of the feature, but also emphasizes extracting its changing trend and mutation rate in the time dimension. It calculates the difference between the current frame feature value and the average feature value of the past N frames, obtaining a set of temporal difference features representing the instantaneous rate of change, thereby capturing the dynamic abnormal mutations of the feature in the time dimension. 3) The above-mentioned static and dynamic features are combined into a set of multi-dimensional feature vectors. This vector can reflect both the feature information of the current frame and the changing trend features such as "sudden increase" and "sudden decrease", further improving the detection accuracy and robustness.

[0109] The business process of the feature processing module is as follows: DC scaling factor: As mentioned above, the values ​​of intra-predicted macroblock ratio (intra_mb_ratio) and high-frequency AC energy ratio (avg_ac_high_ratio) are both around [0,1], while the luminance DC scaling factor ( The DC coefficients are absolute values, which have significantly different dimensions and ranges. Therefore, it is necessary to appropriately scale the DC coefficients to ensure that the three characteristic values ​​are numerically comparable. This invention selects an empirical constant M (M=50) as the normalization denominator. The normalization formula for the mean absolute value of the luminance DC coefficients is as follows: ; After scaling the DC coefficients as described above, each frame can obtain three feature values. For ease of description, the intra_mb_ratio feature value is the F1 coefficient, and the feature value after scaling the mean absolute value of the DC coefficients is... Let F2 be the high-frequency AC energy ratio (avg_ac_high_ratio) and let F3 be the high-frequency AC energy ratio, for further feature processing.

[0110] Initialize the cache queue. Maintain a first-in-first-out (FIFO) cache queue of size N inside the module. Set the queue length N to 30. The queue is used to store the original feature sequence of data within approximately 1 second.

[0111] To calculate the recent feature baseline, for the current frame t, the module first calculates the arithmetic mean of each feature over the past N frames, using this as the baseline for the "recent normal state":

[0112] Calculate the temporal difference value: Calculate the difference between the current frame feature value and this baseline value to obtain a set of temporal difference features representing the relative rate of change.

[0113] Differential values ​​can effectively amplify the sudden change signal of glare. and This will generate a large positive pulse. This will generate a large negative pulse. At the same time, it can filter out slow drift caused by slow changes in ambient light or noise from the camera itself, making it more resistant to interference.

[0114] Multidimensional vector construction: The module combines the above 6 eigenvalues ​​into a six-dimensional eigenvector V(t): ; This vector contains both the static attributes (current state) and dynamic attributes (rate of change relative to recent states) of the image, providing comprehensive discriminative information for training the classifier.

[0115] The model detection module comprises two stages: offline training and real-time detection. In the offline training stage, the aforementioned bitstream parsing and feature processing modules are used to annotate and construct a dataset from a large number of collected video clips of glare overexposure and normal operations in coal mines, ultimately training a Support Vector Machine (SVM) classifier model. In the real-time detection stage, the bitstream parsing module extracts features from the H264 video stream, and the feature processing module performs feature processing. Finally, the resulting multi-dimensional vector is fed into the classifier for judgment, yielding the detection result of whether the video stream is glare overexposure.

[0116] The business process of the model detection module is as follows: Constructing an annotated dataset: Collect multiple typical locations underground in coal mines. Use a miner's lamp to directly irradiate the camera lens at different angles and intensities, and collect the resulting glare overexposure segments as positive samples. Collect a large number of normal operation videos underground as negative samples. Cut the collected video segments frame by frame, and label each frame with "overexposed" or "normal" according to the content of the picture to form an annotated dataset. Constructing a training dataset: Using the bitstream parsing module and the feature processing flow, perform bitstream parsing and feature processing on the H264 bitstreams of all video frames, and combine the feature vectors V and labels Label of each frame. Construct the final training dataset V2: ; Offline training of the model: The system of this invention uses a support vector machine (SVM) as a classifier. SVM has good generalization ability and high accuracy in small-sample and high-dimensional pattern recognition, and the final model is lightweight, suitable for real-time inference with lower resources. Given the possible non-linear relationship between feature data and labels, the radial basis function (RBF) kernel is selected as the kernel function to train the constructed training dataset V2 to obtain a classification model with a relatively high accuracy.

[0117] Real-time inference detection: After the system of this invention is deployed, perform frame-by-frame real-time parsing on the H264 video bitstream collected by the camera underground in the coal mine according to the bitstream parsing module, extract the feature values, and then process the feature values of the current frame according to the feature processing module to obtain a set of multi-dimensional vectors that can characterize the state and change trend of the current frame. Finally, input the vectors into the trained SVM classifier to obtain the detection result of "glare" or "normal" for the current video picture.

[0118] Among them, the output and control module is the "executor" and "interface" for the linkage of this invention with the upper-layer application system. When the "glare" event of the camera detected by this invention occurs, structured event information is reported in real time through a standardized industrial interface. On the one hand, it can be used as an independent alarm event for the business system to query and analyze. On the other hand, the detection result can trigger the linkage control of the business system, such as real-time pop-up alarms in the video monitoring system, and triggering video image enhancement algorithms in the video intelligent analysis system to avoid false detections.

[0119] The business process of the output and control module is as follows: Obtaining real-time detection results: Receive the real-time detection results output by the above model detection module, and obtain the state of the current video stream as "normal" or "glare".

[0120] Constructing "Glare" Event Data: When a glare event is detected, structured glare event data is constructed. The event data is in JSON string format and includes: Camera ID (which can be defined through camera configuration), Camera IP address, Event Type (Glare Detection), Confidence Level, and Occurrence Time.

[0121] Glare event MQTT broadcast: The module publishes "glare" event data to the outside world via MQTT messages. Subsequent business systems such as video surveillance and intelligent analysis can subscribe to the messages for subsequent business processing.

[0122] Glare Event HTTP Callback: The module supports timely notification and distribution of detected events via HTTP callback to business system interfaces.

[0123] Log recording: The module locally records the current "glare" events detected, which is convenient for subsequent querying and analysis.

[0124] Compared with existing technologies, this invention makes significant innovations in the following aspects: This invention realizes a method for detecting glare and overexposure in underground mine cameras based on video bitstream feature analysis; This invention innovatively utilizes the changes in encoding strategy actively adopted by the video encoder when a "glare" event occurs, thereby extracting the original feature information stored in the video stream as a detection basis; This invention differs from the traditional method of "glare" detection based on images after video decoding. The traditional method ignores a lot of useful information in the video encoding stage and requires complete decoding and image reconstruction of the video stream, which consumes a lot of computing resources. This invention utilizes the characteristic that when a video encoder encounters "glare" events such as direct light from a miner's lamp, the encoder's inter-frame prediction fails, forcing it to rely heavily on intra-frame prediction, thereby causing a sharp increase in the proportion of intra-frame prediction macroblocks. This invention utilizes the characteristics of video encoders when encountering "glare" events such as direct light from mining lamps, where the DC coefficient representing overall brightness increases sharply and the proportion of high-frequency AC coefficient representing image texture decreases sharply. In addition to using the feature information of the current frame where a "glare" event occurs, this invention also utilizes the feature information of these features changing drastically over time to prevent false detections caused by "water mist" and "dust" in the mine. This invention uses a lightweight SVG support vector machine as the classifier model; This invention supports the distribution of detection results externally via HTTP or MQTT, facilitating integration and reuse with subsequent business modules such as video surveillance and intelligent analysis.

[0125] This technical solution innovatively extracts three core features from the video stream: the proportion of intra-frame predicted macroblocks, the DC component of DCT coefficients, and the proportion of high-frequency AC energy. These features are combined with temporal difference features to construct a six-dimensional vector, and a lightweight SVM classification model is employed. This enables glare and overexposure detection from underground cameras without fully decoding the video stream, significantly reducing computational resource consumption and power consumption. It is suitable for deployment of underground edge devices and the real-time processing requirements of multiple video streams. Furthermore, leveraging the weak correlation between encoded native features and interference factors such as dust and water mist, and combining this with a detection logic that fuses static and dynamic features, the accuracy and robustness of detection in complex environments are greatly improved, effectively reducing false alarm and false negative rates. It also supports integration with back-end business systems via MQTT and HTTP interfaces, providing reliable decision-making support for video surveillance alarms and image enhancement, demonstrating excellent practicality.

[0126] Example 3 Figure 3 This is a schematic diagram of the overexposure detection device based on video bitstream characteristics provided in Embodiment 3 of this application. Figure 3 As shown, the device includes: Data receiving module 310 is used to receive video stream data; The information extraction module 320 is used to parse the video bitstream data to obtain the prediction type information, brightness coefficient information and image detail information of each macroblock in the current frame; The feature construction module 330 is used to construct multi-dimensional feature vectors based on the information changes of macroblocks between multiple frames; The overexposure detection module 340 is used to input the multidimensional feature vector into a pre-built classifier model to determine the overexposure detection result of the video bitstream data based on the output of the classifier model.

[0127] In this embodiment, a data receiving module receives video stream data; an information extraction module parses the video stream data to obtain prediction type information, brightness coefficient information, and image detail information for each macroblock in the current frame; a feature construction module constructs a multi-dimensional feature vector based on the information changes of each macroblock across multiple frames; and an overexposure detection module inputs the multi-dimensional feature vector into a pre-built classifier model to determine the overexposure detection result of the video stream data based on the output of the classifier model. This technical solution achieves overexposure detection by receiving the video stream, parsing to obtain macroblock prediction type, brightness coefficient, and image detail information, constructing a multi-dimensional feature vector based on multi-frame information changes, and then inputting it into a classifier model. This eliminates the need for complete video stream decoding, reducing computational resource consumption, adapting to complex scenarios such as underground mines, and comprehensively improving the accuracy and robustness of overexposure detection through multi-dimensional features.

[0128] The overexposure detection device based on video bitstream characteristics in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0129] The overexposure detection device based on video bitstream characteristics in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0130] The overexposure detection device based on video bitstream characteristics provided in this application can realize the various processes of the above embodiments, and will not be described again here to avoid repetition.

[0131] Example 4 like Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a program or instructions stored in the memory 402 and executable on the processor 401. When the program or instructions are executed by the processor 401, they implement the various processes of the above-described overexposure detection method embodiment based on video bitstream features and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0132] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0133] Example 5 This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described overexposure detection method based on video bitstream features and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0134] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0135] Example 6 This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described overexposure detection method embodiment based on video bitstream features, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0136] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0140] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. An overexposure detection method based on video bitstream features, characterized in that, The method includes: Receive video stream data; The video stream data is parsed to obtain the prediction type information, brightness coefficient information and image detail information of each macroblock in the current frame; Based on the information changes of macroblocks across multiple frames, a multidimensional feature vector is constructed. The multidimensional feature vector is input into a pre-built classifier model to determine the overexposure detection result of the video bitstream data based on the output of the classifier model.

2. The overexposure detection method based on video bitstream features according to claim 1, characterized in that, The video stream data is parsed to obtain the prediction type information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; The residual coefficient array of each macroblock is reconstructed using the entropy decoder; Based on the information in the NAL unit, the proportion of macroblocks in the current frame whose prediction type is intra-frame prediction is determined.

3. The overexposure detection method based on video bitstream features according to claim 1, characterized in that, The video stream data is parsed to obtain the luminance coefficient information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; Based on the information in the NAL unit and the residual coefficient array, the luminance coefficient information of each macroblock is identified.

4. The overexposure detection method based on video bitstream features according to claim 3, characterized in that, After identifying the luminance coefficient information of each macroblock based on the information in the NAL unit and the residual coefficient array, the method further includes: The brightness coefficient information is subjected to mapping transformation processing so that the brightness coefficient information is within a preset range.

5. The overexposure detection method based on video bitstream features according to claim 1, characterized in that, The video stream data is parsed to obtain the image detail information of each macroblock in the current frame, including: Initialize the internal buffer and entropy decoder; Receive and parse video stream data byte by byte, identify start codes to divide the continuous stream into multiple NAL units; Based on the information in the NAL unit and the residual coefficient array, the image detail information of each macroblock is identified.

6. The overexposure detection method based on video bitstream features according to claim 5, characterized in that, Based on the information in the NAL unit and the residual coefficient array, the image detail information of each macroblock is identified, including: The DC and AC components are extracted based on the residual coefficient array of each macroblock. The DC component in the macroblock is the first element of the residual coefficient array, which represents the average brightness of the current macroblock. The other components are AC components. Calculate the sum of all AC component coefficients in each macroblock and the sum of the high-frequency AC component coefficients, and calculate the high-frequency energy percentage.

7. The overexposure detection method based on video bitstream features according to claim 1, characterized in that, Based on the information changes of macroblocks across multiple frames, a multi-dimensional feature vector is constructed, including: Based on the information changes of each macroblock between multiple frames, as well as the prediction type information, brightness coefficient information and image detail information of the current frame, a six-dimensional feature vector composed of static and dynamic features is constructed.

8. An overexposure detection device based on video bitstream characteristics, characterized in that, The device includes: The data receiving module is used to receive video stream data; The information extraction module is used to parse the video bitstream data to obtain the prediction type information, brightness coefficient information and image detail information of each macroblock in the current frame; The feature construction module is used to construct multi-dimensional feature vectors based on the information changes of macroblocks across multiple frames; An overexposure detection module is used to input the multidimensional feature vector into a pre-built classifier model to determine the overexposure detection result of the video bitstream data based on the output of the classifier model.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the overexposure detection method based on video bitstream features as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the overexposure detection method based on video bitstream features as described in any one of claims 1-7.

Images