Video image jitter processing method and system based on rescue scene

Abstract

The present invention discloses a video image jitter processing method and system based on a rescue scene, and relates to the field of video image processing technology. In order to solve the problem that the prior art has poor jitter compensation effect in complex dynamic scenes, the processing accuracy cannot meet the needs of high-intensity rescue scenes, and it is difficult to effectively improve the clarity of the image in low-light or smoky environments, resulting in insufficient visibility of the video screen in extreme rescue scenes; the present invention effectively improves the visibility of the video screen in extreme rescue scenes through video jitter detection, video stream correction, edge processing and video data transmission steps, can accurately identify and correct the jitter in the video stream data, eliminate the edge distortion caused by jitter compensation through edge processing, brightness compensation, contrast optimization and color correction processing, significantly improve the image clarity in low-light environments, improve the stability and clarity of the video image at the rescue scene, and provide more reliable video support for high-intensity rescue operations.

Description

Technical Field

[0001] The present invention relates to the technical field of video image processing, and in particular to a video image jitter processing method and system based on a rescue scene. Background Art

[0002] In rescue operations, capturing clear and stable video images is essential for accurate situation assessment and effective coordination. Due to the chaotic nature of rescue scenes, video recordings are often affected by significant image jitter and blur, which hinders the ability to analyze and utilize the captured visual data. A Chinese patent application document with publication number CN118175353A discloses a video image jitter processing method, device, equipment and medium. The jitter processing method includes: determining a video image frame to be processed; adjusting a time domain jitter mode according to the image state of the image frame to be processed; determining a target calculation frame number corresponding to the image frame to be processed based on the time domain jitter mode; performing time domain jitter according to the target calculation frame number and combining the pixel position information of the image frame to be processed to obtain a time domain jitter result corresponding to the video, thereby reducing image jitter in static areas and improving the viewing quality of video images.

[0003] Although the above patent can reduce image jitter, it relies on the adjustment of the time domain jitter pattern and has poor jitter compensation effect in complex dynamic scenes, especially in the case of rapid motion or multi-axis jitter. The processing accuracy cannot meet the needs of high-intensity rescue scenes. In addition, in low-light or smoky environments, jitter compensation and image enhancement are not fully combined, making it difficult to effectively improve image clarity, resulting in insufficient video visibility in extreme rescue scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a video image jitter processing method and system based on the rescue scene, which meets the rescue needs of rescue personnel in complex environments, significantly improves the safety and rescue efficiency of rescue personnel, and solves the problems raised in the above-mentioned background technology. It can be used in various application scenarios such as daily fire supervision and inspection, high-speed tunnel rescue, mine rescue, unit safety inspection, and fire, chemical leakage accident rescue and detection.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The video image jitter processing method based on the rescue scene includes the following steps:

[0007] Step 1: Video jitter detection: Based on the video stream data of the rescue scene and the real-time motion data of the recorder captured by the rescue scene recorder, image jitter detection is performed on the acquired video stream data;

[0008] Step 2: Video stream correction: Based on the detection results, the jitter amplitude and jitter frequency of the video stream data are identified, and the jittered video stream data is corrected in combination with the real-time motion data;

[0009] Step 3: Edge processing: In the process of correcting the video stream data, edge processing is performed on the video stream data to obtain stable video data;

[0010] Step 4: Video data transmission: Perform brightness compensation on the edge-processed stable video data, and transmit and store the brightness-compensated stable video data in real time.

[0011] Furthermore, the image jitter detection is performed in step 1, specifically including:

[0012] Decode the video stream data obtained from the recorder and convert it into a continuous frame sequence;

[0013] Dividing the decoded continuous frames into a plurality of data segments according to a preset length, wherein each data segment contains a fixed number of frames;

[0014] Perform key feature point detection on each frame, and match key feature points of each adjacent frame based on the detection results to obtain key feature points of consecutive frames in each data segment;

[0015] Calculate the displacement vectors of key feature points of continuous frames in adjacent frames, calculate the standard deviation and extreme value statistics of all displacement vectors, and perform directional analysis and consistency analysis on the displacement vectors;

[0016] The displacement vector is compared with a preset jitter threshold. If the amplitude of the displacement vector exceeds the threshold and the directional analysis result shows that the direction of the displacement vector changes too drastically, it is considered that jitter exists.

[0017] Furthermore, the jitter amplitude and jitter frequency of the video stream data are identified, specifically including:

[0018] The jitter index is evaluated based on the standard deviation and extreme value statistics of all displacement vectors, and the jitter amplitude is determined according to the jitter index;

[0019] Perform time series analysis on the displacement vectors in each data segment and calculate the average and standard deviation of the time intervals between adjacent displacement vectors. If the standard deviation is small, it is considered that the jitter is periodic.

[0020] Calculate the statistical characteristics of the displacement vector amplitude change, determine the spectrum characteristics of the jitter frequency, and identify non-periodic jitter frequencies.

[0021] Furthermore, the video stream data detected to have jitter is corrected in combination with the real-time motion data. The specific steps are as follows:

[0022] Decoding the real-time motion data of the recorder and converting it into a motion data sequence, wherein the real-time motion data is acquired by an inertial measurement unit in the recorder;

[0023] The inertial measurement unit includes an accelerometer, a gyroscope and a magnetometer, and outputs the attitude, velocity and acceleration data of the device in real time;

[0024] Synchronously matching the decoded motion data sequence with the continuous frame sequence according to the timestamp, and obtaining the motion information corresponding to each frame based on the matching result;

[0025] Convert the real-time motion data obtained from the inertial measurement unit into the device's posture change data in three-dimensional space, and perform coordinate system conversion based on the mapping relationship between the device coordinate system and the video coordinate system;

[0026] Determine the motion trajectory of the recorder based on the motion data sequence, calculate the displacement and rotation angle data corresponding to each frame, and estimate the motion trajectory of the recorder based on the calculation results;

[0027] The displacement vector data based on image jitter detection is integrated with the real-time motion data to optimize the motion trajectory estimation result of the recorder;

[0028] Based on the optimized motion trajectory estimation results, each frame is inversely compensated and each frame image is corrected by geometric transformation.

[0029] Furthermore, the step three: edge processing is specifically as follows:

[0030] Edge detection: Perform edge detection on each frame of image, identify the edge information in each frame of image, determine the edge area in each frame of image based on the detected edge information, and perform edge enhancement and edge smoothing on the edge area;

[0031] Edge repair: Identify abnormalities in the processed edge area, determine whether there are edge breaks and missing edges in the edge area, and repair the abnormal edge area based on the judgment results;

[0032] Image smoothing: Perform noise analysis on the stable video data to determine the type and distribution characteristics of the noise, match the corresponding smoothing filter algorithm according to the type and distribution characteristics of the noise, set the parameters of the filter algorithm according to the severity of the noise, filter the stable video data based on the corresponding smoothing filter algorithm, and perform quality assessment on the filtered stable video data.

[0033] Furthermore, the step 4: video data transmission is specifically as follows:

[0034] Image enhancement: obtaining stable video data after image smoothing, performing brightness compensation on the stable video data, and performing contrast optimization and color correction on the stable video data after brightness compensation;

[0035] Real-time transmission and storage: Based on the network communication channel matching the corresponding transmission mode, the stable video data after contrast optimization and color correction is transmitted in real time based on the transmission protocol corresponding to the transmission mode. At the same time, a circular storage mechanism is constructed.

[0036] Furthermore, brightness compensation is performed on the stable video data, including:

[0037] Performing frame processing on the stable video data to obtain a frame image corresponding to the stable video data;

[0038] Extract the grayscale values ​​corresponding to all pixels contained in each frame image;

[0039] Mark the coordinate data corresponding to each pixel point;

[0040] The grayscale deviation coefficient is obtained by using the deviation between the grayscale values ​​of the pixels with the same coordinate data in each of two adjacent frame images;

[0041] The grayscale deviation coefficient is obtained by the following formula:

[0042]

[0043] Where R represents the grayscale deviation coefficient; n represents the number of pixels contained in each frame image; S bi represents the standard deviation of the gray value of the i-th pixel in all frame images; S bz Represents the standard deviation of the central grayscale value in all frame images; S bnp Represents the average standard deviation of the gray values ​​of n pixels in all frame images; L i L represents the straight-line distance between the i-th pixel of each frame image and the pixel corresponding to the central gray value; b Indicates the diagonal length of each frame image; S bxp Represents the average standard deviation of the grayscale values ​​of the pixels on the two diagonal lines of each frame image;

[0044] Comparing the grayscale deviation coefficient with a preset grayscale deviation coefficient threshold;

[0045] When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, the brightness of the frame image corresponding to the stable video data is adjusted.

[0046] Further, when the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, brightness adjustment is performed on the frame image corresponding to the stable video data, including:

[0047] When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, the grayscale value of the pixel points contained in each frame image of the video image is retrieved;

[0048] Obtaining a brightness adjustment factor corresponding to each frame image by using the grayscale value of the pixel points contained in each frame image of the video image;

[0049] The brightness adjustment factor is obtained by the following formula:

[0050]

[0051] Where K represents the brightness adjustment factor corresponding to each frame image; S bmin and S bmax Indicates the minimum and maximum standard deviations of the grayscale values ​​corresponding to the n pixels in each frame image; S xm Indicates the gray value of the pixel corresponding to the minimum standard deviation of the gray value of each frame image; S ym represents the gray value of the pixel corresponding to the maximum value of the gray value standard deviation of each frame image; n represents the number of pixels contained in each frame image; S bi represents the standard deviation of the gray value of the i-th pixel in all frame images; S bi-1 represents the standard deviation of the gray value of the i-1th pixel in all frame images; S z Represents the central gray value of each frame image; S bz Represents the standard deviation of the central grayscale value in all frame images; S i Represents the gray value of the i-th pixel in all frame images; S i-1 Represents the gray value of the i-1th pixel in all frame images;

[0052] Retrieve the grayscale deviation coefficient;

[0053] The brightness compensation coefficient corresponding to each frame image is obtained by using the brightness adjustment factor and the grayscale deviation coefficient corresponding to each frame image; wherein the brightness compensation coefficient corresponding to each frame image is obtained by the following formula:

[0054]

[0055] Wherein, B represents the brightness compensation coefficient corresponding to each frame image; B0 represents the preset initial brightness compensation coefficient; K represents the brightness adjustment factor corresponding to each frame image; R represents the grayscale deviation coefficient;

[0056] The brightness of each frame image is adjusted using the brightness compensation coefficient corresponding to each frame image, wherein the adjusted brightness value is obtained by the following formula:

[0057]

[0058] Wherein, W represents the brightness value after adjustment corresponding to each frame image; W0 represents the brightness value before adjustment corresponding to each frame image; and B represents the brightness compensation coefficient corresponding to each frame image.

[0059] The present invention provides another technical solution, a video image jitter processing system based on a rescue scene, comprising:

[0060] The recorder is used to obtain the video stream data of the rescue scene in real time, transmit the video stream data to the command platform and smart mobile terminal in real time based on the network communication channel, and obtain the off-site command and control management instructions of the smart mobile terminal. At the same time, it provides infrared thermal imaging pictures and temperature measurement data of the scene;

[0061] The command platform is used to receive the video stream data, infrared thermal imaging images and temperature measurement data collected by the recorder in real time, and locate the recorder in real time;

[0062] Intelligent mobile terminal, used to receive off-site command and control management instructions, play the video stream data of the recorder in real time, and realize real-time voice interaction with rescue personnel based on the network communication channel;

[0063] Peripheral equipment is used to connect to the recorder based on the Bluetooth communication channel to monitor the rescue personnel's vital signs data and environmental data in real time, and issue an alarm in case of abnormalities.

[0064] Furthermore, the recorder comprises:

[0065] Thermal imaging unit, used to obtain infrared thermal imaging data and ambient temperature distribution information at the rescue site;

[0066] A visible light imaging unit, used to obtain visible light images of the rescue scene;

[0067] Audio and video recording unit, used to collect and store on-site audio and video data, and conduct real-time voice intercom based on network communication channels;

[0068] A positioning communication unit, used to obtain positioning data in real time and transmit the positioning data in real time based on a network communication channel;

[0069] The video image jitter processing unit is used to perform real-time jitter detection and jitter compensation processing on the image data collected by the thermal imaging unit and the visible light imaging unit.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] Through video jitter detection, video stream correction, edge processing and video data transmission steps, the visibility of video images in extreme rescue scenarios can be effectively improved. It can not only accurately identify and correct jitter in video stream data, but also eliminate edge distortion caused by jitter compensation through edge processing, and significantly improve image clarity in low-light environments through brightness compensation, contrast optimization and color correction processing. Through real-time transmission and storage mechanism, the integrity of key video data is guaranteed when storage space is limited, which can significantly improve the stability and clarity of video images at the rescue site and provide more reliable video support for high-intensity rescue operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A step diagram of a video image jitter processing method based on a rescue scene according to the present invention;

[0073] Figure 2 It is a module diagram of the video image jitter processing system based on the rescue scene of the present invention;

[0074] Figure 3 This is a module diagram of the recorder of the present invention. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] In order to solve the technical problem that the existing technology has poor jitter compensation effect in complex dynamic scenes, especially in the case of fast movement or multi-axis jitter, the processing accuracy cannot meet the needs of high-intensity rescue scenes, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0077] The video image jitter processing method based on the rescue scene includes the following steps:

[0078] Step 1: Video jitter detection: Based on the video stream data of the rescue scene and the real-time motion data of the recorder captured by the rescue scene recorder, image jitter detection is performed on the acquired video stream data;

[0079] Step 2: Video stream correction: Based on the detection results, the jitter amplitude and jitter frequency of the video stream data are identified, and the jittered video stream data is corrected in combination with the real-time motion data;

[0080] Step 3: Edge processing: In the process of correcting the video stream data, edge processing is performed on the video stream data to obtain stable video data to eliminate edge distortion caused by jitter compensation;

[0081] Step 4: Video data transmission: Brightness compensation is performed on the edge-processed stable video data to improve the image clarity in low-light environments, and the stable video data after brightness compensation is transmitted and stored in real time. Multiple transmission protocols are supported, and the processed stable video is transmitted to the command platform or smart terminal. Specifically:

[0082] Image enhancement: obtaining stable video data after image smoothing, performing brightness compensation on the stable video data, and performing contrast optimization and color correction on the stable video data after brightness compensation;

[0083] Real-time transmission and storage: Based on the network communication channel matching the corresponding transmission mode, the stable video data after contrast optimization and color correction will be transmitted in real time based on the transmission protocol corresponding to the transmission mode. At the same time, a circular storage mechanism is built to ensure that key video data will not be lost when storage space is limited.

[0084] In this embodiment, through video jitter detection, video stream correction, edge processing and video data transmission steps, the visibility of video images in extreme rescue scenarios is effectively improved. Not only can the jitter in the video stream data be accurately identified and corrected, but the edge distortion caused by jitter compensation can also be eliminated through edge processing. The image clarity in low-light environments is significantly improved through brightness compensation, contrast optimization and color correction processing. Through the real-time transmission and storage mechanism, the integrity of key video data is guaranteed when storage space is limited. The stability and clarity of the video image at the rescue site can be significantly improved, providing more reliable video support for high-intensity rescue operations.

[0085] In this embodiment, the step 1 of performing image jitter detection specifically includes:

[0086] Decode the video stream data obtained from the recorder and convert it into a continuous frame sequence;

[0087] Dividing the decoded continuous frames into a plurality of data segments according to a preset length, wherein each data segment contains a fixed number of frames;

[0088] Perform key feature point detection on each frame, and match key feature points of each adjacent frame based on the detection results to obtain key feature points of consecutive frames in each data segment;

[0089] Calculate the displacement vectors of key feature points of continuous frames in adjacent frames, calculate the standard deviation and extreme value statistics (maximum value, minimum value, etc.) of all displacement vectors, and perform directional analysis and consistency analysis on the displacement vectors;

[0090] The displacement vector is compared with a preset jitter threshold. If the amplitude of the displacement vector exceeds the threshold and the directional analysis result shows that the direction of the displacement vector changes too dramatically, for example, by calculating the angular difference of the direction change, it is considered that jitter exists.

[0091] In this embodiment, by detecting and matching key feature points of continuous frame sequences, the jitter in the video stream can be accurately identified, thereby providing a reliable data basis for subsequent jitter compensation, meeting the high real-time requirements of the rescue site, and being able to effectively distinguish between normal camera movement and jitter. Even in complex and changeable rescue scenarios, it can maintain high robustness, and determine whether there is jitter by comparing the jitter threshold combined with the angle difference of the direction change, thereby reducing misjudgment caused by changes in scene content and ensuring that compensation is performed only when jitter actually exists.

[0092] In this embodiment, identifying the jitter amplitude and jitter frequency of the video stream data specifically includes:

[0093] The jitter index is evaluated based on the standard deviation and extreme value statistics of all displacement vectors. The jitter amplitude is determined based on the jitter index. The jitter amplitude is divided into different levels such as mild, moderate, and severe according to the size of the jitter index.

[0094] Jitter index includes:

[0095] The modulus of the average displacement vector, that is, the average value of the lengths of all displacement vectors, reflects the average amplitude of the overall jitter. The longer the modulus, the more obvious the jitter.

[0096] The standard deviation of the displacement vector is a statistic that measures the degree of data dispersion and reflects the degree of fluctuation of the displacement vector. The larger the standard deviation, the more unstable the jitter and the greater the amplitude variation.

[0097] Perform time series analysis on the displacement vectors in each data segment and calculate the average and standard deviation of the time intervals between adjacent displacement vectors. If the standard deviation is small, it is considered that the jitter is periodic, and the jitter frequency is estimated by the following formula:

[0098]

[0099] in, Expressed as the jitter frequency, Expressed as time interval average;

[0100] Calculate the statistical characteristics of the displacement vector amplitude change, such as the rate of change of the displacement vector amplitude over time, the difference between the maximum and minimum values ​​of the displacement vector amplitude, etc., analyze the autocorrelation function of the displacement vector time series to determine the self-similarity and repetitive pattern of the jitter, determine the spectral characteristics of the jitter frequency, and identify non-periodic jitter frequencies.

[0101] In this embodiment, by detecting and correcting the jitter of the video stream, a more stable video image can be obtained, and a time series analysis is performed on the displacement vector in each data segment. The jitter index is evaluated in combination with the standard deviation and extreme value statistics of the displacement vector to help judge the amplitude level of the jitter, so as to determine the self-similarity and repetitive pattern of the jitter, and then identify non-periodic jitter frequencies, which helps rescue personnel better observe and understand the on-site situation in an emergency.

[0102] In this embodiment, the acquired real-time video stream data is corrected in combination with the real-time motion data, and the specific steps are as follows:

[0103] Decoding the real-time motion data of the recorder and converting it into a motion data sequence, wherein the real-time motion data is acquired by an inertial measurement unit in the recorder;

[0104] The inertial measurement unit includes an accelerometer, a gyroscope and a magnetometer, and outputs the attitude, velocity and acceleration data of the device in real time;

[0105] Synchronously matching the decoded motion data sequence with the continuous frame sequence according to the timestamp, and obtaining the motion information corresponding to each frame based on the matching result;

[0106] Convert the real-time motion data obtained from the inertial measurement unit into the device's posture change data in three-dimensional space, and perform coordinate system conversion based on the mapping relationship between the device coordinate system and the video coordinate system;

[0107] Determine the motion trajectory of the recorder based on the motion data sequence, calculate the displacement and rotation angle data corresponding to each frame, and estimate the motion trajectory of the recorder based on the calculation results;

[0108] The displacement vector data based on image jitter detection is integrated with the real-time motion data to optimize the motion trajectory estimation result of the recorder and reduce the error caused by a single data source;

[0109] Based on the optimized motion trajectory estimation results, each frame is inversely compensated and each frame image is corrected by geometric transformation.

[0110] In this embodiment, by dynamically selecting multiple algorithms according to the jitter amplitude, a digital image stabilization algorithm is preferentially used to quickly process slight jitter, a motion vector estimation algorithm is used for moderate jitter, compensation is performed based on the motion vectors of adjacent frames, and a Kalman filter is used for severe jitter. A more accurate compensation is achieved through a prediction and correction mechanism to eliminate image offset and rotation caused by device jitter, ensure image stability, effectively improve compensation accuracy, eliminate image blur caused by device movement, improve image clarity, output the device's posture, speed and acceleration data in real time, and perform correction processing in real time, thereby ensuring the real-time nature of video stream data, providing a better viewing experience, and helping to more accurately understand the on-site situation.

[0111] In this embodiment, the step three: edge processing, specifically includes:

[0112] Edge detection: Perform edge detection on each frame of image, identify the edge information in each frame of image, determine the edge area in each frame of image based on the detected edge information, and perform edge enhancement and edge smoothing on the edge area;

[0113] Edge repair: Identify abnormalities in the processed edge area, determine whether there are edge breaks and missing edges in the edge area, and repair the abnormal edge area based on the judgment results;

[0114] Image smoothing: Perform noise analysis on the stable video data to determine the type and distribution characteristics of the noise. According to the type and distribution characteristics of the noise, match the corresponding smoothing filter algorithm, including mean filtering, median filtering, Gaussian filtering, etc., and set the parameters of the filtering algorithm according to the severity of the noise. For example, in mean filtering, the size and shape of the filter window can be set; in Gaussian filtering, the standard deviation of the Gaussian kernel can be set, etc. The stable video data is filtered based on the corresponding smoothing filter algorithm, and the quality of the filtered stable video data is evaluated.

[0115] In this embodiment, it is ensured that during the video image jitter compensation process, the edge information of the image is effectively protected and enhanced, and the noise of the stable video data is effectively removed, thereby improving the overall quality and visibility of the video data, ensuring that important information in the image is not lost due to edge breakage or missing, and improving the speed of data processing while ensuring image quality, providing more reliable and high-quality visual information for rescue operations.

[0116] Specifically, brightness compensation is performed on the stable video data, including:

[0117] Performing frame processing on the stable video data to obtain a frame image corresponding to the stable video data;

[0118] Extract the grayscale values ​​corresponding to all pixels contained in each frame image;

[0119] Mark the coordinate data corresponding to each pixel point;

[0120] The grayscale deviation coefficient is obtained by using the deviation between the grayscale values ​​of the pixels with the same coordinate data in each of two adjacent frame images;

[0121] The grayscale deviation coefficient is obtained by the following formula:

[0122]

[0123] Where R represents the grayscale deviation coefficient; n represents the number of pixels contained in each frame image; S bi represents the standard deviation of the gray value of the i-th pixel in all frame images; S bz Represents the standard deviation of the central grayscale value in all frame images; S bnp Represents the average standard deviation of the gray values ​​of n pixels in all frame images; L i L represents the straight-line distance between the i-th pixel of each frame image and the pixel corresponding to the central gray value; b Indicates the diagonal length of each frame image; S bxp Represents the average standard deviation of the grayscale values ​​of the pixels on the two diagonal lines of each frame image;

[0124] Comparing the grayscale deviation coefficient with a preset grayscale deviation coefficient threshold;

[0125] When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, the brightness of the frame image corresponding to the stable video data is adjusted.

[0126] The technical effect of the above technical solution is: by calculating the grayscale deviation coefficient, the solution can quantify the fluctuation of the grayscale value of pixels between adjacent frame images in the video data. When the grayscale deviation coefficient exceeds the preset threshold, the brightness of the frame image is adjusted, which helps to reduce the sudden brightness change in the video and improve the stability of the overall brightness. The calculation of the grayscale deviation coefficient takes into account multiple factors, including the standard deviation of the grayscale value of the pixel, the standard deviation of the center grayscale value, the straight-line distance from the pixel to the pixel corresponding to the center grayscale value, etc. These factors work together in the brightness adjustment decision, which helps to retain the image details while adjusting the brightness, and may optimize the contrast to a certain extent, making the video picture clearer. By statistically analyzing the grayscale values ​​of all pixels in the frame image, the solution can identify and reduce the abnormal fluctuation of the grayscale value caused by noise, thereby reducing the impact of noise on the video quality. The solution adopts a method of dynamically calculating the grayscale deviation coefficient and decides whether to adjust the brightness based on the calculation result, which makes the solution have strong adaptability to video data with different lighting conditions and different scenes, and can more flexibly respond to various video processing requirements. Each step of the above technical solution is performed on a frame image of stable video data, and the calculation of the grayscale deviation coefficient depends on the comparison of the grayscale values ​​of pixels between frame images, which largely ensures the feasibility and real-time performance of the calculation. With the support of high-performance computing equipment, this solution can realize real-time processing of stable video data.

[0127] In summary, this technical solution shows good performance indicators and technical effects in terms of brightness stability, detail retention, contrast optimization, noise reduction, adaptability and computational efficiency, which helps to improve the quality and efficiency of video processing.

[0128] Specifically, when the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, brightness adjustment is performed on the frame image corresponding to the stable video data, including:

[0129] When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, the grayscale value of the pixel points contained in each frame image of the video image is retrieved;

[0130] Obtaining a brightness adjustment factor corresponding to each frame image by using the grayscale value of the pixel points contained in each frame image of the video image;

[0131] The brightness adjustment factor is obtained by the following formula:

[0132]

[0133] Where K represents the brightness adjustment factor corresponding to each frame image; S bmin and S bmaxIndicates the minimum and maximum standard deviations of the grayscale values ​​corresponding to the n pixels in each frame image; S xm Indicates the gray value of the pixel corresponding to the minimum standard deviation of the gray value of each frame image; S ym represents the gray value of the pixel corresponding to the maximum value of the gray value standard deviation of each frame image; n represents the number of pixels contained in each frame image; S bi represents the standard deviation of the gray value of the i-th pixel in all frame images; S bi-1 represents the standard deviation of the gray value of the i-1th pixel in all frame images; S z Represents the central gray value of each frame image; S bz Represents the standard deviation of the central grayscale value in all frame images; S i Represents the gray value of the i-th pixel in all frame images; S i-1 Represents the gray value of the i-1th pixel in all frame images;

[0134] Retrieve the grayscale deviation coefficient;

[0135] The brightness compensation coefficient corresponding to each frame image is obtained by using the brightness adjustment factor and the grayscale deviation coefficient corresponding to each frame image; wherein the brightness compensation coefficient corresponding to each frame image is obtained by the following formula:

[0136]

[0137] Wherein, B represents the brightness compensation coefficient corresponding to each frame image; B0 represents the preset initial brightness compensation coefficient; K represents the brightness adjustment factor corresponding to each frame image; R represents the grayscale deviation coefficient;

[0138] The brightness of each frame image is adjusted using the brightness compensation coefficient corresponding to each frame image, wherein the adjusted brightness value is obtained by the following formula:

[0139]

[0140] Wherein, W represents the brightness value after adjustment corresponding to each frame image; W0 represents the brightness value before adjustment corresponding to each frame image; and B represents the brightness compensation coefficient corresponding to each frame image.

[0141] The technical effect of the above technical solution is: when the grayscale deviation coefficient exceeds the preset threshold, the solution can dynamically calculate the brightness adjustment factor according to the grayscale value of each pixel of each frame image, and then obtain the brightness compensation coefficient to achieve accurate adjustment of the brightness of the video frame image. This dynamic adjustment capability helps to adapt to different lighting conditions and scene changes, and maintain the brightness stability and consistency of the video picture. By comprehensively considering the grayscale value standard deviation, grayscale value and its change trend of the pixel points in the frame image, the solution can more accurately identify the area where the brightness needs to be adjusted, so as to retain the image details while adjusting the brightness, and may enhance the contrast to a certain extent, making the video picture more vivid and clear. The calculation of the brightness adjustment factor and the brightness compensation coefficient takes into account multiple factors, including the grayscale value standard deviation, grayscale value and its change trend, center grayscale value, etc., which makes the solution have strong adaptability to video data under different lighting conditions and different scenes. No matter how the video content changes, the solution can adjust the brightness according to the specific situation to maintain the high quality of the video picture. The brightness adjustment factor and the brightness compensation coefficient in the solution can be calculated by the above formula, and the parameters in the formula can be adjusted according to the actual application requirements. This flexibility enables the solution to be adapted to different video processing scenarios and requirements, providing users with more choices and customization space.

[0142] In summary, this technical solution has demonstrated good performance indicators and technical effects in terms of dynamic brightness adjustment capability, detail retention and contrast enhancement, strong adaptability, computational efficiency and real-time performance, and flexibility, which helps to improve the quality and efficiency of video processing and meet actual application needs.

[0143] In order to better demonstrate the video image jitter processing method based on the rescue scene, please refer to Figure 2-Figure 3 The present invention provides a video image jitter processing system based on a rescue scene, comprising:

[0144] The recorder is used to obtain the video stream data of the rescue scene in real time, transmit the video stream data to the command platform and smart mobile terminal in real time based on the network communication channel, and obtain the off-site command and control management instructions of the smart mobile terminal. At the same time, it provides infrared thermal imaging pictures and temperature measurement data of the scene;

[0145] The recorder comprises:

[0146] Thermal imaging unit, used to obtain infrared thermal imaging data and ambient temperature distribution information at the rescue site;

[0147] A visible light imaging unit, used to obtain visible light images of the rescue scene;

[0148] Audio and video recording unit, used to collect and store on-site audio and video data, support real-time playback, and conduct real-time voice intercom based on network communication channels;

[0149] A positioning communication unit, used to obtain positioning data in real time and transmit the positioning data in real time based on a network communication channel;

[0150] The video image jitter processing unit is used to perform real-time jitter detection and jitter compensation processing on the image data collected by the thermal imaging unit and the visible light imaging unit to ensure stable and clear video output;

[0151] The command platform is used to receive the video stream data, infrared thermal imaging images and temperature measurement data collected by the recorder in real time, and locate the recorder in real time;

[0152] Intelligent mobile terminal, used to receive off-site command and control management instructions, play the video stream data of the recorder in real time, and realize real-time voice interaction with rescue personnel based on the network communication channel;

[0153] Peripheral equipment, including toxic gas detectors, air calling equipment and heart rate monitoring bracelets, are used to connect to the recorder based on the Bluetooth communication channel to monitor the vital signs data of rescuers and the environmental data in real time, and issue an alarm in case of abnormalities.

[0154] In this embodiment, through the recorder, the video stream data of the rescue scene can be transmitted to the command platform and the smart mobile terminal in real time, ensuring the timeliness of the information and helping to make rescue decisions quickly. Combined with infrared thermal imaging and visible light imaging, it can provide more comprehensive on-site information, collect and store on-site audio and video data, conduct real-time voice intercom, obtain and transmit positioning data in real time, improve the flexibility and efficiency of on-site command, monitor the vital signs and environmental data of rescue personnel in real time, and promptly alarm under abnormal circumstances to ensure the safety of rescue personnel.

[0155] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A video image jitter processing method based on a rescue scene, characterized in that: The following steps are involved: Step 1: Video jitter detection: Based on the video stream data of the rescue scene and the real-time motion data of the recorder captured by the rescue scene recorder, image jitter detection is performed on the acquired video stream data; Step 2: Video stream correction: Based on the detection results, the jitter amplitude and jitter frequency of the video stream data are identified, and the jittered video stream data is corrected in combination with the real-time motion data; Step 3: Edge processing: In the process of correcting the video stream data, edge processing is performed on the video stream data to obtain stable video data; Step 4: Video data transmission: Brightness compensation is performed on the edge-processed stable video data, and the brightness-compensated stable video data is transmitted and stored in real time; The step 1 of performing image jitter detection specifically includes: Decode the video stream data obtained from the recorder and convert it into a continuous frame sequence; Dividing the decoded continuous frames into a plurality of data segments according to a preset length, wherein each data segment contains a fixed number of frames; Perform key feature point detection on each frame, and match key feature points of each adjacent frame based on the detection results to obtain key feature points of consecutive frames in each data segment; Calculate the displacement vectors of key feature points of continuous frames in adjacent frames, calculate the standard deviation and extreme value statistics of all displacement vectors, and perform directional analysis and consistency analysis on the displacement vectors; The displacement vector is compared with a preset jitter threshold. If the magnitude of the displacement vector exceeds the threshold and the directional analysis result shows that the direction of the displacement vector changes too dramatically, it is considered that jitter exists. Identify the jitter amplitude and jitter frequency of video stream data, including: The jitter index is evaluated based on the standard deviation and extreme value statistics of all displacement vectors, and the jitter amplitude is determined according to the jitter index; Perform time series analysis on the displacement vectors in each data segment and calculate the average and standard deviation of the time intervals between adjacent displacement vectors. If the standard deviation is small, it is considered that the jitter is periodic. Calculate the statistical characteristics of the displacement vector amplitude change, determine the spectrum characteristics of the jitter frequency, and identify non-periodic jitter frequencies; Combined with real-time motion data, correct the video stream data detected to have jitter. The specific steps are as follows: Decoding the real-time motion data of the recorder and converting it into a motion data sequence, wherein the real-time motion data is acquired by an inertial measurement unit in the recorder; The inertial measurement unit includes an accelerometer, a gyroscope and a magnetometer, and outputs the attitude, velocity and acceleration data of the device in real time; Synchronously matching the decoded motion data sequence with the continuous frame sequence according to the timestamp, and obtaining the motion information corresponding to each frame based on the matching result; Convert the real-time motion data obtained from the inertial measurement unit into the device's posture change data in three-dimensional space, and perform coordinate system conversion based on the mapping relationship between the device coordinate system and the video coordinate system; Determine the motion trajectory of the recorder based on the motion data sequence, calculate the displacement and rotation angle data corresponding to each frame, and estimate the motion trajectory of the recorder based on the calculation results; The displacement vector data based on image jitter detection is integrated with the real-time motion data to optimize the motion trajectory estimation result of the recorder; Based on the optimized motion trajectory estimation results, each frame is inversely compensated and each frame image is corrected by geometric transformation.

2. The video image jitter processing method based on the rescue scene according to claim 1, characterized in that: The step three: edge processing, specifically: Edge detection: Perform edge detection on each frame of image, identify the edge information in each frame of image, determine the edge area in each frame of image based on the detected edge information, and perform edge enhancement and edge smoothing on the edge area; Edge repair: Identify abnormalities in the processed edge area, determine whether there are edge breaks and missing edges in the edge area, and repair the abnormal edge area based on the judgment results; Image smoothing: Perform noise analysis on the restored stable video data to determine the type and distribution characteristics of the noise, match the corresponding smoothing filter algorithm according to the type and distribution characteristics of the noise, set the parameters of the filter algorithm according to the severity of the noise, filter the stable video data based on the corresponding smoothing filter algorithm, and perform quality assessment on the stable video data after filtering.

3. The video image jitter processing method based on the rescue scene according to claim 2, characterized in that: The step 4: video data transmission, specifically includes: Image enhancement: obtaining stable video data after image smoothing, performing brightness compensation on the stable video data, and performing contrast optimization and color correction on the stable video data after brightness compensation; Real-time transmission and storage: Based on the network communication channel matching the corresponding transmission mode, the stable video data after contrast optimization and color correction is transmitted in real time based on the transmission protocol corresponding to the transmission mode. At the same time, a circular storage mechanism is constructed.

4. The video image jitter processing method based on the rescue scene according to claim 3, characterized in that: Perform brightness compensation on stable video data, including: Performing frame processing on the stable video data to obtain a frame image corresponding to the stable video data; Extract the grayscale values ​​corresponding to all pixels contained in each frame image; Mark the coordinate data corresponding to each pixel point; The grayscale deviation coefficient is obtained by using the deviation between the grayscale values ​​of the pixels with the same coordinate data in each of two adjacent frame images; The grayscale deviation coefficient is obtained by the following formula: ； Where R represents the grayscale deviation coefficient; n represents the number of pixels contained in each frame image; S bi represents the standard deviation of the gray value of the i-th pixel in all frame images; S bz Represents the standard deviation of the central grayscale value in all frame images; S bnp Represents the average standard deviation of the gray values ​​of n pixels in all frame images; L i L represents the straight-line distance between the i-th pixel of each frame image and the pixel corresponding to the central gray value; d Indicates the diagonal length of each frame image; S bxp Represents the average standard deviation of the grayscale values ​​of the pixels on the two diagonal lines of each frame image; Comparing the grayscale deviation coefficient with a preset grayscale deviation coefficient threshold; When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, the brightness of the frame image corresponding to the stable video data is adjusted.

5. The video image jitter processing method based on the rescue scene according to claim 4, characterized in that: When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, brightness adjustment is performed on the frame image corresponding to the stable video data, including: When the grayscale deviation coefficient exceeds a preset grayscale deviation coefficient threshold, the grayscale value of the pixel points contained in each frame image of the video image is retrieved; Obtaining a brightness adjustment factor corresponding to each frame image by using the grayscale value of the pixel points contained in each frame image of the video image; The brightness adjustment factor is obtained by the following formula: ； Where K represents the brightness adjustment factor corresponding to each frame image; S bmin and S bmax Indicates the minimum and maximum standard deviations of the grayscale values ​​corresponding to the n pixels in each frame image; S xm Indicates the gray value of the pixel corresponding to the minimum standard deviation of the gray value of each frame image; S ym represents the gray value of the pixel corresponding to the maximum value of the gray value standard deviation of each frame image; n represents the number of pixels contained in each frame image; S bi represents the standard deviation of the gray value of the i-th pixel in all frame images; S bi-1 represents the standard deviation of the gray value of the i-1th pixel in all frame images; S z Represents the central gray value of each frame image; S bz Represents the standard deviation of the central grayscale value in all frame images; S i Represents the gray value of the i-th pixel in all frame images; S i-1 Represents the gray value of the i-1th pixel in all frame images; Retrieve the grayscale deviation coefficient; The brightness compensation coefficient corresponding to each frame image is obtained by using the brightness adjustment factor and the grayscale deviation coefficient corresponding to each frame image; wherein the brightness compensation coefficient corresponding to each frame image is obtained by the following formula: ； Wherein, B represents the brightness compensation coefficient corresponding to each frame image; B0 represents the preset initial brightness compensation coefficient; K represents the brightness adjustment factor corresponding to each frame image; R represents the grayscale deviation coefficient; The brightness of each frame image is adjusted using the brightness compensation coefficient corresponding to each frame image, wherein the adjusted brightness value is obtained by the following formula: ； Wherein, W represents the brightness value after adjustment corresponding to each frame image; W0 represents the brightness value before adjustment corresponding to each frame image; and B represents the brightness compensation coefficient corresponding to each frame image.

6. A video image jitter processing system based on a rescue scene, applied in the video image jitter processing method based on a rescue scene as claimed in claim 5, characterized in that: include: The recorder is used to obtain the video stream data of the rescue scene in real time, transmit the video stream data to the command platform and smart mobile terminal in real time based on the network communication channel, and obtain the off-site command and control management instructions of the smart mobile terminal. At the same time, it provides infrared thermal imaging pictures and temperature measurement data of the scene; The command platform is used to receive the video stream data, infrared thermal imaging images and temperature measurement data collected by the recorder in real time, and locate the recorder in real time; Intelligent mobile terminal, used to receive off-site command and control management instructions, play the video stream data of the recorder in real time, and realize real-time voice interaction with rescue personnel based on the network communication channel; Peripheral equipment is used to connect to the recorder based on the Bluetooth communication channel to monitor the rescue personnel's vital signs data and environmental data in real time, and issue an alarm in case of abnormalities.

7. The video image jitter processing system based on the rescue scene as claimed in claim 6, characterized in that: Recorder, including: Thermal imaging unit, used to obtain infrared thermal imaging data and ambient temperature distribution information at the rescue site; A visible light imaging unit, used to obtain visible light images of the rescue scene; Audio and video recording unit, used to collect and store on-site audio and video data, and conduct real-time voice intercom based on network communication channels; A positioning communication unit, used to obtain positioning data in real time and transmit the positioning data in real time based on a network communication channel; The video image jitter processing unit is used to perform real-time jitter detection and jitter compensation processing on the image data collected by the thermal imaging unit and the visible light imaging unit.

Images