A remote video monitoring method, system and device with intelligent zooming
By processing video and audio data in frames, assessing noise interference and depth of field effects, and adjusting filtering strength, the problem of low image acquisition accuracy in harsh environments of traditional monitoring systems is solved, achieving higher monitoring accuracy and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUIZHOU ROAD & BRIDGE GRP
- Filing Date
- 2026-02-28
- Publication Date
- 2026-05-19
AI Technical Summary
Traditional mobile sentry monitoring systems are susceptible to noise interference in harsh weather and complex road and bridge infrastructure environments, which reduces monitoring accuracy. Furthermore, traditional adaptive image denoising algorithms may misjudge inconsistencies in depth of field as noise, leading to feature distortion when denoising intensity is increased.
By processing video and audio data in frames, the degree of external noise interference is evaluated. By combining the temporal and frequency domain energy distribution of audio data and the grayscale changes of video images, the blur coefficient and texture difference coefficient are obtained. The filtering intensity is adjusted to adapt to noise interference, and the blur caused by inconsistent depth of field and noise interference is distinguished.
It improves image denoising performance, preserves monitoring feature information, avoids feature distortion caused by inconsistent depth of field, and enhances the accuracy and reliability of remote video monitoring.
Smart Images
Figure CN121751009B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video surveillance technology, specifically to a remote video surveillance method, system, and device with intelligent zoom capability. Background Technology
[0002] With the continuous development of my country's road and bridge industry, the automation level of road and bridge monitoring systems is constantly improving. Traditional fixed video surveillance systems often fail to meet the needs of remote road and bridge facilities due to long deployment cycles, poor flexibility, and reliance on fixed networks and power supplies. Mobile sentry monitoring, as a highly integrated portable intelligent monitoring solution, integrates video surveillance, power supply, transmission, storage, and AI intelligent analysis functions, making it suitable for temporary or long-term monitoring needs.
[0003] However, in practical applications, the video captured by the mobile sentry monitoring platform can be affected by factors such as severe weather, leading to noise and blurring in the monitored images. Furthermore, due to the complex structure and small defects of road and bridge facilities, zooming and focusing operations are required during monitoring, which may cause blurring in local areas due to inconsistent depth of field. Traditional adaptive image denoising algorithms may misjudge blurring caused by inconsistent depth of field as excessive noise, increasing the denoising intensity and thus distorting features, affecting monitoring accuracy and resulting in missed or false detections during the monitoring of road and bridge facilities. Summary of the Invention
[0004] In view of the above, it is necessary to provide a remote video monitoring method, system, and device with intelligent zoom capability. Compared with traditional remote video monitoring methods with intelligent zoom capability, this method improves the accuracy of assessing the possibility of image interference from real external noise and enhances image denoising performance, thereby improving the accuracy and reliability of remote video monitoring.
[0005] In a first aspect, embodiments of this application provide a remote video monitoring method with intelligent zoom capability, the method comprising the following steps:
[0006] The monitoring equipment is used to collect video of the area to be monitored in real time and perform frame segmentation processing, and the audio data of the location of the monitoring equipment is collected in real time.
[0007] The system presets time intervals for evaluating external noise interference in video images. By analyzing the energy distribution of audio data in the time and frequency domains within each time interval, and the differences in grayscale changes across different regions of the video image over time, it obtains blur coefficients for each time interval to assess the likelihood that the blurring in the video image within each time interval is caused by noise. By analyzing the texture feature differences between different regions in each frame of the video image within each time interval, it obtains texture difference coefficients for each time interval. Combined with the length and positional distribution of edges in the video image within each time interval, it obtains influence coefficients for each time interval to assess the likelihood that the video image blurring is caused by inconsistent depth of field. Finally, by combining the blur coefficients and influence coefficients, it obtains the degree of external noise interference in each time interval.
[0008] By adjusting the filtering intensity of the video images in each time interval based on the external noise interference, remote video monitoring of the area to be monitored can be performed based on the filtered video images.
[0009] In one embodiment, the process of obtaining the fuzzy coefficient is as follows:
[0010] The grayscale dispersion of each region in each frame of video image within each time interval is obtained, the degree of change of the grayscale dispersion of each region in video image within each time interval is measured in time, and the cumulative value of the difference of the measurement results of the degree of change between different regions in video image within each time interval is calculated.
[0011] The noise interference coefficient of audio data in each time interval is obtained by measuring the energy distribution of audio data in the time and frequency domains within each time interval.
[0012] The fuzziness coefficient is positively correlated with the accumulated value and the audio data noise interference coefficient.
[0013] In one embodiment, the process of obtaining the audio data noise interference coefficient is as follows:
[0014] Obtain the goodness of fit of the fitting curve of the audio energy envelope curve of the audio data in each time interval;
[0015] Calculate the signal bandwidth of the audio data within each time interval;
[0016] The audio data noise interference coefficient is directly proportional to the signal bandwidth and inversely proportional to the goodness of fit.
[0017] In one embodiment, the process of obtaining the texture difference coefficient is as follows:
[0018] Obtain the LBP value of each region in each frame of video image, and calculate the sum of the differences in LBP values between different regions in each frame of video image;
[0019] The texture difference coefficient is the summation result of the accumulated sums of all frames of video images within each time interval.
[0020] In one embodiment, the process of obtaining the influence coefficient is as follows:
[0021] Calculate the average length of all edges in all frames of video images within each time interval;
[0022] Calculate the mean of the Euclidean distances between all edge center points and their nearest neighbor edge center points in each frame of video image, and calculate the arithmetic mean of the mean values in all frames of video image within each time interval;
[0023] The influence coefficient is directly proportional to the texture difference coefficient and the arithmetic mean, and inversely proportional to the average value.
[0024] In one embodiment, the method for calculating the degree of influence coefficient is as follows:
[0025] The ratio of the arithmetic mean to the average value is calculated, and the influence coefficient is the product of the ratio and the texture difference coefficient.
[0026] In one embodiment, the external noise interference degree is the normalized value of the ratio of the ambiguity coefficient to the influence degree coefficient.
[0027] In one embodiment, adjusting the filtering intensity of video images within each time interval includes:
[0028] A denoising algorithm is used to denoise each frame of video image within each time interval. The denoising parameter in the denoising algorithm is the product of its preset initial value and the external noise interference degree of the time interval where each frame of video image is located.
[0029] Secondly, embodiments of this application also provide a remote video surveillance system with intelligent zoom capability, the system comprising:
[0030] The information acquisition module is used to acquire video of the area to be monitored in real time using the monitoring equipment, and to perform frame-by-frame processing, and to acquire audio data of the location of the monitoring equipment in real time.
[0031] The noise assessment module is used to preset time intervals for assessing external noise interference in video images. It obtains blur coefficients for each time interval by analyzing the energy distribution of audio data in the time and frequency domains, and the differences in grayscale changes between different regions of the video image over time. This is used to assess the likelihood that the blurring of the video image in each time interval is caused by noise. It also obtains texture difference coefficients for each time interval by analyzing the texture feature differences between different regions in each frame of the video image within each time interval. Combined with the length and positional distribution of edges in the video image within each time interval, it obtains influence coefficients for each time interval. This is used to assess the likelihood that the video image blurring is caused by inconsistent depth of field within each time interval. Finally, it combines the blur coefficients and influence coefficients to obtain the degree of external noise interference in each time interval.
[0032] The noise reduction and monitoring module is used to adjust the filtering intensity of the video images in each time interval based on the external noise interference level, so as to perform remote video monitoring of the area to be monitored based on the filtered video images.
[0033] Thirdly, embodiments of this application also provide a remote video monitoring device with intelligent zoom capability, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any of the above-described remote video monitoring methods with intelligent zoom capability.
[0034] This application has at least the following beneficial effects:
[0035] This application divides video and audio data into multiple time intervals for analysis, which can capture the changes in environmental noise in more detail; by combining the time and frequency domain energy distribution of audio data and the grayscale changes of video images, noise interference is comprehensively evaluated from multiple dimensions, so that the blur coefficient can more accurately represent the degree of image blur caused by noise interference, providing a reliable basis for subsequent noise interference degree assessment.
[0036] Furthermore, by calculating the differences in texture features between different regions in a video image, the blurring caused by inconsistent depth of field can be accurately reflected, which helps to distinguish between blurring caused by inconsistent depth of field and blurring caused by noise interference. Considering that edge features will change significantly when the depth of field is inconsistent, the length and position distribution of edges in the video image, as well as the differences in texture features, are combined to further enhance the accuracy of the assessment of the degree of influence of depth of field. This allows for a more accurate assessment of the degree of blurring caused by inconsistent depth of field, providing a more comprehensive reference for subsequent noise interference assessment.
[0037] Furthermore, by combining the blur coefficient and the influence coefficient, the possibility of the image being interfered with by real external noise in each time interval is evaluated, thereby improving the accuracy of the evaluation of the possibility of the image being interfered with by real external noise. Then, the denoising parameters in the adaptive denoising algorithm are adjusted according to the possibility of the image being interfered with by real external noise. This can retain more feature information while ensuring the denoising effect, and avoid misjudging the blur caused by inconsistent depth of field as noise, which would lead to over-filtering and image feature distortion, thereby improving the accuracy and reliability of remote video monitoring. Attached Figure Description
[0038] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 A flowchart illustrating the steps of a remote video monitoring method with intelligent zoom capability, provided as an embodiment of this application;
[0040] Figure 2 This is a schematic diagram illustrating the process of obtaining external noise interference levels. Detailed Implementation
[0041] In the description of the embodiments in this application, the words "exemplary," "or," and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary," "or," and "for example" is intended to present the relevant concepts in a specific manner.
[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It should be understood that, unless otherwise stated, " / " in this application means "or".
[0043] It should also be noted that the terms "first" and "second" in this application are used to distinguish similar objects, rather than to describe a specific order or sequence.
[0044] The following description, in conjunction with the accompanying drawings, details the specific scheme of the intelligent zoom-enabled remote video monitoring method, system, and device provided in this application.
[0045] Please see Figure 1 The diagram illustrates a flowchart of a remote video monitoring method with intelligent zoom capability according to an embodiment of this application. The method includes the following steps:
[0046] Step 1: Use monitoring equipment to collect video of the area to be monitored in real time and perform frame segmentation processing, and collect audio data of the location of the monitoring equipment in real time.
[0047] For the areas of road and bridge facilities to be monitored, multiple mobile sentry monitoring units are deployed. Visible light cameras on these mobile sentry monitoring units capture video of the monitored area in real time, while audio sensors on the units simultaneously capture audio data from their locations. The captured video and audio data are transmitted wirelessly to a remote control terminal. The video is processed into frames, resulting in individual video images, and each frame undergoes grayscale processing. Grayscale processing is a well-known technique and will not be elaborated upon here. The monitored areas may include the main beams, piers, towers, and stay cables of a cable-stayed bridge. The implementer can set the monitoring targets, i.e., the areas to be monitored, based on the actual situation.
[0048] In this embodiment, when performing frame-segmentation processing on the video, the 1-second video is divided into 60 frames, and the audio data acquisition frequency is 16kHz. Here, 60 and 16kHz are merely one embodiment of this application, and implementers can set their specific values themselves. This application does not impose any special restrictions.
[0049] Step 2: Preset the time interval for evaluating external noise interference in the video image; obtain the blur coefficient of each time interval by the energy distribution of audio data in the time domain and frequency domain within each time interval, and the difference between grayscale changes in different areas of the video image in time sequence, and use it to evaluate the possibility that the blur of the video image in each time interval is caused by noise.
[0050] In remote areas where road and bridge infrastructure is being built, the limited terrain and buildings in the surrounding environment lead to a significant amount of dust being stirred up by the wind, resulting in increased dust concentration in the air. This causes blurring and interference when using mobile sentry surveillance to collect video images of road and bridge facilities, thus reducing the accuracy of monitoring. Furthermore, the monitoring requirements for different types of road and bridge facilities vary. For example, cable-stayed bridges require monitoring the tilt displacement of the pylons and strain cracks in various parts of the main girder. Therefore, mobile sentry surveillance needs to change its focus during monitoring to clearly monitor targets at high or distant locations. However, this zooming operation results in inconsistent depth of field in the acquired video images. Specifically, some areas of the video image are relatively clear, while other areas are significantly blurred, and the proportion of clear and blurred areas is similar. This makes it difficult to distinguish whether the image blurring is caused by increased environmental dust concentration or by the focusing operation, thus requiring differentiation.
[0051] Specifically, when monitoring road and bridge facilities, mobile sentry surveillance typically uses zoom and focus operations to precisely target the monitored area based on different monitoring needs. After zooming and focusing, the overall monitoring image shows the area where the monitored target is located relatively clear, while other areas are relatively blurry. This phenomenon is due to inconsistent depth of field. However, when windy weather occurs in the environment where the mobile sentry surveillance is located, causing an increase in dust concentration, the monitoring image will gradually be obscured by dust. At this time, the overall blurriness of the acquired video image increases, and noise interference intensifies. However, because there are obvious differences in the clarity of local areas in the video image from the beginning, the impact of the increase in dust concentration on the monitoring image shows different characteristics: for areas with high initial clarity, the impact of dust is more significant, with drastic color changes and a faster increase in blurriness; for areas with low initial clarity, the impact of dust is relatively weak, with a slight overall change and a relatively minor change in blurriness. In addition, when wind causes sand and dust concentration to rise, the audio signal in the environment monitored by the mobile sentry will also change, and obvious wind noise and sand and dust collision sounds will appear in the audio signal, which will significantly increase the noise content of the audio signal.
[0052] Based on the above analysis, a time interval for evaluating external noise interference in the video images is preset. Subsequent analysis will be conducted for a single mobile sentry surveillance camera.
[0053] By analyzing the energy distribution of audio data in the time and frequency domains within each time interval, and the differences in grayscale changes between different regions in the video image over time, the blur coefficients for each time interval are obtained. These coefficients characterize the likelihood that the blurring of the video image is caused by wind and sand noise resulting from environmental changes. Specifically:
[0054] By analyzing the energy distribution of audio data in the time and frequency domains within each time interval, the noise interference coefficient of the audio data in each time interval is obtained. This coefficient is used to characterize the noise content characteristics of the audio data in the environment monitored by the mobile sentry within each time interval. The process of obtaining the audio data noise interference coefficient is as follows: obtaining the goodness of fit of the fitting curve of the energy envelope curve of the audio data in the time domain within each time interval; calculating the signal bandwidth of the audio data in each time interval; the audio data noise interference coefficient of each time interval is directly proportional to the signal bandwidth and inversely proportional to the goodness of fit.
[0055] The grayscale dispersion of each region in each frame of video image within each time interval is obtained, the degree of change of the grayscale dispersion of each region in video image within each time interval is measured in time, and the cumulative value of the difference of the measurement results of the degree of change between different regions in video image within each time interval is calculated.
[0056] The ambiguity coefficients for each time interval are positively correlated with the accumulated value and the noise interference coefficient of the audio data.
[0057] It should be noted that positive correlation means that the independent variables change in the same direction; when one independent variable increases, the other independent variable also increases, and when one independent variable decreases, the other independent variable also decreases.
[0058] It should be noted that: dispersion refers to the degree of unevenness in the distribution of data, which can be achieved by calculating information entropy, standard deviation, variance, etc. This application does not impose any special restrictions on this.
[0059] It should be noted that: difference refers to the degree of distinction between data, which can be achieved by calculating the square of the difference, the absolute value of the difference, the ratio, etc. This application does not impose any special restrictions on this.
[0060] In this embodiment, the length of the time interval is 10 minutes. The length of the time interval is preset by the user, and the implementer can set it according to the actual situation. This application does not impose any special restrictions. Adjacent time intervals do not overlap in time sequence.
[0061] In this embodiment, the Hilbert transform is used to obtain the energy envelope curve of the audio data in the time domain. The Hilbert transform is a well-known technique and will not be described in detail here. As other implementation methods, based on the ability to obtain the energy envelope curve of the audio data in the time domain, the implementer may use other existing feasible techniques. This application does not impose any special restrictions.
[0062] In this embodiment, the least squares method is used to perform polynomial fitting on the energy envelope curve to obtain the fitted curve of the energy envelope curve. The least squares method is a well-known technique and will not be described in detail in this application. As other implementation methods, based on the ability to obtain the fitted curve of the energy envelope curve, the implementer may use other existing feasible techniques, and this application does not impose any special restrictions.
[0063] In this embodiment, the goodness of fit is R-squared. The calculation of R-squared is a well-known technique and will not be described in detail here. As other implementation methods, based on the fitting effect of the fitting curve of the energy envelope curve, the implementer may use other existing techniques, such as the reciprocal of the mean square error, the reciprocal of the root mean square error, etc. This application does not impose any special restrictions.
[0064] In this embodiment, to characterize the differences in grayscale values between different regions in multiple video frames within a single time interval, the single-frame video image is equally divided into... In each region, the value of v is 25. The value of v is preset by the user and can be set by the implementer according to the actual situation. This application does not impose any special restrictions.
[0065] In this embodiment, the Fourier transform algorithm is used to convert the audio data to the frequency domain in order to obtain the signal bandwidth of the audio data. The Fourier transform algorithm is a well-known technology and will not be described in detail in this application. As other implementation methods, based on the ability to convert the audio data to the frequency domain, the implementer may use other existing feasible technologies, and this application does not impose any special restrictions.
[0066] In this embodiment, the expression for the audio data noise interference coefficient in each time interval is as follows:
[0067] In the formula, This represents the noise interference coefficient of the audio data in the i-th time interval; This represents the goodness of fit of the energy envelope curve of the audio data in the time domain within the i-th time interval; This represents the signal bandwidth of the audio data within the i-th time interval; This indicates a preset value greater than 0, used to avoid a denominator of 0. The value of affects the calculation result of the noise interference coefficient of audio data. The value range is [0.1, 1]. In this embodiment... The value is 0.1; norm() represents the normalization function. In this embodiment, the normalization function is specifically the Min-Max normalization function. The Min-Max normalization function is a well-known technique and will not be described in detail here.
[0068] It should be noted that the audio data noise interference coefficient is used to characterize the noise content characteristics of the audio data in the environment where the mobile sentry surveillance is located within each time interval. It is reflected by the ratio of the audio data signal bandwidth to the goodness of fit of the audio signal curve within each time interval. Similar to existing methods for characterizing the noise content in audio data, it is represented by analyzing the fluctuation characteristics of the audio data in the time domain and its distribution characteristics in the frequency domain. However, since the purpose of calculating the audio data noise interference coefficient is to reflect the noise situation in the environment where the mobile sentry surveillance is located, thereby assessing the possibility of wind and sand generation, rather than actually considering the magnitude of noise, its actual physical meaning is not considered. Therefore, a normalization function is used to eliminate interference caused by dimensions. The larger the calculated audio data noise interference coefficient, the higher the noise content in the audio data of the environment where the mobile sentry surveillance is located.
[0069] In this embodiment, the dispersion is information entropy; the method for measuring the temporal variation of the grayscale dispersion of each region in the video image within each time interval is as follows: taking the i-th time interval and the j-th region in the video image as an example, the standard deviation of the grayscale dispersion of the j-th region in all frames of the video image within the i-th time interval is calculated. The larger the calculated standard deviation, the greater the temporal variation of the grayscale dispersion of the j-th region in the video image within the i-th time interval; the difference between the measurement results of the degree of variation is the absolute value of the difference; the ambiguity coefficient of each time interval is the product of the accumulated value and the audio data noise interference coefficient.
[0070] In another embodiment, the dispersion is the standard deviation; the method for measuring the temporal variation of the grayscale dispersion of each region in the video image within each time interval is as follows: taking the i-th time interval and the j-th region in the video image as an example, calculate the absolute difference of the grayscale dispersion between the j-th region in any two adjacent frames of the video image within the i-th time interval, and calculate the sum of the absolute differences between the j-th regions in all any two adjacent frames of the video image within the i-th time interval. The larger the sum, the greater the temporal variation of the grayscale dispersion of the j-th region in the video image within the i-th time interval; the difference between the measurement results of the degree of variation is the absolute value of the difference; the ambiguity coefficient of each time interval is the product of the accumulated value and the audio data noise interference coefficient.
[0071] It should be noted that the more significant the noise intensity in the environment where the mobile sentry is located, and the greater the difference in variation between different areas in the video image, the larger the calculated blur coefficient will be. This indicates that the environment where the mobile sentry is located is more likely to be affected by wind and sand noise interference, and the greater the possibility that the blurring of the video image in each time interval is caused by noise.
[0072] Step 3: By obtaining the texture difference coefficient of each time interval through the texture feature differences between different regions in each frame of video image within each time interval, and combining the edge length and edge position distribution in the video image within each time interval, obtain the influence degree coefficient of each time interval, which is used to evaluate the possibility of image blurring caused by inconsistent depth of field in the video image within each time interval.
[0073] In actual monitoring, due to the large amount of content to be monitored on road and bridge facilities, some monitoring targets are relatively easy to capture clear video images, such as some close-range or low-positioned targets. In this case, after focusing, the clarity of a single frame video image is high, and the adaptive image denoising algorithm can effectively identify and process external noise interference in the video image, thus effectively performing denoising processing. However, for some road and bridge facilities in complex locations, there may be other facilities interspersed around them, resulting in some obstruction between monitoring targets. Even if the mobile sentry monitoring adjusts its position, it is difficult to completely avoid interference from non-monitored targets. After the camera zooms, although the clarity of the monitored target is improved, the blurring of non-monitored targets may cause local obstruction of the monitored target, ultimately resulting in a small number of blurred areas still remaining in a single frame video image, with a relatively weak degree of inconsistency in depth of field. If only the calculation is performed using step 2, it may be mistakenly judged that the video image is not affected by environmental noise because the difference in the changes of different areas affected by noise in the video image is small. Therefore, further analysis is needed to more accurately determine the noise interference in the video image.
[0074] Specifically, when inconsistent depth of field occurs in a video image, it manifests as some areas being highly blurred while others are highly sharp. The ratio of blurred to sharp areas varies depending on the specific shooting requirements. If the monitored target occupies a small proportion of the frame, the sharp areas will be relatively few, while the blurred areas will be more numerous. In this case, inconsistent depth of field has a more significant impact on the blurriness of the video image. Blurred areas exhibit extremely high texture distortion, with shorter edges and more dispersed edge distribution. Due to texture distortion, the texture differences between different areas are obvious. Conversely, when inconsistent depth of field has a smaller impact on the video image—that is, when there are more sharp areas and fewer blurred areas—the edge stripes in the sharp areas will be clearer, longer, and more densely distributed. In this case, the texture feature differences between different areas are smaller.
[0075] Based on the above analysis, the texture difference coefficient for each time interval is obtained by analyzing the texture feature differences between different regions in each frame of video image within each time interval. The expression is as follows:
[0076] In the formula, Represents the texture difference coefficient for the i-th time interval; , Let J and Q represent the LBP values of the j-th and (j+1)-th regions in the q-th video image within the i-th time interval, respectively; J is the total number of regions in a single video image; and Q is the total number of video image frames within a single time interval. The LBP values are obtained using the MB-LBP (Multiscale Block LBP) algorithm, which is a well-known technique and will not be described further in this application.
[0077] It should be noted that the texture difference coefficient represents the degree of texture difference between different regions in each frame of video image within each time interval. It is reflected by the absolute difference of texture features between different regions in multiple frames of video image within each time interval, similar to the calculation principle of existing methods for characterizing image texture feature differences. The larger the calculated texture difference coefficient, the greater the degree of texture difference between different regions in the video image within each time interval.
[0078] Furthermore, by combining the texture difference coefficients of each time interval with the edge lengths and positional distributions in the video images within each time interval, an influence coefficient for each time interval is obtained. This coefficient is used to assess the likelihood of image blurring due to inconsistent depth of field in the video images within each time interval. The expression is as follows:
[0079] In the formula, This represents the influence coefficient for the i-th time interval; Let represent the texture difference coefficient for the i-th time interval; calculate the mean Euclidean distance between all edge center points and their nearest neighbor edge center points in each frame of the video image. This represents the arithmetic mean of the values of all video frames within the i-th time interval; This represents the average length of all edges in all frames of video images within the i-th time interval. Where, when When it is 0, first... Mapping data to positive numbers before subsequent calculations can be performed. There are many methods for mapping data to positive numbers, and implementers can choose feasible methods at their own discretion. This embodiment calculates... The sum of the sum with a preset constant greater than 0 will achieve the following: The purpose of mapping to positive numbers is that the value of the constant that is preset to be greater than 0 is 0.01. The value of the constant that is preset to be greater than 0 can be set by the implementer according to the actual situation. This application does not impose any special restrictions.
[0080] In this embodiment, the Canny edge detection algorithm and the dilation erosion algorithm are used to obtain each edge in each frame of video image. The Canny edge detection algorithm and the dilation erosion algorithm are well known technologies and will not be described in detail in this application. As other implementation methods, based on the ability to obtain each edge in each frame of video image, the implementer may adopt other existing feasible technologies, and this application does not impose any special restrictions.
[0081] In this embodiment, the length of each edge is specifically represented by the total number of pixels contained in each edge.
[0082] It should be noted that: by comprehensively considering the edge length and position distribution characteristics of video images in each time interval, as well as the texture difference characteristics between different regions, an influence degree coefficient is obtained to characterize the possibility of image blurring caused by inconsistent depth of field in video images in each time interval; the smaller the calculated influence degree coefficient, the less likely the video images in each time interval are to be blurred due to inconsistent depth of field, and vice versa.
[0083] Step 4: Combine the fuzzy coefficient and the influence degree coefficient to obtain the external noise interference degree for each time interval.
[0084] By combining the ambiguity coefficient and influence coefficient of each time interval, the external noise interference level of each time interval is obtained, which is used to characterize the possibility of external noise interference in the video images acquired within each time interval. The expression is as follows:
[0085] In the formula, This represents the external noise interference level in the i-th time interval; Represents the fuzzy coefficient for the i-th time interval; This represents the influence coefficient for the i-th time interval; norm() represents the normalization function, specifically the Min-Max normalization function in this embodiment. It should be noted that if the influence coefficient is 0, it is first mapped to a positive number before subsequent calculations. There are many methods for mapping data to positive numbers, and implementers can choose any feasible method based on their actual situation. This embodiment achieves the purpose of mapping the influence coefficient to a positive number by calculating the sum of the influence coefficient and a preset constant greater than 0. The preset constant greater than 0 has a value of 0.01, and the value of this preset constant can be set by the implementer based on the actual situation; this application does not impose any special restrictions.
[0086] It should be noted that this application considers different influencing factors when calculating the external noise interference level. It analyzes the noise interference in the environment where the mobile sentry monitoring is located, as well as the impact of potential initial depth-of-field inconsistencies in the video images, and combines these analyses to characterize the probability of external noise interference in the video images acquired within each time interval. Furthermore, since the main purpose of the external noise interference level is to characterize the probability of the acquired video images being affected by real external noise, rather than specific numerical changes, a normalization function is used for normalization. The higher the probability of blurring caused by external noise in the video images acquired within each time interval, and the smaller the impact of depth-of-field inconsistencies in the video images, the greater the calculated external noise interference level, indicating that external noise interference is more likely to exist in the video images acquired within each time interval. A schematic diagram of the process for obtaining the external noise interference level is shown below. Figure 2 As shown.
[0087] Step 5: Adjust the filtering intensity of the video images in each time interval, so as to perform remote video monitoring of the area to be monitored based on the filtered video images.
[0088] The greater the external noise interference in each time interval, the more likely there is image blurring interference caused by real external noise in the video images acquired in each time interval; conversely, the less noise interference, the more likely the blurring in the video images acquired in each time interval is caused by inconsistent depth of field. In this case, the denoising parameters should be reduced to reduce the degree of feature distortion in the clear area where the monitored target is located.
[0089] In this embodiment, the denoising algorithm selected is the adaptive nonlocal mean filtering algorithm. The denoising parameter is the filtering parameter h in the adaptive nonlocal mean filtering algorithm. Taking video images acquired within any time interval as an example, each frame of video image is used as input, along with the external noise interference level of the given time interval. The adaptive nonlocal mean filtering algorithm is used to denoise each frame of video image, and the denoised video image is output. The value of the filtering parameter is the product of its initial value and the external noise interference level of the given time interval. The formula for calculating the initial value of the filtering parameter h is as follows: Where k represents the adjustment coefficient. The standard deviation of the noise is represented by the adjustment coefficient, which is calculated from experimental data. In this embodiment, the adjustment coefficient is 1.2. The adaptive nonlocal mean filtering algorithm is a well-known technique and will not be described further in this application.
[0090] Furthermore, remote video monitoring of the area to be monitored is performed based on the filtered video image, including: identifying edges in the image to help determine the outline of the object; identifying areas with abnormal temperature to help target detection; detecting dynamic targets in the image; extracting the geometric shape features of the target, including aspect ratio and area; analyzing the surface texture of the target, including smoothness and roughness; extracting the color information of the target; and analyzing the temperature distribution of the target to extract features from thermal imaging.
[0091] Based on the same inventive concept as the above method, embodiments of this application also provide a remote video surveillance system with intelligent zoom capability, comprising:
[0092] The information acquisition module is used to acquire video of the area to be monitored in real time using the monitoring equipment, and to perform frame-by-frame processing, and to acquire audio data of the location of the monitoring equipment in real time.
[0093] The noise assessment module is used to preset time intervals for assessing external noise interference in video images. It obtains blur coefficients for each time interval by analyzing the energy distribution of audio data in the time and frequency domains, and the differences in grayscale changes between different regions of the video image over time. This is used to assess the likelihood that the blurring of the video image in each time interval is caused by noise. It also obtains texture difference coefficients for each time interval by analyzing the texture feature differences between different regions in each frame of the video image within each time interval. Combined with the length and positional distribution of edges in the video image within each time interval, it obtains influence coefficients for each time interval. This is used to assess the likelihood that the video image blurring is caused by inconsistent depth of field within each time interval. Finally, it combines the blur coefficients and influence coefficients to obtain the degree of external noise interference in each time interval.
[0094] The noise reduction and monitoring module is used to adjust the filtering intensity of the video images in each time interval based on the external noise interference level, so as to perform remote video monitoring of the area to be monitored based on the filtered video images.
[0095] Based on the same inventive concept as the above method, this application embodiment also provides a remote video monitoring device with intelligent zoom capability, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described remote video monitoring methods with intelligent zoom capability.
[0096] In summary, this application divides video and audio data into multiple time intervals for analysis, which can capture the changes in environmental noise in more detail; by combining the time and frequency domain energy distribution of audio data and the grayscale variation differences of video images, noise interference is comprehensively evaluated from multiple dimensions, so that the blur coefficient can more accurately represent the degree of image blur caused by noise interference, providing a reliable basis for subsequent noise interference degree assessment.
[0097] Furthermore, by calculating the differences in texture features between different regions in a video image, the blurring caused by inconsistent depth of field can be accurately reflected, which helps to distinguish between blurring caused by inconsistent depth of field and blurring caused by noise interference. Considering that edge features will change significantly when the depth of field is inconsistent, the length and position distribution of edges in the video image, as well as the differences in texture features, are combined to further enhance the accuracy of the assessment of the degree of influence of depth of field. This allows for a more accurate assessment of the degree of blurring caused by inconsistent depth of field, providing a more comprehensive reference for subsequent noise interference assessment.
[0098] Furthermore, by combining the blur coefficient and the influence coefficient, the possibility of the image being interfered with by real external noise in each time interval is evaluated, thereby improving the accuracy of the evaluation of the possibility of the image being interfered with by real external noise. Then, the denoising parameters in the adaptive denoising algorithm are adjusted according to the possibility of the image being interfered with by real external noise. This can retain more feature information while ensuring the denoising effect, and avoid misjudging the blur caused by inconsistent depth of field as noise, which would lead to over-filtering and image feature distortion, thereby improving the accuracy and reliability of remote video monitoring.
[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0100] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from its essential characteristics. Therefore, the embodiments described above should be considered exemplary and non-limiting in all respects.
Claims
1. A remote video monitoring method with intelligent zoom capability, characterized in that, The method includes the following steps: The monitoring equipment is used to collect video of the area to be monitored in real time and perform frame segmentation processing, and the audio data of the location of the monitoring equipment is collected in real time. The system presets time intervals for evaluating external noise interference in video images. By analyzing the energy distribution of audio data in the time and frequency domains within each time interval, and the differences in grayscale changes across different regions of the video image over time, it obtains blur coefficients for each time interval to assess the likelihood that the blurring in the video image within each time interval is caused by noise. By analyzing the texture feature differences between different regions in each frame of the video image within each time interval, it obtains texture difference coefficients for each time interval. Combined with the length and positional distribution of edges in the video image within each time interval, it obtains influence coefficients for each time interval to assess the likelihood that the video image blurring is caused by inconsistent depth of field. Finally, by combining the blur coefficients and influence coefficients, it obtains the degree of external noise interference in each time interval. By adjusting the filtering intensity of the video images in each time interval based on the external noise interference, remote video monitoring of the area to be monitored can be performed based on the filtered video images. The process of obtaining the texture difference coefficient is as follows: Obtain the LBP value of each region in each frame of video image, and calculate the sum of the differences in LBP values between different regions in each frame of video image; The texture difference coefficient is the summation result of the accumulated sums in all frames of video images within each time interval; The process for obtaining the influence coefficient is as follows: Calculate the average length of all edges in all frames of video images within each time interval; Calculate the mean of the Euclidean distances between all edge center points and their nearest neighbor edge center points in each frame of video image, and calculate the arithmetic mean of the mean values in all frames of video image within each time interval; The influence coefficient is directly proportional to the texture difference coefficient and the arithmetic mean, and inversely proportional to the average value.
2. The remote video monitoring method with intelligent zoom as described in claim 1, characterized in that, The process of obtaining the fuzzy coefficient is as follows: The grayscale dispersion of each region in each frame of video image within each time interval is obtained, the degree of change of the grayscale dispersion of each region in video image within each time interval is measured in time, and the cumulative value of the difference of the measurement results of the degree of change between different regions in video image within each time interval is calculated. The noise interference coefficient of audio data in each time interval is obtained by measuring the energy distribution of audio data in the time and frequency domains within each time interval. The fuzziness coefficient is positively correlated with the accumulated value and the audio data noise interference coefficient.
3. The remote video monitoring method with intelligent zoom as described in claim 2, characterized in that, The process of obtaining the noise interference coefficient of the audio data is as follows: Obtain the goodness of fit of the fitting curve of the audio energy envelope curve of the audio data in each time interval; Calculate the signal bandwidth of the audio data within each time interval; The audio data noise interference coefficient is directly proportional to the signal bandwidth and inversely proportional to the goodness of fit.
4. The remote video monitoring method with intelligent zoom as described in claim 1, characterized in that, The method for calculating the influence coefficient is as follows: The ratio of the arithmetic mean to the average value is calculated, and the influence coefficient is the product of the ratio and the texture difference coefficient.
5. The remote video monitoring method with intelligent zoom as described in claim 1, characterized in that, The external noise interference degree is the normalized value of the ratio of the ambiguity coefficient to the influence degree coefficient.
6. The remote video monitoring method with intelligent zoom as described in claim 1, characterized in that, The adjustment of the filtering intensity of video images within each time interval includes: A denoising algorithm is used to denoise each frame of video image within each time interval. The denoising parameter in the denoising algorithm is the product of its preset initial value and the external noise interference degree of the time interval where each frame of video image is located.
7. A remote video surveillance system with intelligent zoom capability, employing the remote video surveillance method with intelligent zoom capability as described in claim 1, characterized in that, The system includes: The information acquisition module is used to acquire video of the area to be monitored in real time using the monitoring equipment, and to perform frame-by-frame processing, and to acquire audio data of the location of the monitoring equipment in real time. The noise assessment module is used to preset time intervals for assessing external noise interference in video images. It obtains blur coefficients for each time interval by analyzing the energy distribution of audio data in the time and frequency domains, and the differences in grayscale changes between different regions of the video image over time. This is used to assess the likelihood that the blurring of the video image in each time interval is caused by noise. It also obtains texture difference coefficients for each time interval by analyzing the texture feature differences between different regions in each frame of the video image within each time interval. Combined with the length and positional distribution of edges in the video image within each time interval, it obtains influence coefficients for each time interval. This is used to assess the likelihood that the video image blurring is caused by inconsistent depth of field within each time interval. Finally, it combines the blur coefficients and influence coefficients to obtain the degree of external noise interference in each time interval. The noise reduction and monitoring module is used to adjust the filtering intensity of the video images in each time interval based on the external noise interference level, so as to perform remote video monitoring of the area to be monitored based on the filtered video images.
8. A remote video monitoring device with intelligent zoom capability, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the remote video monitoring method with intelligent zoom as described in any one of claims 1-6.