Method, device, storage medium and software product for detecting physical displacement of camera

Abstract

The application provides a camera physical displacement detection method, device, storage medium and software product. The method comprises the following steps: an initial position camera is used to shoot a first video, a region of interest is divided, image mean values and standard deviations of different video frames in the region are calculated, and a baseline mean value is determined; the camera is used to shoot a second video, a region of interest at the same position is selected, a comparison mean value is calculated, and a tolerance range is calculated according to the baseline mean value; then, a detection video is shot, a region of interest at the same position is selected, and a detection image mean value is calculated; if the detection image mean value is not in the tolerance range, it is determined that the camera has been displaced, otherwise, the camera has not been displaced. The application only relies on video stream shooting by the camera for calculation and analysis, does not need external physical markers or additional sensors, and reduces system deployment cost and maintenance complexity.

Description

Methods, equipment, storage media, and software products for detecting the physical displacement of cameras Technical Field

[0001] This invention relates to the field of camera physical displacement detection technology, and in particular to a method, electronic device, computer-readable storage medium, and computer software product for detecting camera physical displacement. Background Technology

[0002] In modern intelligent video surveillance systems, the physical stability of cameras is fundamental to ensuring the integrity of system functionality. According to the IEEE "Technical White Paper on Intelligent Vision Surveillance Systems," over 73% of video analysis failures stem from physical displacement of the monitoring equipment itself, including mechanical vibration offset, human rotation, and loose mounting base. When a camera's angular offset exceeds 0.5° or its translation exceeds 2% of the field of view, it can cause coordinate reference system drift, multi-camera collaborative calibration failure, etc., increasing the target detection false negative rate by 42% and expanding the cumulative error of trajectory tracking to 3.7 meters per hour (2023 Security Industry Technology Annual Report), thus weakening the reliability of the monitoring system's decision-making. In engineering implementation, the problems caused by camera displacement are multi-dimensional: First, the spatial reference is disrupted. Displacement causes deviations between the camera's internal and external parameters and the preset calibration values, causing video analysis algorithms based on perspective projection models to lose spatial consistency. Second, temporal continuity is disrupted. Displacement breaks the homography constraint between adjacent frames, causing traditional optical flow methods and background modeling techniques to misjudge static scenes, leading to a surge in false alarm rates. Third, there is a system-level trust crisis. A NIST special report points out that in systems without effective displacement detection mechanisms, more than 68% of monitoring events require manual review, reducing the timeliness of security response.

[0003] Currently, several technologies address this problem, mainly falling into the following categories: 1. Sensor-based hardware detection schemes: These use additional hardware such as gyroscopes and accelerometers to detect the physical movement of the camera, judging movement by measuring changes in direction, acceleration, or tilt. The advantage is high-precision detection of minute movements and real-time feedback of physical displacement; the limitations are high cost, increased expenses from installing additional sensors, technical challenges in integrating into existing monitoring systems, and sensor performance degradation requiring regular calibration. 2. Tag-based methods: These rely on identifying and tracking fixed physical markers or features within the camera's field of view, analyzing their displacement relative to the camera to determine movement. This is highly effective in controlled environments with a static background, accurately locating displacement; however, it is environment-dependent, requiring static, visible markers, and is impractical in dynamic or cluttered environments. Environmental changes may lead to false alarms, and the cost of installing and maintaining markers is high. 3. Feature-based image analysis: This compares the visual features of frames in a video stream to detect movement, using algorithms to track key points and analyze displacement. To meet real-time requirements, deep learning models are generally used to extract features, requiring GPU parallel computing, which presents hardware resource constraints in edge computing scenarios. The advantages are that no additional hardware is required, relying only on the video stream, and it can flexibly adapt to different environments; the limitations are that the computational intensity is high, the cost of processing high-resolution video is high and the latency is large in resource-constrained systems, it is not suitable for real-time detection, and it is sensitive to noise, and changes in lighting may affect the reliability of feature tracking. Summary of the Invention

[0004] This invention aims to at least solve the problem of high cost in existing technologies, and innovatively proposes a method, electronic device, computer-readable storage medium, and computer software product for detecting the physical displacement of a camera.

[0005] To achieve the above-mentioned objective of the present invention, the present invention provides a method for detecting the physical displacement of a camera, the method comprising:

[0006] The first video segment is captured using a camera at the initial position, and the region of interest in the first video segment is defined.

[0007] Calculate the image mean and standard deviation of the region of interest in different video frames of the first video segment;

[0008] The baseline mean is determined based on the image mean and standard deviation;

[0009] A second video segment is captured using the camera. A region of interest (ROI) is selected in the second video segment at the same location as the ROI in the first video segment. The average contrast value of the ROI in the second video segment is calculated.

[0010] The tolerance range is calculated based on the comparison mean and the baseline mean;

[0011] The camera is used to capture a detection video. A region of interest (ROI) is selected in the detection video at the same location as the ROI of the first video segment. The average image detection value of the ROI in the detection video is then calculated.

[0012] If the average value of the image detection is not within the tolerance range, then it is determined that the camera has shifted; otherwise, it is not.

[0013] As an optional embodiment of the present invention, optionally, determining the baseline mean based on the image mean and standard deviation includes:

[0014] The video frame with the lowest volatility is selected based on the standard deviation.

[0015] The lowest video frame and the image mean are clustered with the spatial location of the region of interest in the video frame with the lowest volatility to obtain a selected grid in each cluster.

[0016] The mean value of the region of interest image of the selected grid in the first video segment is used as the baseline mean value.

[0017] In one aspect, the present invention also provides an electronic device, including a processor, a memory, and a computer program executable on the processor, wherein the processor, when executing the program, implements the method for detecting the physical displacement of a camera.

[0018] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting the physical displacement of a camera.

[0019] In another aspect, the present invention also provides a computer software product, including a computer program / instructions that, when executed by a processor, implement the method for detecting the physical displacement of a camera.

[0020] This invention achieves significant technical effects through the following technical means:

[0021] 1. By processing image features through color space transformation after removing the luminance component, the interference of ambient lighting changes on the detection results is effectively eliminated, thus improving the robustness of the algorithm;

[0022] 2. By using dimensionality reduction to cluster spatial locations and image means, and by selecting the grid regions with the lowest volatility to determine the baseline mean, misjudgments caused by subtle changes in the video content itself (such as swaying leaves or ripples on the water surface) are significantly reduced.

[0023] 3. The comparison mean is calculated based on the second video segment and the tolerance range is dynamically set accordingly, so that the judgment threshold can adapt to the natural stability fluctuation of the camera at the initial position, avoiding the problems of insufficient sensitivity or high false alarm rate caused by fixed threshold.

[0024] 4. The entire methodology relies solely on video streams captured by cameras for computation and analysis, eliminating the need for external physical markers or additional sensors, thus significantly reducing system deployment costs and maintenance complexity;

[0025] 5. The core calculation steps (such as mean, standard deviation, and cluster screening) have low algorithm complexity and low requirements for hardware computing resources, making them particularly suitable for real-time operation of embedded monitoring devices in edge computing environments.

[0026] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0027] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0028] Figure 1 is a flowchart of a method for detecting the physical displacement of a camera according to the present invention;

[0029] Figure 2 is a schematic diagram of the region of interest (ROI) in one embodiment of this application;

[0030] Figure 3 is a schematic diagram of the region of interest (ROI) in one embodiment of this application;

[0031] Figure 4 is a schematic diagram of the region of interest (ROI) in one embodiment of this application. Detailed Implementation

[0032] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0033] This application provides a method for detecting the physical displacement of a camera, as shown in Figure 1, including:

[0034] S1: Use the camera at the initial position to capture the first video segment and divide the first video segment into a Region of Interest (ROI).

[0035] For example, consider the first video clip captured by a surveillance camera in its normal / default position. Further, each frame of this first video clip can be divided into different Regions of Interest (ROIs) using an n×n pixel grid. Specifically, the division could be done by creating a grid that does not overlap but covers all pixels in each frame, with each grid serving as an ROI. In a simplified example, if each frame of the video is 125×125 pixels, each frame can be divided into 5×5 pixel units, creating a grid that does not overlap but covers all pixels, with each grid serving as an ROI. Here, there are a total of 25 ROIs, as shown in Figure 2. Of course, the division could also be n×m pixels, which will not be elaborated here. Alternatively, the division could be into grids with some coverage covering all pixels in each frame, grids with some coverage covering most pixels in each frame, or grids with no coverage covering most pixels in each frame.

[0036] S2: Calculate the image mean and standard deviation of the region of interest in different video frames of the first video segment.

[0037] Specifically, for the same ROI in different video frames in the first video segment, the mean and standard deviation of the three RGB color channels can be calculated.

[0038] The RGB color model is an industry-standard color model that creates a wide variety of colors by varying the values ​​of the three color channels: Red (R), Green (G), and Blue (B) and their combinations. Each channel typically ranges from 0 to 255, resulting in a total of 16,777,216 (256 × 256 × 256) colors. This model is based on the different sensitivities of the human eye to red, green, and blue light, and is the foundation of additive color mixing.

[0039] The red channel (R) controls the proportion of red light in the image. A value of 0 indicates that there is no red light, while a value of 255 indicates that the red light is strongest.

[0040] Green channel (G): Controls the proportion of green light in the image. Similar to the red channel, 0 represents no green light, and 255 represents the strongest green light.

[0041] Blue channel (B): Controls the proportion of blue light in the image, following the same rules as above.

[0042] When processing digital images, calculating statistics such as the mean and standard deviation of pixel values ​​can help understand the overall brightness and contrast of the image.

[0043] Mean: For a given image or a portion thereof, the mean of the three RGB channels refers to the average of all pixel values ​​within each channel. Assume a channel has... If there are 100 pixels, then the average value of that channel is... It can be calculated using the following formula: ,in, Indicates the first The value of this channel for each pixel. Where:

[0044] Red channel mean: Obviously, It is the first The value of the red channel for each pixel. This represents the mean of the red channel;

[0045] Green channel average: Obviously, It is the first The value of the green channel for each pixel. This represents the average value of the green channel;

[0046] Blue channel mean: Obviously, It is the first The value of the green channel for each pixel. This represents the mean of the blue channel;

[0047] Standard deviation: Standard deviation measures the degree of deviation between a set of values ​​and its mean. For a channel of an RGB image, the standard deviation reflects the dispersion of pixel values ​​within that channel. It can be calculated using the following formula: ;in, It is the average value of the pixel values ​​in that channel. Indicates the first The value of the red channel for each pixel. Where:

[0048] Red channel standard deviation: ,in, Indicates the standard deviation of the red channel;

[0049] Green Channel Standard Deviation: ,in, Indicates the standard deviation of the green channel. Indicates the first The value of the green channel for each pixel;

[0050] Blue channel standard deviation: ,in, This represents the standard deviation of the blue channel. Indicates the first The value of the blue channel for each pixel;

[0051] Calculating the mean and standard deviation of an RGB image helps analyze its color distribution, which is crucial for feature extraction and color correction in image processing and computer vision tasks. For example, understanding these statistics can guide how to better adjust the image to achieve the desired effect when performing image enhancement or color adjustment.

[0052] In the RGB color model, color information and luminance information are intertwined. To reduce the impact of lighting changes on image analysis, the RGB color space can be converted to other spaces that can separate luminance and chrominance (color information). Taking the YUV color space as an example, it is mainly used to optimize the transmission and processing of color video signals. It separates image information into luminance and chrominance (including U and V parts). The following conversion relationship exists between RGB color space channels and YUV color space:

[0053] Y = 0.299R + 0.587G + 0.114B

[0054] U = -0.147R - 0.289G + 0.436B

[0055] V = 0.615R - 0.515G - 0.100B

[0056] In the RGB color space, each pixel's R, G, and B channels simultaneously contain both color and luminance information. For example:

[0057] When the light intensity increases, the values ​​of all channels will increase overall;

[0058] When the color of an object changes, the proportional relationship between the channels changes.

[0059] This coupling characteristic leads to the following problems:

[0060] 1. Confusion between brightness and chromaticity: For example, a red object (R=200, G=50, B=50) may appear (R=255, G=150, B=150) under strong light, and its color perception will be distorted due to the change in brightness;

[0061] 2. Noise sensitivity: Fluctuations in illumination will affect all channels simultaneously, amplifying noise interference.

[0062] Therefore, the RGB color space can be converted to a luminance-free component, thus isolating luminance variations. Furthermore, the aforementioned calculation of the image mean and standard deviation for the same ROI in different video frames of the first video segment can be achieved by calculating the image mean and standard deviation of the same ROI in the color space of different video frames of the first video segment after converting to a luminance-free image. Specifically, this can be done by calculating the image mean and standard deviation of the RGB color space after converting to a luminance-free image, thereby obtaining the average chromaticity level and the fluctuation range of chromaticity variations. For example, this can be converted to the chromaticity components of the YUV color space. In the YUV color space, the value of the luminance component Y directly reflects the light intensity, while chromaticity information (color differences) is stripped to other channels (such as U and V in YUV). Ignoring the Y component and focusing on the U and V components isolates the effects of luminance variations. In addition, other color spaces, such as CMYK, HSV / HSB, HSL, and Lab, can also have their luminance components removed. For example, CMYK retains its color components through conversion, while HSV / HSB ignores the V (Value / Brightness) component and focuses on hue and saturation. HSL and Lab color spaces also ignore the luminance L component while retaining the color components, which will not be elaborated further.

[0063] In S2, the image mean and standard deviation can be calculated for the same ROI in different video frames of the first video segment. For example, the image mean and standard deviation can be calculated for the same ROI in several consecutive video frames, or for the same ROI in several non-consecutive frames. In S2, the image mean and standard deviation can be calculated pixel-by-pixel for the same ROI in different video frames, and then summed to obtain the image mean and standard deviation of the ROI across different video frames; alternatively, the image mean and standard deviation can be calculated pixel-by-pixel for the same ROI in several consecutive video frames, and then summed to obtain the image mean and standard deviation of the ROI across different video frames; or the image mean and standard deviation can be calculated pixel-by-pixel for the same ROI in several non-consecutive frames, and then summed to obtain the image mean and standard deviation of the ROI across different video frames. Further details are omitted here. For RGB, the average of the three channels' means in a specific ROI can be used as the image mean of that ROI across different video frames, and the sum of the standard deviations of the three channels can be used as the standard deviation of that ROI across different video frames. For YUV, the average of the mean values ​​of the U and V channels in a specific ROI can be used as the image mean of that ROI across different video frames, and the sum of the standard deviations of the U and V channels can be used as the standard deviation of that ROI across different video frames, thus removing the influence of brightness.

[0064] S3: Determining the baseline mean based on the image mean and standard deviation includes: selecting the video frame with the lowest volatility based on the standard deviation; clustering the lowest volatility video frame with the spatial location of the region of interest in the video frame with the image mean and the region of interest in the video frame with the lowest volatility to obtain a selected grid in each cluster; and using the image mean of the region of interest in the selected grid in the first video segment as the baseline mean.

[0065] In step S3, it is necessary to explain in detail that the ROI of the video frame with the lowest volatility is selected by the standard deviation, and the selected ROI is clustered based on the mean and spatial location to obtain the selected grid in each cluster.

[0066] Standard deviation reflects the degree of fluctuation in pixel values ​​within a region of an image over time. The smaller the standard deviation, the more stable the pixel values ​​of the ROI are across different video frames, and the lower the volatility.

[0067] Assume a video containing T frames, where several Regions of Interest (ROIs) can be selected within each frame. Each ROI is a rectangular or arbitrarily shaped image region. The position and size of a specific ROI are consistent across all frames (i.e., tracking a fixed ROI). The image has been converted to a luma-stripped form (e.g., retaining only H and S components in HSV / HSB / HSL, or only U and V components in YUV, or even L components in Lab). Ignored, only a and b ), or simply a grayscale image (single channel) or a three-channel image in the original color space such as RGB.

[0068] For the region of interest (ROI) of the first video segment, for example, 25, the standard deviation can be used to select, for example, the ROI with the lowest variability as 8.

[0069] After selecting the Regions of Interest (ROIs) of the video frames with the lowest volatility, further clustering can be performed using the image mean and spatial location of these ROIs. First, for the n ROIs in the video frame, an n-dimensional vector set M = {μ1, μ2, ..., μ...} can be used. i ,..,μ n}, where μ i It is the average image value of several video frames of the i-th ROI. For multi-channel images, as mentioned before, for example for RGB, the vector set can be M={[μ 1R ,μ 1G ,μ 1B ],…,[μ iR ,μ iG ,μ iB ],…,[μ nR ,μ nG ,μnB ]}, where μ 1R μ represents the average image value of several video frames for the i-th ROI in the red channel. 1G μ represents the average image value of several video frames of the i-th ROI in the green channel. 1B This represents the average image value of several video frames for the i-th ROI in the blue channel.

[0070] Each ROI typically has a specific location, which can be represented by Pos. x Pos y To represent the location of the ROI, where Pos x Pos can be the x-coordinate of the center pixel of the ROI. y This could be the ordinate of the center pixel of the ROI. In this way, a high-order feature vector set can be constructed, where the i-th vector F... i =[μ1,Pos xi Pos yi ], or for multi-channel images, such as RGB, F i =[μ iR ,μ iG ,μ iB Pos xi Pos yi ], where Pos xi Pos represents the x-coordinate of the center pixel of the ROI in the i-th vector. yi This represents the ordinate of the center pixel of the ROI in the i-th vector.

[0071] Use an appropriate clustering algorithm to cluster the above feature vectors. Common clustering algorithms include:

[0072] K-means clustering: This is a simple iterative algorithm that attempts to minimize the sum of squared distances from points within a cluster to the cluster center. You can specify the value of k based on the desired number of clusters and run the K-means algorithm on the feature vector F. i Perform clustering.

[0073] Hierarchical clustering: This method does not require pre-specifying the number of clusters. Instead, it progressively merges the most similar objects to form a tree-like structure (dendritic diagram) by calculating the distances between objects (such as Euclidean distance). The dendritic diagram can then be cut as needed to obtain the desired number of clusters.

[0074] DBSCAN: If the dataset contains noise or outliers, and the cluster shapes are irregular, DBSCAN may be a better choice. It defines clusters based on density and can discover clusters of arbitrary shapes. After clustering, each cluster contains a group (or more) of Areas of Interest (ROIs) with similar means and close spatial locations. The so-called "selected grid" can be understood as one or more ROIs in this cluster.

[0075] In this way, we can not only find relatively stable regions in the video (i.e., ROIs with low volatility), but also identify which stable regions are spatially clustered together.

[0076] Clustering can be combined with dimensionality reduction techniques to improve efficiency and effectiveness. Using techniques such as Principal Component Analysis (PCA) can reduce data dimensionality while retaining the most important information, which is very helpful for subsequent clustering analysis. The following section will detail how to combine PCA for clustering in the above tasks.

[0077] First, for the selected ROIs of the video frames, their image mean (which may contain information from multiple channels, such as the chroma components in RGB or HSV) and spatial coordinates are calculated for each ROI. This information is then combined into a feature vector F. i , where i represents the i-th ROI.

[0078] Assume that the feature vector of each ROI includes:

[0079] Chromaticity information: μ i =[μ iR ,μ iG ,μ iB ] or μ i =[μ iH ,μ iS ] etc., depending on the color space used, where μ iR μ represents the average image value of several video frames for the i-th ROI in the red channel. iG μ represents the average image value of several video frames for the i-th ROI in the green channel. iB μ represents the average image value of several video frames for the i-th ROI in the blue channel. iH μ represents the mean value of the i-th ROI in the hue channel, which is a basic property of color (e.g., red corresponds to 0°, green to 120°, and blue to 240°). iS This represents the mean value of the i-th ROI in the saturation channel.

[0080] Spatial location: (x i ,y i (x) i ,y i ).

[0081] Therefore, the feature vector of each ROI may be five-dimensional (for the RGB color space, plus two spatial coordinates), or more / less, depending on the chosen color space and whether other information is included.

[0082] The main purpose of applying PCA for dimensionality reduction is to transform the original high-dimensional data into low-dimensional data through linear transformation, while preserving as much variation information as possible from the original data.

[0083] The steps include the following:

[0084] Standardizing data: Since the scales of different features (such as chromaticity information and spatial location) can vary greatly, it is usually necessary to standardize the data first to make each feature have the same scale. For example, this can be achieved by subtracting the mean and dividing by the standard deviation.

[0085] ,in, It is the average of all eigenvectors. Indicates the first 1 eigenvector That is the standard deviation. This indicates the number after standardization. 1 eigenvector.

[0086] Calculate the covariance matrix: Based on the standardized data, calculate the covariance matrix.

[0087]

[0088] in, Represents the covariance matrix. Indicates the number of feature vectors. This indicates transpose.

[0089] Finding eigenvalues ​​and eigenvectors: Locate the eigenvalues ​​and corresponding eigenvectors of the covariance matrix. Eigenvalues ​​represent the degree of variation of the data along the direction of the corresponding eigenvector.

[0090] Principal components are selected by sorting the eigenvalues ​​and choosing the eigenvectors corresponding to the top k eigenvalues ​​as the new coordinate system (principal components). The choice of k depends on the proportion of data variation that you want to retain. Usually, the smallest k value that can cover most of the variation (e.g., 95%) is chosen.

[0091] Projecting to a new coordinate system: Projecting the original data onto the selected k principal components to obtain the dimensionality-reduced dataset.

[0092]

[0093] in, It is a matrix composed of the first k eigenvectors. This represents the feature vector after standardization.

[0094] Cluster analysis is performed using the dimensionality-reduced data Z. With lower data dimensionality, clustering algorithms such as K-means, hierarchical clustering, or DBSCAN can be applied more efficiently and effectively.

[0095] After clustering is completed, as described earlier, the results can be used to identify which stable regions are spatially clustered together, thus determining the "selected grid".

[0096] The advantages of combining PCA are:

[0097] Reduce computational complexity: By reducing the data dimensionality, the computational burden on subsequent clustering algorithms is reduced.

[0098] Removing redundant information: PCA can help remove information that contributes little to distinguishing different ROIs, making the clustering results more focused on the truly important differences.

[0099] Improving clustering results: In some cases, the original high-dimensional data may contain noisy or irrelevant features, which can interfere with the clustering results. PCA helps to mitigate these problems and provides a clearer clustering structure.

[0100] In summary, by introducing PCA as a dimensionality reduction tool, we can more effectively handle complex feature vector sets, optimize the subsequent clustering process, and thus more accurately identify physically stable regions in video frames. This method is particularly suitable for processing large-scale datasets or situations with high feature dimensions.

[0101] It should be noted that, besides PCA, there are many other dimensionality reduction techniques that can be used for data preprocessing to reduce data dimensionality while retaining the most important information. These techniques can be selected and used depending on the application scenario and data characteristics.

[0102] For clusters containing multiple selected grids, one or more selected grids with the lowest standard deviation can be further filtered. Specifically, the selected ROIs can be clustered based on the mean and spatial location, resulting in three clusters. These three clusters contain 3, 3, and 2 ROIs respectively, as shown in the shaded background ROIs in Figure 3. For example, if the third preset quantity is 1, then each ROI in these three clusters can be filtered to have the ROI with the lowest standard deviation, resulting in 1, 1, 1 selected grids in these three clusters, as shown in Figure 4.

[0103] In step S3, the mean value of the ROI image of the selected grid in the first video segment is used as the baseline mean value.

[0104] Since the above S1 to S3 are selected grids obtained by processing the first video segment captured by the surveillance camera in the normal / default position as the first video segment, and the images in the normal / default position are relatively stable, that is, these selected grids are more stable, the mean of the ROI image of the selected grid in the first video segment can be used as the baseline mean.

[0105] S4: Use the camera to capture a second video segment, select a region of interest in the second video segment that is at the same position as the region of interest in the first video segment, and calculate the average comparison value of the region of interest in the second video segment;

[0106] Using the second video segment captured by the surveillance camera as comparison data, similarly, the ROI of the selected grid at the same location can be selected in the comparison data.

[0107] Next, the average ROI image value of the selected grid in different video frames of the comparison data can be calculated as the comparison mean. Specifically, for RGB, the average value of the three RGB color channels can be calculated. Similarly, to reduce the impact of lighting changes on image analysis, the RGB color space can be converted to other spaces that can separate luminance and chrominance (color information). Calculating the average image value of the same ROI in different video frames of the comparison data can be done by converting the color space of the same ROI in different video frames of the comparison data to the average image value after removing the luminance component. Specifically, this can be, for example, calculating the average image value of the RGB color space after removing the luminance component, thereby obtaining the average chrominance level. As mentioned above, for example, converting to the chrominance component of the YUV color space.

[0108] Similarly, the image mean can be calculated for the same ROI in different video frames of the comparison data. For example, the image mean can be calculated for the same ROI in several consecutive video frames, or the image mean can be calculated for the same ROI in several non-consecutive frames. In S4, the image mean can be calculated pixel by pixel for the same ROI in different video frames of the comparison data, and then summed to obtain the image mean of the ROI in different video frames; alternatively, the image mean can be calculated pixel by pixel for the same ROI in several consecutive video frames, and then summed to obtain the image mean of the ROI in different video frames; or the image mean can be calculated pixel by pixel for the same ROI in several non-consecutive frames, and then summed to obtain the image mean of the ROI in different video frames. Further details are omitted here. For RGB, the average of the three channels in a specific ROI can be used as the image mean of that ROI in different video frames. For YUV, the average of the U and V channels in a specific ROI can be used as the image mean of that ROI in different video frames, thereby removing the influence of brightness.

[0109] The mean ROI image of the selected grid in different video frames in the comparison data calculated above can be used as the comparison mean.

[0110] S5: Calculate the tolerance range based on the comparison mean and the baseline mean;

[0111] A stability index is calculated based on the comparison mean and the baseline mean, and a tolerance range is obtained based on the stability index.

[0112] The stability index can be calculated based on the comparison mean and the baseline mean.

[0113] One method for calculating stability metrics is, for example, to calculate Friction (stability metric) for each selected grid using the following formula:

[0114]

[0115] Therefore, the Friction value within the range of the lower and upper water levels can be used as a stability indicator for the surveillance camera in its normal / default position. For example, the lower water level could be 25%, and the upper water level could be 75%. By setting the upper and lower water levels and taking the Friction value between them, outliers at both ends can be filtered out.

[0116] Furthermore, a tolerance range can be obtained based on this stability index. For example, the tolerance range can be set based on the baseline mean and standard deviation corresponding to this stability index.

[0117] Specifically, a fixed multiple of the standard deviation can be used to set the tolerance range. The tolerance range can be set as the baseline mean plus or minus n times the standard deviation (n is a coefficient chosen based on the specific application scenario). This is typically suitable for setting a relatively lenient or strict tolerance range.

[0118] Formula expression:

[0119] Lower limit of tolerance range = baseline mean - n Standard deviation

[0120] Upper limit of tolerance range = baseline mean + n Standard deviation

[0121] Application examples:

[0122] Suppose that for a selected ROI, its baseline mean μ base =120, standard deviation σ=5. If we choose n=3 (meaning approximately 99.7% of the data falls within this range, based on a normal distribution), then:

[0123] Lower limit of tolerance range = 120 - 3 × 5 = 105

[0124] Upper limit of tolerance range = 120 + 3 × 5 = 135

[0125] This means that if the average value of the image at the same ROI falls between 105 and 135, it is considered that the camera has not undergone significant physical displacement; otherwise, if it exceeds this range, it may indicate that the camera has undergone physical displacement.

[0126] Another example is adaptive tolerance range based on statistical methods. In this approach, statistical principles are used to adaptively adjust the tolerance range. For instance, the long-term standard deviation can be calculated based on historical data (i.e., the first video clip across multiple time periods), and this information can be used to dynamically adjust the tolerance range. This method is particularly suitable for applications with significant environmental changes or those requiring more precise monitoring.

[0127] Specifically, multiple sets of initial video segments can be collected: in addition to the first video segment captured the first time, more video segments are captured and analyzed regularly to obtain the baseline mean and standard deviation of the ROI for each selected grid segment.

[0128] Calculating the long-term standard deviation: A representative long-term standard deviation σ can be calculated by averaging all obtained standard deviations, or by using other statistical methods (such as averaging after removing extreme values). long .

[0129] Set a tolerance range: Use the long-term standard deviation combined with the baseline mean to set the tolerance range. For example, you can use a similar approach as the first method, but use the long-term standard deviation instead of the standard deviation of a single measurement.

[0130] In one application example, suppose that after multiple measurements, the long-term standard deviation σ of the ROI of the selected grid is obtained. long =4, while the baseline mean remains at μ base =120. If n=3 is still chosen, then:

[0131] Lower limit of tolerance range = 120 − 3 × 4 = 108

[0132] Upper limit of tolerance range = 120 + 3 × 4 = 132

[0133] This method, based on long-term standard deviation, can better adapt to environmental changes and provide a more accurate tolerance range, thereby improving the accuracy of detecting the physical displacement of the camera.

[0134] The above two methods allow for flexible setting of tolerance ranges based on different needs and application scenarios, ensuring the accuracy and reliability of detecting the physical displacement of the camera.

[0135] S6: Use the camera to capture a detection video, select a region of interest in the detection video that is at the same position as the region of interest in the first video segment, and calculate the average image detection value of the region of interest in the detection video;

[0136] The detection video captured by the surveillance camera is used as detection data, and the ROI of the selected grid at the same position is selected in the detection data; the average value of the ROI image of the selected grid in the video frame in the detection data is calculated as the detection value.

[0137] The detection video captured by the surveillance camera is used as detection data. Similarly, the ROI of the selected grid at the same location can be selected in the detection data.

[0138] Subsequently, the average ROI image value of the selected grid in different video frames of the detection data can be calculated as the detection value. Specifically, for RGB, the average value of the three RGB color channels can be calculated. Similarly, to reduce the impact of illumination changes on image analysis, the RGB color space can be converted to other spaces that can separate luminance and chrominance (color information). Calculating the average image value of the same ROI in different video frames of the detection data can be done by converting the color space of the same ROI in different video frames of the detection data to the average image value after removing the luminance component. Specifically, this can be, for example, calculating the average image value of the RGB color space after removing the luminance component, thereby obtaining the average chrominance level. As mentioned above, for example, converting to the chrominance component of the YUV color space.

[0139] Similarly, the image mean can be calculated for the same ROI in different video frames of the detection data. For example, the image mean can be calculated for the same ROI in several consecutive video frames, or the image mean can be calculated for the same ROI in several non-consecutive frames. In S6, the image mean can be calculated pixel by pixel for the same ROI in different video frames of the detection data, and then summed to obtain the image mean of the ROI in different video frames; alternatively, the image mean can be calculated pixel by pixel for the same ROI in several consecutive video frames, and then summed to obtain the image mean of the ROI in different video frames; or the image mean can be calculated pixel by pixel for the same ROI in several non-consecutive frames, and then summed to obtain the image mean of the ROI in different video frames. Further details are omitted here. For RGB, the average of the three channels in a specific ROI can be used as the image mean of that ROI in different video frames. For YUV, the average of the U and V channels in a specific ROI can be used as the image mean of that ROI in different video frames, thereby removing the influence of brightness.

[0140] The mean ROI image of the selected grid in different video frames of the detection data calculated above can be used as the detection value.

[0141] S7: If the average value of the image detection is not within the tolerance range, then it is determined that the camera has shifted; otherwise, it is not.

[0142] A detection index is calculated based on the detected value and the baseline mean. If the detection index exceeds the tolerance range, it is determined that the surveillance camera has moved.

[0143] The detection index is calculated based on the detected value. Specifically, the detection index can be calculated based on the detected value and the baseline mean.

[0144] Similarly, the detection index is calculated, for example, for each selected grid, using the following formula. (Detection indicators):

[0145]

[0146] Furthermore, it can be The value is used as a detection indicator. If the detection indicator exceeds the tolerance range, it can be determined that the surveillance camera has moved.

[0147] The above embodiments have advantages in three main aspects: high efficiency, accuracy, and simplicity.

[0148] High efficiency:

[0149] No hardware dependency: This method operates entirely on video data and requires no additional physical sensors, thereby reducing hardware dependency and related costs.

[0150] Lightweight computing: By anchoring grid points and calculating statistical changes, the processing power required for real-time detection is minimized.

[0151] Low resource requirements: The algorithm can run on resource-constrained devices such as edge devices or embedded systems.

[0152] accuracy:

[0153] Robustness: By decomposing the luminance (L) component and using statistical indicators, it effectively distinguishes between camera movement and environmental changes (such as changes in lighting).

[0154] Noise robustness: Clustering and percentile-based score calculations enhance the system's robustness to noise and outliers.

[0155] Experimental verification: Experimental results show that the algorithm has high sensitivity and specificity in detecting camera movement.

[0156] Simplicity: By anchoring grid points in the frame and monitoring deviations, it is intuitive and easy to understand, and the algorithm is easy to implement and debug. Moreover, it can be expanded to different resolutions and grid configurations, adapting to various monitoring systems.

[0157] The embodiments provided in this application, after experimental evaluation, show that the algorithm processes each frame in less than 10 milliseconds on a modern CPU, and can achieve real-time analysis even without GPU support. In controlled tests, the detection accuracy exceeds 95%, and the false alarm rate is extremely low under environmental conditions such as changes in lighting or partial occlusion.

[0158] This application also provides an electronic device, including a processor, a memory, and a computer program that can run on the processor, wherein the processor executes the program to implement the method for detecting the physical displacement of a camera.

[0159] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for detecting the physical displacement of a camera.

[0160] This application also provides a computer software product, including a computer program / instructions, which, when executed by a processor, implement the method for detecting the physical displacement of a camera.

[0161] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0163] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0166] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0167] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0169] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0170] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting the physical displacement of a camera, characterized in that, The method includes: capturing a first video segment using a camera at an initial position, and dividing the first video segment into regions of interest (ROIs); calculating the image mean and standard deviation of the ROIs for different video frames in the first video segment; determining a baseline mean based on the image mean and standard deviation, selecting the video frame with the lowest standard deviation fluctuation as the baseline mean; capturing a second video segment using the camera, selecting a ROI at the same position as the ROI in the first video segment, and calculating the contrast mean of the ROIs in the second video segment; calculating a tolerance range based on the contrast mean and the baseline mean; capturing a detection video using the camera, and... A region of interest (ROI) is selected at the same location as the ROI in the first video segment. The image detection mean of the ROI in the detected video is calculated. If the image detection mean is not within the tolerance range, it is determined that the camera has shifted; otherwise, it is determined that the camera has not shifted. Determining the baseline mean based on the image mean and standard deviation includes: selecting the video frame with the lowest volatility based on the standard deviation; clustering the lowest volatility video frame with the spatial location of the ROI in the video frame with the lowest image mean and the lowest volatility video frame to obtain a selected grid in each cluster; and using the image mean of the ROI in the selected grid in the first video segment as the baseline mean.

2. The method for detecting the physical displacement of a camera as described in claim 1, characterized in that, Clustering the lowest video frame and the image mean with the spatial location of the region of interest in the video frame with the lowest volatility includes: using a dimensionality reduction method to cluster the lowest video frame and the image mean with the spatial location of the region of interest in the video frame with the lowest volatility.

3. The method for detecting the physical displacement of a camera as described in claim 1, characterized in that, Obtaining the selected grid in each cluster involves selecting the region of interest from the video frames with the lowest standard deviation within each cluster as the selected grid.

4. The method for detecting the physical displacement of a camera as described in claim 1, characterized in that, Calculating the image mean and standard deviation of the region of interest in different video frames of the first video segment includes: calculating the image mean and standard deviation of the same region of interest in different video frames of the first video segment after converting the color space to the image after removing the luminance component.

5. The method for detecting the physical displacement of a camera as described in claim 1, characterized in that, Calculating the contrast mean of the region of interest in the second video segment includes: calculating the color space mean of the region of interest in the selected grid of different video frames in the second video segment after converting it to the image mean after removing the luminance component, and using it as the contrast mean.

6. The method for detecting the physical displacement of a camera as described in claim 1, characterized in that, Calculating the tolerance range based on the comparison mean and the baseline mean includes: calculating a score value based on the comparison mean and the baseline mean; using the score value in the range from the lower water level to the upper water level as the stability index of the camera at the initial position; and setting the tolerance range based on the baseline mean and standard deviation corresponding to the stability index.

7. An electronic device, characterized in that, The device includes a processor, a memory, and a computer program that can run on the processor, wherein the processor executes the program to implement the method for detecting physical displacement of a camera as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method for detecting the physical displacement of a camera as described in any one of claims 1 to 6.

9. A computer software product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the method for detecting the physical displacement of a camera as described in any one of claims 1 to 6.

Images