Camera state judgment method based on road surface covering and Hash comparison

Abstract

The invention relates to the technical field of intelligent traffic, and discloses a camera state judgment method based on road surface covering and Hash comparison, and the method sequentially comprises the steps: extracting key frame pairs from a video stream at intervals; performing road surface region extraction on each frame of image to obtain a road surface mask; carrying out covering processing on the road surface area in each frame of image based on the mask to obtain a covered image; calculating a difference hash feature of each frame of covered image to obtain a hash code; calculating the Hamming distance between the Hash codes of the key frame pair; and judging whether the camera is in a rotating or stable state according to a comparison result of the Hamming distance and a threshold value. The method only depends on the video image data, does not need an external sensor, effectively eliminates the dynamic interference of the vehicle through covering the road surface, achieves the efficient and robust judgment of the state of the camera through combining the lightweight difference hash calculation, is low in calculation cost, and is suitable for edge equipment and complex road environments.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, specifically a camera state determination method based on road surface occlusion and hash comparison. Background Technology

[0002] In video perception systems such as road monitoring, cameras may rotate or shift due to human adjustment, wind load, or equipment loosening, causing changes in their field of view. To ensure the accuracy of subsequent analysis algorithms such as lane line recognition and vehicle tracking, it is necessary to promptly and accurately determine whether the camera is rotating or has regained stability in order to trigger scene recalibration.

[0003] Currently, common solutions for camera status determination mainly rely on external sensors such as inertial measurement units (IMUs), feature-point-based optical flow estimation, or visual SLAM (simultaneous localization and mapping) technology. These methods generally have the following limitations: the introduction of additional hardware leads to increased costs, the computational complexity of the algorithms is high, real-time performance is limited, and their performance is heavily dependent on the richness of scene textures. In real-world complex environments such as varying lighting, occlusion, or simple road textures (such as clean asphalt roads), their stability and robustness are often insufficient.

[0004] Therefore, there is an urgent need for a purely visual judgment method that does not rely on additional hardware, is computationally lightweight, and is highly robust to environmental interference. Summary of the Invention

[0005] The purpose of this application is to provide a camera state determination method based on road surface occlusion and hash comparison to solve the technical problems mentioned in the background art.

[0006] To achieve the above objectives, this application discloses the following technical solution: a camera state determination method based on road surface occlusion and hash comparison, comprising: Step S1: Extract two frames that constitute a keyframe pair from the video stream captured by the camera at preset time intervals; Step S2: Extract the road surface region for each frame of the keyframe pair and obtain the road surface mask for each frame. Step S3: Based on the road surface mask corresponding to each frame image, perform masking processing on the road surface area in the frame image respectively to obtain the corresponding masked image; Step S4: Calculate the difference hash feature for each frame of the masked image, and obtain the corresponding hash code for each frame of the masked image; Step S5: Calculate the Hamming distance between the hash codes corresponding to the two frames in the keyframe pair as an inter-frame difference metric; Step S6: Based on the comparison result between the Hamming distance and the preset threshold, determine whether the camera is in a rotating state or a stable state.

[0007] Optionally, when performing step S1 for the first time, the first frame image in the acquired video stream is taken as a historical frame; steps S2 to S4 are performed on the historical frame to obtain and store its corresponding hash code; after waiting for a preset initial interval, a current frame image is extracted from the video stream; the historical frame and the current image are combined to form the first keyframe pair.

[0008] Optionally, in step S1, extracting keyframe pairs at preset time intervals includes: When it is determined that the camera is not rotating, after waiting for the first interval, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair. When it is determined that the camera is rotating, after waiting for a second interval, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair. Wherein, the first interval duration is longer than the second interval duration.

[0009] Optionally, step S2 includes: The semantic segmentation model is used to segment each frame of the keyframe pair, extract the road surface region in the image, and output the road surface mask corresponding to each frame.

[0010] Optionally, the semantic segmentation model is an encoder-decoder structure; The encoder includes a backbone network and a spatial pyramid pooling module; the backbone network is used to receive the input image and extract features, and output high-level feature maps and low-level feature maps; the spatial pyramid pooling module is used to receive the high-level feature maps, perform dilated convolution with multiple dilation rates and global average pooling operations in parallel, and concatenate and convolve the resulting multiple feature maps to generate encoded features. The decoder is used to receive the low-level feature map and the encoded features, perform channel dimensionality reduction on the low-level feature map to obtain dimensionality-reduced features, and perform upsampling on the encoded features to obtain upsampled features; concatenate the dimensionality-reduced features and the upsampled features, perform convolution on the concatenation result, and upsample the convolution result to the same resolution as the input image, and output the road surface mask.

[0011] Optionally, in step S3, the process of masking the road surface region in each frame of the image based on the road surface mask corresponding to that frame includes: The images in the keyframe pairs are converted into grayscale images and used as the images to be masked. In the image to be masked, the pixel value at the pixel position marked by the road surface mask is modified to a preset background value to obtain the masked image.

[0012] Optionally, the grayscale image is obtained by weighting the red channel value, green channel value, and blue channel value of each pixel in the keyframe pair to obtain the grayscale value of that pixel, and then forming the corresponding grayscale image from the grayscale values ​​of all pixels.

[0013] Optionally, in step S4, calculating the difference hash feature includes: S41. Scale the masked image to an intermediate image of a preset size; S42. Convert the intermediate image into a grayscale image and use it as the image to be compared. S43. Compare the adjacent pixels in each row of the image to be compared. If the pixel value on the left is greater than the pixel value on the right, record it as the first binary value; otherwise, record it as the second binary value. Generate the binary sequence of that row. S44. Concatenate the binary sequences of all rows to form the hash code.

[0014] Optionally, step S6 includes: If the Hamming distance is less than the preset threshold, it is determined that the camera has not rotated or the camera has stabilized again. If the Hamming distance is greater than or equal to the preset threshold, it is determined that the camera has rotated or the camera is unstable.

[0015] Optionally, the method further includes: when it is determined that the camera has entered a restabilized state from a rotating state, outputting a trigger signal to trigger calibration of the road scene captured by the camera.

[0016] Beneficial effects: The camera state judgment method based on road surface occlusion and hash comparison in this application adopts a combination of road surface area occlusion and differential hash calculation, which realizes the goal of efficiently and accurately judging the rotation or stable state of the camera by relying solely on video image data, avoiding dependence on external sensors and reducing system cost and deployment complexity. By occluding the road surface area where dynamically passing vehicles are located, the main motion interference is effectively eliminated, and the judgment is focused on the changes in the static part of the scene, improving robustness and accuracy in real traffic environments. At the same time, the hash calculation method has low computational cost and fast response speed, making it particularly suitable for deployment in edge computing devices or monitoring systems with high real-time requirements. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the camera state determination method based on road surface occlusion and hash comparison provided in this application embodiment. Detailed Implementation

[0019] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0020] In intelligent video analytics systems based on fixed cameras, such as road monitoring, traffic flow statistics, and violation detection, cameras are typically pre-positioned at specific locations and angles to ensure their field of view covers the target road area and provides a stable spatial reference for subsequent video analytics algorithms (such as lane detection, vehicle tracking and classification, and event recognition). However, during actual deployment and long-term operation, cameras may experience changes in physical position or angle due to various reasons: for example, maintenance personnel may manually rotate the camera for cleaning, repair, or adjusting the monitoring range; environmental factors such as strong winds and vibrations may cause the mounting bracket to loosen, resulting in a slow camera shift; even unexpected events such as vehicle collisions with light poles may cause a significant change in the camera's orientation. These rotations or shifts, even if small, can cause a deviation between the camera's actual field of view and the system's pre-set calibration parameters. If this change in state is not detected in a timely and accurate manner and recalibrated, all subsequent analysis algorithms relying on a fixed viewpoint will operate based on an incorrect spatial reference, leading to a series of problems such as incorrect lane recognition, distorted vehicle trajectory tracking, and event detection failure, severely impacting the reliability and availability of the entire system.

[0021] Currently, the industry primarily relies on the following technical approaches to perceive the motion state of cameras: The first approach uses physical sensors such as inertial measurement units (IMUs). This method integrates gyroscopes, accelerometers, and other devices into the camera body to directly measure the camera's angular velocity and acceleration, and then integrates these to calculate its attitude changes. However, introducing additional hardware not only increases the material cost and power consumption of the equipment but also places higher demands on installation processes and sensor calibration, hindering large-scale deployment and upgrades in existing, massive surveillance equipment. The second approach is based on computer vision-based feature point optical flow estimation methods. This method detects and tracks feature points (such as corners and edges) between consecutive video frames, calculates the motion vectors (optical flow) of these feature points, and then infers the overall motion of the camera. While this method does not require additional hardware, its computational complexity is high, placing significant pressure on real-time processing. More importantly, its performance is highly dependent on the presence of rich, stable, and repeatable texture features in the scene. In road monitoring scenarios, especially under complex conditions such as low light at night, blurred images due to rain, snow, fog, or haze, large-area obstructions (e.g., large vehicles occupying the field of view for extended periods), or roads with monotonous textures (e.g., newly paved black asphalt), the number of reliable feature points decreases drastically, leading to optical flow calculation failures or significant errors, rendering state judgment completely ineffective. The third category utilizes Visual Simultaneous Localization and Mapping (SLAM) technology. This approach aims to simultaneously estimate camera motion and build a scene map, theoretically providing richer motion information. However, its algorithms are more extensive and complex, consuming enormous computational resources, making it difficult to meet the general requirements of real-time performance and lightweight design in road monitoring systems. It is also limited by scene texture features and faces challenges in map initialization and dynamic object processing, making it unsuitable as a simple camera state monitoring module embedded in existing systems.

[0022] In summary, existing technical solutions often struggle to achieve a good balance between cost, computational efficiency, environmental robustness, and ease of deployment when dealing with the specific application scenario of road monitoring. Therefore, this embodiment aims to overcome these shortcomings by providing a method for determining camera rotation or stable state that utilizes only existing video stream data, is computationally lightweight, and is highly adaptable to complex road environments (especially dynamic vehicle interference and a lack of static textures).

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application. Secondly, in this document, the term "comprising" is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0024] A camera state determination method based on road surface occlusion and hash comparison, such as Figure 1 As shown, the steps are as follows: Step S1: Extract two frames that constitute a keyframe pair from the video stream captured by the camera at preset time intervals. The video stream is continuously captured by network cameras or high-definition cameras deployed at road monitoring points and transmitted to a processing unit (such as an edge computing device or server). The keyframe pair consists of a historical frame and a current frame, used for inter-frame difference comparison.

[0025] Step S2: Extract the road surface region for each frame in the keyframe pair and obtain the road surface mask for each frame. This step aims to accurately identify the pixel regions in the image that belong to the road (such as asphalt or cement road surface).

[0026] Step S3: Based on the road surface mask corresponding to each frame of the image, perform masking processing on the road surface region in that frame of the image to obtain the corresponding masked image. This processing excludes the area where dynamic vehicles are usually located from subsequent calculations.

[0027] Step S4: Calculate the difference hash feature for each masked image frame, and obtain the corresponding hash code for each masked image frame. The hash code is a compact and efficient fingerprint representation of image content.

[0028] Step S5: Calculate the Hamming distance between the hash codes of the two frames in the keyframe pair as an inter-frame difference metric. The Hamming distance directly reflects the difference in the number of bits between the hash codes of the two frames.

[0029] Step S6: Based on the comparison result between the Hamming distance and the preset threshold, determine whether the camera is in a rotating state or a stable state.

[0030] Based on the above, this embodiment combines road surface occlusion with differential hashing. Occlusion effectively isolates the interference from dynamic objects such as frequently moving vehicles on the road, allowing the difference measurement to focus on changes in the "static background" such as buildings, vegetation, and static traffic facilities on both sides of the road. The differential hashing algorithm is fast and has a certain tolerance for minor changes in image brightness, but it is sensitive to geometric changes such as rotation and scaling of image content. The synergy of these two methods enables this method to robustly perceive changes in the visual content of the static background caused by camera rotation with extremely low computational cost, thereby achieving efficient and accurate camera status judgment without the need for any external sensors.

[0031] In determining the state of a continuous video stream, the algorithm faces a specific problem upon startup: the lack of initial historical reference frames for comparison. If two frames are selected directly to begin calculation, the differences between the first pair of keyframes lack a benchmark significance, which may lead to misjudgment at the moment of startup.

[0032] Specifically, to solve this initialization problem, as an optional implementation method in this embodiment, algorithm initialization is required when step S1 is executed for the first time. This specifically includes: taking the first frame image in the acquired video stream as a historical frame, denoted as... ; Regarding this historical frame Perform steps S2 to S4 to obtain and store the corresponding hash codes. Waiting for a preset initial interval. Then, extract a frame of the current image from the video stream, denoted as... ; to historical frames With the current image Form the first keyframe pair Among them, the initial interval duration An empirical value can be used, such as 0.5 seconds.

[0033] Based on the above, this embodiment establishes a correct starting point and reference benchmark for the entire continuous judgment process. By preprocessing and storing the hash code of the first frame, it ensures that the difference calculation of the first key frame pair can be performed normally, enabling the algorithm to enter an effective monitoring state from the start, thus solving the problem that the algorithm cannot perform effective comparison or is prone to misjudgment due to the lack of historical reference during the startup phase.

[0034] In long-term monitoring scenarios, sampling and calculation at a fixed high frequency will place an unnecessary computational burden on edge devices; while if the sampling frequency is too low, it may be unable to capture the camera's brief rotation or accurately determine its stable moment in time, resulting in a problem of difficulty in balancing computational efficiency and the timeliness of state judgment.

[0035] Furthermore, to balance computational load and detection response speed, as an optional implementation method in this embodiment, keyframe pairs are extracted at preset time intervals, employing an adaptive time interval strategy, specifically including: When it is determined from step S6 that the camera is in a non-rotating (i.e., stable) state, the first interval duration is adopted. After waiting, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair.

[0036] When it is determined from step S6 that the camera is in a rotating (or unstable) state, the second interval duration is adopted. After waiting, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair.

[0037] Among them, the first interval duration Greater than the second interval duration .For example, Two seconds are acceptable. 0.5 seconds is acceptable.

[0038] Based on the above, the adaptive time interval strategy adopted in this embodiment optimizes the allocation of system computing resources while ensuring the continuity of status monitoring. During the camera's stabilization period, a longer sampling interval is used, significantly reducing the computational frequency and system load, making it suitable for long-term monitoring. Once rotation or instability is detected, it immediately switches to a high-frequency (short-interval) sampling mode, which can densely capture the camera's motion process, thereby capturing the moment of re-stabilization more accurately and promptly, improving the timeliness and response speed of status judgment. Thus, while ensuring lightweight overall computation, the timeliness of capturing camera rotation events and their stabilization moments is improved.

[0039] Accurately separating the road surface area from the static background is a prerequisite for eliminating vehicle interference. Traditional segmentation methods based on color or texture thresholds have poor adaptability and are prone to failure under different lighting, weather, and road types, thus failing to provide reliable input for subsequent steps.

[0040] Specifically, to obtain robust and accurate road surface region identification, as an optional implementation method in this embodiment, step S2 specifically includes: segmenting each frame image in the keyframe pair using a semantic segmentation model, extracting the road surface region in the image, and outputting the road surface mask corresponding to each frame image. Road surface mask It is a binary image in which pixel values ​​belonging to the road surface area are marked as 1 (or 255), and pixel values ​​of non-road surface areas are marked as 0.

[0041] Semantic segmentation is a key technology for achieving pixel-level classification in computer vision. The road segmentation models used in this embodiment (such as DeepLabV3+, UNet, etc.) are typically pre-trained on large road scene datasets with fine pixel-level annotations (such as Cityscapes, Mapillary Vistas) to give them strong generalization capabilities, enabling them to accurately and robustly segment the "road" category from images containing complex backgrounds, different lighting and weather conditions.

[0042] Based on the above, this embodiment employs a deep learning-based semantic segmentation model to automatically obtain road surface masks, overcoming the limitations of traditional manual rule-based methods such as color and texture. This method can adapt with high precision to various real-world road environments (such as different types of asphalt roads, cement roads, slippery surfaces, partially obscured surfaces, etc.) and complex environmental conditions, achieving high-precision and automated extraction of road surface areas. This provides a reliable and accurate foundation for subsequent effective area masking, and is a key prerequisite for the high robustness of this method.

[0043] Although semantic segmentation models can extract road surface regions, road scenes often contain scale variations (narrow road surfaces in the distance) and similar-looking distractions (such as gray sidewalks). General models may lack sufficient detail and boundary accuracy, affecting the accuracy of occlusion.

[0044] To improve segmentation accuracy, especially boundary details and multi-scale adaptability, the semantic segmentation model is an encoder-decoder structure as a further optional implementation method in this embodiment, based on the aforementioned semantic segmentation model.

[0045] The encoder consists of a backbone network and a spatial pyramid pooling module. The backbone network (e.g., using a ResNet101 architecture with dilated convolutions replaced in its final stage) receives the input image and performs multi-level feature extraction, outputting a high-level feature map containing rich semantic information. and low-level feature maps containing more spatial details The spatial pyramid pooling module is used to receive high-level feature maps. The algorithm performs dilated convolutions with various dilation rates (e.g., learning rates of 6, 12, and 18) and a global average pooling operation in parallel. The resulting feature maps are then concatenated and fused using a 1×1 convolutional layer for dimensionality reduction, ultimately generating encoded features that incorporate multi-scale contextual information. .

[0046] The decoder is used to receive the low-level feature map $F_{low}$ and the encoded features $F_{enc}$. First, the low-level feature map... Perform 1×1 convolution to achieve channel dimensionality reduction and obtain dimensionality-reduced features. At the same time, for coding features Perform bilinear interpolation upsampling to make its spatial size consistent with... The same, yielding upsampled features Next, the dimensionality reduction features will be... With upsampling features The image is then stitched together along the channel dimension. Next, a 3×3 convolution is performed on the stitched result to further fuse features. Finally, bilinear interpolation upsampling is performed again until the resolution is restored to the same as the input image, outputting the road surface mask. .

[0047] Based on the above, this embodiment employs a semantic segmentation model with an encoder and decoder structure, particularly incorporating a spatial pyramid pooling module. This allows the model to capture contextual information at different scales within the image while maintaining the spatial resolution of the feature map. This is crucial for accurate road segmentation, as road areas may appear at different scales (e.g., wide in the foreground, narrow in the background) and need to be distinguished from areas with similar appearances (e.g., gray sidewalks, parking lots). The decoder effectively recovers the details of the segmentation boundaries by fusing low-level features. This design improves the boundary accuracy of road segmentation and its adaptability to complex scenes, ensuring high precision and boundary clarity in road mask extraction. In particular, it better distinguishes between different areas with similar appearances, ensuring mask quality and laying a solid foundation for the reliability of subsequent steps, indirectly improving the accuracy of the final state determination.

[0048] Even with accurate road surface masks, effectively eliminating the influence of dynamic vehicles and focusing calculations on the static background requires a clear processing flow. Processing directly on a full-color image is computationally intensive, and the mask information needs to be efficiently converted into operations on the image data.

[0049] Specifically, to achieve efficient interference elimination, as an optional implementation method in this embodiment, in step S3, based on the road surface mask corresponding to each frame of the image, the road surface area in that frame of the image is masked, specifically including: First, the original color image in the keyframe pair is converted to a grayscale image, which is then used as the image to be masked. .

[0050] Then, in the image to be masked In the middle, the road surface mask The identified pixel position (i.e. The pixel values ​​at the specified location are uniformly modified to the preset background values. (For example, pure white 255), thus obtaining the final masked image. For the non-road surface areas marked in the mask ( ), its pixel value is The middle remains unchanged.

[0051] Based on the above, this embodiment reduces data dimensionality by converting the image to grayscale. More importantly, it uses precise road surface masks to transform the dynamic area where vehicles are traveling into a background with constant brightness. This fundamentally eliminates the interference of vehicle movement and color changes on inter-frame difference calculations, forcing subsequent hash difference calculations to fully reflect the relative changes of static backgrounds (such as buildings, guardrails, and green belts) on both sides of the road. This makes the algorithm extremely sensitive to background displacement caused by camera rotation, while being almost immune to changes in traffic flow on the road, improving the specificity and robustness of the judgment.

[0052] Converting color images to grayscale is a common operation, but different conversion methods can affect the visual information retained in the grayscale image. A simple averaging method may weaken important visual features, necessitating a conversion method that better aligns with human visual perception so that subsequent comparisons based on grayscale values ​​can more accurately reflect the true differences in visual content.

[0053] Therefore, based on the aforementioned masking process, as a further optional implementation method of this embodiment, the grayscale image is obtained in the following way: for each pixel of the image in the keyframe pair, its red channel value is... Green channel value and blue channel value Perform a weighted calculation to obtain the grayscale value of the pixel. The grayscale values ​​of all pixels together form the corresponding grayscale image. .

[0054] In some implementations, the grayscale value calculation formula used is as follows: Among them, 0.299, 0.587, and 0.114 are the industry standard weights for converting the conventional RGB color space to grayscale; This represents the original pixel value in the RGB channel of the pixel whose grayscale value is to be calculated.

[0055] Based on this, this embodiment utilizes the physiological characteristics of the human eye's sensitivity to different colors of light to make the generated grayscale image more closely resemble the subjective visual experience of humans in terms of brightness distribution. This ensures that the brightness relationship between pixels, upon which the subsequent differential hash calculation relies, can more realistically represent the scene structure and content changes perceptible to the human eye, thus improving the consistency between the state judgment result and visual perception.

[0056] Furthermore, to further improve the distinction between static backgrounds and dynamic vehicle areas in road monitoring scenarios and enhance the sensitivity of subsequent hash calculations to background changes, this embodiment proposes a scene-adaptive weighted grayscale method. This method is first based on road surface masks. By statistically analyzing the color distribution characteristics of road surface areas and non-road surface (static background) areas in the image, and dynamically adjusting the grayscale weights, the texture of the static background is more significantly preserved in the grayscale image.

[0057] Specifically, the calculation of the regional color mean involves calculating the RGB channel mean of the non-road surface area (static background), denoted as... And the average RGB channel values ​​of the road surface area, denoted as .

[0058] The weight adjustment vector generation process involves calculating the color difference vector between the static background and the road surface area, and using this vector to generate the weight adjustment bias. A specific calculation formula is as follows: in, To adjust the strength coefficient (for example, it can be taken as 0.1), For a very small constant (e.g., 10) -8 This is used to prevent division by zero errors. The formula normalizes color differences and then proportionally converts them into weighted adjustment amounts. This is the R-channel color difference vector between the static background and the road surface area. This is the G-channel color difference vector between the static background and the road surface area. This is the B-channel color difference vector between the static background and the road surface area. To adjust the bias for the weights on the R channel, To adjust the bias for weights on the G channel, Adjust the bias for the weights on channel B.

[0059] Therefore, the adaptive grayscale calculation process is as follows: The bias adjustment is applied to the standard weights for RGB-to-grayscale conversion to obtain adaptive weights, and the adaptive grayscale value of the target pixel is calculated. The specific formula is: in, To integrate the adaptive weights of the RGB channels after adjusting the bias; The grayscale value is obtained through adaptive calculation.

[0060] Finally, for Perform truncation (i.e., limit its value to the range of 0-255 to match the valid range of pixel values ​​in an 8-bit grayscale image).

[0061] Based on the above, this embodiment dynamically adjusts the weights of the RGB channels by analyzing the inherent color differences between the static background and the road surface in the current scene. Its design principle is to enhance the contribution of the color component in the static background area that differs most significantly from the road surface in grayscale conversion. For example, if the static background contains a large amount of green vegetation (high G component), while the road surface is gray (R, G, B are similar), the algorithm will automatically increase the weight of the G component, making the vegetation texture in the background more prominent in the grayscale image. Thus, when the camera rotates, these enhanced background texture changes are more easily captured by the differential hashing algorithm, effectively improving the sensitivity and robustness of camera rotation detection in complex or low-contrast scenes, and solving the problem of difficulty in detecting minute rotations when the background and road surface colors are similar.

[0062] To quickly determine the state, an image feature representation method that is computationally efficient and sensitive to camera rotation is needed. Directly calculating features on high-resolution images is computationally complex, and a feature encoding mechanism that is insensitive to lighting changes but sensitive to geometric changes is required.

[0063] To meet the requirements of lightweight design and sensitivity, as an optional implementation method in this embodiment, step S4 involves calculating the difference hash feature, specifically including the following sub-steps: S41. Mask the image Scale the image to a preset size. For example, scaling to 9 pixels wide and 8 pixels high (9×8). Scaling operations typically use a fast bilinear interpolation algorithm.

[0064] S42, Transfer the intermediate image Convert to grayscale image (if) (Non-grayscale) and used as the image to be compared. .

[0065] S43. Comparison Images Each row of pixels is compared pairwise. For each row, the grayscale values ​​of two adjacent pixels are compared from left to right. If the left pixel value is greater than the right pixel value, a binary '1' is recorded at that position; otherwise, a '0' is recorded. Each row comparison will produce a binary sequence of length 8.

[0066] S44. Concatenate the binary sequences generated from all 8 rows in row order to form a 64-bit binary string. This string is the difference hash code of the image. .

[0067] Based on the above, this embodiment, by scaling to an extremely small size (e.g., 8×9), significantly compresses the data volume while preserving the macroscopic structural information of the image, making the computation extremely lightweight. By comparing adjacent pixels to generate a binary string, the hash code is insensitive to changes in the overall brightness of the image (uniform changes in illumination do not affect the size relationship of adjacent pixels), but is extremely sensitive to geometric transformations of the image content (such as changes in pixel position relationships caused by rotation and translation). This characteristic aligns with the requirements of camera rotation detection: ignoring gradual changes in illumination and focusing on capturing the displacement of background structures. The entire calculation process is simple and fast, making it ideal for quickly detecting geometric changes in image content caused by camera rotation on resource-constrained edge devices (in resource-constrained environments).

[0068] After calculating the hash difference (Hamming distance), a clear and stable decision rule is needed to determine the camera's state. How to set a reasonable threshold and make a binary decision of "rotate" or "remain stable" based on it is a key decision logic problem connecting computation and application.

[0069] Specifically, as an optional implementation of this embodiment, step S6 specifically includes: If Hamming distance Less than the preset threshold If the camera does not rotate, it is determined that the camera has either returned to a stable state or has not rotated.

[0070] If Hamming distance Greater than or equal to the preset threshold If the camera rotates, it is determined that the camera has not yet stabilized.

[0071] Among them, the preset threshold It is a key parameter whose value can be obtained through experimental calibration by collecting a large amount of video data in both stable and rotating states in the target scene, aiming to achieve the best balance between sensitivity and noise resistance.

[0072] Based on the above, the Hamming distance in this embodiment It directly quantifies the difference in the hash codes between two image frames, reflecting the degree of visual content change in a static background area. When the camera is stationary, the background remains unchanged. Maintained at a low level close to 0 (less than) When the camera rotates, the background shifts on the imaging plane, causing a significant change in the hash code. Increase (greater than or equal to) By using a clearly defined threshold for binary judgment, continuous difference measurement is transformed into discrete state decision-making. The logic is clear, the decision-making speed is fast, and it is easy to integrate into various control systems, realizing efficient mapping from image differences to physical states.

[0073] In addition, a fixed preset threshold It is difficult to adapt to changes in different monitoring locations, lighting conditions (such as day-night cycles), and static background complexity. Therefore, this embodiment proposes an adaptive threshold calculation method based on historical frame sequence statistics to replace or initialize a fixed threshold. The specific process is as follows: (1) Acquisition of differential sequences during the stable period: After system initialization or during a period when the system is continuously judged to be stable, continuous acquisition is carried out. One (e.g.) The Hamming distances of consecutive keyframe pairs constitute a difference sequence. This sequence reflects the baseline level of background hash differences caused by factors such as gradual changes in lighting, slight movement of leaves, and encoded noise when there is no camera rotation.

[0074] (2) Adaptive threshold calculation: Based on the sequence, calculate the adaptive threshold. The calculation formula is: in, The mean of the differential sequences. Its standard deviation, This is the sensitivity coefficient (typically between 2.5 and 3.5). Mean. The standard reflects the central location of baseline differences. It reflects the degree of baseline fluctuation.

[0075] (3) Threshold update mechanism: In subsequent operation, when the state is continuously judged to be stable for more than a certain number of frames, new historical difference data can be used to update the threshold. and Thus achieving Dynamic adjustments.

[0076] At this point, in step S6, the Hamming distance is adaptively compared with the adaptive threshold. This is used to compare and determine the camera's status. Therefore, based on this adaptive threshold design, the threshold is set to a value based on this baseline statistical characteristic (…). The dynamic value of ). This allows the judgment threshold to automatically adapt to the inherent noise level of different scenes: in calm scenes ( At low thresholds (small), sensitivity is improved; under conditions with more dynamic interference (such as swaying trees), the threshold is lower, thus increasing sensitivity; under conditions with more dynamic interference (such as swaying trees), In scenarios with large scale, the threshold is automatically increased to enhance anti-interference capabilities and prevent false positives. This design improves the method's adaptability and overall robustness in various complex real-world environments.

[0077] If subsequent processing cannot be automatically triggered after the camera is detected to have stabilized, manual intervention is still required. The system's automated closed-loop capability is insufficient, and the value of state awareness is not fully realized.

[0078] Furthermore, to achieve fully automatic adaptation of the system, as an optional implementation method in this embodiment, the method also includes an application closed-loop step: when it is determined that the camera has entered a re-stabilized state from a rotating state, a trigger signal is generated and output. This signal is used to trigger the subsequent system to calibrate the new road scene currently captured by the camera. The calibration may include updating the lane line model, re-delineating the detection area, calibrating camera parameters, etc., so that subsequent intelligent traffic analysis algorithms (such as vehicle detection, traffic statistics, and event detection) can adapt to the new camera field of view.

[0079] Based on the above, this embodiment seamlessly integrates the "state awareness" function of the method with the "adaptive calibration" function of the intelligent transportation system. The output trigger signal enables the entire monitoring system to possess self-adjustment and continuous optimization capabilities. After automatically detecting changes in camera state and confirming its stability, the system proactively initiates a recalibration process without manual intervention. This improves the automation level, long-term operational stability, and maintenance efficiency of large-scale roadside monitoring systems, ensuring that the upper-level video analysis algorithm is always based on accurate spatial environmental information. This guarantees the autonomy, long-term reliability, and maintenance efficiency of the overall system performance, achieving a complete automated closed loop of "perception-decision-execution."

[0080] In the embodiments provided in this application, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any suitable combination thereof. For hardware implementation, the processor may be implemented in one or more of the following: application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to implement the functions described herein, or combinations thereof. For software implementation, some or all of the processes of the embodiments may be performed by a computer program instructing the associated hardware. During implementation, the program may be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available medium accessible to a computer. Computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer.

[0081] Finally, it should be noted that the above description is only a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A camera state determination method based on road surface occlusion and hash comparison, characterized in that, In order, they include: Step S1: Extract two frames that constitute a keyframe pair from the video stream captured by the camera at preset time intervals; Step S2: Extract the road surface region for each frame of the keyframe pair and obtain the road surface mask for each frame. Step S3: Based on the road surface mask corresponding to each frame image, perform masking processing on the road surface area in the frame image respectively to obtain the corresponding masked image; Step S4: Calculate the difference hash feature for each frame of the masked image, and obtain the corresponding hash code for each frame of the masked image; Step S5: Calculate the Hamming distance between the hash codes corresponding to the two frames in the keyframe pair as an inter-frame difference metric; Step S6: Based on the comparison result between the Hamming distance and the preset threshold, determine whether the camera is in a rotating state or a stable state.

2. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, When step S1 is executed for the first time, the first frame image in the acquired video stream is taken as a historical frame; steps S2 to S4 are executed on the historical frame to obtain and store its corresponding hash code; after waiting for a preset initial interval, a current frame image is extracted from the video stream; the historical frame and the current image are combined to form the first keyframe pair.

3. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, In step S1, extracting keyframe pairs at preset time intervals includes: When it is determined that the camera is not rotating, after waiting for the first interval, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair. When it is determined that the camera is rotating, after waiting for a second interval, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair. Wherein, the first interval duration is longer than the second interval duration.

4. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, Step S2 includes: The semantic segmentation model is used to segment each frame of the keyframe pair, extract the road surface region in the image, and output the road surface mask corresponding to each frame.

5. The camera state determination method based on road surface occlusion and hash comparison according to claim 4, characterized in that, The semantic segmentation model is an encoder-decoder structure; The encoder includes a backbone network and a spatial pyramid pooling module; the backbone network is used to receive the input image and perform feature extraction, outputting high-level feature maps and low-level feature maps. The spatial pyramid pooling module is used to receive the high-level feature map, perform dilated convolution with multiple dilation rates and global average pooling operations in parallel, and concatenate and convolve the resulting multiple feature maps to generate encoded features. The decoder is used to receive the low-level feature map and the encoded features, perform channel dimensionality reduction on the low-level feature map to obtain dimensionality-reduced features, and perform upsampling on the encoded features to obtain upsampled features; concatenate the dimensionality-reduced features and the upsampled features, perform convolution on the concatenation result, and upsample the convolution result to the same resolution as the input image, and output the road surface mask.

6. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, In step S3, the process of masking the road surface region in each frame of the image based on the road surface mask corresponding to that frame includes: The images in the keyframe pairs are converted into grayscale images and used as the images to be masked. In the image to be masked, the pixel value at the pixel position marked by the road surface mask is modified to a preset background value to obtain the masked image.

7. The camera state determination method based on road surface occlusion and hash comparison according to claim 6, characterized in that, The grayscale image is obtained by weighting the red, green, and blue channel values ​​of each pixel in the keyframe pair to obtain the grayscale value of that pixel, and then forming the corresponding grayscale image from the grayscale values ​​of all pixels.

8. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, In step S4, calculating the difference hash feature includes: S41. Scale the masked image to an intermediate image of a preset size; S42. Convert the intermediate image into a grayscale image and use it as the image to be compared. S43. Compare the adjacent pixels in each row of the image to be compared. If the pixel value on the left is greater than the pixel value on the right, record it as the first binary value; otherwise, record it as the second binary value. Generate the binary sequence of that row. S44. Concatenate the binary sequences of all rows to form the hash code.

9. The camera state determination method based on road surface occlusion and hash comparison according to claim 1 or 3, characterized in that, Step S6 includes: If the Hamming distance is less than the preset threshold, it is determined that the camera has not rotated or the camera has stabilized again. If the Hamming distance is greater than or equal to the preset threshold, it is determined that the camera has rotated or the camera is unstable.

10. The camera state determination method based on road surface occlusion and hash comparison according to claim 9, characterized in that, The method further includes: when it is determined that the camera has entered a restabilized state from a rotating state, outputting a trigger signal to trigger the calibration of the road scene captured by the camera.

Images