A camera state judgment method based on road surface masking and hash comparison

Abstract

The application relates to the technical field of intelligent transportation, and discloses a camera state judgment method based on road surface masking and hash comparison, which sequentially comprises the following steps: key frame pairs are extracted from a video stream at intervals; a road surface area of each frame of image is extracted to obtain a road surface mask; the road surface area in each frame of image is subjected to masking treatment based on the mask to obtain a masked image; a difference hash feature of each frame of masked image is calculated to obtain a hash code; a Hamming distance between the hash codes of the key frame pairs is calculated; and whether the camera is in a rotating or stable state is judged according to a comparison result of the Hamming distance and a threshold value. The application only relies on video image data, does not need external sensors, effectively eliminates vehicle dynamic interference by masking the road surface, and realizes efficient and robust judgment of the camera state by combining light difference hash calculation, so that the calculation cost is low and the application is suitable for edge devices and complex road environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a camera state judgment method based on road surface masking and hash comparison. BACKGROUND

[0002] In a video perception system such as road monitoring, the camera may rotate or deviate due to human adjustment, wind load or equipment loosening, resulting in changes in its field of view. In order to ensure the accuracy of subsequent lane line recognition, vehicle tracking and other analysis algorithms, it is necessary to timely and accurately judge whether the camera is in the process of rotation or has been re-stabilized, so as to trigger the re-calibration of the scene.

[0003] At present, the common scheme to realize the camera state judgment mainly relies on external sensors such as inertial measurement unit (IMU), optical flow estimation based on feature points or visual SLAM (simultaneous localization and mapping) technology. These methods generally have the following limitations: the introduction of additional hardware leads to increased cost, large algorithm calculation complexity, limited real-time performance, and their performance is severely dependent on the richness of scene texture, and in actual complex environments such as light changes, occlusions or single road surface texture (such as clean asphalt road surface), the stability and robustness are often insufficient.

[0004] Therefore, there is an urgent need for a pure visual judgment method that does not rely on additional hardware, has light calculation and strong robustness to environmental interference. SUMMARY

[0005] The purpose of the present application is to provide a camera state judgment method based on road surface masking and hash comparison to solve the technical problems proposed in the background.

[0006] In order to achieve the above purpose, the present application discloses the following technical scheme: a camera state judgment method based on road surface masking and hash comparison, comprising the following steps in sequence:

[0007] Step S1: extracting two frames of images constituting a key frame pair from a video stream collected by a camera at a preset time interval;

[0008] Step S2: performing road surface region extraction on each frame of image in the key frame pair, and obtaining a road mask for each frame of image;

[0009] Step S3: based on the road mask corresponding to each frame of image, respectively masking the road surface region in the frame of image to obtain the corresponding masked image;

[0010] Step S4: calculating the difference hash feature of each frame of the masked image, and obtaining the corresponding hash code for each frame of the masked image;

[0011] Step S5: calculating the Hamming distance between the hash codes corresponding to the two images in the key frame pair as the inter-frame difference measure;

[0012] Step S6: judging whether the camera is in a rotating state or a stable state according to the comparison result of the Hamming distance and a preset threshold.

[0013] Optionally, when the step S1 is performed for the first time, a first frame image in the acquired video stream is taken as a history frame; the steps S2 to S4 are performed on the history frame to obtain and store the hash code corresponding thereto; after waiting for a preset initial interval duration, a current frame image is extracted from the video stream; and the history frame and the current frame image are taken as a first key frame pair.

[0014] Optionally, in the step S1, the key frame pair is extracted at a preset time interval, including:

[0015] When it is judged that the camera is in a non-rotating state, the next frame image is extracted from the video stream after waiting for a first interval duration, and a new key frame pair is formed with the image processed last time;

[0016] When it is judged that the camera is in a rotating state, the next frame image is extracted from the video stream after waiting for a second interval duration, and a new key frame pair is formed with the image processed last time;

[0017] The first interval duration is greater than the second interval duration.

[0018] Optionally, the step S2 includes:

[0019] Each frame image in the key frame pair is segmented using a semantic segmentation model, a road surface region in the image is extracted, and the road surface mask corresponding to each frame image is output.

[0020] Optionally, the semantic segmentation model is an encoder-decoder structure.

[0021] The encoder includes a backbone network and a spatial pyramid pooling module; the backbone network is configured to receive an input image and perform feature extraction, output high-level feature maps and low-level feature maps; and the spatial pyramid pooling module is configured to receive the high-level feature maps, perform a plurality of dilated convolution operations at different expansion rates and a global average pooling operation in parallel, and perform splicing and convolution processing on a plurality of feature maps obtained to generate encoded features.

[0022] The decoder is configured to receive the low-level feature map and the encoded feature, perform channel dimension reduction on the low-level feature map to obtain a reduced dimension feature, and perform up-sampling on the encoded feature to obtain an up-sampled feature; splice the reduced dimension feature and the up-sampled feature, perform convolution on the spliced result, and up-sample the convolution result to the same resolution as the input image, and output the road mask.

[0023] Optionally, in the step S3, based on the road mask corresponding to each frame of image, the road area in the frame of image is respectively masked, including:

[0024] Converting the image in the key frame pair into a gray image as a to-be-masked image;

[0025] In the to-be-masked image, the pixel value at the pixel position identified by the road mask is modified into a preset background value, and the masked image is obtained.

[0026] Optionally, the gray image is obtained by performing weighted calculation on the red channel value, the green channel value and the blue channel value of each pixel point in the image in the key frame pair to obtain the gray value of the pixel point, and the corresponding gray image is formed by the gray values of all pixel points.

[0027] Optionally, in the step S4, the difference value hash feature is calculated, including:

[0028] S41, scaling the masked image to an intermediate image of a preset size;

[0029] S42, converting the intermediate image into a gray image and taking it as a to-be-compared image;

[0030] S43, comparing adjacent pixels in each row of the to-be-compared image, if the left pixel value is greater than the right pixel value, recording it as a first binary value, otherwise recording it as a second binary value, and generating a binary sequence of the row;

[0031] S44, splicing the binary sequences of all rows to form the hash code.

[0032] Optionally, the step S6 includes:

[0033] If the Hamming distance is less than the preset threshold, it is judged that the camera does not rotate or the camera has been re-stabilized;

[0034] If the Hamming distance is greater than or equal to the preset threshold, it is judged that the camera rotates or the camera is not stabilized.

[0035] Optionally, the method further comprises: outputting a trigger signal to trigger calibration of the road scene photographed by the camera when it is judged that the camera enters the re-stabilization state from the rotation state.

[0036] Beneficial effects: the camera state judgment method based on road surface masking and hash comparison provided by the present application adopts the technical means of combining road surface area masking and difference hash calculation, realizes the purpose of efficiently and accurately judging the rotation or stabilization state of the camera only relying on video image data, avoids the dependence on external sensors, reduces the system cost and deployment complexity; by masking the road surface area where the dynamic passing vehicles are located, the main motion interference is effectively excluded, the focus of the judgment is focused on the change of the static part of the scene, and the robustness and accuracy in the real traffic environment are improved; at the same time, the hash calculation method has small calculation cost and fast response speed, and is especially suitable for deployment and application in edge computing devices or monitoring systems with high real-time requirements. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0038] Figure 1 The flow chart of the camera state judgment method based on road surface masking and hash comparison provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0039] In order to facilitate the understanding of the technical solutions provided by the embodiments of the present application, the background art related to the embodiments of the present application will be described first.

[0040] In intelligent video analysis systems for road monitoring, traffic flow statistics, violation detection, etc. based on fixed cameras, the cameras are usually preset at specific positions and angles to ensure that their fields of view cover the target road area and provide a stable spatial reference for subsequent video analysis algorithms (such as lane line detection, vehicle tracking and classification, event identification, etc.). However, during actual deployment and long-term operation, the cameras may change their physical positions or angles due to various reasons: for example, maintenance personnel may manually rotate the cameras for cleaning, maintenance or adjustment of the monitoring range; environmental factors such as strong winds, vibrations, etc. may cause the installation bracket to loosen, causing the camera to slowly shift; even accidental events such as vehicle collisions with lamp posts may cause significant changes in the direction of the camera. These rotations or shifts, even if the angles are not large, will cause the actual field of view of the camera to deviate from the system's preset calibration parameters. If this state change cannot be sensed in a timely and accurate manner and a re-calibration is triggered, all subsequent analysis algorithms that rely on a fixed angle will run based on incorrect spatial references, resulting in a series of problems such as lane line recognition errors, vehicle trajectory tracking distortions, event detection failures, etc., seriously affecting the reliability and usability of the entire system.

[0041] Currently, the industry mainly relies on the following technical routes for the perception of the motion state of the camera itself: The first type is to use inertial measurement units (IMU) and other physical sensors. This method directly measures the angular velocity and acceleration of the camera by integrating gyroscopes, accelerometers and other devices in the camera body, and then calculates the attitude change. However, the introduction of additional hardware not only increases the material cost and power consumption of the device, but also puts higher requirements on installation technology and sensor calibration, which is not conducive to large-scale promotion and upgrade in the stock of massive monitoring equipment. The second type is a feature point optical flow estimation method based on computer vision. This method detects and tracks feature points (such as corners and edges) between consecutive video frames, calculates the motion vector (optical flow) of these feature points, and then infers the overall motion of the camera. Although this method does not require additional hardware, its computational complexity is high and real-time processing is difficult. More importantly, its performance is extremely dependent on whether there are rich, stable and repeatedly detectable texture features in the scene. In road monitoring scenarios, especially in common complex situations such as night low light, image blur caused by rain, snow and fog weather, large area occlusion (such as large vehicles occupying the field of view for a long time), or single texture road surface (such as newly paved black asphalt road), the number of reliable feature points will decrease dramatically, causing optical flow calculation to fail or produce large errors, making the state judgment completely ineffective. The third type is to use visual simultaneous localization and mapping (SLAM) technology. This type of solution aims to estimate camera motion and build a scene map simultaneously, which can theoretically provide more rich motion information. However, its algorithm is more complex and consumes a lot of computing resources, making it difficult to meet the general requirements of real-time and lightweight for road monitoring systems, and is also limited by scene texture features, and has challenges in map initialization, dynamic object processing, etc. It is not suitable for embedding into existing systems as a simple camera state monitoring module.

[0042] In summary, the existing technical solutions often struggle to achieve a good balance between cost, computational efficiency, environmental robustness and deployment convenience when dealing with the specific application scenario of road monitoring. Based on this, the present embodiment aims to overcome the above-mentioned defects and provide a camera rotation or stable state judgment method that only uses existing video stream data, is computationally lightweight, and is strongly adaptive to complex road environments, especially dynamic vehicle interference and lack of static texture.

[0043] For those skilled in the technical field, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. Secondly, in this paper, the term "includes" is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or equipment.

[0044] A camera state judgment method based on road surface masking and hash comparison, as shown in Figure 1 The steps include the following steps in sequence:

[0045] Step S1: Extract two frames of images constituting a key frame pair from the video stream collected by the camera at a preset time interval. The video stream is continuously collected and transmitted to the processing unit (such as edge computing device or server) by the network camera or high-definition camera deployed at the road monitoring point. The key frame pair is composed of a historical frame and a current frame, which is used for inter-frame difference comparison.

[0046] Step S2: Extract the road surface area of each frame image in the key frame pair, and obtain the road surface mask for each frame image. This step aims to accurately identify the pixel area belonging to the road (such as asphalt, cement pavement) in the image.

[0047] Step S3: Based on the road surface mask corresponding to each frame image, mask the road surface area in the frame image respectively, and obtain the corresponding masked image. This processing excludes the area where the dynamic vehicle usually stays from the subsequent calculation.

[0048] Step S4: Calculate the difference hash feature of each frame of masked image, and obtain the corresponding hash code for each frame of masked image. Hash code is a compact and efficient fingerprint representation of image content.

[0049] Step S5: Calculate the Hamming distance between the hash codes corresponding to the two frames of images in the key frame pair as the inter-frame difference measure. The Hamming distance directly reflects the number of different bits between the hash codes of the two frames of images.

[0050] Step S6: According to the comparison result of the Hamming distance and the preset threshold, judge whether the camera is in a rotating state or a stable state.

[0051] Based on the above, the embodiment combines the road surface area masking with the difference hash calculation. Through masking, the interference of the frequently moving vehicles and other dynamic objects on the road to the background change detection is effectively isolated, so that the difference measurement focuses on the changes of the "static background" such as buildings, vegetation and static traffic facilities on both sides of the road. The difference hash algorithm has fast calculation speed and certain tolerance to small changes in image brightness, but is sensitive to geometric changes such as rotation and scaling of image content. The cooperation of the two makes the method be able to robustly perceive the visual content changes of the static background caused by the rotation of the camera at extremely low calculation cost, so as to realize efficient and accurate camera state judgment without any external sensor.

[0052] In the state judgment of continuous video stream, the algorithm faces a specific problem when it starts: lack of initial historical reference frame for comparison. If two frames are directly selected to start calculation, the difference of the first pair of key frames lacks reference significance and may lead to misjudgment at the start moment.

[0053] Specifically, to solve this initialization problem, as an optional embodiment of the embodiment, when step S1 is performed for the first time, algorithm initialization is needed. Specifically, the first frame image in the acquired video stream is taken as a historical frame, denoted as ; steps S2 to S4 are performed on the historical frame to obtain and store the corresponding hash code ; after waiting for a preset initial interval time , a current image is extracted from the video stream, denoted as ; the historical frame and the current image form the first key frame pair . The initial interval time may be an empirical value, for example, 0.5 seconds.

[0054] Based on the above, the embodiment establishes a correct starting point and reference benchmark for the entire continuous judgment process. By pre-processing and storing the hash code of the first frame, it ensures that the difference calculation of the first key frame pair can proceed normally, so that the algorithm can enter the effective monitoring state from the beginning, solving the problem that the algorithm cannot effectively compare or is prone to misjudgment in the starting stage due to lack of historical reference.

[0055] In a long-term monitoring scene, if sampling calculation is performed at a fixed high frequency, unnecessary calculation burden will be brought to the edge device; and if the sampling frequency is too low, the short-term rotation of the camera or the stable moment thereof may not be captured in time, and there is a problem that calculation efficiency and state judgment timeliness are difficult to balance.

[0056] Further, to balance the calculation load and the detection response speed, as an optional implementation of the embodiment, the key frame pairs are extracted at preset time intervals, and an adaptive time interval strategy is adopted, which specifically includes:

[0057] When it is judged according to step S6 that the camera is in a non-rotating (i.e., stable) state, a first interval duration After waiting, the next frame of image is extracted from the video stream to form a new key frame pair with the last processed image.

[0058] When it is judged according to step S6 that the camera is in a rotating (or unstable) state, a second interval duration After waiting, the next frame of image is extracted from the video stream to form a new key frame pair with the last processed image.

[0059] The first interval duration is greater than the second interval duration . For example, 2 seconds can be taken, 0.5 seconds can be taken.

[0060] Based on the above, the adaptive time interval strategy adopted by the embodiment optimizes the allocation of system calculation resources while ensuring the continuity of state monitoring. During the stable period of the camera, longer interval sampling is adopted, which greatly reduces the calculation frequency and system load, and is suitable for long-term monitoring. Once rotation or unstable period is detected, it immediately switches to high-frequency (short interval) sampling mode, which can densely capture the motion process of the camera, so as to more accurately and timely capture the moment of its re-stabilization, and improve the timeliness and response speed of state judgment. Thus, under the premise of ensuring overall lightweight calculation, the timeliness of capturing the rotation event of the camera and its stable moment is improved.

[0061] Accurate separation of the road surface region and the non-road surface static background in the image is the premise of excluding vehicle interference. The traditional segmentation method based on color or texture threshold has poor adaptability and is easily invalid under different light, weather and road types, which cannot provide reliable input for the subsequent steps.

[0062] Specifically, to obtain a robust and accurate road surface region identifier, as an optional implementation of the embodiment, step S2 specifically includes: using a semantic segmentation model to segment each frame of image in the key frame pair, extracting the road surface region in the image, and outputting a road mask corresponding to each frame of image . The road mask is a binary image, in which the pixel value belonging to the road surface region is marked as 1 (or 255), and the pixel value of the non-road surface region is marked as 0.

[0063] Semantic segmentation is a key technology for achieving pixel-level classification in computer vision. The road segmentation models used in this embodiment (such as DeepLabV3+, UNet, etc.) are typically pre-trained on large road scene datasets with fine pixel-level annotations (such as Cityscapes, Mapillary Vistas) to give them strong generalization capabilities, enabling them to accurately and robustly segment the "road" category from images containing complex backgrounds, different lighting and weather conditions.

[0064] Based on the above, this embodiment employs a deep learning-based semantic segmentation model to automatically obtain road surface masks, overcoming the limitations of traditional manual rule-based methods such as color and texture. This method can adapt with high precision to various real-world road environments (such as different types of asphalt roads, cement roads, slippery surfaces, partially obscured surfaces, etc.) and complex environmental conditions, achieving high-precision and automated extraction of road surface areas. This provides a reliable and accurate foundation for subsequent effective area masking, and is a key prerequisite for the high robustness of this method.

[0065] Although semantic segmentation models can extract road surface regions, road scenes often contain scale variations (narrow road surfaces in the distance) and similar-looking distractions (such as gray sidewalks). General models may lack sufficient detail and boundary accuracy, affecting the accuracy of occlusion.

[0066] To improve segmentation accuracy, especially boundary details and multi-scale adaptability, the semantic segmentation model is an encoder-decoder structure as a further optional implementation method in this embodiment, based on the aforementioned semantic segmentation model.

[0067] The encoder consists of a backbone network and a spatial pyramid pooling module. The backbone network (e.g., using a ResNet101 architecture with dilated convolutions replaced in its final stage) receives the input image and performs multi-level feature extraction, outputting a high-level feature map containing rich semantic information. and low-level feature maps containing more spatial details The spatial pyramid pooling module is used to receive high-level feature maps. The algorithm performs dilated convolutions with various dilation rates (e.g., learning rates of 6, 12, and 18) and a global average pooling operation in parallel. The resulting feature maps are then concatenated and fused using a 1×1 convolutional layer for dimensionality reduction, ultimately generating encoded features that incorporate multi-scale contextual information. .

[0068] The decoder is used to receive the low-level feature map $F_{low}$ and the encoded features $F_{enc}$. First, the low-level feature map... Perform 1×1 convolution to achieve channel dimensionality reduction and obtain dimensionality-reduced features. Meanwhile, the encoded feature is bilinearly upsampled to have the same spatial size as the input image, resulting in an upsampled feature . Next, the reduced-dimension feature is concatenated with the upsampled feature in the channel dimension. Then, the concatenated result is subjected to 3x3 convolution to further fuse the features, and bilinear upsample is performed again until the resolution is restored to the same as the input image, outputting the road mask .

[0069] Based on the above, the embodiment adopts a semantic segmentation model with an encoder-decoder structure, especially in combination with the design of a spatial pyramid pooling module, so that the model can capture context information of different scales in the image while maintaining the spatial resolution of the feature map. This is crucial for accurate road segmentation, as road areas can appear at different scales (e.g., wide at close range, narrow at long range) and need to be distinguished from visually similar areas (e.g., gray sidewalks, parking lots). The decoder effectively restores the details of the segmentation boundary by fusing low-level features. This design improves the boundary accuracy of road segmentation and adaptability to complex scenes, ensuring high precision and clear boundaries in road mask extraction, especially better distinguishing different areas with similar appearances, ensuring mask quality, laying a solid foundation for the reliability of subsequent steps, and indirectly improving the accuracy of the final state judgment.

[0070] Even if an accurate road mask is obtained, how to effectively exclude the influence of dynamic vehicles and focus the calculation on the static background requires a clear processing flow. Directly processing on the full-color image has a large amount of calculation, and the mask information needs to be effectively converted into operations on image data.

[0071] Specifically, to achieve efficient interference exclusion, as one of the optional embodiments of the present embodiment, in step S3, based on the road mask corresponding to each frame of image, the road area in the frame of image is masked respectively, specifically including:

[0072] First, the original color image in the key frame pair is converted into a grayscale image as the to-be-masked image .

[0073] Then, in the to-be-masked image , the pixel value at the pixel position identified by the road mask (that is, the position of ) is uniformly modified to a preset background value (for example, pure white 255), thereby obtaining the final masked image . For the non-road area identified in the mask whose pixel value remains unchanged in .

[0074] Based on the above, the embodiment reduces the data dimension by converting the image into a grayscale image. More importantly, the dynamic area where the vehicle travels is converted into a constant brightness background using an accurate road mask. This fundamentally eliminates the interference of vehicle movement and color changes on the inter-frame difference calculation, forcing the subsequent hash difference calculation to fully reflect the relative changes of the static background (such as buildings, guardrails, and green belts) on both sides of the road. This makes the algorithm extremely sensitive to background displacement caused by camera rotation, and almost immune to changes in traffic flow on the road, improving the specificity and robustness of the judgment.

[0075] Converting a color image to a grayscale image is a common operation, but different conversion methods can affect the visual information retained in the grayscale image. A simple average method may result in important visual features being weakened, and a conversion method that better matches human eye perception is needed to make subsequent grayscale value-based comparisons more accurately reflect real visual content differences.

[0076] To this end, based on the aforementioned masking process, as a further optional implementation of the embodiment, the grayscale image is obtained by: for each pixel point of the images in the key frame pair, performing weighted calculation on the red channel value , green channel value and blue channel value of the pixel point to obtain the grayscale value of the pixel point. The grayscale values of all pixel points collectively form the corresponding grayscale image .

[0077] In some embodiments, the grayscale value calculation formula used is:

[0078]

[0079] where 0.299, 0.587, and 0.114 are the industry standard weights for converting RGB color space to grayscale; is the original pixel value of the current pixel to be calculated in the RGB channel.

[0080] Based on this, the embodiment is based on the physiological characteristics of human eye sensitivity to different colors, making the generated grayscale image more closely match human subjective visual experience in brightness distribution. This ensures that the brightness relationship between pixels relied on by subsequent difference hash calculation can more truly represent the scene structure and content changes that can be perceived by the human eye, improving the consistency of the state judgment result and visual perception.

[0081] In addition, in order to further improve the distinction between static background and dynamic vehicle area in the road monitoring scene, and enhance the sensitivity of subsequent hash calculation to background changes, the embodiment further proposes a scene-adaptive weighted grayscale method. The method first calculates the color distribution characteristics of the road surface area and the non-road surface (static background) area in the statistical image, dynamically adjusts the grayscale weight, and makes the texture of the static background more significantly reserved in the grayscale image.

[0082] Among them, the calculation of the color mean value of the region is: the RGB channel mean value of the non-road surface area (static background) is calculated respectively, denoted as , and the RGB channel mean value of the road surface area is denoted as .

[0083] The weight adjustment vector generation is: the color difference vector of the static background and the road surface area is calculated, and the weight adjustment bias is generated therefrom. A specific calculation formula is as follows:

[0084]

[0085] Among them, is an adjustment intensity coefficient (for example, 0.1 can be taken), is a very small constant (such as 10 -8 ) to prevent division by zero error. The formula converts the color difference into weight adjustment amount in proportion after normalization. is the R channel color difference vector of the static background and the road surface area, is the G channel color difference vector of the static background and the road surface area, is the B channel color difference vector of the static background and the road surface area, is the weight adjustment bias on the R channel, is the weight adjustment bias on the G channel, is the weight adjustment bias on the B channel.

[0086] Therefore, the process of adaptive grayscale calculation is: the adjustment bias is applied to the standard weight of RGB to grayscale to obtain the adaptive weight, and the adaptive grayscale value of the target pixel is calculated. The specific formula is:

[0087]

[0088] Among them, is the RGB channel adaptive weight after fusion adjustment bias; is the adaptive grayscale value calculated.

[0089] Finally, the ​Truncation (i.e. limit its value in the range of 0-255, to match the effective range of pixel values of 8-bit grayscale image).

[0090] Based on the above, the embodiment dynamically adjusts the weight of the RGB channel by analyzing the inherent color difference between the static background and the road surface in the current scene. The design principle is to strengthen the contribution of the color component that is most different from the road surface in the static background area in the grayscale. For example, if the static background contains a large amount of green vegetation (G component is high), and the road surface is gray (R, G, B is close), the algorithm will automatically increase the weight of the G component, so that the texture of the vegetation in the background is more prominent in the grayscale image. In this way, when the camera rotates, these enhanced background textures are more easily captured by the difference hash algorithm, thereby effectively improving the sensitivity and robustness of camera rotation detection in complex or low-contrast scenes, and solving the problem of difficulty in detecting small rotations when the background and road surface colors are similar.

[0091] In order to quickly determine the state, a highly efficient image feature representation method sensitive to camera rotation is needed. Directly calculating features on high-resolution images is complex and requires a feature encoding mechanism that is not sensitive to illumination changes but sensitive to geometric changes.

[0092] To meet the requirements of lightness and sensitivity, as an optional embodiment of the embodiment, in step S4, the difference hash feature is calculated, specifically including the following sub-steps:

[0093] S41, the masked image is scaled to an intermediate image of a predetermined size , for example, 9 pixels wide and 8 pixels high (9x8). The scaling operation usually uses a fast bilinear interpolation algorithm.

[0094] S42, the intermediate image is converted into a grayscale image (if non-gray) and used as the image to be compared .

[0095] S43, each row of the image to be compared is compared with its adjacent pixels. For each row, the gray values of the adjacent two pixels are compared from left to right, and if the left pixel value is greater than the right pixel value, a binary '1' is recorded at that position, otherwise a '0' is recorded. Each row comparison will produce a binary sequence of length 8.

[0096] S44, the binary sequences generated by all rows (a total of 8 rows) are spliced together in order to form a binary string of length 64, which is the difference hash code of the image .

[0097] Based on the above, the embodiment greatly compresses the data amount and makes the calculation extremely lightweight by scaling to a very small size (such as 8x9) while preserving the macrostructure information of the image. By comparing adjacent pixels to generate a binary string, the hash code is not sensitive to the overall brightness change of the image (uniform changes in illumination do not affect the size relationship of adjacent pixels), but is extremely sensitive to geometric transformations of the image content (such as rotation, translation resulting in changes in pixel position relationship). This feature meets the needs of camera rotation detection: ignore the gradual change of light, and focus on capturing the displacement of the background structure. The entire calculation process is simple and fast, and is very suitable for the needs of quickly detecting geometric changes in image content caused by camera rotation on resource-constrained edge devices (in a resource-constrained environment).

[0098] After calculating the hash difference (Hamming distance), a clear and stable decision rule is needed to determine the camera state. How to set a reasonable threshold and make a binary decision of "rotation" or "stable" is a key decision-making problem that connects calculation and application.

[0099] Specifically, as an optional implementation of the embodiment, step S6 specifically includes:

[0100] If the Hamming distance is less than a preset threshold , it is determined that the camera has not rotated or the camera has reached a stable state again.

[0101] If the Hamming distance is greater than or equal to the preset threshold , it is determined that the camera has rotated or the camera has not stabilized.

[0102] Wherein, the preset threshold is a key parameter, and its value can be obtained by experimental calibration on a large amount of video data of stable state and rotation state in the target scene, aiming to achieve the best balance between sensitivity and noise resistance.

[0103] Based on the above, the Hamming distance of the embodiment directly quantifies the number of difference bits between the hash codes of the two images, that is, reflects the degree of change in the visual content of the static background area. When the camera is stationary, the background is unchanged, maintained at a low level close to 0 (less than ); when the camera rotates, the background is displaced on the imaging surface, causing the hash code to change significantly, increased (greater than or equal to ). The binary decision by a clear threshold converts the continuous difference measure into discrete state decision, with clear logic, fast decision speed, and easy integration into various control systems, realizing efficient mapping from image difference to physical state.

[0104] In addition, the fixed preset threshold is difficult to adapt to changes in different monitoring points, different lighting conditions (such as day and night alternation), and different static background complexity. Therefore, the embodiment proposes an adaptive threshold calculation method based on historical frame sequence statistics to replace or initialize the fixed threshold. The specific process is as follows:

[0105] (1) Stable period difference sequence acquisition: after system initialization or during the stable stage, the Hamming distance of (e.g. ) consecutive key frame pairs is continuously acquired to form a difference sequence . This sequence reflects the baseline level of background hash difference caused by gradual changes in lighting, leaf movement, coding noise, etc. when there is no camera rotation.

[0106] (2) Adaptive threshold calculation: based on the sequence, the adaptive threshold is calculated, and the calculation formula is:

[0107]

[0108] wherein is the mean of the difference sequence, is the standard deviation thereof, is the sensitivity coefficient (usually 2.5-3.5). The mean reflects the center position of the baseline difference, and the standard reflects the degree of baseline fluctuation.

[0109] (3) Threshold update mechanism: in subsequent operation, when the stable state is continuously determined for more than a certain number of frames, the new historical difference data can be used to update and , so as to dynamically adjust .

[0110] At this time, in step S6, the Hamming distance is compared with the adaptive threshold to determine the camera state. Thus, based on the design of this adaptive threshold, the threshold is set to a dynamic value based on the baseline statistical characteristics ( ). This enables the judgment threshold to automatically adapt to the inherent noise level of different scenes: in a calm scene ( small), the threshold is low, improving sensitivity; in a dynamic interference (such as tree swaying, In the case of large scene, the threshold is automatically increased to enhance the anti-interference ability and prevent misjudgment. This design improves the adaptive ability and overall robustness of the method in various real-world complex environments.

[0111] After detecting the camera re-stabilization, if the subsequent processing cannot be automatically triggered, manual intervention is still needed, and the automatic closed-loop capability of the system is insufficient, and the value of state awareness is not fully realized.

[0112] In addition, to realize the full automatic adaptation of the system, as an optional embodiment of the present embodiment, the method further comprises an application closed loop step: when it is judged that the camera enters the re-stabilization state from the rotation state, a trigger signal is generated and output. The signal is used to trigger the subsequent system to calibrate the new road scene currently shot by the camera. The calibration content can include updating the lane line model, re-drawing the detection area, and calibrating the camera parameters, so as to adapt the subsequent intelligent traffic analysis algorithm (such as vehicle detection, flow statistics, event detection) to the new camera field of view.

[0113] Based on the above, the present embodiment seamlessly connects the "state awareness" function in the method with the "adaptive calibration" function of the intelligent traffic system. The output trigger signal enables the entire monitoring system to have the ability of self-adjustment and continuous optimization. The system can automatically detect the camera state change and confirm its stability, and then actively initiate the re-calibration process without manual intervention. This improves the automation level, long-term operation stability and maintenance efficiency of the large-scale roadside monitoring system, ensures that the upper video analysis algorithm is always based on accurate spatial environment information, and thus guarantees the autonomy, persistent reliability and maintenance efficiency of the overall system performance, realizes the complete automatic closed loop of "perception-decision-execution".

[0114] In the embodiments provided by the present application, it should be understood that the embodiments described herein can be realized by hardware, software, firmware, middleware, codes or any proper combination thereof. For hardware implementation, the processor can be realized in one or more of the following components: an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor, other electronic units designed to perform the functions described herein, or a combination thereof. For software implementation, the procedures described herein can be implemented with a computer program that is written in any suitable programming language. The program can be stored in a computer readable storage medium or transmitted as one or more instructions or codes on the computer readable storage medium. The computer readable storage medium includes any storage medium that can be accessed by a computer. The computer readable storage medium can include but is not limited to the following media: a RAM, a ROM, an EEPROM, a CD-ROM or other optical disc storage, a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer.

[0115] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, modifications or equivalent replacements of some technical features described in the foregoing embodiments can be made by those skilled in the art, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A camera state determination method based on road surface occlusion and hash comparison, characterized in that, In order, they include: Step S1: Extract two frames that constitute a keyframe pair from the video stream captured by the camera at preset time intervals; Step S2: Extract the road surface region for each frame of the keyframe pair and obtain the road surface mask for each frame. Step S3: Based on the road surface mask corresponding to each frame image, perform masking processing on the road surface area in the frame image respectively to obtain the corresponding masked image; Step S4: Calculate the difference hash feature for each frame of the masked image, and obtain the corresponding hash code for each frame of the masked image; Step S5: Calculate the Hamming distance between the hash codes corresponding to the two frames in the keyframe pair as an inter-frame difference metric; Step S6: Based on the comparison result between the Hamming distance and the preset threshold, determine whether the camera is in a rotating state or a stable state.

2. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, When step S1 is executed for the first time, the first frame image in the acquired video stream is taken as a historical frame; steps S2 to S4 are executed on the historical frame to obtain and store its corresponding hash code; after waiting for a preset initial interval, a current frame image is extracted from the video stream; the historical frame and the current image are combined to form the first keyframe pair.

3. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, In step S1, extracting keyframe pairs at preset time intervals includes: When it is determined that the camera is not rotating, after waiting for the first interval, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair. When it is determined that the camera is rotating, after waiting for a second interval, the next frame image is extracted from the video stream and combined with the most recently processed image to form a new keyframe pair. Wherein, the first interval duration is longer than the second interval duration.

4. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, Step S2 includes: The semantic segmentation model is used to segment each frame of the keyframe pair, extract the road surface region in the image, and output the road surface mask corresponding to each frame.

5. The camera state determination method based on road surface occlusion and hash comparison according to claim 4, characterized in that, The semantic segmentation model is an encoder-decoder structure; The encoder includes a backbone network and a spatial pyramid pooling module; the backbone network is used to receive the input image and perform feature extraction, outputting high-level feature maps and low-level feature maps. The spatial pyramid pooling module is used to receive the high-level feature map, perform dilated convolution with multiple dilation rates and global average pooling operations in parallel, and concatenate and convolve the resulting multiple feature maps to generate encoded features. The decoder is used to receive the low-level feature map and the encoded features, perform channel dimensionality reduction on the low-level feature map to obtain dimensionality-reduced features, and perform upsampling on the encoded features to obtain upsampled features; concatenate the dimensionality-reduced features and the upsampled features, perform convolution on the concatenation result, and upsample the convolution result to the same resolution as the input image, and output the road surface mask.

6. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, In step S3, the process of masking the road surface region in each frame of the image based on the road surface mask corresponding to that frame includes: The images in the keyframe pairs are converted into grayscale images and used as the images to be masked. In the image to be masked, the pixel value at the pixel position marked by the road surface mask is modified to a preset background value to obtain the masked image.

7. The camera state determination method based on road surface occlusion and hash comparison according to claim 6, characterized in that, The grayscale image is obtained by weighting the red channel value, green channel value, and blue channel value of each pixel in the keyframe pair to obtain the grayscale value of that pixel, and then forming the corresponding grayscale image from the grayscale values ​​of all pixels.

8. The camera state determination method based on road surface occlusion and hash comparison according to claim 1, characterized in that, In step S4, calculating the difference hash feature includes: S41. Scale the masked image to an intermediate image of a preset size; S42. Convert the intermediate image into a grayscale image and use it as the image to be compared. S43. Compare the adjacent pixels in each row of the image to be compared. If the pixel value on the left is greater than the pixel value on the right, record it as the first binary value; otherwise, record it as the second binary value. Generate the binary sequence of that row. S44. Concatenate the binary sequences of all rows to form the hash code.

9. The camera state determination method based on road surface occlusion and hash comparison according to claim 1 or 3, characterized in that, Step S6 includes: If the Hamming distance is less than the preset threshold, it is determined that the camera has not rotated or the camera has stabilized again. If the Hamming distance is greater than or equal to the preset threshold, it is determined that the camera has rotated or the camera is unstable.

10. The camera state determination method based on road surface occlusion and hash comparison according to claim 9, characterized in that, The method further includes: when it is determined that the camera has entered a restabilized state from a rotating state, outputting a trigger signal to trigger the calibration of the road scene captured by the camera.

Images