Camera labeling area automatic correction algorithm based on feature point registration

Abstract

The invention relates to a feature point registration-based camera labeling area automatic correction algorithm, and relates to the technical field of image data processing. The method comprises the following steps: acquiring a reference image before offset of a camera and an offset image after offset of the camera, wherein the reference image comprises a pre-labeled region of interest; performing feature extraction and feature point matching on the reference image and the offset image to obtain feature point position information and a matching relationship; on the basis of the feature point matching relationship, carrying out region correction on the pre-marked region of interest to obtain a corrected region of interest; and outputting a correction result, wherein the correction result comprises the corrected coordinate information of the region of interest. The method has the effect of solving the problem of camera offset as much as possible.

Description

Technical Field

[0001] This application relates to the technical field of image data processing, and in particular to an automatic correction algorithm for camera annotation areas based on feature point registration. Background Technology

[0002] With the digital transformation of enterprises, machine vision systems are playing an increasingly important role in industrial production. As a crucial component of machine vision systems, the stability of the camera's installation position directly affects the accuracy and reliability of visual inspection. However, in real industrial environments, due to factors such as equipment vibration, temperature changes, and human impact, cameras often experience varying degrees of displacement. This displacement causes the detection area to deviate from its original set position, resulting in missed and false detections, and necessitates manual recalibration and re-calibration, increasing maintenance costs.

[0003] Currently, the main solutions to the camera offset problem include manual calibration, fixed reference points, and template matching. However, manual calibration requires professional personnel, is time-consuming and inefficient; fixed reference point solutions require additional markers and are easily affected by ambient lighting; and template matching solutions are sensitive to changes in lighting, have high computational requirements, and poor real-time performance. Summary of the Invention

[0004] In order to solve the problem of camera offset as much as possible, this application provides an automatic correction algorithm for camera annotation area based on feature point registration.

[0005] The automatic correction algorithm for camera labeled regions based on feature point registration provided in this application adopts the following technical solution: An automatic correction algorithm for camera labeled regions based on feature point registration includes the following steps: Acquire a reference image before camera offset and an offset image after camera offset, wherein the reference image contains a pre-labeled region of interest; Feature extraction and feature point matching are performed on the reference image and the offset image to obtain feature point location information and matching relationships; Based on the feature point matching relationship, the pre-labeled region of interest is corrected to obtain the corrected region of interest; Output the correction result, which includes the coordinate information of the corrected region of interest.

[0006] By adopting the above technical solution and using the "feature matching-coordinate transformation" technical path, the pre-annotated region of interest in the reference image can be automatically and quickly mapped accurately to the current image where the offset has occurred. This eliminates the time-consuming and laborious manual re-annotation work in traditional methods, ensuring that video surveillance, visual analysis and other systems can continue to operate stably even after the camera has shifted slightly.

[0007] Preferably, the method further includes preprocessing the reference image and the offset image, wherein the preprocessing of the reference image and the offset image includes the following steps: The RGB channels of the reference image and the offset image are normalized respectively; Check whether the resolution of the reference image and the offset image are consistent; if they are inconsistent, terminate the process. By analyzing the statistical characteristics of the RGB channel histograms, images with abnormal lighting or low contrast are evaluated and filtered out.

[0008] By adopting the above technical solutions, normalization processing can eliminate color and brightness deviations caused by lighting and sensor differences, allowing the feature extraction network to focus more on image structure and texture rather than absolute color values, thus improving the model's generalization ability in different environments. Resolution consistency checks can minimize subsequent calculation errors or crashes caused by image size mismatches, enhancing system stability. Image quality assessment can proactively filter out invalid images that are too dark, overexposed, or have low contrast, ensuring the reliability and accuracy of the final correction results.

[0009] Preferably, the step of performing feature extraction and feature point matching on the preprocessed image includes the following steps: The reference image and the offset image are processed in parallel batches using a pre-trained deep learning model to simultaneously complete feature point detection and feature descriptor calculation. The deep learning model is optimized and accelerated using the TensorRT inference engine; Output the feature point location information and matching relationship of the reference image and the offset image.

[0010] By adopting the above technical solutions, the features extracted by the deep learning model have stronger invariance and discriminative power against changes in illumination, viewpoint, and slight occlusion, and can generate richer and more stable feature matching pairs, laying a solid foundation for subsequent calculations. Through model optimization, layer fusion, and quantization, several times the inference speed and lower latency have been achieved.

[0011] Preferably, the region correction based on feature point matching relationships includes the following steps: Based on the boundary of the pre-labeled region of interest, a buffer region is formed by expanding a preset pixel range outward, and feature matching point pairs located within the buffer region are selected. The number of feature matching point pairs used for calculation is controlled within a preset range.

[0012] By adopting the above technical solution and introducing spatial constraints, the geometric transformation calculation is made more focused on the target area, thereby significantly improving the local accuracy and reliability of the correction.

[0013] Preferably, the number of feature matching point pairs used for calculation is within a preset range, specifically: If the number of selected feature matching point pairs is less than the first threshold, the pixel range of the buffer area is expanded or global feature points are used. If the condition is still not met, the correction is deemed to have failed. If the number of selected feature matching point pairs is higher than the second threshold, the number of point pairs will be reduced to the second threshold or below by uniform sampling or random sampling.

[0014] By adopting the above technical solution and through intelligent control of the number of points, the computational efficiency and stability of the algorithm are greatly improved while ensuring the accuracy of the calculation results.

[0015] Preferably, the regional correction step further includes: Based on the selected feature matching point pairs, the affine transformation matrix between the reference image and the offset image is calculated; The coordinates of the pre-labeled region of interest are mapped using the affine transformation matrix to obtain the corrected region of interest in the offset image.

[0016] By adopting the above technical solution, it is possible to fully describe various geometric changes such as translation, rotation, scaling and shearing that may be caused by camera offset. The model has strong expressive power and wide applicability.

[0017] Preferably, the method further includes deformation constraints, which include the following steps: Calculate the area ratio of the pre-labeled region of interest to the corrected region of interest; Determine whether the area ratio is within a preset reasonable threshold range; If the area ratio exceeds the reasonable threshold range, the correction result is deemed invalid.

[0018] By adopting the above technical solution, abnormal deformation results caused by serious feature matching errors or model mismatch can be effectively identified and intercepted, preventing the algorithm from outputting obviously illogical results.

[0019] Preferably, the output results also include a correction status flag and / or offset parameters; the offset parameters include the horizontal offset, vertical offset, and rotation angle obtained through the decomposition coordinate transformation process.

[0020] By adopting the above technical solutions, the output information of the algorithm is made richer and more structured, which facilitates intelligent decision-making and human-computer interaction in the upstream system.

[0021] In summary, this application includes at least one of the following beneficial technical effects: 1. By acquiring reference and offset images and performing preprocessing, high-quality image data can be provided for subsequent feature extraction and matching, thereby improving the accuracy of corrected annotation areas; 2. By extracting features from the reference image and the offset image and establishing feature point matching relationships, combined with the region correction step, the camera-annotated area can be effectively corrected, solving the problem of mismatch between the annotated area and the actual scene caused by camera offset; 3. By employing methods such as neighbor point selection, feature point quantity control, and affine transformation calculation for regional correction, this method can improve the accuracy and efficiency of labeled region correction compared to traditional methods. It avoids the subjectivity and inefficiency of manual adjustment, as well as the inflexibility of preset fixed parameter methods and the coarseness of simple image feature matching methods. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of an embodiment of this application. Detailed Implementation

[0023] The following combination Figure 1 This application will be described in further detail.

[0024] This application discloses an automatic correction algorithm for camera annotation areas based on feature point registration.

[0025] Reference Figure 1 An automatic correction algorithm for camera labeled regions based on feature point registration includes the following steps: S1: Obtain the reference image before camera offset and the offset image after camera offset, including the following steps: The reference image is a template image saved after the camera has been installed, debugged, and its field of view and angle have been confirmed to fully meet the application requirements. The reference image includes a pre-marked region of interest, which represents the correct viewpoint and range of the scene and remains fixed after initialization.

[0026] Offset images are the current frames captured by the system periodically or by events from the real-time video stream after the camera has shifted due to physical disturbances during operation. They reflect the real-time state of the camera's field of view.

[0027] S2: Preprocess the reference image and the offset image, including the following steps: S21: Normalization. The RGB channels of the acquired reference and offset images are normalized separately. The mean of the R, G, and B channels is set to 0, and the standard deviation is set to 255 to eliminate overall differences in brightness and contrast between different images, allowing the deep learning model to focus more on texture and structural information rather than absolute color values.

[0028] S22: Resolution Consistency Check. Read the height and width of the reference image and the offset image and compare them. If the height and / or width of the reference image and the offset image are inconsistent, immediately terminate the process and report an exception to prompt the user to check the camera acquisition configuration, thereby ensuring that the acquired reference image and offset image have the same resolution.

[0029] S23: Image quality assessment. Image quality is assessed by analyzing the histogram statistics of the RGB channels of the reference image and the offset image.

[0030] Specifically, for lighting issues, the system determines the problem by detecting whether the histogram distribution is severely skewed to the left (pixel values ​​concentrated around 0, indicating the image is generally too dark) or severely skewed to the right (pixel values ​​concentrated around 255, indicating the image is generally overexposed). For low contrast issues, the system calculates the variance or dynamic range (the difference between the maximum and minimum values) of the histogram distribution; an excessively narrow distribution range indicates a lack of detail and texture. For exposure issues, the system comprehensively evaluates the overall position and shape characteristics of the histogram distribution. The system sets thresholds for these quality indicators; when an image's quality score falls below the threshold, it is identified as an abnormal image and filtered out, triggering a resampling mechanism or an alarm, thus eliminating abnormal images caused by lighting, exposure, and other problems.

[0031] S3: Perform feature extraction and feature point matching on the preprocessed reference image and offset image to obtain feature point location information and matching relationships, including the following steps: S31: Use a deep learning model to perform batch processing on the reference image and the offset image.

[0032] The system uses pre-trained deep learning models (such as end-to-end networks like SuperPoint, D2-Net, or LoFTR) on large-scale datasets to perform parallel batch processing on pre-processed reference and offset images. The deep learning model can simultaneously perform two tasks: feature point detection and feature descriptor computation. First, it locates salient keypoints in the deep convolutional feature maps of the network, then generates a high-dimensional, highly discriminative feature vector for each keypoint. Finally, it establishes the correspondence between feature points in the two images through similarity calculations between feature vectors (such as cosine similarity or nearest neighbor matching).

[0033] By employing batch processing instead of image-by-image processing, the parallel computing power of the GPU can be fully utilized, significantly improving overall processing efficiency. This approach is particularly suitable for real-time applications requiring processing multiple cameras or consecutive frames. This deep learning-based method is more adaptable to complex scenes such as changes in lighting, perspective shifts, and partial occlusion, providing high-quality and sufficient feature matching pairs for subsequent precise geometric transformation calculations.

[0034] S32: Accelerate the model using the TensorRT inference engine.

[0035] The deep learning model used in step S31 is transformed into a highly optimized inference form to improve the computational efficiency of feature extraction. The TensorRT inference engine processes the trained model: first, it performs model parsing and graph optimization, merging operations such as convolution, batch normalization, and activation functions into a single kernel through layer fusion, significantly reducing memory access overhead and kernel startup latency; second, it performs accuracy calibration, quantizing the FP32 model to INT8 precision while maintaining model accuracy, using integer calculations to significantly improve computational throughput and reduce memory usage; simultaneously, it automatically selects the optimal kernel implementation for the specific GPU hardware architecture, maximizing the utilization of hardware computing units. The engine optimized by TensorRT achieves several times the inference speed and lower latency while maintaining feature extraction accuracy, enabling complex deep learning feature extraction models to meet the stringent performance requirements of real-time video streaming processing (e.g., reaching 30 FPS or higher). This acceleration optimization ensures that the entire correction system can operate efficiently and stably in actual deployments, serving as a crucial bridge between theoretical research and engineering applications.

[0036] S33: Output feature point location information and matching relationships of the reference image and offset image.

[0037] The system receives the output from the deep learning model and converts it into structured feature information. It generates two sets of key data: first, feature point location information, recording the precise positions (x, y coordinates) of all detected salient feature points in both the reference and offset images in two-dimensional pixel coordinates; second, feature point matching relationships, establishing a correspondence table between feature points by comparing the similarity of feature descriptors between the two images, clearly identifying which feature point in the offset image matches each feature point in the reference image (or marking it as unmatched). This output data is typically organized in matrix or list form, containing the coordinate correspondence of matched point pairs and their matching confidence scores. High-quality feature point matching results should exhibit clear spatial consistency, meaning the relative positional relationship between matched point pairs maintains a certain geometric regularity in both images, providing a reliable data foundation for subsequent calculation of the affine transformation matrix.

[0038] S4: Based on feature point matching relationships, perform region correction on the pre-annotated region of interest, including the following steps: S41: The region of interest (ROI) is determined, and neighboring points are filtered. Feature points are filtered within a 20-pixel range around the ROI.

[0039] By introducing spatial constraints on the Region of Interest (ROI), a subset of feature points most relevant to the current correction task is selected from global feature points. Specifically, the system uses the original labeled ROI boundary as a baseline, extending outwards by 20 pixels to form a buffer region. Only feature matching point pairs falling within this buffer region are retained, while matching points far from the ROI are discarded. The advantages of this approach are: first, it prioritizes feature points closest to the target region because the local scene of these points most closely resembles the geometric deformation characteristics of the ROI, more accurately reflecting the true transformation relationship of the ROI; second, by excluding matching points far from the region, it effectively reduces the interference of inconsistent movements (such as the movement of multiple independent objects in the scene) between different regions on the overall transformation calculation; it also significantly reduces the proportion of abnormal matching points, improving the robustness of subsequent calculations. This spatial proximity-based selection mechanism ensures that the geometric transformation calculation focuses on the feature changes in the local region where the ROI is located, thus obtaining more accurate and reliable region correction results.

[0040] S42: Feature Point Count Control. Set a minimum and a maximum number of feature points to ensure the number of feature points in each area is within the set range. In this embodiment, the minimum threshold is set to 10, and the maximum threshold is set to 200, resulting in a feature point count in each area between 10 and 200. A two-way threshold management system is implemented for the number of selected feature points. This ensures sufficient sample points for calculating a stable transformation model while avoiding unnecessary computational overhead due to excessive points. Specifically, a dual upper and lower limit control mechanism is employed: a minimum threshold ensures enough sample points for robust estimation algorithms like RANSAC. These algorithms require multiple random samplings to eliminate false matches; too few points cannot provide a sufficient statistical sample size, potentially leading to transformation matrix calculation failures or erroneous results. A maximum threshold is set for computational efficiency. When the number of matching points is too large, the number of random samplings required by the random sampling consensus algorithm increases significantly, resulting in an exponential increase in computation time. Limiting the maximum number of points can significantly improve computational efficiency with almost no loss of accuracy.

[0041] When the number of points exceeds the upper limit, the system will select a representative point set using uniform or random sampling. When the number of points is less than the lower limit, the system will expand the search range in step S41 or try to use global feature points. If the requirements are still not met, the correction is deemed a failure. Through the above intelligent point number control mechanism, the real-time performance and stability of the algorithm are significantly improved while ensuring computational accuracy.

[0042] S43: Affine transformation calculation. Calculate the affine transformation matrix based on the selected feature points.

[0043] Using the high-quality feature matching point pairs selected in steps S41 and S42, an optimal affine transformation matrix is ​​calculated through mathematical optimization to describe the geometric mapping relationship between the reference image and the offset image. Affine transformation is a linear two-dimensional transformation model that can fully describe the four basic transformations: translation, rotation, scaling, and shearing. Its mathematical form is a 2×3 transformation matrix containing six degrees of freedom. The calculation process typically employs the least squares method for direct solution or combines it with the RANSAC (Random Sample Consensus) algorithm for robust estimation: the RANSAC algorithm calculates candidate transformation matrices by repeatedly randomly sampling the minimum point set (3 pairs of non-collinear points) and evaluates their fit with all matching points, ultimately selecting the transformation matrix with the most interior points and the smallest fitting error as the optimal solution. This obtained transformation matrix establishes a mathematical relationship between the two image coordinate systems, allowing the coordinates of any pixel in the reference image to be mapped to the corresponding position in the offset image through matrix operations. The output of this step is an accurate mathematical model, providing the mathematical foundation for subsequent transformation of the original ROI region coordinates to the current image coordinate system, and is the core computational step of the entire region correction algorithm.

[0044] S44: Deformation constraint, which ensures the rationality of the correction result through area ratio constraint.

[0045] By setting area ratio constraints, abnormal deformation results caused by feature point matching errors or unsuitable transformation models can be prevented as much as possible. In practice, the system calculates the area ratio of the original ROI region to the ROI region after affine transformation and sets a reasonable threshold range (e.g., 0.95-1.05, i.e., ±5% allowable variation). In scenarios with slight camera shifts, the projected area of ​​the same physical region in the image should not change drastically. When the calculated area ratio exceeds the preset threshold, it indicates that the correction has produced unreasonable deformation (such as over-scaling or distortion). The system will reject the calculation result and can adopt various response strategies according to business needs. Specific application strategies can be set to retain the valid correction result of the previous frame, use the original uncorrected ROI region, or trigger a system alarm requiring manual intervention. This constraint mechanism effectively prevents errors in the algorithm under extreme conditions, greatly improving the reliability and practicality of the entire system and ensuring that the output results always remain within a reasonable range.

[0046] S5: Output the correction results.

[0047] The correction result includes a status flag, offset parameters, and the corrected region coordinates. The status flag provides immediate feedback on the process execution result, indicating success or failure. Success means the affine transformation matrix was successfully calculated and passed the deformation constraint check; failure may be due to reasons such as substandard image quality, insufficient feature points, or excessive deformation.

[0048] Offset parameters are intuitive physical quantities decomposed from the calculated affine transformation matrix, including horizontal and vertical offsets in pixels and camera rotation angle in degrees. These parameters greatly facilitate the quantitative monitoring and diagnosis of camera physical displacement by maintenance personnel.

[0049] The corrected region coordinate information is obtained by applying an affine transformation matrix to the vertex coordinates of the original region of interest, resulting in a new region boundary defined in the offset image coordinate system. This boundary is typically represented as a rectangular or polygonal region in the form of a vertex list. All this information is usually encapsulated in a structured data format such as JSON, and supplemented with metadata such as timestamps and the number of feature points used. This ensures that subsequent video analysis modules, alarm systems, or visualization platforms can accurately and reliably utilize the correction results for further processing and decision-making.

[0050] The implementation principle of the automatic correction algorithm for camera labeled regions based on feature point registration in this application is as follows: Feature points are extracted from the camera reference image and the offset image using a deep learning model and matched to establish a correspondence between the two images. Based on this, the algorithm calculates an affine transformation matrix based on the matching points to accurately describe the camera offset. Finally, this matrix is ​​used to geometrically transform the pre-labeled region of interest (ROI) in the reference image, directly mapping it to the offset image, thereby outputting the corrected ROI coordinates and achieving automatic and accurate correction of the labeled region.

[0051] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. An automatic correction algorithm for camera annotation regions based on feature point registration, characterized in that: Includes the following steps: Acquire a reference image before camera offset and an offset image after camera offset, wherein the reference image contains a pre-labeled region of interest; Feature extraction and feature point matching are performed on the reference image and the offset image to obtain feature point location information and matching relationships; Based on the feature point matching relationship, the pre-labeled region of interest is corrected to obtain the corrected region of interest; Output the correction result, which includes the coordinate information of the corrected region of interest.

2. The automatic correction algorithm for camera annotation regions based on feature point registration according to claim 1, characterized in that: The process also includes preprocessing the reference image and the offset image, which includes the following steps: The RGB channels of the reference image and the offset image are normalized respectively; Check whether the resolution of the reference image and the offset image are consistent; if they are inconsistent, terminate the process. By analyzing the statistical characteristics of the RGB channel histograms, images with abnormal lighting or low contrast are evaluated and filtered out.

3. The automatic correction algorithm for camera annotation areas based on feature point registration according to claim 1, characterized in that: The process of feature extraction and feature point matching of the preprocessed image includes the following steps: The reference image and the offset image are processed in parallel batches using a pre-trained deep learning model to simultaneously complete feature point detection and feature descriptor calculation. The deep learning model is optimized and accelerated using the TensorRT inference engine; Output the feature point location information and matching relationship of the reference image and the offset image.

4. The automatic correction algorithm for camera annotation areas based on feature point registration according to claim 1, characterized in that: The region correction based on feature point matching relationships includes the following steps: Based on the boundary of the pre-labeled region of interest, a buffer region is formed by expanding a preset pixel range outward, and feature matching point pairs located within the buffer region are selected. The number of feature matching point pairs used for calculation is controlled within a preset range.

5. The automatic correction algorithm for camera annotation areas based on feature point registration according to claim 4, characterized in that: The control is used to calculate the number of feature matching point pairs within a preset range, specifically: If the number of selected feature matching point pairs is less than the first threshold, the pixel range of the buffer area is expanded or global feature points are used. If the condition is still not met, the correction is deemed to have failed. If the number of selected feature matching point pairs is higher than the second threshold, the number of point pairs will be reduced to the second threshold or below by uniform sampling or random sampling.

6. The automatic correction algorithm for camera annotation areas based on feature point registration according to claim 4, characterized in that: The regional correction steps also include: Based on the selected feature matching point pairs, the affine transformation matrix between the reference image and the offset image is calculated; The coordinates of the pre-labeled region of interest are mapped using the affine transformation matrix to obtain the corrected region of interest in the offset image.

7. The automatic correction algorithm for camera annotation areas based on feature point registration according to claim 6, characterized in that: It also includes deformation constraints, which include the following steps: Calculate the area ratio of the pre-labeled region of interest to the corrected region of interest; Determine whether the area ratio is within a preset reasonable threshold range; If the area ratio exceeds the reasonable threshold range, the correction result is deemed invalid.

8. The automatic correction algorithm for camera annotation areas based on feature point registration according to claim 1, characterized in that: The output results also include a correction status flag and / or offset parameters; the offset parameters include the horizontal offset, vertical offset, and rotation angle obtained through the decomposition coordinate transformation process.

Images