A Preset Automatic Calibration and Deviation Correction Method and System

Abstract

A preset position automatic calibration and deviation correction method and system, which relates to the technical field of preset position calibration. In order to solve the defects that the existing preset position correction technology has a large deviation correction error and cannot effectively process the situation when the observed target deviates out of the observation range, a preset position reference image with clarity meeting the preset confidence level is obtained; a to-be-detected image with clarity meeting the preset confidence level is obtained; each deviation amount is calculated respectively by using a feature point matching algorithm process, a template matching algorithm process, and / or a target detection method; the weight coefficient of each deviation amount is adjusted, and the final deviation amount is calculated according to each deviation amount and its corresponding weight coefficient; the preset position deviation correction is performed according to the final deviation amount. The present invention is mainly used for automatically calibrating and correcting the preset position.

Description

Technical Field

[0001] The present invention relates to the technical field of preset calibration, and particularly relates to a method and system for automatically calibrating and correcting a preset position. Background Art

[0002] With the rapid development and popularization of video surveillance systems, image recognition technology is increasingly used in various industrial fields. The traditional inspection method in the substation field can no longer meet the needs of modern society, and the rapid development of artificial intelligence has made intelligent inspection technology gradually become an important means in the substation field. Among them, the quality of preset position images seriously affects the accuracy of intelligent recognition results.

[0003] At present, cameras with pan-tilt control such as dome cameras are one of the important image acquisition devices for intelligent inspection in substations. However, due to reasons such as the mechanical gear spacing of the driving motor and the accuracy of structural parts, errors in preset position accuracy will occur. When the pan-tilt camera runs for a long time and the preset positions are frequently retrieved, the preset position errors will continue to accumulate, resulting in an obvious deviation between the current preset position and the initially set preset position. Especially when the camera focal length is magnified, the deviation is more obvious, and even the observed target deviates out of the observation range, seriously affecting the recognition accuracy of devices such as pointer meters in intelligent inspection.

[0004] Currently, the calibration and correction of preset positions mainly rely on manual operation, which has large correction errors, is time-consuming and laborious, and wastes a lot of manpower and material resources. And the existing preset position correction technologies are basically implemented by a single method based on SIFT feature point matching, with relatively large correction errors and unable to effectively handle the situation where the observed target deviates out of the observation range.

[0005] Therefore, there is an urgent need for a method and system for automatically calibrating and correcting the preset position of a pan-tilt camera, which can effectively solve the problem of preset position deviation, improve the recognition accuracy of intelligent inspection, and reduce the manual operation and maintenance cost. Summary of the Invention

[0006] In order to solve the defects that the existing preset position correction technology has relatively large correction errors and cannot effectively handle the situation where the observed target deviates out of the observation range, the present invention provides a method and system for automatically calibrating and correcting the preset position of a pan-tilt camera, which can effectively solve the problem of preset position deviation, improve the recognition accuracy of intelligent inspection, and reduce the manual operation and maintenance cost.

[0007] A method for automatically calibrating and correcting a preset position according to the present invention includes the following steps:

[0008] S1. Obtain a preset position reference image with clarity meeting the preset confidence level;

[0009] S2. Obtain the image to be detected with clarity meeting the preset confidence level;

[0010] S3. Calculate each offset respectively by using the feature point matching algorithm process, the template matching algorithm process and / or the object detection method;

[0011] S4. Adjust the weight coefficient of each offset, and calculate the final offset according to each offset and its corresponding weight coefficient;

[0012] S5. Perform pre-set position correction according to the final offset.

[0013] Furthermore: In S1, the specific steps of obtaining the pre-set position reference image with clarity meeting the preset confidence level are as follows:

[0014] S11. Adjust the camera position according to the position information of the pre-set position to make it correspond to the position information of the pre-set position, and capture the current pre-set position image;

[0015] S12. Use the video image quality diagnosis model to judge the clarity of the pre-set position image. If the confidence level of the pre-set position image is less than the preset threshold, it means that the current pre-set position image is not clear, and then repeat S11 until a pre-set position reference image with clarity meeting the preset confidence level is obtained;

[0016] S13. Take the central area of the pre-set position reference image as the first template image, and obtain the regional coordinate information of the first template image; Use the multi-object detection model to obtain the target area with the highest confidence level in the current pre-set position reference image, take the target area as the second template image, and obtain the regional coordinate information of the second template image; Use the sign detection model and the character recognition model to obtain the sign area in the current pre-set position reference image, and obtain the coordinate information and character information of the sign area;

[0017] S14. Save the pre-set position reference image, the first template image and its regional coordinate information, the second template image and its regional coordinate information, the sign area and its regional coordinate information, and the character information of the sign area.

[0018] Furthermore: In S2, the specific steps of obtaining the image to be detected with clarity meeting the preset confidence level are as follows:

[0019] S21. Adjust the camera position according to the position information of the pre-set position to be detected to make it correspond to the position information of the pre-set position to be detected, and capture the current image to be detected;

[0020] S22. Use the video image quality diagnosis model to judge the clarity of the image to be detected. If the confidence level of the image to be detected is less than the preset threshold, it indicates that the current image to be detected is not clear. Then repeat S21 until an image to be detected with clarity meeting the preset confidence level is obtained.

[0021] Further: When the image to be detected contains signboard information, use the following steps to obtain the signboard offset:

[0022] S23. Use the signboard detection model and the character recognition model to obtain the signboard area in the current image to be detected, and obtain the coordinate information and character information of the signboard area.

[0023] S24. Judge whether the character information of the image to be detected is consistent with the character information of the preset reference image. If they are consistent, obtain the coordinate information and confidence level of the signboard area in the current image to be detected, and calculate the signboard offset.

[0024] Further: In S3, the specific calculation steps of the offset include:

[0025] S31. Use the feature point matching algorithm process to calculate the feature point offset; use the template matching algorithm process to calculate the template area offset; use the target detection method to calculate the target offset.

[0026] S32. Judge whether each offset is valid. If there are two sets of offsets that are valid, execute S4; otherwise, execute S33.

[0027] S33. Readjust the camera position to make it correspond to the preset position information to be detected, re-capture the current image to be detected, and repeat the execution of S2 and S3.

[0028] Further: In S31, the specific calculation steps of using the feature point matching algorithm process to calculate the feature point offset include:

[0029] Perform downsampling on the preset reference image and the image to be detected.

[0030] Obtain the feature points of the preset reference image and the image to be detected.

[0031] Obtain the optimal matching point pairs.

[0032] Calculate the single mapping matrix and confidence coefficient of the preset reference image and the image to be detected.

[0033] According to the image center point coordinates, use the perspective transformation to calculate the feature point offsets in the x and y directions. .

[0034] Further: In S31, the specific calculation steps for calculating the template region offset using the template matching algorithm process include:

[0035] The template matching algorithm evaluates the effectiveness of image matching by calculating the confidence levels of two images, and then determines the template region offset of the image. The specific implementation process of the algorithm includes:

[0036] Preprocessing the first template image and the image to be detected;

[0037] Performing template matching using the normalized squared difference matching method;

[0038] Analyzing the matching results to obtain the region with the highest matching degree as the target region;

[0039] Calculating the correlation value of each matching window and evaluating the matching correlation coefficient;

[0040] Calculating the template region offsets in the x and y directions based on the coordinate information of the template region of the reference image and the coordinate information of the matched region .

[0041] Further: In S31, the specific calculation steps for calculating the target offset using the target detection method include:

[0042] Using a multi-target detection model to obtain the target region;

[0043] Performing similarity recognition on the target image and the second template image to obtain the matching correlation coefficient;

[0044] Obtaining the region that is the same as the second template image as the final detection region;

[0045] Calculating the target offsets in the x and y directions based on the coordinate information of the region of the second template image and the coordinate information of the currently detected target region .

[0046] Further: In S4, a non-linear function is introduced to automatically adjust the weight coefficient corresponding to each offset, and the final offsets in the x and y directions are calculated based on each offset and its corresponding weight coefficient.

[0047] A preset position automatic calibration and deviation correction system according to the present invention includes a preset position reference image processing module, an image to be detected processing module, an offset calculation module, a final offset calculation module, and a preset position deviation correction module;

[0048] The preset position reference image processing module is used to obtain a preset position reference image with clarity meeting the preset confidence level;

[0049] The image to be detected processing module is used to obtain an image to be detected with clarity meeting the preset confidence level;

[0050] An offset calculation module, configured to calculate respective offsets and their corresponding correlation coefficients by using a feature point matching algorithm process, a template matching algorithm process, and / or an object detection method respectively;

[0051] A final offset calculation module, configured to adjust the weight coefficients of each offset and calculate a final offset according to each offset and its corresponding weight coefficient;

[0052] A preset deviation correction module, configured to perform preset deviation correction according to the final offset.

[0053] The beneficial effects of the present invention are:

[0054] Aiming at the problem that the preset position of a pan-tilt control camera is significantly offset due to long-term operation, the present invention designs a preset position calibration and deviation correction method and system from the perspectives of practicability, high efficiency, high precision, full automation, etc. It can be used as an intelligent operation and maintenance tool for cameras in an intelligent patrol system or a video surveillance system, effectively solve the problem of preset position offset, improve the recognition accuracy of intelligent patrol, and reduce the manual operation and maintenance cost. Description of the Drawings

[0055] Figure 1 It is a schematic flowchart of a preset position automatic calibration and deviation correction method in an embodiment of the present invention;

[0056] Figure 2 It is a block diagram of each module in an embodiment of the present invention;

[0057] Figure 3 It is a flowchart of a feature matching algorithm;

[0058] Figure 4 It is a flowchart of a template matching algorithm;

[0059] Figure 5 It is a flowchart of object detection offset calculation;

[0060] Figure 6 It is a flowchart of identification plate offset calculation. Detailed Embodiments

[0061] The following are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The following embodiments are only used to explain the present invention and cannot be construed as a limitation of the present invention. The protection scope of the present invention should be subject to the protection scope of the claims. The embodiments of the present invention are described in detail below. For the convenience of describing the present invention and simplifying the description, the technical terms used in the specification of the present invention should be interpreted in a broad sense, including but not limited to the conventional replacement schemes not mentioned in this application, and including both direct implementation methods and indirect implementation methods.

[0062] Embodiment 1

[0063] Combined with Figures 1 - 6 To illustrate this embodiment, a preset position automatic calibration and deviation correction method disclosed in this embodiment includes the following steps:

[0064] S1. Obtain a preset position reference image with clarity meeting the preset confidence level; automatically use the image that meets the clarity requirements and is first obtained with the position information of this preset position as the reference image.

[0065] In S1, the specific steps for obtaining a preset position reference image with clarity meeting the preset confidence level are as follows:

[0066] S11. Adjust the position of the camera according to the position information of the preset position so that it corresponds to the position information of the preset position, and capture the current preset position image.

[0067] Referring to Figure 1 , taking a fixed preset position of a certain camera as an example, there are multiple monitoring cameras installed in the substation that builds an intelligent patrol system, which are used to monitor all target devices to be observed, such as pointer meters, oil level gauges, breathing apparatuses, etc. Since there are many devices to be observed, it is unrealistic and costly to install specific cameras for each device. Therefore, each camera is configured with multiple preset positions to observe different devices. According to the position information of the preset positions imported for each camera, the pan-tilt is controlled to adjust the camera to this preset position, and after waiting for 10 - 15 seconds, the corresponding preset position image can be captured.

[0068] S12. Use the video image quality diagnosis model to judge the clarity of the preset position image. If the confidence level of the preset position image is less than the preset threshold, it means that the current preset position image is not clear, and S11 is repeated until a preset position reference image with clarity meeting the preset confidence level is obtained.

[0069] Collect training sample data in the substation scenario. The video image quality diagnosis model is a classification model trained by the Inception ResNet v2 (i.e., Inception Residuality Network v2, a deep convolutional neural network) network of the TensorFlow framework (an open-source machine learning framework developed by the Google team), and can support 11 types of image quality diagnosis, including black and white images, video occlusion, over-dark images, over-bright images, blurred images, signal loss, noise interference, color cast, stripe interference, signal anomaly, and low contrast.

[0070] When it is determined that the image is over-bright, the recognition result returns that the image exposure is too high and cannot be analyzed;

[0071] When it is determined that the image is blurred, wait for 5 seconds to re-capture the preset position image, and then proceed to step 2. If it is determined that the image is blurred three times in a row, the recognition result returns that the image is blurred and cannot be analyzed;

[0072] When it is determined that the signal is lost or the signal is abnormal, the recognition result returns that the signal is abnormal and cannot be analyzed;

[0073] When it is determined that there is noise interference, color cast, or stripe interference, the recognition result returns that there is an image decoding problem and cannot be analyzed.

[0074] S13. Take the central area of the preset position reference image as the first template image, and obtain the regional coordinate information of the first template image; use the multi-object detection model to obtain the target area with the highest confidence in the current preset position reference image, take the target area as the second template image, and obtain the regional coordinate information of the second template image; use the sign detection model and the text recognition model to obtain the sign area in the current preset position reference image, and obtain the coordinate information and text information of the sign area;

[0075] Automatically extract an image with a size of 200*200 pixels in the center of the preset position reference image (which can be adjusted according to the actual scenario) as the first template image, and obtain the regional coordinate information of the first template image; use the multi-object detection model to obtain the target area with the highest confidence as the second template image, and obtain the regional coordinate information of the second template image; use the sign detection and text recognition models to obtain the sign area coordinate information and text information; the sign can be added next to the device as needed, or the device signs in the substation can be directly used;

[0076] Such as Figure 6As shown, the multi-object detection model and the sign detection model adopt the yolov5 algorithm, which supports the detection of various substation equipment such as pointer instruments, changeover switches, oil level gauges, breather, oil conservator oil level, bushings, coolers, voltage transformers, current transformers, etc. and signs, and can be applied to all substation scenarios.

[0077] The text recognition model uses PaddleOCR (a powerful open-source OCR, Optical Character Recognition tool developed by Baidu based on its deep learning framework PaddlePaddle).

[0078] S14. Save the preset reference image, the first template image and its region coordinate information, the second template image and its region coordinate information, the sign region and its region coordinate information, and the text information of the sign region.

[0079] The preset reference image is saved to the local in JPG format, with the naming format of "point code.jpg", such as a953286e-ca38-4aa5-8839-8402f3d13c9e.jpg, where the point code is a randomly generated value with uniqueness and is the unique identification code of this preset point; the first template image is saved to the local in JPG format, with the naming format of "point code_templet.jpg"; the second template image region coordinate information, the sign region coordinate information and the text information are stored in the local in json format, with the naming format of "point code.json", and the defined json contains fields as shown in Table 1:

[0080] Table 1 Meanings of json fields

[0081]

[0082] S2. Obtain the image to be detected with clarity meeting the preset confidence level; the specific implementation method is the same as that of S1;

[0083] In S2, the specific steps for obtaining the image to be detected with clarity meeting the preset confidence level are as follows:

[0084] S21. Adjust the camera position according to the point information of the preset position to be detected, so that it corresponds to the point information of the preset position to be detected, and capture the current image to be detected;

[0085] S22. Use the video image quality diagnosis model to judge the clarity of the image to be detected. If the confidence level of the image to be detected is less than the preset threshold, it means that the current image to be detected is not clear, and then repeat S21 until an image to be detected with clarity meeting the preset confidence level is obtained.

[0086] S23. Use the sign detection model and the text recognition model to obtain the sign area in the current image to be detected, and obtain the coordinate information and text information of the sign area;

[0087] S24. Determine whether the text information of the image to be detected is consistent with the text information of the preset reference image. If they are consistent, obtain the coordinate information and confidence level of the sign area in the current image to be detected, and calculate the sign offset .

[0088] First, use the yolov5 sign detection model to detect the image to be detected to obtain a small image of the sign area. Then, input the small sign image into the PaddleOCR text recognition model to obtain the sign text information, and determine whether it is consistent with the text information of the preset reference image. If they are not consistent, it means that there is no target to be observed within the preset observation range, and the camera position needs to be adjusted to find the target to be observed.

[0089] The sign detection and text recognition algorithm process includes:

[0090] Use the yolov5 sign detection model to detect the image to be detected, screen candidate targets with a confidence level greater than 0.6, and use the text recognition model for text recognition;

[0091] Match the text information of all candidate targets with the text information in the reference image one by one. The target with the highest matching degree and a similarity greater than 0.9 is the final matching target. Among them, the similarity matching algorithm selects a string similarity evaluation method based on Jaccard similarity;

[0092] The specific implementation process of Jaccard similarity is as follows:

[0093] Preprocess the two strings to be compared and For example, convert to a unified format (consistent case); remove punctuation marks; retain or perform word segmentation on Chinese characters and numbers to improve the accuracy of similarity calculation;

[0094] Construct character sets, and convert the processed strings into character sets, including Chinese characters and numbers:

[0095] ;

[0096] ;

[0097] Among them, A is the character set of the string , including Chinese characters, numbers and other characters; B is the character set of the string , including Chinese characters, numbers and other characters, is a single character in the string ; is a single character in the string ;

[0098] Calculate the intersection and union of character sets:

[0099] The number of common characters;

[0100] The number of all different characters;

[0101] Calculate the Jaccard similarity using the number of intersection and union :

[0102] ;

[0103] Correlation coefficient is the optimal similarity, and the coordinates of the optimal matching area are the area positions corresponding to the optimal similarity;

[0104] ;

[0105] Assume that the coordinate position of the reference map sign is , and calculate the sign offset based on the sign detection and text recognition algorithm by the difference between the optimal matching position and the original position , the formula is as follows:

[0106] .

[0107] S3. Calculate the offset obtained by using the feature point matching algorithm process, template matching algorithm process, and / or object detection method respectively;

[0108] In S3, the specific calculation steps of the offset include:

[0109] S31. Calculate the feature point offset using the feature point matching algorithm process ; calculate the template area offset using the template matching algorithm process ; calculate the object offset using the object detection method ;

[0110] As Figure 3 shown, the feature point matching algorithm is a method based on SIFT feature matching, which calculates the feature point offset between the image to be detected and the preset reference image, and calculates the correlation (confidence) according to the feature point matching quality to evaluate the credibility of image matching. The specific implementation process of the algorithm includes:

[0111] Downsample the preset reference image and the image to be detected, with the downsampling factor P = 2;

[0112] Obtain the feature points of the preset reference image and the image to be detected; Use SIFT::create() to create a SIFT detector, and use detectAndCompute (a function for feature detection in OpenCV) to obtain the feature point information of the reference image and the image to be detected respectively;

[0113] Obtain the optimal matching point pairs;

[0114] Use FlannBasedMatcher to obtain the feature point matching pairs. Under the condition that matches[i].distance < 0.61 * (matches[i + 1].distance), obtain the optimal matching point pair set M, and eliminate the invalid point pairs to improve the matching accuracy; where matches[i] represents the i-th feature point, matches[i + 1] represents the (i + 1)-th feature point, and matches[i].distance represents the distance between the matching feature points;

[0115] Calculate the feature point offset The formula is as follows:

[0116] ;

[0117] where N is the number of matching pairs in the optimal matching point pair set M; and are the coordinates of the j-th matching point in the source image and the template image respectively; is the feature point offset calculated by the feature point matching algorithm.

[0118] Calculate the single mapping matrix and the correlation coefficient;

[0119] Calculate the correlation coefficient by calculating the mean and standard deviation of the matching point distances, and normalize it to the range of 0 to 1. The specific implementation formula is as follows:

[0120] The formula for calculating the correlation coefficient is as follows:

[0121] ;

[0122] ;

[0123] ;

[0124] where N is the number of matching pairs in the optimal matching point pair set M; is the distance of the j-th matching point; and are the average distance and the standard deviation; The correlation coefficient calculated by the feature point matching algorithm, where 0 indicates unreliable matching and 1 indicates very reliable matching.

[0125] Of course, the correlation coefficient of the matching of two images can also be evaluated according to the quality of the matching points. The specific formula is as follows:

[0126] ;

[0127] Where N is the number of matching pairs in the set M of optimal matching point pairs; ALL represents the number of all matching points.

[0128] According to the coordinates of the center point of the image, the offset of the feature points in the x and y directions is calculated using projective transformation .

[0129] As Figure 4 shown, the template matching algorithm evaluates the effectiveness of image matching by calculating the correlation (confidence) of two images, and then determines the offset of the template area of the image;

[0130] Preprocessing of the first template image and the image to be detected mainly includes operations such as grayscale conversion, smoothing filtering, and image enhancement of the image, which can reduce the noise and interference in the image and make the matching result more accurate;

[0131] Use the sliding window method to search for the best position of the template image in the image to be detected, traverse each possible position of the image to be detected, place the template image at this position, and calculate the matching degree between the image at the sliding window position and the template image;

[0132] The specific implementation process of the algorithm includes:

[0133] Preprocessing of the first template image and the image to be detected;

[0134] Perform template matching using the normalized square difference matching method;

[0135] Analyze the matching result, and the region with the highest matching degree is the target region;

[0136] Calculate the correlation value of each matching window using indicators such as correlation or mean square error (MSE), and evaluate the matching correlation coefficient;

[0137] According to the coordinate information of the template area of the reference image and the coordinate information of the matched area, calculate the offset of the template area in the x and y directions .

[0138] The specific reasoning process is described as follows:

[0139] Define the size of the sliding window to be the same as that of the template image;

[0140] Traverse the image to be detected row by row and column by column;

[0141] For each window position, calculate the matching degree with the template image.

[0142] Assume that the upper left corner coordinates of the window are , and the formula for calculating the matching degree of this window is as follows:

[0143] ;

[0144] Among them, is the matching degree at position ;

[0145] are two optional similarity measurement methods.

[0146] Optionally, use correlation as the similarity measurement method, and the calculation formula is as follows:

[0147] ;

[0148] ;

[0149] ;

[0150] Among them, N is the number of pixels in the template image; is the coordinate of the i-th pixel in the image; is the sliding window in the image to be detected that is the same size as the template image; is the template image; is the pixel value at position in the sliding window; is the pixel value at position in the template image; is the window position with the optimal similarity, that is, the best matching position; is the correlation coefficient of the template matching algorithm, that is, the optimal matching similarity.

[0151] Optionally, use the mean square error as the similarity measurement method, and the calculation formula is as follows:

[0152] ;

[0153] ;

[0154] ;

[0155] Assume that the original position of the template image is , and calculate the template area offset , the formula is as follows:

[0156] .

[0157] like Figure 5 As shown, the multi-target detection model is used to obtain the target area. The target here can be a signboard or a common device in a substation;

[0158] The target image and the second template image are similarly identified to obtain a matching correlation coefficient;

[0159] Acquire the area consistent with the second template image as the final detection area;

[0160] According to the coordinate information of the second template image area and the coordinate information of the currently detected target area, the target offset in the x and y directions is calculated. .

[0161] Use the yolov5 multi-target detection model to detect the image to be detected, and select targets with a confidence greater than 0.6 and a target category consistent with the second template image type as candidate targets;

[0162] All candidate targets are matched with the second template image one by one, and the target with the highest matching degree and a similarity greater than 0.9 is the final matching target. The similarity matching algorithm uses the histogram equalization method for similarity recognition;

[0163] The specific similarity implementation process is as follows:

[0164] Get candidate target area image and template image Histogram of :

[0165] ;

[0166] in, is the histogram of image I; Represents the number of pixels with gray value i.

[0167] Normalize the histogram:

[0168] ;

[0169] in, is the normalized histogram.

[0170] Calculate similarity:

[0171] ;

[0172] in, and are two matched target area images; is the Euclidean norm of the normalized histogram, and the formula is as follows:

[0173] ;

[0174] correlation coefficient is the optimal similarity, and the coordinates of the optimal matching template area are the area positions corresponding to the optimal similarity:

[0175] ;

[0176] Assume that the coordinate position of the second template image is , and calculate the target offset based on the multi-object detection algorithm through the difference between the optimal matching template position and the original position , and the formula is as follows:

[0177] .

[0178] S32. Judge whether each offset , , , is valid. If there are two sets of offsets that are valid, execute S4; otherwise, execute S33;

[0179] S33. Readjust the camera position, automatically adjust the pan-tilt to capture images up, down, left, and right multiple times according to the threshold size, so that it corresponds to the preset point information to be detected, re-capture the current image to be detected, and repeat S2 and S3.

[0180] First, according to the final offset in the x and y axis offset directions, with the set threshold as the step size for each adjustment, control the pan-tilt camera to turn to the set position, wait for 10 - 15 seconds to capture an image, adjust the capture four times according to the set threshold step size. If the requirements are not met, terminate the pan-tilt control in this direction. Then, in this way, control the pan-tilt to capture images up, down, left, and right respectively.

[0181] At least ensure that when there are two sets of valid values for the offsets , , , can the weight coefficients of each offset be adjusted; automatically adjust the pan-tilt up, down, left, and right multiple times according to the threshold size to obtain the preset position image. This step is mainly used to ensure that there are targets to be observed within the observation range.

[0182] Specifically, the pan-tilt control offset setting rules are as follows:

[0183] First, according to the final offset direction, adjust the pan-tilt head with a step size of 200 pixels, control the pan-tilt head camera to turn to the set position, wait for 10 - 15 seconds to capture an image, execute steps 5 - 9 again, set the maximum number of automatic adjustments to 4 times. If it fails, restore the preset position to the initial position; then, execute the pan-tilt head adjustment with offsets (0, -200), (0, 200), (-200, 0), (200, 0) respectively, with a maximum number of 4 times and a step size of 200 pixels, and execute steps 5 - 9. If it is successful, exit. If all fail, stop the preset position deviation correction, indicating that the area to be observed is not found.

[0184] S4. Adjust the weight coefficient of each offset, and calculate the final offset according to each offset and its corresponding weight coefficient;

[0185] In S4, introduce a non-linear function to automatically adjust each offset 、 、 、 corresponding weight coefficient, and calculate the final offsets in the x and y directions according to each offset and its corresponding weight coefficient;

[0186] The non-linear function can be an exponential function or a Sigmoid function. In this embodiment, the Sigmoid function is used to adjust the offset weight coefficients calculated by different algorithms, which can effectively enhance or weaken the influence of a certain offset, effectively enhance the fusion ability of different algorithm results, and improve the accuracy of the final offset. In addition, a dynamic weight coefficient adjustment mechanism can also be defined to update the weights. This real-time feedback and adjustment mechanism can further adjust the trust degree and weight coefficients of the offsets calculated by each algorithm according to the results of each calculation and the newly obtained data to adapt to environmental changes and improve the accuracy of the final offset.

[0187] According to the offsets 、 、 、 and the corresponding correlation coefficients 、 、 、 , use the non-linear function of the Sigmoid function to adjust the correlation coefficients to obtain the weight coefficients corresponding to each offset, and finally adopt the weighted fusion method to calculate the final offsets in the x and y directions .

[0188] Normalize the correlation coefficients 、 、 、 , and define them as weight coefficients:

[0189] ;

[0190] Among them, is the normalized weight coefficient.

[0191] Introduce the Sigmoid function to adjust :

[0192] ;

[0193] Among them, is the adjusted weight coefficient; k is the coefficient controlling the degree of nonlinearity, and the initial value is 10. This coefficient is not fixed. When changes greatly, the value of k needs to be appropriately increased, and vice versa, the value of k needs to be decreased;

[0194] Calculate the final offset :

[0195] ;

[0196] ;

[0197] ;

[0198] Among them, represents the offset (x, y) calculated by the i-th method; represents the final offset.

[0199] S5. Perform preset deviation correction according to the final offset .

[0200] After obtaining the reference image and the image to be detected, the image quality is first diagnosed to evaluate the rationality of the current preset position and the quality of the acquired image, ensuring the best current observation screen. Then, the offsets are calculated respectively through three different dimensions of the feature point matching algorithm, the template matching algorithm, and the object detection. Finally, the offset closer to the true offset degree is calculated through the weighted fusion method, so as to realize the automatic, high-precision calibration and deviation correction of the preset position. At the same time, in the whole implementation process, the situation that the observed target is not within the observation range due to the large offset degree is also considered, and the preset position where the observed target is located is found by automatically adjusting and controlling the pan-tilt capture up, down, left, and right multiple times according to the set parameter adjustment step.

[0201] Embodiment 2

[0202] This embodiment is described in conjunction with Example 1. This embodiment discloses a preset position automatic calibration and correction system, including a preset position reference image processing module, a to-be-detected image processing module, an offset calculation module, a final offset calculation module, and a preset position correction module;

[0203] A preset position reference image processing module, used for obtaining a preset position reference image whose clarity meets a preset confidence level;

[0204] The camera preset position information import module is used to add camera position code, IP, preset position number and other information to prepare for the subsequent capture of preset position pictures; the import method can use Excel batch import, and the storage method can use MySql database;

[0205] The preset position image capture module is used to capture the current camera preset position image. After successfully calling the preset position, it is generally necessary to wait for 10 to 15 seconds before starting the capture service to capture the image, so as to avoid the situation where the captured image is blurred or there is no observed target in the captured image due to the preset position not being transferred to the right position or the camera focusing not being completed.

[0206] The preset position picture receiving module is used to receive the preset position picture, which can convert the picture into base64 encoding format and transmit and receive it using TCP;

[0207] The preset position image quality diagnosis module is used to determine whether the current preset position is available, whether there is video blocking, image too dark, image too bright, image blur, signal loss, noise interference, image color cast, stripe interference, signal abnormality, etc.

[0208] The data information storage module is used to save the reference image, the first template image, the second template image and the coordinate information, the sign text information and the regional coordinate information; here, the image is saved locally in JPG format, and the coordinate, text and other text information are saved in json format in the local .json file, and of course it can also be stored in the MySql database;

[0209] The image processing module to be detected is used to obtain the image to be detected whose clarity meets the preset confidence level;

[0210] An offset calculation module, used to calculate respective offsets and their corresponding correlation coefficients using a feature point matching algorithm process, a template matching algorithm process and / or a target detection method;

[0211] An offset calculation module based on feature point matching algorithm is used to calculate feature point offset ;

[0212] An offset calculation module based on the template matching algorithm is used to calculate the template area offset ;

[0213] Offset calculation module based on multi-object detection, used to calculate the object offset ;

[0214] Offset calculation module based on signboard object detection and text recognition, used to calculate the signboard offset ;

[0215] Final offset calculation module, used to adjust the weight coefficient of each offset, and calculate the final offset according to each offset and its corresponding weight coefficient;

[0216] Preset automatic query module, used to automatically control the camera pan-tilt to search for the target area up, down, left, and right according to the set rules when there is no target device within the observation range with a large preset offset;

[0217] Final offset calculation module based on information fusion, used to calculate the effective final offset ;

[0218] Preset deviation correction module, used to control the camera pan-tilt, adjust the preset to the ideal position, and perform preset deviation correction according to the final offset. According to the offset parameter Adjust the preset. It can be corrected multiple times until the preset is adjusted to the ideal position. The allowable offset threshold can be set to (20, 20). As long as the calculated offset x ≤ 20 and y ≤ 20, it is considered that the preset deviation correction is completed. Of course, the maximum number of correction times also needs to be set. Here, it is set to 5.

Claims

1. A preset position automatic calibration and correction method, characterized in that: The steps include: S1, obtaining a preset position reference image whose clarity meets a preset confidence level; S2, obtaining an image to be detected whose clarity meets a preset confidence level; S3, respectively calculating each offset using a feature point matching algorithm process, a template matching algorithm process and / or a target detection method; In S3, the specific steps of calculating the offset include: S31, using a feature point matching algorithm process to calculate a feature point offset; using a template matching algorithm process to calculate a template area offset; using a target detection method to calculate a target offset; S32, determine whether each offset is valid, if there are two sets of offsets that are valid, execute S4, otherwise execute S33; S33, readjust the camera position to make it correspond to the position information of the preset position to be detected, recapture the current image to be detected, and repeat S2 and S3; S4, adjusting the weight coefficient of each offset, and calculating the final offset according to each offset and its corresponding weight coefficient; introducing a nonlinear function to automatically adjust the weight coefficient corresponding to each offset, and calculating the final offset in the x and y directions according to each offset and its corresponding weight coefficient; S5. Perform preset position correction according to the final offset.

2. The method for automatic calibration and correction of preset positions according to claim 1, characterized in that: In S1, the specific steps of obtaining a preset position reference image whose clarity meets a preset confidence level are as follows: S11, adjusting the camera position according to the preset position information so that it corresponds to the preset position information, and capturing the current preset position image; S12, using the video image quality diagnosis model to determine the clarity of the preset position image, if the confidence of the preset position image is less than a preset threshold, it means that the current preset position image is not clear, and S11 is repeated until a preset position reference image with a clarity that meets the preset confidence is obtained; S13, taking the central area of ​​the preset position reference image as the first template image, and obtaining the area coordinate information of the first template image; Using a multi-target detection model to obtain the target area with the highest confidence in the current preset position reference image, using the target area as the second template image, and obtaining the area coordinate information of the second template image; using a sign detection model and a text recognition model to obtain the sign area in the current preset position reference image, and obtaining the coordinate information and text information of the sign area; S14, saving the preset reference image, the first template image and its area coordinate information, the second template image and its area coordinate information, the sign area and its area coordinate information, and the text information of the sign area.

3. The method for automatic calibration and correction of preset positions according to claim 2, characterized in that: In S2, the specific steps of obtaining the image to be detected whose clarity meets the preset confidence level are as follows: S21, adjusting the camera position according to the position information of the preset position to be detected, so that it corresponds to the position information of the preset position to be detected, and capturing the current image to be detected; S22. Use the video image quality diagnosis model to determine the clarity of the image to be detected. If the confidence of the image to be detected is less than a preset threshold, it means that the current image to be detected is not clear. Repeat S21 until an image to be detected whose clarity meets the preset confidence is obtained.

4. The method for automatic calibration and correction of preset positions according to claim 3, characterized in that: If the image to be detected contains identification information, the following steps are used to obtain the identification offset: S23, using the sign detection model and the text recognition model to obtain the sign area in the current image to be detected, and obtain the coordinate information and text information of the sign area; S24, determining whether the text information of the image to be detected is consistent with the text information of the preset reference image, if consistent, obtaining the coordinate information and confidence of the sign area in the current image to be detected, and calculating the sign offset.

5. The method for automatic calibration and correction of preset positions according to claim 4, characterized in that: In S31, the specific calculation steps of calculating the feature point offset using the feature point matching algorithm process include: Down-sampling the preset reference image and the image to be detected; Acquire feature points of a preset reference image and an image to be detected; Get the best matching point pair; Calculate the single mapping matrix and confidence coefficient of the preset reference image and the image to be detected; According to the coordinates of the center point of the image, the feature point offset in the x and y directions is calculated using the transmission transformation .

6. The method for automatic calibration and correction of preset positions according to claim 1, characterized in that: In S31, the specific calculation steps of calculating the template area offset using the template matching algorithm process include: The template matching algorithm evaluates the effectiveness of image matching by calculating the confidence of two images, and then determines the offset of the template area of ​​the image. The algorithm implementation process specifically includes: Preprocessing of the first template image and the image to be detected; Template matching is performed using the normalized square difference matching method; Analyze the matching results and obtain the area with the highest matching degree as the target area; Calculate the correlation value of each matching window and evaluate the matching correlation coefficient; According to the reference image template area coordinate information and the matching area coordinate information, calculate the template area offset in the x and y directions .

7. The method for automatic calibration and correction of preset positions according to claim 1, characterized in that: In S31, the specific calculation steps of calculating the target offset using the target detection method include: Use the multi-target detection model to obtain the target area; The target image and the second template image are similarly identified to obtain a matching correlation coefficient; Acquire the area consistent with the second template image as the final detection area; According to the coordinate information of the second template image area and the coordinate information of the currently detected target area, the target offset in the x and y directions is calculated. .

8. A preset position automatic calibration and correction system for implementing a preset position automatic calibration and correction method according to any one of claims 1 to 7, characterized in that: It includes a preset position reference image processing module, a to-be-detected image processing module, an offset calculation module, a final offset calculation module and a preset position deviation correction module; A preset position reference image processing module, used for obtaining a preset position reference image whose clarity meets a preset confidence level; The image processing module to be detected is used to obtain the image to be detected whose clarity meets the preset confidence level; An offset calculation module, used to calculate respective offsets and their corresponding correlation coefficients using a feature point matching algorithm process, a template matching algorithm process and / or a target detection method; The final offset calculation module is used to adjust the weight coefficient of each offset and calculate the final offset according to each offset and its corresponding weight coefficient; The preset position correction module is used to perform preset position correction according to the final offset.

Images