Camera calibration method, device and electronic equipment for substation inspection

Abstract

The application discloses a kind of camera calibration method, device and electronic equipment of substation inspection.The method comprises: in the process of carrying out inspection to power equipment in substation, the image template of power equipment is acquired, and the target image of power equipment based on the camera of current preset position is collected, wherein the image template is the image of known accurate position and definition for power equipment collection;Image quality evaluation is carried out based on image template and target image, and target image quality score is obtained;It is judged whether target image quality score is less than preset score threshold;In the case where target image quality score is less than preset score threshold, the target image positioning offset between image template and target image is determined;Parameter calibration is carried out to camera based on target image positioning offset.The application solves the technical problems of position deviation and insufficient definition of power equipment image collected based on camera in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of smart grid, in particular to a camera calibration method and device for substation inspection and an electronic device. BACKGROUND

[0002] With the continuous expansion of the power system scale and the continuous progress of technology, the traditional power inspection method has been difficult to meet the demand of modern power grid efficient and safe operation. Artificial inspection not only consumes time and effort, but also has certain safety hazards. Especially in remote areas or bad weather conditions, the inspection work becomes more difficult and dangerous. Therefore, the power grid field begins to actively explore and promote the use of advanced camera inspection technology to improve the inspection efficiency and safety. In recent years, the development of artificial intelligence, Internet of Things and big data technologies has provided strong support for the intelligentization of the power system. By installing high-definition cameras on key facilities such as transmission lines and substations and combining intelligent analysis algorithms, real-time monitoring and fault warning of power equipment status can be realized, thereby greatly improving the level of power grid operation and management.

[0003] At present, in the actual use process of a large number of substation online inspection systems, it is found that the camera is easily affected by natural factors (such as wind, rain, dust, temperature, vibration, etc.) and its own factors (unstable fixation, mechanical part wear, control precision difference, firmware version difference, etc.), resulting in problems such as inaccurate positioning of the photographed picture, preset position offset, and blurred image picture.

[0004] In view of the problems in the above related technologies that the power equipment image collected based on the camera is prone to position offset and insufficient clarity, no effective solution has been proposed at present. SUMMARY

[0005] The embodiments of the present application provide a camera calibration method, device and electronic equipment for substation inspection, to at least solve the technical problems that the power equipment image collected based on the camera is prone to position offset and insufficient clarity in the related technologies.

[0006] According to an aspect of an embodiment of the present application, a camera calibration method for substation inspection is provided, comprising: acquiring an image template of a power equipment in a substation, and a target image of the power equipment collected based on a current preset position of a camera, wherein the image template is an image of a known accurate position and clarity collected for the power equipment; performing image quality evaluation based on the image template and the target image to obtain a target image quality score; determining whether the target image quality score is less than a preset score threshold; in the case that the target image quality score is less than the preset score threshold, determining a target image positioning offset between the image template and the target image; and performing parameter calibration on the camera based on the target image positioning offset.

[0007] According to another aspect of the embodiments of the present application, there is also provided a camera calibration device for substation inspection, comprising: an image acquisition module configured to acquire an image template of a power equipment in a substation, and a target image of the power equipment acquired by a camera based on a current preset position, wherein the image template is an image of a known accurate position and a clear definition acquired for the power equipment; a quality assessment module configured to perform image quality assessment based on the image template and the target image to obtain a target image quality score; a threshold judgment module configured to judge whether the target image quality score is less than a preset score threshold; an offset determination module configured to determine a target image positioning offset between the image template and the target image in a case where the target image quality score is less than the preset score threshold; and a parameter calibration module configured to perform parameter calibration on the camera based on the target image positioning offset.

[0008] According to another aspect of the embodiments of the present application, there is also provided a non-volatile storage medium storing a plurality of instructions, the instructions being adapted to be loaded and executed by a processor to implement any of the camera calibration methods for substation inspection.

[0009] According to another aspect of the embodiments of the present application, there is also provided an electronic device comprising one or more processors and a memory, the memory being configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement any of the camera calibration methods for substation inspection.

[0010] In the embodiments of the present application, by acquiring an image template of a power equipment in a substation, and a target image of the power equipment acquired by a camera based on a current preset position, wherein the image template is an image of a known accurate position and a clear definition acquired for the power equipment; performing image quality assessment based on the image template and the target image to obtain a target image quality score; judging whether the target image quality score is less than a preset score threshold; determining a target image positioning offset between the image template and the target image in a case where the target image quality score is less than the preset score threshold; and performing parameter calibration on the camera based on the target image positioning offset, the purpose of acquiring an image template and comparing and evaluating a real-time image, automatically determining an image quality deviation, and accurately adjusting camera parameters based on deviation information is achieved, thereby realizing fast and accurate calibration of the camera, ensuring power equipment inspection image quality, and improving inspection efficiency and accuracy, and further solving the technical problems of position offset and insufficient definition of power equipment images acquired based on a camera in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0012] Figure 1 is a flow chart of a camera calibration method for substation inspection according to an embodiment of the application;

[0013] Figure 2 is a flow chart of an alternative camera calibration method for substation inspection according to an embodiment of the application;

[0014] Figure 3 is an alternative system framework diagram according to an embodiment of the application;

[0015] Figure 4 is a schematic diagram of a camera calibration device for substation inspection according to an embodiment of the application. DETAILED DESCRIPTION

[0016] In order to make the personnel in the art better understand the application scheme, the technical scheme in the embodiments of the application will be described clearly and completely below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the protection scope of the application.

[0017] It should be noted that the terms "first", "second", and the like in the specification and claims of the application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] According to an embodiment of the application, a method embodiment of camera calibration for substation inspection is provided. It should be noted that the steps shown in the flow chart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flow chart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0019] Figure 1 is a flowchart of a camera calibration method for substation inspection according to an embodiment of the present application, as shown in the figure, the method comprises the following steps: Figure 1

[0020] In step S102, an image template of the power equipment in the substation is acquired, and a target image of the power equipment captured by the camera based on the current preset position is acquired.

[0021] Optionally, the target image can be a picture taken by the camera for the power equipment to be inspected in the substation. The image template is an image with known accurate position and clarity collected for the power equipment, and the image template includes a reference map of the target detection object (such as the power equipment) and coordinate information.

[0022] Optionally, the execution subject of steps S102 to S110 can be a camera calibration system. The camera calibration system acquires the image template from the online intelligent inspection system of the substation. Before step S102 is executed, it can be detected whether the image template of the power equipment exists, if it exists, the operation of step S102 is executed, if it does not exist, the image template can be generated by online labeling through the camera calibration system, and then the operation of step S102 is executed. If the online intelligent inspection system already has the image template of the power equipment, it is synchronized to the camera calibration system through online or offline mode, and the power equipment can include but is not limited to pointer table, oil level table, digital table, knife gap, split, indicator light, etc. If the online intelligent inspection system has no image template, the image template can be generated by online labeling through the camera calibration system, for example, a large number of substation equipment appearances without templates.

[0023] Optionally, the camera calibration system can indirectly acquire the camera preset position identification ID and the PTZF parameter corresponding to the preset position ID from the online intelligent inspection system or directly from the camera. P (Pan) corresponds to the rotation movement parameter of the camera in the horizontal direction; T (Tilt) corresponds to the tilt rotation parameter of the camera in the vertical direction; Z (Zoom) corresponds to the zoom parameter of the camera, which realizes the near-far adjustment of the lens field of view; F (Focus) corresponds to the zoom parameter of the camera, which realizes the focusing adjustment of the lens. The camera calibration system formulates an automatic calibration scheme plan to realize automatic calibration without human intervention. The camera calibration can be performed in the process of inspecting the power equipment in the substation. In addition, to avoid the conflict between the calibration task and the intelligent inspection task, the task plan comparison and linkage of the calibration system and the intelligent inspection task can be realized. For example, in the case that the identification rate is lower than the threshold value during the idle period after the inspection task is completed, the related operations of steps S102 to S110 are performed.

[0024] ​Optionally, the camera calibration system captures pictures of power equipment in the substation based on the obtained camera preset position ID. Since there are many cameras in the substation, multiple presets can be set for one camera. To improve efficiency, the strategy for obtaining pictures can be implemented in parallel using multiple cameras, and pictures can be captured in the order of preset position ID within a single camera to achieve the collection of images corresponding to power equipment.

[0025] In step S104, image quality evaluation is performed based on the image template and the target image to obtain a target image quality score.

[0026] Optionally, the quality of the target image can be evaluated by comparing the image template and the target image. The comparison can include, but is not limited to, the clarity of the target image, the degree of offset of the target image relative to the image template, and the like.

[0027] In an optional embodiment, the image quality evaluation based on the image template and the target image to obtain a target image quality score includes: comparing the clarity of the image template and the target image to obtain a clarity comparison result; performing feature registration on the image template and the target image to obtain a feature registration result, wherein the feature registration result indicates the degree of offset of feature points included in the target image relative to the image template; and obtaining the target image quality score based on the clarity comparison result and the feature registration result.

[0028] Optionally, the quality of the target image is evaluated through clarity comparison and feature registration. This comprehensive evaluation method not only considers the clarity of the image, but also analyzes the position deviation of key feature points in the image, thereby providing a more comprehensive image quality evaluation standard. Compared with a single evaluation method that relies only on clarity or only on feature registration, the above method can more accurately reflect the overall quality of the target image and make calibration decisions more reliably. The clarity comparison result and the feature registration result evaluate the visual quality and feature positioning accuracy of the image, respectively, and can determine whether calibration is needed and which type of calibration is needed (such as adjusting the focal length to improve clarity or adjusting the positioning parameters to correct the offset). The above targeted calibration helps to improve the efficiency and effectiveness of camera calibration and can avoid unnecessary calibration and resource waste. By automatically evaluating the quality of the target image and determining the calibration requirement, camera calibration without human intervention can be achieved. This not only improves the intelligent level of the inspection system, but also reduces labor costs, especially for environments such as substations that require frequent and accurate monitoring. Automatic calibration can ensure that the camera is always in the best working condition, improving the efficiency and safety of the inspection.

[0029] In an optional embodiment, the image template and the target image are subjected to sharpness comparison to obtain a sharpness comparison result, including: the image template and the target image are subjected to sharpness comparison by using a structural similarity index measurement method to obtain the sharpness comparison result; the image template and the target image are subjected to feature registration to obtain a feature registration result, including: the image template and the target image are subjected to feature registration by using a speeded-up robust features method to obtain the feature registration result.

[0030] Optionally, the sharpness comparison is a process of evaluating the difference in sharpness between the target image and the image template. By using the structural similarity index measurement (SSIM) algorithm, the structural similarity between two images can be measured, rather than just a direct comparison of pixels. The SSIM algorithm takes into account brightness, contrast, and structural information, and calculates the similarity of these information to obtain the sharpness comparison result. If the SSIM value is high, it indicates that the target image and the image template have high structural similarity, i.e., the image sharpness is good; otherwise, the SSIM value is low, which means the image sharpness is poor and needs to be calibrated. Feature registration is a process of evaluating the positional deviation by detecting and matching key feature points in the image template and the target image. The speeded-up robust features (SURF) algorithm can quickly detect feature points in the image and calculate the relative positional deviation of these points in two images, thereby obtaining the feature registration result. The SURF algorithm is robust to changes in light and radiation, so it can more accurately judge the feature point deviation. If the average deviation of the feature points is low, it indicates that the target image and the image template are well aligned in position; otherwise, it indicates that calibration is needed to reduce the positional deviation. By using the SSIM and SURF algorithms, not only can the accuracy and efficiency of camera calibration be improved, but also the robustness to changes in light, scale, and rotation can be enhanced, thereby ensuring that the camera images in the complex environment of the substation can always maintain high sharpness and correct position, and improving the reliability and automation level of the inspection system.

[0031] Optionally, the registration accuracy and stability of the feature algorithm SURF with acceleration mode and robust characteristics are evaluated. The SURF algorithm not only maintains the scale-invariant and rotation-invariant characteristics of the Scale-Invariant Feature Transform (SIFT), but also has strong robustness to illumination changes and radiation changes. The SURF algorithm uses the concept of HAAR features and integral images, greatly accelerating the running time of the program. The HAAR features are based on the wavelet transform principle, and the features are described by calculating the gray difference between different regions in the image. These features are usually step patterns in vertical, horizontal or diagonal directions. The pyramid image constructed by SURF is very different from SIFT, and this difference speeds up the detection. SIFT uses a Difference of Gaussians (DOG) image, while SURF uses a Hessian matrix determinant approximation image. The Hessian matrix is the core of the SURF algorithm, and the Hessian matrix is composed of functions and partial derivatives. The Hessian matrix of a pixel point in the image is where x represents the horizontal coordinate of the corresponding pixel point, and y represents the vertical coordinate of the corresponding pixel point. The Hessian determinant is where the in the Hessian determinant is the Gaussian convolution of the original image. The Gaussian kernel is subject to normal distribution, and the coefficient decreases from the center point to the outside. In order to improve the operation speed, a box filter is used to approximate the Gaussian filter. The Hessian determinant is transformed into where the gray difference of the adjacent pixels of a pixel point in the image is , and the derivative of the gray difference is

[0032] When the determinant obtains a local maximum value, it is determined that the current point is brighter or darker than other points in the neighborhood, and the position of the key feature point is determined. Comparing the key feature points of the two images can obtain the registration offset, and the average offset can be determined by calculating the offset of all key feature points. ​​​​​​​​​standard deviation of is the covariance of the images , and is a constant to avoid the special case that the denominator is zero; the constant , wherein , , is determined by the bit number of the image, such as 8-bit image, and takes the value of 255. The larger the value is, the more similar the two images are. When calculating the difference between two images at each position, SSIM does not take one pixel from each image at the position, but takes the pixels in a region. SSIM mainly considers three key features of the image: brightness, contrast, and structure. The brightness similarity index is . The brightness is measured by the average gray level. The brightness of the template image is calculated by the average pixel value in the region as wherein is the average brightness of the image corresponding to the pixel in the image region, is the total number of pixels in the image region; the brightness of the image taken based on the preset bit ID is wherein is the average brightness of the image corresponding to the pixel in the image region. The contrast similarity index is . The contrast is measured by the gray standard deviation. The gray standard deviation of the template image is , wherein is the average brightness of the image , is the average brightness of the image . The structure of the image refers to the description of the relative position and scale relationship between the pixels. The structure similarity index is wherein , is a constant.

[0033] In an optional embodiment, the target image quality score is obtained based on the clarity comparison result and the feature registration result, including: determining a first weight value corresponding to the clarity comparison result and a second weight value corresponding to the feature registration result; and performing weighted calculation based on the clarity comparison result, the first weight value, the feature registration result, and the second weight value to obtain the target image quality score.

[0034] Optionally, the target image quality score is derived by combining the sharpness comparison result and the feature registration result. First, the quantification results of sharpness and feature offset are obtained using the SSIM and SURF algorithms, respectively. Then, these two results are weighted and averaged according to pre-defined weights to generate the target image quality score (i.e., the sharpness comparison result and the feature registration result). The choice of weights reflects the relative importance of sharpness and positioning accuracy in the overall evaluation. If the weight of sharpness is higher, the sharpness comparison result has a greater impact on the score; conversely, the feature registration result has a greater impact on the score. By assigning different weight values ​​to the sharpness comparison result and the feature registration result and performing weighted calculations, a comprehensive target image quality score is generated. This method ensures that the scoring system considers the relative importance of the two key indicators when evaluating image quality, thereby improving the accuracy and efficiency of camera calibration decisions.

[0035] Step S106: Determine whether the target image quality score is less than a preset score threshold;

[0036] Optionally, the sharpness comparison and feature registration evaluation of the target image acquired from the power equipment and the corresponding image template will be performed to obtain a comprehensive evaluation score (i.e., image quality score). Threshold determination, It is derived from the combined image quality SSIM value (i.e., sharpness contrast result) and the average offset of key feature points (i.e., feature registration result) according to certain weights. For example, when When the value is >90, the image quality corresponding to the current preset ID is considered good, and no calibration is required. When the value is less than 90, the image corresponding to the current preset ID needs to be calibrated in terms of clarity and offset.

[0037] Step S108: If the target image quality score is less than a preset score threshold, determine the target image positioning offset between the image template and the target image.

[0038] Optionally, if the target image quality score is less than a preset score threshold, it indicates that the image quality captured by the camera is poor and the camera needs to be calibrated. In this case, the target image positioning offset between the image template and the target image can be obtained, and the camera parameters can be calibrated based on the target image positioning offset.

[0039] In one optional embodiment, determining the target image positioning offset between the image template and the target image includes: using an image feature detection algorithm to identify target feature points included in the image template and the target image respectively; matching the target feature points included in the image template and the target image respectively to determine the target image positioning offset.

[0040] Optionally, the key feature points in the image template and the target image are automatically identified and compared using image feature detection algorithms (such as SURF, etc.), to accurately determine the positioning offset of the target image. This process is a crucial step in the camera calibration system, as it quantifies the displacement of the target image relative to the template image, providing specific data for subsequent camera parameter adjustments. Specifically, image feature detection algorithms can detect key points in the image, which can be unique structures in the image (such as corner points, edges, or texture change points), and thus can be stably identified in different images. By matching these key points, the translation, rotation, and scale changes of the target image relative to the template image, i.e., the positioning offset, can be calculated. This feature point matching-based method not only handles changes in lighting, perspective, and scale, but also has good robustness to partial occlusion, distortion, and noise in images. Through the above means, even under the influence of environmental factors, the key feature offset in the image can be accurately detected, thereby providing accurate guidance for the automatic calibration of the camera, improving the efficiency and accuracy of calibration, reducing the need for manual intervention, and ultimately achieving automatic and accurate correction of the camera preset position, ensuring that the patrol image quality of power equipment meets the requirements of remote monitoring and intelligent analysis.

[0041] In an optional embodiment, matching the target feature points respectively included in the image template and the target image to determine the positioning offset of the target image includes: matching the corresponding target feature points of the image template and the target image to obtain a list of matching feature points; calculating the offset of each pair of matching feature points in the horizontal coordinate direction and the vertical coordinate direction in the list of matching feature points; and identifying the maximum offset from all the offsets of the matching feature points as the positioning offset of the target image.

[0042] Optionally, by matching the feature points on the image template and the target image, a list containing all pairs of matching feature points is generated. This step utilizes the robustness and positioning ability of image feature detection algorithms (such as SURF) to ensure that a sufficient number of matching points can be found even in cases of poor image quality. Based on the list of matching feature points, the offset of each pair of matching feature points in the horizontal coordinate (x-coordinate) and vertical coordinate (y-coordinate) direction is calculated, thereby quantifying the relative movement between the target image and the template image, including translation and rotation. From all the calculated offsets, the maximum offset is identified as the positioning offset of the target image. In camera calibration, this method is particularly suitable for handling cases where the preset position offset is large. By determining the maximum offset, the system can prioritize adjusting this offset, ensuring accurate alignment of key feature points, thereby improving the efficiency and accuracy of calibration.

[0043] Optionally, the camera calibration system calculates the value of each image feature key offset of the current preset position based on the image feature registration algorithm, which includes two offset values of x coordinate and y coordinate, and the overall image offset value adopts the maximum offset method, that is, the maximum offset is taken as . The key feature point pixel offset is converted into the offset of PTZ , , wherein the field of view angle , is the image width, is the image height, is the magnification.

[0044] Step S110, based on the target image positioning offset, the camera is calibrated.

[0045] Optionally, when calibrating the camera, it can but is not limited to include parameter calibration of Pan, Tilt, Zoom and Focus parameters (referred to as PTZF parameters). That is, after the camera calibration system determines that the current preset position ID based on the obtained camera shooting picture (i.e. target image) and the image template comparison comprehensive evaluation score is low, the camera calibration process is entered. The camera calibration system can adjust to the corresponding preset position to take a photo and obtain the PTZF parameters of the current preset position.

[0046] Optionally, when the camera calibration system is adjusted according to the target image positioning offset (i.e. the maximum offset), the image feature registration offset score (i.e. the feature registration result ) is used to determine whether the correction is successful. For example, when >95, it is determined that the picture offset corresponding to the current preset position ID is within the acceptable range and does not need to be offset calibrated; when <95, it is determined that the picture corresponding to the current preset position ID is still large in terms of offset. When the offset cannot be corrected to the ideal range, the target detection based on the deep learning algorithm YOLO is continued to be used for feature value registration, which can not only ensure the detection accuracy but also improve the detection speed. Before target detection, due to the possibility that the target detection object (i.e. the power equipment) has been partially or completely offset, the camera magnification is first reduced to the minimum, and then the deep learning algorithm is used to confirm whether the target object is within the field of view. If it is confirmed that the target object is within the field of view (the confidence is greater than the set threshold), the method of converting the key feature point pixel offset into the offset of PTZF parameter is continuously adjusted until the set threshold is reached, the offset calibration is successful, and the adjusted camera PTZF parameter is saved; if the target detection object cannot be identified or has exceeded the field of view and cannot be detected (the confidence is less than the set threshold), the offset calibration fails.

[0047] Optionally, after the camera calibration system successfully performs offset calibration, a sharpness score is calculated based on the Structural Similarity Index (SSIM) image quality algorithm. Evaluation. For example, when When the resolution is >95, the image clarity corresponding to the current preset ID is determined to be within an acceptable range, and no clarity calibration is required; when When the score is less than 95, the image corresponding to the current preset ID is considered to have relatively high sharpness. The camera's sharpness is mainly determined by adjusting the focal length (f) parameter. The sharpness deviation is mainly based on a combination of an appropriate fixed focus range and a semi-automatic zoom mode. An adjustment range is set based on the obtained focal length (f) parameter. Within the adjustment range, combined with semi-automatic zoom, multiple screenshots are taken to obtain the image with the best sharpness. The focal length value with the highest sharpness score is then selected and saved.

[0048] In an optional embodiment, after calibrating the camera parameters based on the target image positioning offset, the method further includes: acquiring a new image of the power equipment captured by the parameter-calibrated camera; performing image quality assessment based on the image template and the new image to obtain a new image quality score; determining whether the new image quality score is less than a preset score threshold; if the new image quality score is less than the preset score threshold, determining a new image positioning offset between the image template and the new image; and continuing to calibrate the parameters of the parameter-calibrated camera based on the new image positioning offset.

[0049] Optionally, after calibrating the camera, images of the power equipment are reacquired to assess the calibration effect in real time and provide a basis for possible subsequent adjustments. Based on the image template, the newly acquired images are quality-assessed to obtain a new image quality score, which provides immediate feedback on the actual image quality after calibration, ensuring that the calibration process does not rely on a single calibration result but can be adjusted according to the actual effect. The new image quality score is compared with a preset scoring threshold. If the new image quality score is lower than the preset threshold, it indicates that the image quality after the initial calibration still does not meet the requirements and further optimization is needed. If the image quality score is substandard, the positioning offset is re-determined based on the new image, thereby more accurately identifying the difference between the current image and the template, providing data support for secondary or multiple calibrations. Based on the new image positioning offset, the camera parameters are recalibrated. This process can be repeated until the image quality score reaches or exceeds the preset threshold, ensuring that the final image meets the needs of inspection and fault early warning.

[0050] In the above manner, through iterative calibration, the continuity of the camera calibration process and the stability of the final image quality can be ensured, even if the image quality is not ideal after the initial adjustment, subsequent optimization can be automatically performed to avoid manual intervention. The above manner can adapt to changes in environmental conditions such as lighting, weather, device status, etc., and maintain the optimal state of image quality through continuous image quality evaluation and adjustment. By accurately calculating the new image positioning offset, the accuracy of the calibration can be ensured, thereby improving the accuracy and reliability of the entire camera calibration system. The automated iterative calibration process can reduce the need for manual adjustment, improve the operation and maintenance efficiency of the power equipment inspection system, reduce labor costs, and also improve the efficiency of the substation online inspection and the accuracy of data collection.

[0051] Through the above steps S102 to S110, the purpose of obtaining image templates and real-time images for comparison and evaluation, automatically determining image quality deviation, and accurately adjusting camera parameters based on deviation information can be achieved, thereby realizing fast and accurate camera calibration, ensuring power equipment inspection image quality, and improving inspection efficiency and accuracy. Technical effects, thereby solving the technical problems of position deviation and insufficient clarity of power equipment images based on camera acquisition in related technologies.

[0052] Based on the above embodiments and optional embodiments, the present application proposes an optional implementation, Figure 2 is a flowchart of an optional substation inspection camera calibration method according to an embodiment of the present application, Figure 3 is an optional system framework diagram according to an embodiment of the present application, which can be applied to a system framework as shown in Figure 3 as shown, the method comprises: Figure 2

[0053] Step S1: The camera calibration system obtains image templates from the substation online intelligent inspection system. The image templates include reference images and coordinate information of labeled target detection objects (such as power equipment). If the online intelligent inspection system already has image templates of power equipment, it can be synchronized to the camera calibration system through online or offline methods. The power equipment can include but is not limited to pointer tables, oil level tables, digital tables, knife gates, split and combined, indicator lights, etc., and all of the above equipment have image templates. If the online intelligent inspection system has no image templates, image templates can be generated by online labeling through the camera calibration system, for example, a large number of substation equipment appearances, etc. without templates.

[0054] ​Step S2: The camera calibration system can indirectly obtain the camera preset position ID and the PTZF parameters corresponding to the preset position ID from the online intelligent patrol system or directly from the camera. P (Pan) corresponds to the rotation movement parameter of the camera in the horizontal direction; T (Tilt) corresponds to the tilt rotation parameter of the camera in the vertical direction; Z (Zoom) corresponds to the zoom parameter of the camera, which realizes the near-far adjustment of the lens field of view; F (Focus) corresponds to the zoom parameter of the camera, which realizes the clear adjustment of the lens focus.

[0055] Step S3: The camera calibration system formulates an automatic calibration scheme plan to realize automatic calibration without human intervention. To avoid conflicts between calibration tasks and intelligent patrol tasks, the task plan comparison and linkage between the calibration system and the intelligent patrol task can be realized. For example, regular correction is realized in the case that the idle period after the patrol task is over and the recognition rate is lower than the threshold.

[0056] Step S4: The camera calibration system captures pictures of power equipment in the substation based on the obtained camera preset position ID. Since there are many cameras in the substation, multiple presets can be set for one camera. To improve efficiency, the strategy for obtaining pictures can be executed in parallel by multiple cameras, and pictures can be captured in the order of preset position ID within a single camera.

[0057] Step S5: Based on the pictures obtained in step S4 and the corresponding image templates obtained in step S1, the clarity comparison and feature registration evaluation are performed, and the comprehensive evaluation score (i.e. image quality score ) threshold value is determined. The comprehensive image quality SSIM value (i.e. clarity comparison result) and the average offset degree of key feature points (i.e. feature registration result) are obtained according to certain weights. For example, when > 90, it is determined that the picture quality corresponding to the current preset position ID is good, and calibration is not needed. When < 90, it is determined that the picture corresponding to the current preset position ID needs to be calibrated in terms of clarity and offset. The feature registration result of the image is evaluated by the registration accuracy and stability of the SURF algorithm which is an accelerated mode and has robust characteristics; the clarity comparison result is evaluated by the SSIM image quality algorithm based on the contrast of the picture.

[0058] Step S6: After the camera calibration system determines that the comprehensive evaluation score of the pictures captured based on the obtained camera preset position ID is low compared with the image template, the camera calibration process is entered. The calibration system adjusts to capture the picture of the corresponding preset position and obtains the PTZF parameters of the current preset position.

[0059] Step S7: The camera calibration system calculates the value of each image feature key offset of the current preset position based on the image feature registration algorithm, which includes x coordinate and y coordinate offset values. The overall image offset value uses the maximum offset method, that is, the maximum offset is taken as . The image feature registration offset score ( ) in step S5 is used to determine the correction success. For example, when >95, it is determined that the picture offset corresponding to the current preset position ID is within the acceptable range and does not need to be offset calibrated; when <95, it is determined that the picture corresponding to the current preset position ID is still large in terms of offset. When the offset cannot be corrected to the ideal range, the target detection based on the deep learning algorithm YOLO is continued to be used for feature value registration, which can ensure the detection accuracy and improve the detection speed. Before target detection, since there is a possibility that the target detection object has been partially or completely offset, the camera magnification is first reduced to the minimum, and then the deep learning algorithm is used to confirm whether the target object is within the field of view. If it is confirmed that the target object is within the field of view (confidence greater than the set threshold), the method of converting the key feature point pixel offset to the PTZ offset is continuously used for adjustment until the set threshold is reached, and the offset calibration is successful. At the same time, the adjusted camera PTZ parameters are saved; if the target detection object cannot be recognized or has exceeded the field of view and cannot be detected (confidence less than the set threshold), the offset calibration fails.

[0060] Step S8: After the camera calibration system successfully completes the offset calibration, the structural similarity index image quality algorithm SSIM based on step S5 is used to evaluate the sharpness score ( ). For example, when >95, it is determined that the picture sharpness corresponding to the current preset position ID is within the acceptable range and does not need to be calibrated for sharpness; when <95, it is determined that the picture corresponding to the current preset position ID is still large in terms of sharpness. The main adjustment of the camera sharpness is the focal length (f) parameter. The deviation of the sharpness is mainly based on the combination of the fixed focus range of the camera and the semi-automatic zoom mode. Based on the obtained focal length (f) parameter, the adjustment range is set. Within the adjustment range, the semi-automatic zoom is combined, and the best sharpness picture is obtained by multiple screenshots. The focal length value with the maximum sharpness score is taken and saved.

[0061] Step S9: After the camera calibration system completes the preset position correction of all cameras according to the task setting, the parameter update is performed according to the set automatic update mode. If it is automatic update, the saved PTZF parameters can be updated to the online intelligent patrol system or each camera; if it is not automatic update, the update can be performed manually.

[0062] After the above nine steps, a camera calibration system based on image feature registration and deep learning visual fusion is formed, so as to realize batch, fast, accurate, efficient and automatic correction of the camera.

[0063] It should be noted that in the related art, the image target offset detection is basically realized by using a scale-invariant feature transform method based on image features. On the one hand, when there is a large amount of calculation, the calculation speed is slow, the resource demand is large, and the like, and the substation scene camera cannot meet the requirements of a large number of cameras and a large number of preset sites. On the other hand, when the light changes and the radiation changes, the detection accuracy is low. In addition, when the preset site has a large displacement deviation, the scale-invariant feature transform method cannot effectively correct the offset, and cannot meet the actual application requirements.

[0064] The feature algorithm SURF provided in the embodiment is based on an acceleration mode and has a robust characteristic, adopts HAAR features and integral images, greatly accelerates the detection speed, maintains the scale-invariant and rotation-invariant characteristics of the SIFT algorithm, and has strong robustness to light changes and radiation changes, and improves the accuracy in the case of environmental influence on the picture. The image quality algorithm SSIM provided in the embodiment adopts a detection mechanism based on a template picture comparison mode. When the SSIM calculates the difference between two images at each position, it takes a region of pixels instead of a single pixel from each of the two images. It is superior to a single picture quality analysis method, such as the SMD algorithm based on a gray scale variance function, the algorithm based on an energy gradient function, and the like; and is superior to the same type of template picture comparison mode, such as the MSE algorithm based on a mean square error function, the PSNR algorithm based on a peak signal-to-noise ratio, and the like. When the MSE and PSNR calculate the pixel difference at each position, the result is only related to the two pixel values at the current position, and is irrelevant to the pixels at any other position. This way of calculating the difference only regards the image as a single isolated pixel point, and ignores some visual features contained in the image content, especially the local structure information of the image. In addition, the feature value registration based on the deep learning algorithm YOLO provided in the embodiment compensates for the case where there is a large offset, or even the case where the target has been offset out of the original preset site field of view, and can also find the target to perform one-step offset correction, and has detection accuracy and speed. In addition, the camera calibration system provided in the embodiment introduces the image feature registration offset score (OF-Score), the definition score (DE-Score), and the comprehensive evaluation score (CE-Score), which can be flexibly configured and optimized according to a large amount of calibration data, and can improve the calibration accuracy.

[0065] A substation inspection camera calibration device is also provided in the embodiment, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" "device" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0066] According to the embodiment of the present application, a device for implementing the above-mentioned substation inspection camera calibration method is also provided, Figure 4 is a structural schematic diagram of a substation inspection camera calibration device according to an embodiment of the present application, as Figure 4 The above-mentioned substation inspection camera calibration device includes an image acquisition module 400, a quality evaluation module 402, a threshold judgment module 404, an offset determination module 406, and a parameter calibration module 408, wherein:

[0067] The image acquisition module 400 is configured to acquire an image template of a power equipment in a substation and a target image of the power equipment acquired by a camera based on a current preset position, wherein the image template is an image of a known accurate position and a clear degree acquired for the power equipment;

[0068] The quality evaluation module 402 is configured to perform image quality evaluation based on the image template and the target image to obtain a target image quality score;

[0069] The threshold judgment module 404 is configured to judge whether the target image quality score is less than a preset score threshold;

[0070] The offset determination module 406 is configured to determine a target image positioning offset between the image template and the target image in a case where the target image quality score is less than the preset score threshold;

[0071] The parameter calibration module 408 is configured to perform parameter calibration on the camera based on the target image positioning offset.

[0072] It should be noted that the above-mentioned modules can be implemented by software or hardware, for example, for the latter, the above-mentioned modules can be located in the same processor, or the above-mentioned modules can be located in different processors in any combination.

[0073] It should be noted that the image acquisition module 400, the quality evaluation module 402, the threshold judgment module 404, the offset determination module 406, and the parameter calibration module 408 correspond to steps S102 to S108 in the embodiment, and the above modules have the same instances and application scenarios as the corresponding steps, but are not limited to the above disclosed contents. It should be noted that the above modules can run in a computer terminal as part of the device.

[0074] It should be noted that the optional or preferred embodiments of the present embodiment can refer to the related description in the embodiment, which will not be repeated here.

[0075] The camera calibration device for substation inspection described above can further include a processor and a memory, and the image acquisition module 400, the quality evaluation module 402, the threshold judgment module 404, the offset determination module 406, and the parameter calibration module 408 are stored in the memory as program modules, and the processor executes the above program modules stored in the memory to realize the corresponding functions.

[0076] The processor includes a core, and the core retrieves the corresponding program module from the memory. The above core can be set to one or more. The memory can include a non-persistent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0077] According to the embodiments of the present application, an embodiment of a non-volatile storage medium is also provided. Optionally, in the present embodiment, the non-volatile storage medium includes a stored program, wherein the program controls the device where the non-volatile storage medium is located to execute any of the above camera calibration methods for substation inspection when the program is running.

[0078] Optionally, in the present embodiment, the non-volatile storage medium can be located in any one of a group of computer terminals in a computer network, or in any one of a group of mobile terminals, and the non-volatile storage medium includes a stored program.

[0079] Optionally, the device where the nonvolatile storage medium is located performs the following functions under the control of the program running time: obtaining an image template of a power equipment in a transformer substation, and obtaining a target image of the power equipment collected by a camera at a current preset position, wherein the image template is an image of a known accurate position and a clear degree collected for the power equipment; performing image quality evaluation based on the image template and the target image to obtain a target image quality score; determining whether the target image quality score is less than a preset score threshold; in the case that the target image quality score is less than the preset score threshold, determining a target image positioning offset between the image template and the target image; and performing parameter calibration on the camera based on the target image positioning offset.

[0080] According to the embodiments of the present application, an embodiment of a processor is further provided. Optionally, in the embodiment, the processor is used to run a program, and the program performs any of the above camera calibration methods for transformer substation inspection when running.

[0081] According to the embodiments of the present application, an embodiment of a computer program product is further provided, which is adapted to perform a program that initializes the steps of any of the above camera calibration methods for transformer substation inspection when executed on a data processing device.

[0082] Optionally, the computer program product is adapted to perform a program that initializes the steps of the following method when executed on a data processing device: obtaining an image template of a power equipment in a transformer substation, and obtaining a target image of the power equipment collected by a camera at a current preset position, wherein the image template is an image of a known accurate position and a clear degree collected for the power equipment; performing image quality evaluation based on the image template and the target image to obtain a target image quality score; determining whether the target image quality score is less than a preset score threshold; in the case that the target image quality score is less than the preset score threshold, determining a target image positioning offset between the image template and the target image; and performing parameter calibration on the camera based on the target image positioning offset.

[0083] An electronic device is provided in the embodiments of the present application, which comprises a processor, a memory, and a program stored in the memory and capable of running on the processor. The processor implements the following steps when executing the program: obtaining an image template of a power equipment in a transformer substation, and obtaining a target image of the power equipment collected by a camera at a current preset position, wherein the image template is an image of a known accurate position and a clear degree collected for the power equipment; performing image quality evaluation based on the image template and the target image to obtain a target image quality score; determining whether the target image quality score is less than a preset score threshold; in the case that the target image quality score is less than the preset score threshold, determining a target image positioning offset between the image template and the target image; and performing parameter calibration on the camera based on the target image positioning offset.

[0084] The sequence of the above embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments.

[0085] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0086] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the above-mentioned modules can be a logical function division, and actual implementation can have another division way, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between modules or modules, which can be electrical or other forms.

[0087] The above-mentioned modules described as separate components can be or can not be physically separated, and the components shown as modules can be or can not be physical modules, that is, they can be located in one place, or can be distributed to multiple modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0088] In addition, each functional module in each embodiment of the present application can be integrated in a processing module, or each module can exist physically, or two or more modules can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module.

[0089] The above-mentioned integrated module, if realized in the form of software functional module and sold or used as an independent product, can be stored in a computer readable non-volatile storage medium. Based on this understanding, the technical scheme of the present application or the part of the present application which is essential or contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of software product, which is stored in a non-volatile storage medium, including a plurality of instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the embodiments of the present application. The above-mentioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and various program code storage media.

[0090] The above merely is the preferred embodiment of the present application, it should be pointed out that, for ordinary skilled in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method for calibrating cameras during substation inspection, characterized in that, include: Obtain image templates of power equipment in a substation, and target images of the power equipment captured by a camera at a current preset position, wherein the image templates are images of known accurate location and clarity captured for the power equipment; Image quality assessment based on the image template and the target image to obtain a target image quality score includes: comparing the sharpness of the image template and the target image to obtain a sharpness comparison result; performing feature registration on the image template and the target image to obtain a feature registration result, wherein the feature registration result is used to indicate the offset of feature points included in the target image relative to the image template; determining a first weight value corresponding to the sharpness comparison result and a second weight value corresponding to the feature registration result; and performing a weighted calculation based on the sharpness comparison result, the first weight value, the feature registration result, and the second weight value to obtain the target image quality score. Determine whether the target image quality score is less than a preset score threshold; If the target image quality score is less than the preset score threshold, determine the target image positioning offset between the image template and the target image; Based on the target image positioning offset, the camera parameters are calibrated; The sharpness comparison results are obtained in the following way: ,in, and The image templates are respectively and the target image average brightness and The image templates are respectively and the standard deviation of the brightness of the target image, The image template And the brightness covariance of the target image, and It is a constant used to avoid the special case where the denominator is zero; setting a constant , ,in , , Determined by the number of bits in the image.

2. The method according to claim 1, characterized in that, The step of comparing the sharpness of the image template and the target image to obtain a sharpness comparison result includes: using a structural similarity index measurement method to compare the sharpness of the image template and the target image to obtain the sharpness comparison result; The step of performing feature registration on the image template and the target image to obtain the feature registration result includes: performing feature registration on the image template and the target image using an accelerated robust feature method to obtain the feature registration result.

3. The method according to claim 1, characterized in that, Determining the target image positioning offset between the image template and the target image includes: An image feature detection algorithm is used to identify the target feature points included in the image template and the target image, respectively. The target feature points included in the image template and the target image are matched to determine the positioning offset of the target image.

4. The method according to claim 3, characterized in that, The step of matching the target feature points included in the image template and the target image respectively to determine the target image positioning offset includes: The corresponding target feature points of the image template and the target image are matched to obtain a list of matched feature points; In the list of matching feature points, calculate the offset of each pair of matching feature points in the horizontal and vertical coordinate directions; From the offsets of all matching feature points, the largest offset is identified as the target image positioning offset.

5. The method according to any one of claims 1 to 4, characterized in that, After calibrating the camera parameters based on the target image positioning offset, the method further includes: Acquire new images of the power equipment based on the camera after parameter calibration; Based on the image template and the new image, an image quality assessment is performed to obtain a new image quality score; Determine whether the new image quality score is less than the preset score threshold; If the new image quality score is less than the preset score threshold, a new image positioning offset is determined between the image template and the new image; Based on the new image positioning offset, continue to calibrate the parameters of the camera after parameter calibration.

6. A camera calibration device for substation inspection, characterized in that, include: The image acquisition module is used to acquire image templates of power equipment in the substation, as well as target images of the power equipment acquired by a camera at a current preset position, wherein the image templates are images of known accurate location and clarity acquired for the power equipment; A quality assessment module is used to perform image quality assessment based on the image template and the target image to obtain a target image quality score. The module includes: comparing the sharpness of the image template and the target image to obtain a sharpness comparison result; performing feature registration on the image template and the target image to obtain a feature registration result, wherein the feature registration result indicates the offset of feature points included in the target image relative to the image template; determining a first weight value corresponding to the sharpness comparison result and a second weight value corresponding to the feature registration result; and performing a weighted calculation based on the sharpness comparison result, the first weight value, the feature registration result, and the second weight value to obtain the target image quality score. The threshold determination module is used to determine whether the quality score of the target image is less than a preset score threshold; The offset determination module is used to determine the target image positioning offset between the image template and the target image when the target image quality score is less than the preset score threshold. The parameter calibration module is used to calibrate the parameters of the camera based on the target image positioning offset. The sharpness comparison results are obtained in the following way: ,in, and The image templates are respectively and the target image average brightness and The image templates are respectively and the standard deviation of the brightness of the target image, The image template And the brightness covariance of the target image, and It is a constant used to avoid the special case where the denominator is zero; setting a constant , ,in , , Determined by the number of bits in the image.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores multiple instructions, which are adapted to be loaded by a processor and executed by the camera calibration method for substation inspection as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the camera calibration method for substation inspection as described in any one of claims 1 to 5.

Images