A tower monitoring method and terminal based on digital image processing

Abstract

The application discloses a tower monitoring method and terminal based on digital image processing, and after determining a panoramic image of a monitored tower, when determining a local image, firstly, camera parameters when a global camera is shot are determined according to a target local object subgraph selected by framing, the camera parameters are sent to a local camera to shoot a local image in a direction approximately same as that of the global camera, then, searching is performed in the local image based on an outline of the target local object, the position of the target local object in the local image is determined, finally, the local camera shoots according to the determined position of the target local object, and a target image meeting requirements is obtained by comparing the outline of the target local object, through positioning the target local object in the panoramic image and the local image in sequence and comparing the outline of the target local object, further checking of details in the panoramic image is realized quickly and accurately, and global and local monitoring of towers can be ensured.

Description

Technical Field

[0001] This invention relates to the field of pole monitoring, and in particular to a pole monitoring method and terminal based on digital image processing. Background Technology

[0002] Transmission lines in a power system include underground transmission lines and overhead transmission lines. Overhead transmission lines have always been the primary transmission method used in power systems because they are cheaper to construct, have a shorter construction period, and are easier to maintain and repair compared to underground transmission lines.

[0003] Towers are the main supporting structures in overhead transmission lines, used to support the transmission lines. To ensure the safety and reliability of both the towers and the transmission lines, various types of electrical fittings are used, such as suspension clamps, tension clamps, connecting fittings, splicing fittings, protective fittings, and guy wire fittings. Therefore, the integrity of the towers, fittings, and various components in the transmission lines (such as insulators, cables, and high-voltage bushings) directly affects the normal operation of the entire overhead transmission line system.

[0004] However, since overhead transmission lines are exposed to the atmosphere, they are directly affected by various external conditions, such as weather conditions, industrial pollution, and accidents. Therefore, the towers, their fittings, and the supporting transmission lines inevitably experience wear and tear during actual use, requiring regular inspections by staff for timely maintenance.

[0005] Traditional inspection methods typically involve manual inspections. This approach is labor-intensive, costly, and poses risks to personnel safety. Furthermore, manual inspections have a limited scope and scope, lacking comprehensiveness. With advancements in information technology, automated inspections, such as robot and drone inspections, are gradually replacing manual inspections. These automated inspections eliminate the need for human intervention, can operate in various environments, and offer comprehensive and complete inspection capabilities. However, whether manual, robotic, or drone inspections, they can only be initiated when personnel need to monitor overhead power lines; real-time monitoring is not possible.

[0006] To achieve real-time monitoring, current methods typically involve installing cameras on power poles to capture real-time footage. Monitoring personnel can then retrieve this footage from a backend system to monitor the poles in real time. This allows them to inspect the condition of the poles, nearby hardware, and various components within the transmission lines. Since monitoring requires both a comprehensive view of the poles, transmission lines, and transformers, as well as detailed inspections of hardware, insulators, and high-voltage bushings, real-time monitoring necessitates the ability to assess both the overall picture and the details. However, current pole monitoring methods do not adequately balance both comprehensive and detailed monitoring. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a pole monitoring method and terminal based on digital image processing, which can quickly and accurately view details while performing a global view of the pole.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A method for monitoring power poles based on digital image processing, comprising the following steps:

[0010] Receive panoramic images of the tower to be monitored from the panoramic camera, and send the panoramic images to the client.

[0011] Based on the panoramic image, determine the target local object sub-image located by the client on the panoramic image;

[0012] The contour features of the target local object and the camera parameters when the panoramic camera captures the target local object are determined based on the target local object sub-image.

[0013] The camera parameters are sent to a local camera, and a local image captured by the local camera based on the camera parameters is received.

[0014] Based on the contour features, a search is performed in the local image to locate the position of the target local object in the local image;

[0015] The location and contour features of the target local object are sent to the local camera, and the target image corresponding to the target local object, which meets the preset size and preset resolution, is captured by the local camera based on the location and contour features.

[0016] The target image is sent to the client, and the target image is displayed on the client while the panoramic image is displayed.

[0017] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0018] A pole monitoring terminal based on digital image processing includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps in the aforementioned pole monitoring method based on digital image processing.

[0019] The beneficial effects of this invention are as follows: In tower monitoring, after determining the panoramic image of the tower to be monitored, the user can locate the tower within the panoramic image and select the local image that requires further detailed display. When determining the local image, the camera parameters for the global camera are first determined based on the target local object sub-image selected by the user. The camera parameters are then sent to the local camera to capture a local image with the same approximate direction and orientation as the global camera, achieving coarse positioning. Next, the local camera searches within the local image based on the outline of the target local object to determine its position, achieving fine positioning. Finally, the local camera captures an image based on the determined fine positioning of the target local object and compares it with the outline of the target local object to obtain the desired result. The target image is obtained by locating the target local object in the panoramic image and the local image in turn, and comparing it with the outline of the target local object to ensure the accuracy of the acquired local object. This simplifies the tedious and complicated parameter conversion process between the two cameras when determining the local image in the panoramic image, and also avoids the inaccuracy of the acquired local object caused by the deviation between the two cameras in actual use when directly converting between the two cameras. This enables a fast and accurate further inspection of the details in the panoramic image. Therefore, it can ensure both global and local considerations in tower monitoring. In tower monitoring, it can not only perform global inspection of towers, channels, transformers, etc., but also perform local inspection of hardware, insulators, high-voltage bushings, etc. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of a pole monitoring method based on digital image processing according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of a tower monitoring terminal based on digital image processing according to an embodiment of the present invention;

[0022] Figure 3 This invention provides a display method for simultaneously presenting a global view and a detailed view in tower monitoring.

[0023] Figure 4 This is another display method for simultaneously presenting a global view and a detailed view in the tower monitoring of this invention;

[0024] Figure 5 This is a schematic diagram of a tower monitoring system based on digital image processing according to an embodiment of the present invention. Detailed Implementation

[0025] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0026] The pole monitoring method and terminal based on digital image processing described above are applicable to any scenario that requires monitoring of poles and their surrounding environment. The following is a detailed description of the specific implementation methods.

[0027] Example 1

[0028] Please refer to Figure 1 A method for monitoring power poles based on digital image processing, described in this embodiment from the perspective of a power pole monitoring terminal based on digital image processing, specifically includes the following steps:

[0029] S1. Receive the panoramic image of the tower to be monitored sent by the panoramic camera, and send the panoramic image to the client;

[0030] The panoramic camera is mounted on a pan-tilt unit, which supports horizontal rotation of -180° to +180° and vertical rotation of -90° to +90°. This allows for all-around monitoring of the tower by rotating the panoramic camera in both horizontal and vertical directions.

[0031] The panoramic image is stitched together from panoramic sub-images captured by the panoramic camera at preset angles, and each panoramic sub-image is bound to corresponding camera parameters.

[0032] The camera parameters include the horizontal rotation angle, vertical rotation angle, focal length, and convergence position corresponding to the panoramic sub-image captured by the panoramic camera and bound to the camera parameters.

[0033] That is, after rotating to a certain angle, once the panoramic camera captures the corresponding panoramic sub-image, the camera parameters at this time will be stored, and the panoramic sub-image will be bound and stored with the corresponding camera parameters. When sending the panoramic image to the tower monitoring terminal based on digital image processing, the corresponding camera parameters will also be sent.

[0034] After receiving the panoramic sub-images and their corresponding camera parameters sent by the panoramic camera, the tower monitoring terminal based on digital image processing can first determine the approximate stitching position of each panoramic sub-image according to the rotation angle in the camera parameters, perform preliminary stitching of each panoramic sub-image, and then perform feature point extraction, feature point similarity calculation and fusion on the overlapping parts of the adjacent panoramic sub-images after preliminary stitching, and finally generate a panoramic image that meets the requirements. The terminal also stores the positional relationship between each panoramic sub-image stitched into the panoramic image and their respective camera parameters, that is, establishes the binding relationship between each panoramic sub-image and its corresponding camera parameters.

[0035] S2. Determine the target local object sub-image located by the client on the panoramic image based on the panoramic image;

[0036] After receiving the panoramic image of the tower to be monitored sent by the tower monitoring terminal based on digital image processing, the client displays it on the screen for the user to view. The user can use the mouse to select a box in the panoramic image to determine the target local object to be magnified locally, and send the coordinate range corresponding to the selected target local object to the tower monitoring terminal based on digital image processing.

[0037] S3. Determine the contour features of the target local object and the camera parameters when the panoramic camera captures the target local object based on the target local object sub-image;

[0038] After receiving the coordinate range of the target local object sent by the client, the tower monitoring terminal based on digital image processing extracts the corresponding target local object sub-image from the panoramic image according to the coordinate range.

[0039] Given that this involves the field of pole and tower monitoring, and the main focus of pole and tower monitoring is on poles and the hardware and transmission lines installed near them, and that the objects of interest typically have rich contour features, in one optional implementation, when searching and matching target local objects, the contour features of the target local objects can be used to locate them quickly and accurately. Furthermore, hardware and transmission lines are usually erected on poles tens of meters above the ground; therefore, when extracting the contour features of the target local objects, the foreground and background images of the target local object sub-image can be separated first, then the foreground image can be sharpened, and finally the contour features of the sharpened foreground image can be extracted to determine the contour features of the target local object.

[0040] When determining the camera parameters when the panoramic camera captures the target local object, the target panoramic sub-image to which the target local object sub-image belongs in the panoramic image is first determined according to the coordinate range and the positional relationship between the various panoramic sub-images that are stitched together to form the panoramic image.

[0041] Next, the target camera parameters bound to the target panoramic sub-image are determined based on the correspondence between the target panoramic sub-image, the stored panoramic sub-images stitched together to form the panoramic image, and their respective camera parameters.

[0042] Finally, the camera parameters for capturing the target local object by the panoramic camera are determined based on the target camera parameters.

[0043] S4. Send the camera parameters to the local camera and receive the local image captured by the local camera according to the camera parameters;

[0044] The local camera, like the global camera, is also mounted on the gimbal and can support horizontal rotation of -180° to +180° and vertical rotation of -90° to +90°.

[0045] After receiving the camera parameters, the local camera can adjust its own camera parameters to match the received camera parameters. For example, if the received camera parameters are horizontal rotation angle w1, vertical rotation angle w2, focal length F, and convergence position L, the local camera can directly adjust its horizontal rotation angle w01 to w1, vertical rotation angle w02 to w2, focal length F0 to F, and convergence position L0 to L. In this way, the local image captured by the local camera is roughly the same as the target panoramic sub-image captured by the global camera. The main difference is in the size of the image. To map the local image and the target global sub-image to the same coordinate system, it is only necessary to perform simple scaling based on the camera parameters of the local camera and the panoramic camera, which greatly reduces the cumbersome and complex calculations required for direct conversion between the two cameras.

[0046] S5. Based on the contour features, search the local image to locate the position of the target local object in the local image;

[0047] In this embodiment, the local image is mapped to the same coordinate system as the target global sub-image, which facilitates the search in the mapped local image based on contour features;

[0048] S6. Send the position and contour features of the target local object to the local camera, and receive the target image corresponding to the target local object that meets the preset size and preset resolution, captured by the local camera according to the position and contour features.

[0049] The global image can be formed by stitching together various panoramic sub-images obtained by adjusting the angle of the panoramic camera, or it can be formed by setting the panoramic camera as a wide-angle camera. In this case, the image captured by the wide-angle camera in one direction will be an image with a large angle range. The image captured in that direction can be sent directly to the client as a panoramic image. If the user finds the target local object after viewing the panoramic image, they can directly select the target local object in the panoramic image. If not, the adjusted angle can be sent to the wide-angle camera by the tower monitoring terminal based on digital image processing, so that it can change its direction and take pictures again until the target local object is found in the received panoramic image.

[0050] Because the angle range captured by a wide-angle camera is greater than that of a local camera, the outline of the target local object in the image captured by the local camera based on the determined position may not be complete. Therefore, after capturing an initial image that meets the preset size and preset resolution based on the position, it is determined whether the initial image contains the complete outline features based on the outline features. If yes, the initial image is determined as the target image corresponding to the target local object. If not, the missing part of the outline features in the initial image is determined based on the outline features. The rotation angle, focal length, and convergence position of the local camera are adjusted based on the missing part to obtain the initial image. The process is then repeated until the initial image meets the preset size and preset resolution and contains the complete outline features. Therefore, after determining the position of the target local object, the outline features are further sent to the local camera to verify whether the local camera has indeed captured the required complete target local object.

[0051] Another possible scenario is that the contour feature cannot be found in the local image in step S5. In this case, the angle range not covered by the local image can be determined based on the comparison between the target global sub-image and the local image. Then, the rotation angle of the local camera is adjusted accordingly to obtain a new local image. The contour feature is then searched in the newly determined local image to determine its corresponding position. This adjustment can be repeated until the position corresponding to the contour feature is located in the newly captured local image.

[0052] S7. Send the target image to the client, and display the target image on the client while displaying the panoramic image;

[0053] When displaying a global image or a target image on the client, it can be done as follows: Figure 3 As shown, the global view and the detail view are displayed with the same image size. The content selected by the rectangle in the global view corresponds to the content in the detail view; it can also be displayed as follows: Figure 4 As shown, it is displayed in a picture-in-picture format, that is, the main screen displays the detail screen, and the upper right corner of the detail screen displays the global screen containing the detail screen. The content corresponding to the detail screen is selected by the rectangle. By double-clicking the global screen, the main screen can display the global screen. In other words, double-clicking can easily switch between the global screen and the detail screen.

[0054] Example 2

[0055] This embodiment addresses how to determine the corresponding camera parameters when a panoramic image is stitched together from multiple panoramic sub-images, and the target local object sub-image happens to belong to one of these multiple panoramic sub-images. Specifically:

[0056] The step of determining the camera parameters for capturing the target local object using the panoramic camera based on the target camera parameters includes:

[0057] Determine the number of target camera parameters. If the number is multiple, determine the area of ​​each sub-region occupied by the target local object sub-image in the multiple target panoramic sub-images bound to the determined multiple target camera parameters.

[0058] For example, the target local object sub-image is composed of four sub-regions Sub-S1, Sub-S2, Sub-S3 and Sub-S4, and Sub-S1, Sub-S2, Sub-S3 and Sub-S4 belong to four target global sub-images Sub-P1, Sub-P2, Sub-P3 and Sub-P4 respectively. Then, the areas S1, S2, S3 and S4 of the four sub-regions are determined in turn.

[0059] After determining the area of ​​each sub-region of the target local object sub-image, there are two ways to determine the camera parameters when the panoramic camera captures the target local object:

[0060] In one alternative implementation, the camera parameters when the panoramic camera captures the target local object are determined based on the ratio of the areas of the respective sub-regions and the parameters of the plurality of target cameras.

[0061] For example, if the area ratio of each sub-region is a%:b%:c%:d%, and a+b+c+d=100, and their corresponding camera parameters are par1, par2, par3 and par4, then the camera parameters when the panoramic camera captures the target local object can be determined as tar-par=a%*par1+b%*par2+c%*par3+d%*par4.

[0062] In another optional implementation, the camera parameters for the panoramic camera to capture the target local object are determined based on the target camera parameters bound to the target panoramic sub-image corresponding to the largest sub-region.

[0063] For example, assuming that S3 has the largest area, the camera parameters corresponding to the target global sub-image where its corresponding sub-region is located can be directly determined as the final camera parameters, thereby realizing the control of the local camera.

[0064] Since only the camera parameters corresponding to the target global sub-image containing the largest sub-region are taken as the final camera parameters, it is possible that after the local camera adjusts and captures the image according to the final camera parameters, the captured local image may not contain the complete target local object. Therefore, further:

[0065] When capturing a target image corresponding to the target local object that meets the preset size and preset clarity based on the location and the contour features, an initial image that meets the preset size and preset clarity is captured based on the location. The initial image is then used to determine whether it contains the complete contour features. If it does, the initial image is identified as the target image corresponding to the target local object. If not, the missing portion of the contour features in the initial image is determined based on the contour features. The rotation angle, focal length, and focus position of the local camera are adjusted based on the missing portion to obtain the initial image. The process then returns to executing the step of determining whether the initial image contains the complete contour features based on the contour features until the initial image meets the preset size and preset clarity and contains the complete contour features.

[0066] This embodiment takes into account that when the target local object selected by the user is stitched together from the corresponding sub-regions of multiple panoramic sub-images, the camera parameters determined at the end need to be adjusted according to each sub-region to ensure that the local image captured by the local camera contains the target local object. If the target local object is not complete, the image captured by the local camera is further adjusted by contour features until the contour features of the complete target local object are contained, thereby further ensuring the accuracy of the obtained local image.

[0067] Example 3

[0068] This embodiment further specifies that after each target image is determined, the contour features and camera parameters corresponding to the determined target image can be stored. Specifically:

[0069] After receiving the target image, the process further includes the following steps:

[0070] The contour features of the target local objects contained in the target image are added to the candidate contour feature set, the camera parameters corresponding to the local camera capturing the target image are saved, and a mapping relationship is established between the contour features of the target local objects contained in the target image and the camera parameters corresponding to the local camera capturing the target image.

[0071] In the field of pole monitoring, the objects of interest in the captured global images are usually relatively stable, with minimal positional changes. The objects that inspectors need to check are also relatively fixed, and they need to be checked periodically. Therefore, preserving the checked local objects facilitates quick and easy subsequent review. Accordingly:

[0072] After determining the contour features of the target local object based on the target local object sub-image, it is determined whether there is a target contour feature in the candidate contour feature set that matches the contour features of the target local object. If so, the camera parameters corresponding to the target contour feature and the contour features of the target local object are directly sent to the local camera, so that the local camera directly captures a target image corresponding to the target local object that meets the preset size and preset clarity based on the camera parameters corresponding to the target contour feature and the contour features of the target local object.

[0073] In determining whether a target contour feature matching the contour feature of the target local object exists in the candidate contour feature set, a similarity threshold can be set. When a target contour feature with a similarity greater than the similarity threshold exists in the candidate contour feature set, it indicates that a target contour feature exists. Since the matching target contour feature is determined based on the similarity threshold, when setting the local camera based on the camera parameters corresponding to the target contour feature, after obtaining the initial image captured by the local camera, the contour feature of the target local object is further used for verification. If the initial contour feature obtained through the initial image deviates from the contour feature of the target local object, the camera parameters are fine-tuned until the initial contour feature obtained through the initial image matches the contour feature of the target local object, thereby obtaining the target image.

[0074] If not, then the camera parameters when the panoramic camera captures the target local object are determined based on the target local object sub-image, and the camera parameters when the panoramic camera captures the target local object are sent to the local camera. That is, if there are no contour features similar to the target local object in the candidate contour feature set, then steps S3-S7 are executed to determine the target image.

[0075] This embodiment adaptively establishes a candidate contour feature set during the monitoring process based on the characteristics of pole images captured in the field of pole monitoring. This facilitates the rapid determination of local camera parameters when subsequent inspection personnel select similar target local objects, further improving the speed of presenting local images.

[0076] Example 4

[0077] This embodiment further specifies how to handle situations where the area selected by the client for local magnification is too large, making it impossible to clearly and completely present the target object using only a local camera. Specifically:

[0078] The local cameras include multiple cameras;

[0079] If the target image cannot reach the preset resolution during the process of capturing the target image by the local camera, then the target image is captured as an image whose size meets the preset size and whose resolution is closest to the preset resolution.

[0080] After receiving the target image sent by the local camera, the method further includes the following steps:

[0081] Determine whether the clarity of the target image meets the preset requirements. If not, decompose the contour features of the target local object into multiple contour sub-features, determine the position of each contour sub-feature in the local image, and determine the corresponding local camera for each contour sub-feature.

[0082] Each of the contour sub-features and its corresponding position in the local image is sent to its corresponding local camera, so that its corresponding local camera captures a target sub-image of the target local object that meets the preset size and preset clarity according to the contour sub-features and their positions.

[0083] Receive target sub-images sent by each local camera, stitch together and scale the target sub-images to obtain a target image corresponding to the target local object that meets the preset size and preset resolution.

[0084] When the target local object range is too large, this embodiment further decomposes it. Multiple local cameras locate and focus on the decomposed contour sub-features to obtain multiple target sub-images. The obtained target sub-images are then stitched together to form the required target image. This ensures the clarity of the acquired target image, further improves the robustness of local image display, is applicable to various situations, has high flexibility, and ensures the high quality of the displayed local image.

[0085] In an optional implementation, after a local camera completes a capture, it is not reset, but its current state and parameter settings are maintained. When only one local camera is needed for local capture, but there are multiple local cameras, the required target camera parameters can be determined, the target camera parameters can be compared with the current camera parameters of each local camera, and the local camera whose current camera parameters are closest to the target camera parameters can be selected to perform the local capture.

[0086] In another optional implementation, a control can be set on the client to allow inspectors to enable the display of multiple target local objects. When the inspector enables the control and sets the number of objects to be displayed simultaneously, the local camera that matches each of the multiple target local objects can be selected from multiple local cameras according to the camera parameters corresponding to each of the multiple target local objects selected by the inspector, and the target images captured by each local camera can be sent to the client for simultaneous display.

[0087] Example 5

[0088] Since both the panoramic and partial cameras mentioned above are mounted on power poles, and the power lines and various hardware on these poles may obstruct the view, blind spots may exist. This embodiment further optimizes the shooting to address the issue of blind spots, specifically:

[0089] The process includes the following steps before receiving the panoramic image of the tower to be monitored from the panoramic camera:

[0090] The panoramic camera is rotated within a horizontal range of -180° to +180° and a vertical range of -90° to +90° to determine whether there is a shooting blind spot. If so, a blind spot camera is installed at the position corresponding to the shooting blind spot.

[0091] It also includes the following steps:

[0092] The system receives a blind spot image corresponding to the blind spot area sent by the blind spot camera, sends the blind spot image to the client, and displays the blind spot image on the client while displaying the panoramic image and the target image.

[0093] In another alternative implementation, since panoramic images, partial images, and blind spot images need to be transmitted, if only 4G technology is used for image transmission, the image quality will be poor and unstable due to insufficient uplink bandwidth and large latency. Therefore, in order to improve the image transmission quality, the images captured by the panoramic camera and the partial camera can be received and sent based on 5G technology. 5G technology ensures high-speed and low-latency image transmission, improves image quality, and truly achieves real-time monitoring.

[0094] Example 6

[0095] Please refer to Figure 2 A pole monitoring terminal based on digital image processing includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements each step of the pole monitoring method based on digital image processing described in any one of Embodiments 1 to 5 above.

[0096] The aforementioned digital image processing-based pole monitoring terminal can be considered the control terminal in the pole monitoring process. It handles all relevant steps in the monitoring process and can interact with panoramic cameras, partial cameras, and the client. The digital image processing-based pole monitoring system, composed of these components, is as follows: Figure 5 As shown.

[0097] In summary, the present invention provides a pole monitoring method and terminal based on digital image processing. In pole monitoring, after determining the panoramic image of the monitored pole, inspection personnel can locate the pole within the panoramic image and select local images requiring further detail display. When determining the local images, the camera parameters for the global camera are first determined based on the target local object sub-image selected by the user. These camera parameters are then sent to the local camera to capture a local image with roughly the same direction and orientation as the global camera, achieving coarse positioning. Next, based on the outline of the target local object, a search is performed within the local image to determine its position, achieving fine positioning. Finally, the local camera captures an image based on the determined fine positioning of the target local object and compares it with the target local object's outline to obtain a target image that meets the requirements. By sequentially locating the target local object in both the panoramic and local images and comparing it with the target local object's outline, the accuracy of the acquired local object is ensured, simplifying the process. To eliminate the tedious and complex parameter conversion process between two cameras when determining local images in a panoramic image, and to avoid the inaccuracy of the acquired local objects caused by the deviation between the two cameras in actual use when directly converting between them, this method comprehensively considers various situations that may occur during pole monitoring and takes corresponding solutions. For example, it addresses situations where the contour features of the target local object cannot be matched in the local image, or the matched contour features are incomplete, or the final target image has insufficient clarity. It is extremely robust. In addition, it adaptively adds a candidate contour feature set for the characteristics of pole monitoring, which facilitates the direct application of camera parameters corresponding to similar contour features in later stages. This enables rapid and accurate further examination of details in the panoramic image. Therefore, it can ensure both global and local coverage in pole monitoring. In pole monitoring, it can perform global viewing of poles, channels, transformers, etc., as well as local viewing of hardware, insulators, high-voltage bushings, etc.

[0098] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A tower monitoring method based on digital image processing, characterized in that, Including the following steps: Receive panoramic images of the tower to be monitored sent by the panoramic camera, and send the panoramic images to the client. The panoramic images are stitched together from panoramic sub-images taken by the panoramic camera at preset angles, and each panoramic sub-image is bound to the corresponding camera parameters. Based on the panoramic image, determine the target local object sub-image located by the client on the panoramic image; The contour features of the target local object and the camera parameters when the panoramic camera captures the target local object are determined based on the target local object sub-image. The camera parameters include the horizontal rotation angle, vertical rotation angle, focal length, and convergence position corresponding to the panoramic sub-image bound to the camera parameters captured by the panoramic camera. The camera parameters are sent to a local camera, and a local image captured by the local camera based on the camera parameters is received. Based on the contour features, a search is performed in the local image to locate the position of the target local object in the local image; The location and contour features of the target local object are sent to the local camera, and the target image corresponding to the target local object, which meets the preset size and preset resolution, is captured by the local camera based on the location and contour features. The target image is sent to the client, and the target image is displayed on the client simultaneously with the panoramic image. After receiving the target image, the process further includes the following steps: The contour features of the target local objects contained in the target image are added to the candidate contour feature set, the camera parameters corresponding to the local camera when capturing the target image are saved, and a mapping relationship is established between the contour features of the target local objects contained in the target image and the camera parameters corresponding to the local camera when capturing the target image. After determining the contour features of the target local object based on the target local object sub-image, it is determined whether there is a target contour feature in the candidate contour feature set that matches the contour features of the target local object. If so, the camera parameters corresponding to the target contour feature and the contour features of the target local object are directly sent to the local camera, so that the local camera directly captures a target image corresponding to the target local object that meets the preset size and preset clarity based on the camera parameters corresponding to the target contour feature and the contour features of the target local object. If not, then the camera parameters for capturing the target local object by the panoramic camera are determined based on the target local object subgraph, and the camera parameters for capturing the target local object by the panoramic camera are sent to the local camera.

2. The tower monitoring method based on digital image processing according to claim 1, characterized in that, Both the panoramic camera and the partial camera are mounted on a gimbal, which supports horizontal rotation of -180° to +180° and vertical rotation of -90° to +90°. The step of determining the camera parameters when the panoramic camera captures the target local object based on the target local object sub-image includes: Determine the target panoramic sub-image to which the target local object sub-image belongs in the panoramic image; Determine the target camera parameters bound to the target panoramic sub-image based on the target panoramic sub-image; The camera parameters for capturing the target local object are determined based on the target camera parameters.

3. The tower monitoring method based on digital image processing according to claim 2, characterized in that, The step of determining the camera parameters for capturing the target local object using the panoramic camera based on the target camera parameters includes: Determine the number of target camera parameters. If the number is multiple, determine the area of ​​each sub-region occupied by the target local object sub-image in the multiple target panoramic sub-images bound to the determined multiple target camera parameters. The camera parameters for capturing the target local object by the panoramic camera are determined based on the ratio of the areas of each sub-region and the parameters of the multiple target cameras.

4. The tower monitoring method based on digital image processing according to claim 3, characterized in that, The camera parameters used to determine the panoramic camera's ability to capture the target local object based on the ratio of the areas of each sub-region and the parameters of the plurality of target cameras are replaced with: The camera parameters for capturing the target local object by the panoramic camera are determined based on the target camera parameters bound to the target panoramic sub-image corresponding to the largest sub-region. When the local camera captures a target image corresponding to the target local object that meets the preset size and preset clarity based on the position and the contour features, it captures an initial image that meets the preset size and preset clarity based on the position. It then determines whether the initial image contains the complete contour features based on the contour features. If yes, the initial image is determined as the target image corresponding to the target local object. If no, it determines the missing part of the contour features in the initial image based on the contour features. Based on the missing part, it adjusts the rotation angle, focal length, and focus position of the local camera to obtain the initial image, and returns to execute the step of determining whether the initial image contains the complete contour features based on the contour features until the initial image meets the preset size and preset clarity and contains the complete contour features.

5. A tower monitoring method based on digital image processing according to any one of claims 1 to 4, characterized in that, The local cameras include multiple cameras; If the target image cannot reach the preset resolution during the process of capturing the target image by the local camera, then the target image is captured as an image whose size meets the preset size and whose resolution is closest to the preset resolution. After receiving the target image sent by the local camera, the method further includes the following steps: Determine whether the clarity of the target image meets the preset requirements. If not, decompose the contour features of the target local object into multiple contour sub-features, determine the position of each contour sub-feature in the local image, and determine the corresponding local camera for each contour sub-feature. Each of the contour sub-features and its corresponding position in the local image is sent to its corresponding local camera, so that its corresponding local camera captures a target sub-image of the target local object that meets the preset size and preset clarity according to the contour sub-features and their positions. Receive target sub-images sent by each local camera, stitch together and scale the target sub-images to obtain a target image corresponding to the target local object that meets the preset size and preset resolution.

6. A tower monitoring method based on digital image processing according to any one of claims 2 to 4, characterized in that, The process includes the following steps before receiving the panoramic image of the tower to be monitored from the panoramic camera: The panoramic camera is rotated within a horizontal range of -180° to +180° and a vertical range of -90° to +90° to determine whether there is a shooting blind spot. If so, a blind spot camera is installed at the position corresponding to the shooting blind spot. It also includes the following steps: The system receives a blind spot image corresponding to the blind spot area sent by the blind spot camera, sends the blind spot image to the client, and displays the blind spot image on the client while displaying the panoramic image and the target image.

7. A tower monitoring method based on digital image processing according to any one of claims 1 to 4, characterized in that, Images captured by the panoramic camera and the partial camera are received and transmitted based on 5G technology.

8. A tower monitoring terminal based on digital image processing, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the pole monitoring method based on digital image processing as described in any one of claims 1 to 7.

Images