Oil taking port positioning method and system based on monocular vision and laser positioning

Abstract

The invention provides an oil taking port positioning method and system based on monocular vision and laser positioning. The method comprises the steps that laser point cloud data and a monocular image near an oil taking port are acquired; constructing a three-dimensional model of the oil taking port based on the laser point cloud data, and projecting the three-dimensional model of the oil taking port into a two-dimensional slice image according to an acquisition position label of the monocular camera; matching the two-dimensional slice image with the monocular image to determine the two-dimensional feature position of the oil extraction port; calculating the feature distance between the oil taking port and the camera based on the matched two-dimensional feature position of the oil taking port and the focal length label and the size label of the monocular image; on the basis of the position parameters of the oil taking port in the laser point cloud data, the feature distance is combined, and initial three-dimensional coordinates of the oil taking port are generated; and unifying the pixel coordinates of the monocular vision and the laser point cloud coordinates into the same coordinate system, and carrying out preliminary three-dimensional coordinate data fusion to obtain the final three-dimensional coordinates of the oil extraction port. The positioning precision of the transformer oil taking robot on the oil taking port is improved.

Description

Technical Field

[0001] This invention relates to the field of transformer oil and gas detection technology, and more specifically, to a method and system for locating oil sampling ports based on monocular vision and laser positioning. Background Technology

[0002] In the field of power system operation and maintenance, regular testing of transformer oil is crucial for preventing transformer failures and ensuring the stable operation of the power grid. Traditional manual oil sampling methods have many drawbacks, such as high operational risks, low efficiency, and susceptibility to human error leading to inconsistent accuracy. With the development of intelligent robot technology, the development of robots capable of automatically, accurately, and efficiently completing transformer oil sampling tasks has become an industry trend. Achieving precise positioning of the robot at the transformer oil sampling port is one of the key technical challenges in ensuring the successful completion of this task.

[0003] Therefore, it is necessary to provide a reliable and accurate positioning scheme for transformer oil inlets to support automatic detection of transformer oil and gas. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a method and system for locating oil sampling ports based on monocular vision and laser positioning, thereby improving the positioning accuracy of oil sampling robots for oil sampling ports.

[0005] According to a first aspect of the present invention, a method for locating an oil sampling port based on monocular vision and laser positioning is provided, comprising:

[0006] S1, acquire laser point cloud data and monocular image near the oil intake port;

[0007] S2. Construct a 3D model of the oil intake port based on laser point cloud data. Project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera. Through shape analysis, match the 2D slice image with the monocular image and select the projection slice with the highest similarity to determine the 2D feature position of the oil intake port.

[0008] S3. Based on the two-dimensional feature position of the oil extraction port after matching, the focal length label and size label of the monocular image, calculate the feature distance between the oil extraction port and the camera; based on the position parameters of the oil extraction port in the laser point cloud data, combined with the feature distance, generate the preliminary three-dimensional coordinates of the oil extraction port.

[0009] S4. The pixel coordinates of monocular vision and the laser point cloud coordinates are unified into the same coordinate system. The data is fused by combining the feature matching accuracy of monocular vision and the distance measurement accuracy of laser radar to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

[0010] Based on the above technical solution, the present invention can also be improved as follows.

[0011] Optionally, the method also includes:

[0012] The S5 uses a monocular camera to capture real-time images of the oil intake port, compares the deviation between the current monocular image and the preset template, and combines the real-time distance update of the lidar to dynamically correct the oil intake port positioning result.

[0013] Optionally, step S1 includes:

[0014] The monocular camera lens and lidar installed at the front of the oil sampling robot are directed toward the transformer oil sampling port area to ensure that the oil sampling port is fully in the field of view;

[0015] The robot is initially located near the oil intake port using lidar. If the distance between the robot and the oil intake port is determined to be within a certain distance threshold based on the initial lidar positioning data, the monocular camera is triggered to capture a preset frame of monocular image at a preset shooting frequency. At the same time, the lidar is triggered to acquire the laser point cloud data of the oil intake port at a preset scanning frequency.

[0016] The acquired monocular images and laser point cloud data are corrected and denoised. Based on time synchronization and spatial coordinates, the preprocessed monocular images and laser point cloud data are correlated and aligned.

[0017] Optionally, step S2 includes:

[0018] S201. After denoising and smoothing the laser point cloud data, a three-dimensional model of the oil intake port is constructed and key parameters are labeled. The three-dimensional model includes at least geometric dimensions, contour features and key feature points.

[0019] S202, based on the acquisition position label and lens deflection angle when the monocular camera acquires images, calculate the relative orientation between the camera and the oil intake port to ensure that the projection angle of the 3D model is consistent with the shooting angle of the monocular image; use 3D modeling software to project the 3D model of the oil intake port in two dimensions along the projection angle to generate multiple sets of two-dimensional slice images with different levels of detail. The two-dimensional slice images include at least the outline of the oil intake port and the pixel coordinates of key feature points.

[0020] S203, The edge detection algorithm is used to extract the contour edge of the oil intake port from the preprocessed monocular image, and the corner detection algorithm is used to identify the key corner points of the oil intake port. Based on the extracted contour edge and key corner points, the geometric parameters of the oil intake port in the monocular image are calculated.

[0021] S204: Select key corner points in the two-dimensional projection slice image and corner points extracted from the monocular image for preliminary alignment. Correct the image scale and rotation deviation through affine transformation to ensure that the two are comparable in the same coordinate system.

[0022] The Hausdorff distance algorithm is used to calculate the matching degree of the oil extraction port contour in the two-dimensional projected slice image and the monocular image. At the same time, the number of feature points matched between the two-dimensional projected slice image and the monocular image is compared to generate a comprehensive similarity score.

[0023] S205, compare the similarity score with the similarity threshold to determine whether a match is successful:

[0024] If a match is successful, the two-dimensional feature position of the oil extraction port in the two-dimensional projection slice image is output. The two-dimensional feature position includes at least the pixel coordinates of the oil extraction port in the monocular image, the pose of the three-dimensional model corresponding to the selected two-dimensional projection slice image, and the similarity score.

[0025] If the matching fails, adjust the projection angle of the 3D model, regenerate the 2D slice image, and repeat the matching until the optimal result is found or the iteration stops.

[0026] Optionally, step S3 includes:

[0027] S301, based on the selected two-dimensional projection slice image, obtain the actual size of the oil sampling port, and collect the focal length label and image size label of the monocular camera;

[0028] S302, based on the principle of geometric imaging, calculates the straight-line distance between the oil intake port and the monocular camera according to the actual size of the oil intake port, the focal length of the monocular camera, and the image size;

[0029] By using the camera's intrinsic parameters, the pixel coordinates of the oil extraction port in the image are converted into two-dimensional coordinates in the camera coordinate system. Combined with the straight-line distance between the oil extraction port and the monocular camera, the three-dimensional coordinates of the oil extraction port in the camera coordinate system are obtained based on monocular vision calculation.

[0030] S303, the point set corresponding to the oil intake port is extracted from the laser point cloud data by a clustering algorithm, and the three-dimensional coordinates of the oil intake port in the laser coordinate system are calculated based on the key position parameters of the point set corresponding to the oil intake port.

[0031] Optionally, step S4 includes:

[0032] S401, Based on the external parameter calibration results of the oil sampling robot, the three-dimensional coordinates of the oil sampling port in the camera coordinate system described in S302 and the three-dimensional coordinates of the oil sampling port in the laser coordinate system described in S303 are respectively transformed to the world coordinate system.

[0033] S402 uses a weighted fusion algorithm to fuse the two transformation results of S401 to obtain the three-dimensional coordinates of the oil intake port in the world coordinate system.

[0034] Optionally, step S5 includes:

[0035] S501, acquire the real-time monocular image of the oil extraction port captured by the monocular camera: if it is determined that the current monocular image is occluded, then reduce the positioning weight of the oil extraction port in the camera coordinate system and increase the positioning weight of the oil extraction port in the laser coordinate system in S402.

[0036] If it is determined that the laser point cloud data is interfered with by metal reflection, then the positioning weight of the oil sampling port in the camera coordinate system in S402 is increased, and the positioning weight of the oil sampling port in the laser coordinate system is decreased.

[0037] S502, using monocular vision to identify fixed landmark feature points around the oil intake port, and calculating the relative pixel distance between the fixed landmark feature points and the oil intake port;

[0038] The actual physical distance between the fixed landmarks around the oil intake and the oil intake is measured using lidar.

[0039] If the relative distance deviation between the relative pixel distance and the actual physical distance is greater than the preset distance deviation, then the three-dimensional coordinates of the fused oil extraction port in the robot chassis coordinate system are corrected.

[0040] According to a second aspect of the present invention, an oil sampling port positioning system based on monocular vision and laser positioning is provided, comprising:

[0041] The acquisition module is used to acquire laser point cloud data and monocular images near the oil intake port;

[0042] The matching module is used to construct a 3D model of the oil intake port based on laser point cloud data, and project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera; it is also used to match the 2D slice image with the monocular image through shape analysis, and filter out the projection slice with the highest similarity to determine the 2D feature position of the oil intake port.

[0043] The preliminary positioning module is used to calculate the feature distance between the oil intake port and the camera based on the two-dimensional feature position of the oil intake port after matching, the focal length label and size label of the monocular image; and to generate the preliminary three-dimensional coordinates of the oil intake port based on the position parameters of the oil intake port in the laser point cloud data and the feature distance.

[0044] The fine positioning module is used to combine the feature matching accuracy of monocular vision and the distance measurement accuracy of lidar for data fusion, unifying the pixel coordinates of monocular vision and the lidar point cloud coordinates into the same coordinate system, so as to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

[0045] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the processor is configured to execute a computer management program stored in the memory to implement the steps of the above-described oil extraction port positioning method based on monocular vision and laser positioning.

[0046] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management program is stored, wherein when executed by a processor, the computer management program implements the steps of the above-described method for locating an oil extraction port based on monocular vision and laser positioning.

[0047] This invention provides a method, system, electronic device, and storage medium for oil tap location based on monocular vision and laser positioning. It integrates the detailed feature recognition of monocular vision (such as oil tap edge, bolt hole, etc.) with the three-dimensional spatial ranging capability of laser radar, solving the problems of lack of depth information in monocular vision and lack of fine features in monocular laser. It significantly improves positioning accuracy, meets the automated operation requirements of transformer oil tapping robots, and enhances environmental adaptability and robustness, adapting to complex scenarios, and providing high operational safety and efficiency. Attached Figure Description

[0048] Figure 1 A flowchart illustrating an oil sampling port positioning method based on monocular vision and laser positioning, provided in a certain embodiment of the present invention;

[0049] Figure 2 A flowchart of an oil sampling port positioning method based on monocular vision and laser positioning is provided as another embodiment of the present invention;

[0050] Figure 3 A block diagram of an oil sampling port positioning system based on monocular vision and laser positioning, provided in a certain embodiment of the present invention;

[0051] Figure 4 A block diagram of an oil sampling port positioning system based on monocular vision and laser positioning is provided as another embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0053] Figure 6 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0054] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0055] Figure 1A flowchart of an oil sampling port positioning method based on monocular vision and laser positioning provided by the present invention is shown below. Figure 1 As shown, the method includes steps S1 to S4:

[0056] S1, acquire laser point cloud data and monocular image near the oil intake port;

[0057] S2. Construct a 3D model of the oil intake port based on laser point cloud data. Project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera. Through shape analysis, match the 2D slice image with the monocular image and select the projection slice with the highest similarity to determine the 2D feature position of the oil intake port.

[0058] S3. Based on the two-dimensional feature position of the oil extraction port after matching, the focal length label and size label of the monocular image, calculate the feature distance between the oil extraction port and the camera; based on the position parameters of the oil extraction port in the laser point cloud data, combined with the feature distance, generate the preliminary three-dimensional coordinates of the oil extraction port.

[0059] S4. The pixel coordinates of monocular vision and the laser point cloud coordinates are unified into the same coordinate system. The data is fused by combining the feature matching accuracy of monocular vision and the distance measurement accuracy of laser radar to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

[0060] Understandably, given the deficiencies in the background technology, this invention proposes an oil sampling port positioning method based on monocular vision and laser positioning. This method integrates the detailed feature recognition of monocular vision (e.g., oil sampling port edges, bolt holes, etc.) with the three-dimensional spatial ranging capability of laser radar, solving the problems of insufficient depth information from single vision and insufficient fine features from single laser. This significantly improves positioning accuracy, meets the automated operation requirements of transformer oil sampling robots, and enhances environmental adaptability and robustness, adapting to complex scenarios while maintaining high operational safety and efficiency.

[0061] In one possible embodiment, step S1 mainly includes the following sub-steps: device selection and deployment, parameter configuration, data acquisition triggering, and data preprocessing.

[0062] S101, Equipment Selection and Deployment:

[0063] The monocular camera lens and lidar installed at the front of the oil-taking robot are directed toward the transformer oil-taking port area to ensure that the oil-taking port (including valves, interfaces, and other features) is fully within the field of view.

[0064] In this embodiment, a 16-line lidar is installed on the top of the robot, and the scanning range covers the side of the transformer where the oil intake is located (e.g., a horizontal viewing angle of 120° and a vertical viewing angle of 30°).

[0065] S102, parameter configuration:

[0066] For monocular cameras, set image size labels, focal length labels, and camera acquisition position labels. For example, set the resolution to 1920×1080 pixels and the focal length to 50mm. Record the coordinates at the time of shooting (i.e., acquisition position labels, such as X=10.2m, Y=5.3m) through the robot's positioning unit.

[0067] For lidar, set the scanning frequency (e.g., 10Hz), point cloud density 0.5°×0.5° (angular resolution), and ranging range (e.g., 0.5-50m) to ensure that the point cloud data error of the oil sampling port is ≤±2mm (error threshold) within the preset distance threshold (e.g., 1-2m distance).

[0068] S103, Data Acquisition Trigger:

[0069] The robot is initially located near the oil intake port using lidar. If the distance between the robot and the oil intake port is determined to be within a certain threshold based on the initial lidar positioning data, the monocular camera is triggered to capture a preset frame of monocular image at a preset shooting frequency. Simultaneously, the lidar is triggered to acquire the laser point cloud data of the oil intake port at a preset scanning frequency.

[0070] S104, Preprocessing:

[0071] The acquired monocular images and laser point cloud data are corrected and denoised. Based on time synchronization and spatial coordinates, the preprocessed monocular images and laser point cloud data are correlated and aligned.

[0072] More specifically, in a particular implementation scenario, the preprocessing of monocular images mainly includes:

[0073] 1. Geometric correction: Based on camera intrinsic parameters (focal length, principal point coordinates) and distortion coefficients, the image is distorted to eliminate lens perspective error and ensure that the edge of the oil intake port is a straight line (e.g., the circular valve interface is corrected to a perfect circle).

[0074] 2. Noise Reduction Processing: Gaussian filtering is used to remove image noise, and then bilateral filtering is used to preserve edge features (such as the boundary between the oil intake and the pipeline) to solve the image blurring problem caused by dust in the substation environment;

[0075] 3. Region of Interest (ROI) Extraction: By pre-setting the oil inlet features (such as setting the diameter of the circular interface), the possible region (such as a 100×100 pixel range) is selected in the image to reduce background interference (such as transformer casing, heat sink).

[0076] The preprocessing of laser point cloud data mainly includes:

[0077] 1. Point cloud filtering:

[0078] Voxel grid filtering (e.g., voxel size 5mm×5mm×5mm) is used to downsample the original point cloud to reduce the amount of data (retaining about 30% of key points);

[0079] By using statistical outlier filtering (setting K=20 neighborhood and standard deviation threshold 1.0), dynamic interference points such as birds and falling objects in the substation environment are removed.

[0080] 2. Clustering and segmentation:

[0081] Based on Euclidean distance clustering (e.g., distance threshold of 10mm), the point cloud is segmented into different objects, distinguishing between oil extraction port (metal material, dense point cloud) and background (such as concrete wall, cable, sparse point cloud).

[0082] 3. Ground removal:

[0083] The RANSAC algorithm is used to fit the ground plane, remove the ground point cloud, and focus on the vertical plane (the side of the transformer) where the oil intake is located.

[0084] S105, Data Association and Alignment:

[0085] 1. Time synchronization: By using a unified clock in the control system of the oil-collecting robot, the acquisition time difference between the monocular image and the laser point cloud is ensured to be ≤ a preset time difference threshold (e.g., 50ms), thus avoiding spatiotemporal deviations caused by the robot's micro-movements.

[0086] 2. Coordinate transformation: Based on the extrinsic parameters of the camera and LiDAR (such as pre-calibrated rotation matrices and translation vectors), the pixel coordinates of the monocular image are associated with the three-dimensional coordinates of the LiDAR point cloud, realizing the mapping of "two-dimensional image features - three-dimensional point cloud position".

[0087] Understandably, through the various sub-steps of step S1, the preprocessed monocular image can clearly extract features such as the edge and shape of the oil intake port, and the laser point cloud can accurately reflect the spatial location and surrounding structure of the oil intake port, laying the foundation for feature matching and localization calculation in subsequent steps.

[0088] In one possible embodiment, step S2 includes sub-steps S201 to S205.

[0089] S201. After denoising and smoothing the laser point cloud data, a three-dimensional model of the oil intake port is constructed and key parameters are labeled. The three-dimensional model includes at least geometric dimensions, contour features and key feature points.

[0090] More specifically, including:

[0091] Based on the standard design parameters of the transformer oil tap (such as a tap diameter of 50mm, a tap flange thickness of 10mm, and a tap valve protrusion height of 20mm), the standard oil tap is scanned by lidar to detect the target shape and generate point cloud data. Point cloud processing software (such as CloudCompare) is used to reduce noise and smooth the data to construct a three-dimensional model of the oil tap, including geometric dimensions, contour features (such as circular interfaces and rectangular valve edges), and key feature points (such as the location of flange bolt holes).

[0092] Mark key parameters in the 3D model of the oil inlet: the coordinates of the interface center, the diameter of the outer edge of the flange, the position of the valve rotation axis, etc., as the reference for subsequent matching.

[0093] S202, based on the acquisition position label and lens deflection angle when the monocular camera acquires the image, calculate the relative orientation between the camera and the oil intake port to ensure that the projection angle of the 3D model is consistent with the shooting angle of the monocular image; use 3D modeling software to project the 3D model of the oil intake port in two dimensions along the projection angle to generate multiple sets of two-dimensional slice images with different levels of detail. The two-dimensional slice images include at least the outline of the oil intake port and the pixel coordinates of key feature points.

[0094] More specifically, based on the acquisition position label (e.g., camera coordinates X=10m, Y=5m, height Z=1.2m) and lens deflection angle (obtained through robot posture sensor) when the monocular camera acquires images, the relative orientation of the camera and the oil intake port is calculated, and the projection angle of the 3D model (e.g., 30° horizontally and 15° vertically) is determined to ensure that the projection angle is consistent with the monocular image acquisition angle.

[0095] Then, using 3D modeling software (such as Blender), the 3D model of the oil intake port is projected in 2D along the above projection viewpoint to generate multiple sets of slice images with different levels of detail (resolution consistent with the monocular image, 1920×1080 pixels), including the outline of the oil intake port and the pixel coordinates of key feature points.

[0096] S203. The Canny edge detection algorithm is used to extract the contour edge of the oil inlet from the preprocessed monocular image, focusing on preserving the arc of the circular interface, the straight edge of the flange and the rectangular contour of the valve, while filtering background noise (such as irrelevant textures of the transformer shell).

[0097] The Harris corner detection algorithm is used to identify key corners of the oil intake (such as the four corners of the flange, the junction of the valve and the interface), and local texture features (such as the operation marks on the valve surface) are extracted through ORB feature descriptor to form a feature point set.

[0098] Based on the extracted contour edges and key corner points, the geometric parameters of the oil intake port in the monocular image are calculated, such as the interface center coordinates (pixel coordinates), contour diameter (pixel size), and flange edge angle, as quantitative indicators for matching.

[0099] S204, Corner Alignment: Select key corner points (such as the upper left and lower right corners of the flange) in the 2D projection slice image and perform preliminary alignment with the corner points extracted from the monocular image. Correct the image scale and rotation deviation through affine transformation to ensure that the two are comparable in the same coordinate system.

[0100] Shape similarity calculation: The contour Hausdorff distance algorithm is adopted. Based on shape similarity analysis, the matching degree of the oil extraction port contour in the two-dimensional projection slice image and the monocular image is calculated. At the same time, the number of feature point matching (such as ORB feature matching pairs) of the two-dimensional projection slice image and the monocular image are compared, and a similarity score (range 0-1) is generated.

[0101] S205, Matching Result Determination: The similarity score is compared with a similarity threshold to determine whether a match is successful, specifically including:

[0102] Set a similarity threshold (also known as a confidence threshold, 0.75 is used as an example here): If the similarity score is ≥0.75, it is determined to be a successful match, and the two-dimensional projection slice image is output as the "selected two-dimensional projection slice image". The two-dimensional feature position of the selected two-dimensional projection slice image is also output. The two-dimensional feature position includes at least the pixel coordinates of the oil extraction port in the monocular image, the pose of the three-dimensional model corresponding to the selected two-dimensional projection slice image, and the similarity score.

[0103] If the score is less than 0.75, the match is considered unsuccessful. In this case, the projection angle of the 3D model is adjusted (within ±5°), the 2D slice image is regenerated and the matching is repeated until the optimal result is found or the iteration stopping condition is met (e.g., a maximum of 5 iterations are preset).

[0104] After matching is complete, the matching results can be verified and optimized, including:

[0105] 1. Laser point cloud-assisted verification:

[0106] Extract clusters of oil intake regions from the lidar point cloud, project them onto the monocular image plane, and verify whether the matched oil intake position coincides with the point cloud projection area (deviation ≤ 5 pixels) to eliminate false matches (such as misidentifying transformer bolts as oil intakes).

[0107] 2. Feature Parameter Correction (Optimization)

[0108] If the match is successful but there are slight deviations (such as a deviation of 3-5 pixels in the center pixel coordinates of the interface), the feature parameters of the oil sampling port in the monocular image are corrected by combining the three-dimensional dimensions of the laser point cloud (such as the actual diameter of 50mm) to ensure consistency with the three-dimensional model.

[0109] Understandably, after a successful match in step S2, the output should include at least: the precise location (pixel coordinates) of the oil extraction port in the monocular image, the pose of the 3D model corresponding to the selected 2D projected slice image (relative angle to the camera), and the matching confidence score (similarity rating). This output provides the core input for the geometric localization calculation in subsequent steps, ensuring the accuracy of the oil extraction port location.

[0110] In one possible embodiment, step S3 includes:

[0111] S301, based on the selected two-dimensional projection slice image, obtain the actual size of the oil sampling port, and collect the focal length label and image size label of the monocular camera;

[0112] S302, based on the principle of geometric imaging, calculates the straight-line distance between the oil intake port and the monocular camera according to the actual size of the oil intake port, the focal length of the monocular camera and the image size, that is, it calculates the feature distance based on monocular ranging.

[0113] By using the camera's intrinsic parameters, the pixel coordinates of the oil extraction port in the image are converted into two-dimensional coordinates in the camera coordinate system. Combined with the straight-line distance between the oil extraction port and the monocular camera, the three-dimensional coordinates of the oil extraction port in the camera coordinate system are obtained based on monocular vision calculation.

[0114] S303, the point set corresponding to the oil intake port is extracted from the laser point cloud data by a clustering algorithm, and the three-dimensional coordinates of the oil intake port in the laser coordinate system are calculated based on the key position parameters of the point set corresponding to the oil intake port.

[0115] Understandably, step S3 calculates and outputs the preliminary three-dimensional coordinates of the oil intake port through geometric positioning, providing input data for data fusion and fine positioning in subsequent steps.

[0116] In one possible embodiment, step S4 includes sub-steps S401 to S402:

[0117] S401, Establishment of coordinate system one and mapping relationship:

[0118] Based on the external parameter calibration results of the oil sampling robot, the three-dimensional coordinates of the oil sampling port in the camera coordinate system described in S302 and the three-dimensional coordinates of the oil sampling port in the laser coordinate system described in S303 are transformed to the world coordinate system; specifically including:

[0119] 1. Definition of coordinate system

[0120] Establish a world coordinate system with the center of the transformer base as the origin (X-axis: parallel to the ground and pointing towards the oil intake, Y-axis: perpendicular to the ground and upward, Z-axis: perpendicular to the X-axis and horizontally to the right).

[0121] Regarding the monocular vision coordinate system: with the camera optical center as the origin, the pixel coordinates (x,y) are converted into preliminary three-dimensional coordinates in the image coordinate system using camera intrinsic parameters (e.g., focal length f = 50mm, principal point coordinates (e.g., u0 = 960, v0 = 540)).

[0122] Regarding the lidar coordinate system: with the lidar rotation center as the origin, the initial three-dimensional coordinates (x_l, y_l, z_l) of the lidar point cloud are transformed to the world coordinate system through external parameter calibration (e.g., rotation matrix R, translation vector T).

[0123] 2. Spatiotemporal synchronization correction

[0124] Based on the unified clock of the oil-collecting robot control system, the acquisition timestamps of the monocular image and the laser point cloud are aligned (e.g., to ensure an error ≤ 50ms). If a time deviation exists, the laser point cloud position is corrected using an interpolation algorithm (e.g., by setting the robot's moving speed v and displacement Δs within 50ms to compensate for the laser coordinates).

[0125] S402, Feature-level fusion of monocular vision and laser data: The two transformation results of S401 are fused by a weighted fusion algorithm to obtain the three-dimensional coordinates of the oil intake port in the world coordinate system.

[0126] Specifically, it includes:

[0127] The confidence levels of the preliminary 3D coordinates from monocular vision (based on the world coordinate system) and the preliminary 3D coordinates from laser point cloud are used to assign weights to both. For example, the weight for monocular vision is set to α based on feature matching similarity, such as α = 0.7 for a similarity of 0.8. The weight for laser radar is set to 1-α based on point cloud density, such as α = 0.7 for a point cloud density of 50 points / cm². 2 This corresponds to 1-α = 0.3.

[0128] Based on the above weight allocation results, the fused coordinates P = α·P_v + (1-α)·P_l, where P_v is the initial monocular vision positioning coordinate, P_l is the initial laser positioning coordinate, and P is the final 3D coordinate after fusion.

[0129] In one possible embodiment, such as Figure 2 As shown, the method of the present invention further includes:

[0130] The S5 uses a monocular camera to capture real-time images of the oil intake port, compares the deviation between the current monocular image and the preset template, and combines the real-time distance update of the lidar to dynamically correct the oil intake port positioning result.

[0131] Step S5 specifically includes sub-steps S501 to S502:

[0132] S501, acquire the real-time monocular image of the oil extraction port captured by the monocular camera: if it is determined that the current monocular image is occluded, then reduce the positioning weight α of the oil extraction port in the monocular camera coordinate system in S402 and increase the positioning weight 1-α of the oil extraction port in the laser coordinate system.

[0133] If it is determined that the laser point cloud data is interfered with by metal reflection, then increase the positioning weight α of the oil sampling port in the camera coordinate system in S402 and decrease the positioning weight 1-α of the oil sampling port in the laser coordinate system.

[0134] S502, using monocular vision to identify fixed landmark feature points around the oil intake port, and calculating the relative pixel distance between the fixed landmark feature points and the oil intake port;

[0135] The actual physical distance between the fixed landmarks around the oil intake and the oil intake is measured using lidar.

[0136] If the relative distance deviation between the relative pixel distance and the actual physical distance is greater than the preset distance deviation, then the three-dimensional coordinates of the fused oil extraction port in the robot chassis coordinate system are corrected.

[0137] Understandably, when a deviation is found in the positioning verification (such as a deviation of more than 3mm between monocular vision and laser point cloud positioning results), the oil-taking robot's oil-taking actuator is adjusted in real time to ensure the final positioning accuracy.

[0138] Step S5 enables dynamic positioning adjustments in various special scenarios with strong anti-interference capabilities. For example, to address sudden lighting interference: if feature point matching fails (similarity < 0.7) due to strong light or shadows in the monocular image, the system automatically switches to laser point cloud-based positioning, directly adjusting based on the point cloud fitting results until image verification is reactivated after lighting is restored. To address metal reflection interference: when point cloud data is missing due to reflections from the metal surface of the oil intake port, the monocular vision weight is enhanced, and positioning is achieved through geometric parameter verification and template matching, while simultaneously controlling the oil intake robot to fine-tune its angle to reduce reflections. To address near-field occlusion: if the oil intake port is partially obstructed by cables or dust (e.g., obstruction area < 30%), a reflector-assisted detection method (a reflector is installed on the inner side of the robot's shell) can be used to obtain images of the unobstructed area, extracting local features (such as parts of the flange edge) for matching, and combining this with the laser point cloud to complete the 3D information of the obstructed area.

[0139] After the final positioning verification is passed, the precise final three-dimensional coordinates of the oil intake port and the adjustment parameters of the oil intake device (such as X-axis movement +1mm, rotation -0.5°) are output, and the actuator of the oil intake robot is triggered to perform the docking action to ensure the sealed docking of the oil intake port and the oil intake device.

[0140] Figure 3 A structural diagram of an oil sampling port positioning system based on monocular vision and laser positioning provided in an embodiment of the present invention is shown below. Figure 3 As shown, an oil sampling port positioning system based on monocular vision and laser positioning includes an acquisition module, a matching module, a preliminary positioning module, and a fine positioning module, wherein:

[0141] The acquisition module is used to acquire laser point cloud data and monocular images near the oil intake port;

[0142] The matching module is used to construct a 3D model of the oil intake port based on laser point cloud data, and project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera; it is also used to match the 2D slice image with the monocular image through shape analysis, and filter out the projection slice with the highest similarity to determine the 2D feature position of the oil intake port.

[0143] The preliminary positioning module is used to calculate the feature distance between the oil intake port and the camera based on the two-dimensional feature position of the oil intake port after matching, the focal length label and size label of the monocular image; and to generate the preliminary three-dimensional coordinates of the oil intake port based on the position parameters of the oil intake port in the laser point cloud data and the feature distance.

[0144] The fine positioning module is used to combine the feature matching accuracy of monocular vision and the distance measurement accuracy of lidar for data fusion, unifying the pixel coordinates of monocular vision and the lidar point cloud coordinates into the same coordinate system, so as to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

[0145] It is understood that the oil intake port positioning system based on monocular vision and laser positioning provided by the present invention corresponds to the oil intake port positioning method based on monocular vision and laser positioning provided in the foregoing embodiments. The relevant technical features of the oil intake port positioning system based on monocular vision and laser positioning can be referred to the relevant technical features of the oil intake port positioning method based on monocular vision and laser positioning, and will not be repeated here.

[0146] As a preferred embodiment, such as Figure 4 As shown, the system also includes:

[0147] The dynamic adjustment module is used to capture images of the oil intake port in real time using a monocular camera, compare the deviation between the current monocular image and the preset template, and dynamically correct the oil intake port positioning result by combining the real-time distance update of the lidar.

[0148] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 5As shown, this embodiment of the invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, it performs the following steps:

[0149] S1, acquire laser point cloud data and monocular image near the oil intake port;

[0150] S2. Construct a 3D model of the oil intake port based on laser point cloud data. Project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera. Through shape analysis, match the 2D slice image with the monocular image and select the projection slice with the highest similarity to determine the 2D feature position of the oil intake port.

[0151] S3. Based on the two-dimensional feature position of the oil extraction port after matching, the focal length label and size label of the monocular image, calculate the feature distance between the oil extraction port and the camera; based on the position parameters of the oil extraction port in the laser point cloud data, combined with the feature distance, generate the preliminary three-dimensional coordinates of the oil extraction port.

[0152] S4. Unify the pixel coordinates of monocular vision and the laser point cloud coordinates into the same coordinate system, and perform data fusion by combining the feature matching accuracy of monocular vision and the distance measurement accuracy of laser radar, so as to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

[0153] The S5 uses a monocular camera to capture real-time images of the oil intake port, compares the deviation between the current monocular image and the preset template, and combines the real-time distance update of the lidar to dynamically correct the oil intake port positioning result.

[0154] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, it performs the following steps:

[0155] S1, acquire laser point cloud data and monocular image near the oil intake port;

[0156] S2. Construct a 3D model of the oil intake port based on laser point cloud data. Project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera. Through shape analysis, match the 2D slice image with the monocular image and select the projection slice with the highest similarity to determine the 2D feature position of the oil intake port.

[0157] S3. Based on the two-dimensional feature position of the oil extraction port after matching, the focal length label and size label of the monocular image, calculate the feature distance between the oil extraction port and the camera; based on the position parameters of the oil extraction port in the laser point cloud data, combined with the feature distance, generate the preliminary three-dimensional coordinates of the oil extraction port.

[0158] S4. Unify the pixel coordinates of monocular vision and the laser point cloud coordinates into the same coordinate system, and perform data fusion by combining the feature matching accuracy of monocular vision and the distance measurement accuracy of laser radar, so as to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

[0159] The S5 uses a monocular camera to capture real-time images of the oil intake port, compares the deviation between the current monocular image and the preset template, and combines the real-time distance update of the lidar to dynamically correct the oil intake port positioning result.

[0160] This invention provides a method, system, and storage medium for locating oil extraction ports based on monocular vision and laser positioning. Through multi-sensor fusion and precise algorithm design, it offers the following main advantages:

[0161] 1. Positioning accuracy is significantly improved, meeting the needs of automated operations.

[0162] By integrating the detailed feature recognition of monocular vision (such as the edge of the oil intake port, bolt holes, etc.) with the three-dimensional spatial ranging capability of lidar, the problems of lack of depth information in monocular vision and lack of fine features in monocular lidar are solved.

[0163] 2. Enhanced environmental adaptability and robustness, enabling adaptation to complex scenarios.

[0164] To address interference from substation lighting variations (backlighting, cloudy days), localized obstructions (cables, dust), and metal reflections, a dynamic "vision-laser" weight adjustment is implemented (e.g., increasing the laser weight during sudden lighting changes and relying on visual features during metal reflections) to ensure positioning stability. Image denoising and point cloud filtering in the preprocessing stage, along with multi-scale templates and similarity threshold control during feature matching, further reduce the impact of environmental noise on positioning.

[0165] 3. Improved work efficiency and reduced reliance on manual labor

[0166] The automated positioning process (requiring no manual intervention from data acquisition to dynamic adjustment) reduces the positioning time for a single oil sampling port to less than 10 seconds, improving efficiency by over 90% compared to traditional manual positioning (approximately 3-5 minutes). The positioning results are directly output to the robot control system, driving the automatic docking device to complete the oil sampling operation, providing core support for the full automation of transformer oil sample collection.

[0167] 4. Reduce operational risks and ensure operational safety.

[0168] Replacing manual close-range operation (traditional manual positioning requires proximity to high-voltage equipment), robots enable remote positioning and operation, reducing the risk of personnel exposure to high-voltage electric fields and mechanical injuries. Positioning verification and dynamic adjustment mechanisms (such as secondary matching and deviation correction) prevent safety accidents such as equipment collisions and oil leaks caused by positioning errors.

[0169] 5. Highly innovative and versatile in technology

[0170] The innovative application of the "3D model projection - 2D feature matching - 3D coordinate fusion" logic to oil intake port positioning achieves accurate mapping between the "virtual model and the actual scene".

[0171] 6. Promote the intelligent upgrading of power operation and maintenance.

[0172] This invention provides key support for the automation and unmanned operation of transformer condition monitoring. Combined with an automatic oil sample collection and analysis system, it can construct an intelligent operation and maintenance closed loop of "condition perception - automatic decision-making - precise execution" to improve the reliability of power grid equipment.

[0173] In summary, this invention achieves breakthroughs in accuracy, efficiency, safety, and versatility through multi-dimensional optimization, providing core technical support for the intelligent upgrade of transformer oil sample collection in power systems.

[0174] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0175] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0177] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0178] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0179] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0180] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for locating an oil sampling port based on monocular vision and laser positioning, characterized in that, include: S1, acquire laser point cloud data and monocular image near the oil intake port; S2. Construct a 3D model of the oil intake port based on laser point cloud data. Project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera. Through shape analysis, match the 2D slice image with the monocular image and select the projection slice with the highest similarity to determine the 2D feature position of the oil intake port. S3. Based on the two-dimensional feature position of the oil extraction port after matching, the focal length label and size label of the monocular image, calculate the feature distance between the oil extraction port and the camera; based on the position parameters of the oil extraction port in the laser point cloud data, combined with the feature distance, generate the preliminary three-dimensional coordinates of the oil extraction port. S4. The pixel coordinates of monocular vision and the laser point cloud coordinates are unified into the same coordinate system. The data is fused by combining the feature matching accuracy of monocular vision and the distance measurement accuracy of laser radar to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

2. The oil extraction port positioning method based on monocular vision and laser positioning according to claim 1, characterized in that, Also includes: The S5 uses a monocular camera to capture real-time images of the oil intake port, compares the deviation between the current monocular image and the preset template, and combines the real-time distance update of the lidar to dynamically correct the oil intake port positioning result.

3. The oil extraction port positioning method based on monocular vision and laser positioning according to claim 1, characterized in that, Step S1 includes: The monocular camera lens and lidar installed at the front of the oil sampling robot are directed toward the transformer oil sampling port area to ensure that the oil sampling port is fully in the field of view; The robot is initially located near the oil intake port using lidar. If the distance between the robot and the oil intake port is determined to be within a certain distance threshold based on the initial lidar positioning data, the monocular camera is triggered to capture a preset frame of monocular image at a preset shooting frequency. At the same time, the lidar is triggered to acquire the laser point cloud data of the oil intake port at a preset scanning frequency. The acquired monocular images and laser point cloud data are corrected and denoised. Based on time synchronization and spatial coordinates, the preprocessed monocular images and laser point cloud data are correlated and aligned.

4. The oil sampling port positioning method based on monocular vision and laser positioning according to claim 1, characterized in that, Step S2 includes: S201. After denoising and smoothing the laser point cloud data, a three-dimensional model of the oil intake port is constructed and key parameters are labeled. The three-dimensional model includes at least geometric dimensions, contour features and key feature points. S202, based on the acquisition position label and lens deflection angle when the monocular camera acquires images, calculate the relative orientation between the camera and the oil intake port to ensure that the projection angle of the 3D model is consistent with the shooting angle of the monocular image; use 3D modeling software to project the 3D model of the oil intake port in two dimensions along the projection angle to generate multiple sets of two-dimensional slice images with different levels of detail. The two-dimensional slice images include at least the outline of the oil intake port and the pixel coordinates of key feature points. S203, The edge detection algorithm is used to extract the contour edge of the oil intake port from the preprocessed monocular image, and the corner detection algorithm is used to identify the key corner points of the oil intake port. Based on the extracted contour edge and key corner points, the geometric parameters of the oil intake port in the monocular image are calculated. S204: Select key corner points in the two-dimensional projection slice image and corner points extracted from the monocular image for preliminary alignment. Correct the image scale and rotation deviation through affine transformation to ensure that the two are comparable in the same coordinate system. The Hausdorff distance algorithm is used to calculate the matching degree of the oil extraction port contour in the two-dimensional projected slice image and the monocular image. At the same time, the number of feature points matched between the two-dimensional projected slice image and the monocular image is compared to generate a comprehensive similarity score. S205, compare the similarity score with the similarity threshold to determine whether a match is successful: If a match is successful, the two-dimensional feature location of the oil extraction port in the two-dimensional projection slice image is output. The two-dimensional feature location includes at least the pixel coordinates of the oil extraction port in the monocular image, the pose of the three-dimensional model corresponding to the selected two-dimensional projection slice image, and the similarity score. If the matching fails, adjust the projection angle of the 3D model, regenerate the 2D slice image, and repeat the matching until the optimal result is found or the iteration stops.

5. The oil extraction port positioning method based on monocular vision and laser positioning according to claim 4, characterized in that, Step S3 includes: S301, based on the selected two-dimensional projection slice image, obtain the actual size of the oil sampling port, and collect the focal length label and image size label of the monocular camera; S302, based on the principle of geometric imaging, calculates the straight-line distance between the oil intake port and the monocular camera according to the actual size of the oil intake port, the focal length of the monocular camera, and the image size; By using the camera intrinsic parameters, the pixel coordinates of the oil extraction port in the image are converted into two-dimensional coordinates in the camera coordinate system. Combined with the straight-line distance between the oil extraction port and the monocular camera, the three-dimensional coordinates of the oil extraction port in the camera coordinate system are obtained based on monocular vision calculation. S303, the point set corresponding to the oil intake port is extracted from the laser point cloud data by a clustering algorithm, and the three-dimensional coordinates of the oil intake port in the laser coordinate system are calculated based on the key position parameters of the point set corresponding to the oil intake port.

6. The oil extraction port positioning method based on monocular vision and laser positioning according to claim 5, characterized in that, Step S4 includes: S401, Based on the external parameter calibration results of the oil sampling robot, the three-dimensional coordinates of the oil sampling port in the camera coordinate system described in S302 and the three-dimensional coordinates of the oil sampling port in the laser coordinate system described in S303 are respectively transformed to the world coordinate system. S402 uses a weighted fusion algorithm to fuse the two transformation results of S401 to obtain the three-dimensional coordinates of the oil intake port in the world coordinate system.

7. The oil extraction port positioning method based on monocular vision and laser positioning according to claim 6, characterized in that, Step S5 includes: S501, acquire the real-time monocular image of the oil extraction port captured by the monocular camera: if it is determined that the current monocular image is occluded, then reduce the positioning weight of the oil extraction port in the camera coordinate system and increase the positioning weight of the oil extraction port in the laser coordinate system in S402. If it is determined that the laser point cloud data is interfered with by metal reflection, then the positioning weight of the oil sampling port in the camera coordinate system in S402 is increased, and the positioning weight of the oil sampling port in the laser coordinate system is decreased. S502, using monocular vision to identify fixed landmark feature points around the oil intake port, and calculating the relative pixel distance between the fixed landmark feature points and the oil intake port; The actual physical distance between the fixed landmarks around the oil intake and the oil intake is measured using lidar. If the relative distance deviation between the relative pixel distance and the actual physical distance is greater than the preset distance deviation, then the three-dimensional coordinates of the fused oil extraction port in the robot chassis coordinate system are corrected.

8. An oil sampling port positioning system based on monocular vision and laser positioning, characterized in that, include: The acquisition module is used to acquire laser point cloud data and monocular images near the oil intake port; The matching module is used to construct a 3D model of the oil intake port based on laser point cloud data, and project the 3D model of the oil intake port into a 2D slice image according to the acquisition position label of the monocular camera; it is also used to match the 2D slice image with the monocular image through shape analysis, and filter out the projection slice with the highest similarity to determine the 2D feature position of the oil intake port. The preliminary positioning module is used to calculate the feature distance between the oil intake port and the camera based on the two-dimensional feature position of the oil intake port after matching, the focal length label and size label of the monocular image; and to generate the preliminary three-dimensional coordinates of the oil intake port based on the position parameters of the oil intake port in the laser point cloud data and the feature distance. The fine positioning module is used to combine the feature matching accuracy of monocular vision and the distance measurement accuracy of lidar for data fusion, unifying the pixel coordinates of monocular vision and the lidar point cloud coordinates into the same coordinate system, so as to calculate the final three-dimensional coordinates of the oil intake port based on the preliminary three-dimensional coordinates.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the processor is used to execute computer management programs stored in the memory to implement the steps of the oil extraction port positioning method based on monocular vision and laser positioning as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer management program, which, when executed by a processor, implements the steps of the oil extraction port positioning method based on monocular vision and laser positioning as described in any one of claims 1-7.

Images