Image feature point extraction method and system

Abstract

The application provides an image feature point extraction method and system, comprising: constructing a feature map, and obtaining the center coordinates of all circular horn mouths of heat insulation sleeves; image acquisition and preprocessing; target recognition and feature point extraction; feature point matching and position and posture calculation; positioning result optimization. The application identifies a circle (the heat insulation sleeve on the top of a nuclear power plant storage room has a circular horn mouth shape and faces downward) from a perception image, calculates the center position of the circle as a stable feature point, and matches the constructed map for robot positioning. The robot can overcome the problem of influence of light and view angle change on a traditional algorithm, and realizes stable positioning and navigation in a completely dark and simple structure storage room.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image feature point extraction method and system. Background Technology

[0002] In robot vision navigation control systems, accurate estimation of the robot's position and attitude provides crucial foundational data for subsequent path planning and obstacle avoidance decisions, directly impacting the overall system's accuracy, reliability, and efficiency. While meeting high precision and real-time requirements, localization algorithms and path planning need to possess greater robustness and stability. Existing navigation technologies typically employ multi-sensor fusion methods such as visual SLAM and LiDAR for localization and mapping. However, in scenarios with sparse feature points, the localization accuracy and stability of existing algorithms are often insufficient, potentially leading to navigation interruptions, path planning deviations, or untimely system responses, thereby affecting the smooth execution and safety of navigation tasks.

[0003] In existing visual navigation and positioning systems, feature points are extracted based on pixel changes to identify contours or other methods. These feature points often lack actual physical meaning. In such cases, if the lighting or camera position changes, the feature points will shift. If it is a wide-angle camera, there will also be the effect of image distortion. The shift of feature points will directly affect the robot's visual matching and positioning results.

[0004] Furthermore, in the unique environment of a nuclear power plant's roof storage area, it is even more difficult for robots to find stable feature points in their visual perception images, leading to inaccurate positioning. The nuclear power plant's roof storage area is a special environment characterized by its complete enclosure and darkness, lack of ground features, confined space, and high-intensity radiation, among other unfavorable conditions. These factors not only make it difficult for humans to easily enter and set up light sources, but also pose a severe challenge to the applicability of conventional visual navigation methods. Due to the lack of ambient light, operations inside can only be completed by robots equipped with small light sources. However, as the robot moves, the direction of the light source and camera constantly changes, resulting in complex and non-uniform lighting conditions. Traditional visual positioning methods based on feature point extraction (such as SIFT or ORB algorithms) exhibit significant limitations in this scenario. These algorithms typically rely on local texture or edge features of the image; however, under conditions of drastic changes in lighting and constantly shifting camera angles, the extracted feature points are prone to significant shifts, or even complete failure, directly leading to a decrease in robot positioning accuracy or even failure.

[0005] Furthermore, due to the spacious interior of the storage room and the lack of obvious features on the ground, the robot can only complete its positioning task by relying on the overhead heat insulation sleeve. The heat insulation sleeve has a uniform structural feature; its lower flared end is circular and remains on the same horizontal line, providing a basis for geometry-based visual positioning methods. However, traditional algorithms are not robust enough to edge details or geometric centers when extracting circular feature points, and are easily affected by changes in viewing angle, lighting conditions, and noise, thus limiting their application in complex environments.

[0006] Patent document CN114723977A discloses a stable feature point recognition method for visual SLAM systems. This method identifies highly repeatable visual feature points in irregular environments by constructing a large number of training samples and establishing a corresponding predictive neural network. However, this method requires the prior collection of a large number of data samples for training. Furthermore, its training results cannot perfectly adapt to significant changes in the overall environment because it essentially still extracts some feature points from image pixels.

[0007] Patent document CN113269207A discloses an image feature point extraction method for visual measurement of mesh structured light. It proposes to extract the skeleton of the mesh structured light pattern, then extract multiple corner points for each single pixel of the mesh structured light skeleton pattern, determine the extraction region of the light stripe center based on the corner point coordinates, extract the light stripe center and fit a straight line, and finally obtain the pixel coordinates of the feature points. This method provides a stable feature point extraction method only for visual measurement of mesh structured light, but it cannot be applied to general cameras. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide an image feature point extraction method and system.

[0009] An image feature point extraction method provided by the present invention includes:

[0010] Step S1: Construct a feature map and obtain the center coordinates of all circular flared openings of the insulation sleeves;

[0011] Step S2: Image acquisition and preprocessing;

[0012] Step S3: Target recognition and feature point extraction;

[0013] Step S4: Feature point matching and position / pose calculation;

[0014] Step S5: Optimize the positioning results.

[0015] Preferably, step S2 includes:

[0016] Step S2.1: Use RTSP to acquire the camera video stream and obtain the latest image as the raw image for perception;

[0017] Step S2.2: Convert the color image to a grayscale image, retaining only the brightness information;

[0018] Step S2.3: Apply Gaussian blur to the grayscale image to smooth high-frequency noise while preserving edge features;

[0019] Step S2.4: Use Canny edge detection to extract salient edges in the image;

[0020] Step S2.5: Use the closing operation to fill the break in the circular edge, and use the opening operation to remove small isolated noise points;

[0021] Step S2.6: Set a threshold and use binarization to convert the image into a black and white image.

[0022] Preferably, step S3 includes:

[0023] Step S3.1: Adjust the Hough algorithm detection parameters according to the different nuclear power plant reactor sizes and camera parameters;

[0024] Step S3.2: Use the Hough algorithm to identify all circular parameters in the binarized image, including pixel center coordinates and pixel radius;

[0025] Step S3.3: Verify whether the circular results identified in the image are standard, optimize the pixel center coordinates, and save the identification result data in memory.

[0026] Preferably, step S3.3 includes: calculating the mean square error (MSE) of the circular edge points, using the following formula:

[0027]

[0028] In the formula, MSE represents the mean square error of the edge points of the circle, n represents the number of data points, h and k represent the coordinates of the center of the fitted circle, r represents the radius of the fitted circle, and x... i y i These represent the coordinates of the i-th data point;

[0029] If the mean square error (MSE) of the edge points of the circle is greater than the threshold, it indicates that the circle is not standard. Gradient descent is used to fit an ellipse to the non-standard circle, minimizing the objective function to obtain more accurate pixel center coordinates. The formula is as follows:

[0030]

[0031] In the formula, a represents the major axis radius of the ellipse, and b represents the minor axis radius of the ellipse.

[0032] Preferably, step S4 includes:

[0033] Step S4.1: Calculate the scale relationship s based on the camera parameters, using the following formula:

[0034] s = f / Z

[0035] Where f represents the camera's focal length, and Z represents the distance between the camera and the target;

[0036] Step S4.2: Scale the pixel center coordinates (u, v) according to the scale relationship s to obtain the scaled center coordinates (x, y), as shown in the following formula:

[0037]

[0038] In the formula, u and v represent the x and y coordinates of the center of the circle, respectively, and x and y represent the x and y coordinates of the center of the circle after scaling.

[0039] Step S4.3: Complete the pixel center coordinates with a Z value of 0 to facilitate the subsequent ICP algorithm in calculating the matching relationship;

[0040] Step S4.4: Use the improved ICP algorithm to calculate the matching relationship between the pixel center and the coordinates of the insulation sleeve center in the map;

[0041] Step S4.5: Using the ICP algorithm matching results as input feature points, solve for the camera's rotation matrix R and translation vector T to obtain the camera's position and pose information, as shown in the following formula:

[0042] [RT] = PnP({p i}, {q i})

[0043] Step S4.6: Calculate the robot's position and orientation information based on the relative positional relationship between the camera and the robot.

[0044] Preferably, step S4.4 includes:

[0045] Step S4.4.1: Perform nearest neighbor pairing on the source point set src and the target point set dst to find the nearest neighbor for each source point p. i The nearest neighbor pairing formula for the nearest point in the target point set is as follows:

[0046]

[0047] In the formula, Q represents the target point set, q j Represents each source point p i The nearest point in the target point set;

[0048] Step S4.4.2: Record the number of matches for each target point and count each point;

[0049] Step S4.4.3: If a target point is matched by multiple source points, randomly select a matching result, record the information of other matching results, and then rematch, ignoring the target points that have already been matched during the rematch.

[0050] Step S4.4.4: After obtaining valid matching pairs, use the least squares method to calculate the rigid transformation of the source point set and the target point set to obtain the rotation matrix R and the translation vector T, as shown in the following formula:

[0051]

[0052] Step S4.4.5: Update the source point set coordinates based on the calculated R and T;

[0053] Step S4.4.6: Calculate the error value ∈ for each iteration. iter If the error value is less than the threshold, the iteration is terminated; otherwise, return to step S4.4.1, as shown in the following formula:

[0054]

[0055] Preferably, in each iteration, if there are duplicate matches, the error is recorded recursively and the optimal matching result is selected.

[0056] The error values ​​of different matching schemes are recorded recursively, and the matching result with the smallest error is finally selected.

[0057] Preferably, step S4.6 includes:

[0058] Step S4.6.1: Calculate the robot's position vector using the following formula:

[0059] t robot-world =t cam-world -R cam-world ·t robot-cam Among them, t cam-world Let T and R represent the camera's translation vectors. cam-world R,t represents the rotation matrix of the camera. robot-cam This represents the position vector of the camera relative to the robot;

[0060] Step S4.6.2: Calculate the robot rotation matrix, using the following formula:

[0061]

[0062] in, This represents the transpose of the rotation matrix of the camera relative to the robot.

[0063] Preferably, step S5 includes:

[0064] Step S5.1: Read the sliding window size parameter and initialize the sliding window;

[0065] Step S5.2: Input the robot position vector into the sliding window;

[0066] Step S5.3: Determine whether there is enough data in the sliding window and whether the fluctuation value of the input data exceeds the threshold. If there is insufficient data or the input data is abnormal, end the current step directly and wait for subsequent data; otherwise, proceed to step S5.4.

[0067] Step S5.4: Calculate the smoothed position vector using the moving average formula, as follows:

[0068]

[0069] In the formula, Let represent the smoothed position vector, n represent the size of the sliding window, k represent the starting index of the sliding window, i represent the time ith, and t represent the position vector. k This represents the k-th position vector within the sliding window;

[0070] Step S5.6: Update the sliding window data and output the optimized positioning results.

[0071] An image feature point extraction system provided by the present invention includes:

[0072] Module M1: Construct a feature map and obtain the center coordinates of all circular flared openings of the insulation sleeves;

[0073] Module M2: Image acquisition and preprocessing;

[0074] Module M3: Target recognition and feature point extraction;

[0075] Module M4: Feature point matching and position / pose calculation;

[0076] Module M5: Optimization of positioning results.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] 1. This invention proposes a novel image feature point extraction method that can meet the needs of robot visual navigation and localization in the special scenario of a nuclear power plant roof storage room. Furthermore, it supports the extraction of stable feature points under the condition that the flared opening of the insulation sleeve is the only feature object, and can quickly construct a scene map based on these feature points, thereby achieving high-precision localization.

[0079] 2. This invention can effectively avoid feature point shifts caused by changes in the position and angle of the camera or light source, thereby improving the stability and accuracy of robot visual navigation and positioning.

[0080] 3. This invention can quickly generate environmental maps, providing support for the planning and execution of subsequent tasks. Experiments show that the method proposed in this invention is significantly superior to traditional methods in terms of stability and accuracy, providing a reliable technical solution for robotic operations in the special environment of nuclear power plants. Attached Figure Description

[0081] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0082] Figure 1 This is a schematic diagram of the workflow of the present invention. Detailed Implementation

[0083] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0084] This invention identifies circles (such as the circular insulated sleeve at the top of the storage room under the roof of a nuclear power plant, with the insulated sleeve having a circular, downward-facing flared opening) from a perceived image, calculates the center position of the circle as a stable feature point, and matches it with a constructed map for robot localization. This allows the robot to overcome the limitations of traditional algorithms caused by changes in lighting and viewing angle, achieving stable localization and navigation in complete darkness and a structurally simple storage room under the roof.

[0085] Example 1

[0086] According to the present invention, an image feature point extraction method is provided, such as... Figure 1 As shown, it includes:

[0087] Step S1: Construct a feature map. Step S1 includes the following steps:

[0088] Step S1.1: Obtain the distribution drawing of the thermal insulation sleeve for the storage compartment of the nuclear power plant roof and establish a coordinate system. Establishing the coordinate system includes determining the coordinate system direction and origin. The origin (0, 0, 0) is taken as the entrance of the storage compartment of the nuclear power plant roof. The direction from the entrance to the center of the storage compartment is taken as the positive X-axis. The positive Y-axis is taken as the direction of the positive X-axis rotated 90 degrees counterclockwise. The positive Z-axis is the direction perpendicular to the horizontal ground and pointing upwards.

[0089] Step S1.2: Calculate the center coordinates of the circular flared end of each insulation sleeve in the coordinate system based on the drawing data. The center coordinates include the specific XY coordinate position, and the Z-axis coordinate is uniformly defined as the height of the flared end from the ground.

[0090] Step S1.3: Organize the center coordinates of all the circular flared openings of the insulation sleeves, and save the center coordinates in a .map file according to the text format of insulation sleeve serial number + coordinate value (X, Y, Z).

[0091] Step S2: Image acquisition and preprocessing. Step S2 includes the following steps:

[0092] Step S2.1: Use RTSP to acquire the camera video stream and obtain the latest image as the raw image for perception.

[0093] Step S2.2: Convert the color image to a grayscale image, retaining only the brightness information to reduce data complexity.

[0094] Step S2.3: Apply Gaussian blur to the grayscale image to smooth high-frequency noise while preserving edge features.

[0095] Step S2.4: Use Canny edge detection to extract salient edges in the image.

[0096] Step S2.5: Use the closing operation to fill the break in the circular edge and the opening operation to remove small isolated noise points.

[0097] Step S2.6: Set a threshold and use binarization to convert the image into a black and white image.

[0098] Step S3: Target recognition and feature point extraction. Step S3 includes the following steps:

[0099] Step S3.1: Adjust the Hough algorithm detection parameters according to the different nuclear power plant reactor sizes and camera parameters.

[0100] Step S3.2: Use the Hough algorithm to identify all circular parameters in the binarized image, including the pixel center coordinates and pixel radius.

[0101] Step S3.3: Verify whether the identified circle in the image is standard, optimize the pixel center coordinates, and save the identification result data in memory. Step S3.3 includes: determining the standardity of the circle by calculating the mean square error (MSE) of the circle's edge points. The calculation formula is:

[0102]

[0103] In the formula, MSE represents the mean square error of the edge points of the circle, n represents the number of data points, h and k represent the coordinates of the center of the fitted circle, r represents the radius of the fitted circle, and x... i y i Let represent the coordinates of the i-th data point.

[0104] If the MSE is greater than the threshold, it indicates that the circle is not standard, possibly due to a change in viewing angle making it resemble an ellipse. Gradient descent is used to fit the non-standard circle to an ellipse, minimizing the objective function to obtain more accurate pixel center coordinates. The formula is as follows:

[0105]

[0106] In the formula, a represents the major axis radius of the ellipse, and b represents the minor axis radius of the ellipse.

[0107] Step S4: Feature point matching and position / pose calculation. Step S4 includes:

[0108] Step S4.1: Calculate the scale relationship s based on the camera parameters, using the following formula:

[0109] s = f / Z

[0110] Where f represents the camera's focal length, and Z represents the distance between the camera and the target.

[0111] Step S4.2: Scale the pixel center coordinates (u, v) according to the scale relationship s to obtain the scaled center coordinates (x, y), as shown in the following formula:

[0112]

[0113] In the formula, u and v represent the x and y coordinates of the center of the circle, respectively, and x and y represent the x and y coordinates of the center of the circle after scaling.

[0114] Step S4.3: Complete the pixel center coordinates with a Z value of 0 to facilitate the subsequent ICP algorithm in calculating the matching relationship;

[0115] Step S4.4: Calculate the matching relationship between the pixel center and the coordinates of the insulation sleeve center in the map using the improved ICP algorithm. Step S4.4 includes:

[0116] Step S4.4.1: Perform nearest neighbor pairing on the source point set src and the target point set dst to find the nearest neighbor for each source point p. i The nearest point in the target point set. The nearest neighbor pairing formula is as follows:

[0117]

[0118] In the formula, Q represents the target point set, q j Represents each source point pi The closest point in the target point set.

[0119] Step S4.4.2: Record the number of matches for each target point and count each point.

[0120] Step S4.4.3: If a target point is matched by multiple source points, randomly select a matching result, record the information of other matching results, and then rematch, ignoring the target points that have already been matched during the rematch.

[0121] Step S4.4.4: After obtaining valid matching pairs, use the least squares method to calculate the rigid transformation of the source point set and the target point set to obtain the rotation matrix R and the translation vector T, as shown in the following formula:

[0122]

[0123] Step S4.4.5: Update the source point set coordinates based on the calculated R and T;

[0124] Step S4.4.6: Calculate the error value ∈ for each iteration. iter If the error value is less than the threshold, the iteration is terminated; otherwise, return to step S4.4.1, as shown in the following formula:

[0125]

[0126] In each iteration, if duplicate matches occur, the error is recorded recursively, and the optimal matching result is selected. The error values ​​of different matching schemes are recorded recursively, and the matching result with the smallest error is ultimately selected.

[0127] Step S4.5: Using the ICP algorithm matching results as input feature points, solve for the camera's rotation matrix R and translation vector T to obtain the camera's position and pose information, as shown in the following formula:

[0128] [RT] = PnP({p i}, {q i})

[0129] Step S4.6: Calculate the robot's position and pose information based on the relative positional relationship between the camera and the robot. Step S4.6 includes:

[0130] Step S4.6.1: Calculate the robot's position vector using the following formula:

[0131] t robot-world =t cam-world -R cam-world ·t robot-cam

[0132] Among them, t cam-worldLet T and R represent the camera's translation vectors. cam-world R,t represents the rotation matrix of the camera. robot-cam This represents the position vector of the camera relative to the robot.

[0133] Step S4.6.2: Calculate the robot rotation matrix, using the following formula:

[0134]

[0135] in, This represents the transpose of the rotation matrix of the camera relative to the robot.

[0136] Step S5: Optimize the localization results. Step S5 includes:

[0137] Step S5.1: Read the sliding window size parameter and initialize the sliding window.

[0138] Step S5.2: Input the robot position vector into the sliding window.

[0139] Step S5.3: Determine whether there is enough data in the sliding window and whether the fluctuation value of the input data exceeds the threshold. If there is insufficient data or the input data is abnormal, end the current step directly and wait for subsequent data.

[0140] Step S5.4: Calculate the smoothed position vector using the moving average formula, as follows:

[0141]

[0142] In the formula, Let represent the smoothed position vector, n represent the size of the sliding window, k represent the starting index of the sliding window, i represent the time ith, and t represent the position vector. k This represents the k-th position vector within the sliding window.

[0143] Step S5.6: Update the sliding window data and output the optimized positioning results.

[0144] Furthermore, the specific implementation of this invention in the visual navigation control scenario of special robots is described as follows:

[0145] First, target recognition and feature point extraction are performed: target detection is performed on the original perceived image, and the Hough transform algorithm is used to identify all circular regions in the image that resemble the flared opening of the insulation sleeve. The Hough transform has strong geometric feature recognition capabilities and can accurately extract circular contours under uneven lighting conditions. After extraction, the geometric center (center of the circle) is used as the feature point of the image.

[0146] Then, the position and pose are determined: using the ICP (Iterative Closest Point) algorithm, the extracted 2D image feature points are registered with the actual 3D coordinate points to solve the mapping relationship between the 2D and 3D points. Subsequently, combined with the PnP (Perspective-n-Point) algorithm, the current camera pose information, i.e., the camera's rotation and translation parameters, are calculated. These parameters can be further converted into the robot's overall position and pose data, providing real-time and accurate feedback for robot navigation.

[0147] This invention aims to solve the problem of feature point shift caused by changes in camera or light source position in traditional visual feature point extraction methods by extracting the center position of a circular target as a feature point. Traditional methods rely on small or complex image features (such as corner points or edge points), which lack physical meaning and are easily affected by changes in lighting and viewing angle, leading to a decrease in positioning accuracy. This invention, specifically for the special scenario of a nuclear power plant roof storage room, identifies the center of the circular flared opening of the top insulation sleeve as a stable latent feature point. Even if the camera angle, light source position, or the image is incomplete (only showing part of the circle), the center position remains stable, thereby significantly improving the stability and positioning accuracy of the robot's visual navigation system.

[0148] Example 2

[0149] The present invention also provides an image feature point extraction system, which can be implemented by executing the process steps of the image feature point extraction method. That is, those skilled in the art can understand the image feature point extraction method as a preferred embodiment of the image feature point extraction system.

[0150] An image feature point extraction system provided by the present invention includes:

[0151] Module M1: Construct a feature map and obtain the center coordinates of all circular flared openings of the insulation sleeves;

[0152] Module M2: Image Acquisition and Preprocessing; Module M2 includes: Module M2.1: Using RTSP to acquire camera video stream, acquiring the latest image as the raw image for perception; Module M2.2: Converting the color image to a grayscale image, retaining only brightness information; Module M2.3: Applying Gaussian blur to the grayscale image to smooth high-frequency noise while preserving edge features; Module M2.4: Using Canny edge detection to extract significant edges in the image; Module M2.5: Using a closing operation to fill breaks in circular edges and an opening operation to remove small isolated noise points; Module M2.6: Setting a threshold and using binarization to convert the image to a black and white image.

[0153] Module M3: Target Recognition and Feature Point Extraction; Module M3 includes: Module M3.1: Adjusting Hough algorithm detection parameters according to different nuclear power plant reactor sizes and camera parameters; Module M3.2: Using the Hough algorithm to identify all circular parameters in the binarized image, including pixel center coordinates and pixel radius; Module M3.3: Verifying whether the identified circular results in the image are standard, optimizing pixel center coordinates, and storing the recognition result data in memory. Module M3.3 includes: Calculating the mean square error (MSE) of the circular edge points, using the following formula:

[0154]

[0155] In the formula, MSE represents the mean square error of the edge points of the circle, n represents the number of data points, h and k represent the coordinates of the center of the fitted circle, r represents the radius of the fitted circle, and x... i y i Let represent the coordinates of the i-th data point. If the mean square error (MSE) of the edge points of the circle is greater than the threshold, it indicates that the circle result is not standard. Gradient descent is used to fit an ellipse to the non-standard circle, minimizing the objective function to obtain more accurate pixel center coordinates. The formula is as follows:

[0156]

[0157] In the formula, a represents the major axis radius of the ellipse, and b represents the minor axis radius of the ellipse.

[0158] Module M4: Feature point matching and position / pose calculation; Module M4 includes: Module M4.1: Calculates the scale relationship s based on camera parameters, using the following formula:

[0159] s = f / Z

[0160] Where f represents the camera's focal length, and Z represents the distance between the camera and the target; Module M4.2: Scales the pixel center coordinates (u, v) according to the scale relationship s to obtain the scaled center coordinates (x, y), as shown in the following formula:

[0161]

[0162] In the formula, u and v represent the x and y coordinates of the center of the circle, respectively, and x and y represent the x and y coordinates of the center of the circle after scaling. Module M4.3: Completes the pixel center coordinates with a Z value of 0 to facilitate the subsequent ICP algorithm calculation of the matching relationship. Module M4.4: Uses the improved ICP algorithm to calculate the matching relationship between the pixel center and the center coordinates of the heat insulation sleeve in the map. Module M4.4 includes: Module M4.4.1: Performs nearest neighbor pairing on the source point set src and the target point set dst to find each source point p. i The nearest neighbor pairing formula for the nearest point in the target point set is as follows:

[0163]

[0164] In the formula, Q represents the target point set, q j Represents each source point p i The nearest point in the target point set; Module M4.4.2: Records the number of matches for each target point and counts each point; Module M4.4.3: If a target point is matched by multiple source points, randomly selects a match result, records the information of other match results, and then rematches, ignoring target points that have already been matched during rematching; Module M4.4.4: After obtaining valid matching pairs, uses the least squares method to calculate the rigid transformation of the source point set and the target point set to obtain the rotation matrix R and the translation vector T, as shown in the following formula:

[0165]

[0166] Module M4.4.5: Updates the source point set coordinates based on the calculated R and T; Module M4.4.6: Calculates the error value ∈ [value missing] for each iteration. iter If the error value is less than the threshold, the iteration stops; otherwise, return to module M4.4.1, as shown in the following formula:

[0167]

[0168] In each iteration, if there are duplicate matches, the error is recorded recursively and the optimal matching result is selected. The error values ​​of different matching schemes are recorded recursively, and the matching result with the smallest error is finally selected.

[0169] Module M4.5: Using the ICP algorithm matching results as input feature points, the camera's rotation matrix R and translation vector T are solved to obtain the camera's position and pose information, as shown in the following formula:

[0170] [RT] = PnP({p i}, {q i})

[0171] Module M4.6: Calculates the robot's position and pose information based on the relative positions of the camera and the robot. Module M4.6 includes: Module M4.6.1: Calculates the robot's position vector using the following formula:

[0172] t robot-world =t cam-world -R cam-world ·t robot-cam

[0173] Among them, t cam-world Let T and R represent the camera's translation vectors. cam-world R,t represents the rotation matrix of the camera.robot-cam Represents the camera's position vector relative to the robot; Module M4.6.2: Calculates the robot's rotation matrix, using the following formula:

[0174]

[0175] in, This represents the transpose of the rotation matrix of the camera relative to the robot.

[0176] Module M5: Positioning Result Optimization. Module M5 includes: Module M5.1: Reading the sliding window size parameter and initializing the sliding window; Module M5.2: Inputting the robot position vector into the sliding window; Module M5.3: Determining whether the data in the sliding window is sufficient and whether the fluctuation value of the input data exceeds the threshold. If the data is insufficient or the input data is abnormal, the current module is terminated and subsequent data is awaited; otherwise, module M5.4 is triggered; Module M5.4: Calculating the smoothed position vector using the moving average formula, as follows:

[0177]

[0178] In the formula, Let represent the smoothed position vector, n represent the size of the sliding window, k represent the initial index of the sliding window, i represent the time ith, and t represent the position vector. k This represents the k-th position vector in the sliding window; Module M5.6: Updates the sliding window data and outputs the optimized positioning result.

[0179] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of method step S, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0180] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An image feature point extraction method characterized by, include: Step S1: Construct a feature map and obtain the center coordinates of all circular flared openings of the insulation sleeves; Step S2: Image acquisition and preprocessing; Step S3: Target recognition and feature point extraction; Step S4: Feature point matching and position / pose calculation; Step S5: Optimize the positioning results.

2. The image feature point extraction method according to claim 1, characterized in that, Step S2 includes: Step S2.1: Use RTSP to acquire the camera video stream and obtain the latest image as the raw image for perception; Step S2.2: Convert the color image to a grayscale image, retaining only the brightness information; Step S2.3: Apply Gaussian blur to the grayscale image to smooth high-frequency noise while preserving edge features; Step S2.4: Use Canny edge detection to extract salient edges in the image; Step S2.5: Use the closing operation to fill the break in the circular edge, and use the opening operation to remove small isolated noise points; Step S2.6: Set a threshold and use binarization to convert the image into a black and white image.

3. The image feature point extraction method according to claim 1, characterized in that, Step S3 includes: Step S3.1: Adjust the Hough algorithm detection parameters according to the different nuclear power plant reactor sizes and camera parameters; Step S3.2: Use the Hough algorithm to identify all circular parameters in the binarized image, including pixel center coordinates and pixel radius; Step S3.3: Verify whether the circular results identified in the image are standard, optimize the pixel center coordinates, and save the identification result data in memory.

4. The image feature point extraction method according to claim 3, characterized in that, Step S3.3 includes: calculating the mean square error (MSE) of the circular edge points, using the following formula: In the formula, MSE represents the mean square error of the circular edge points, n represents the number of data points, h and k represent the coordinates of the center of the fitted circle, r represents the radius of the fitted circle, x i , and y i respectively represent the coordinates of the i-th data point. If the mean square error (MSE) of the edge points of the circle is greater than the threshold, it indicates that the circle is not standard. Gradient descent is used to fit an ellipse to the non-standard circle, minimizing the objective function to obtain more accurate pixel center coordinates. The formula is as follows: In the formula, a represents the major axis radius of the ellipse, and b represents the minor axis radius of the ellipse.

5. The image feature point extraction method according to claim 1, characterized in that, Step S4 includes: Step S4.1: Calculate the scale relationship s based on the camera parameters, using the following formula: s = f / Z Where f represents the camera's focal length, and Z represents the distance between the camera and the target; Step S4.2: Scale the pixel center coordinates (u, v) according to the scale relationship s to obtain the scaled center coordinates (x, y), as shown in the following formula: In the formula, u and v represent the x and y coordinates of the center of the circle, respectively, and x and y represent the x and y coordinates of the center of the circle after scaling. Step S4.3: Complete the pixel center coordinates with a Z value of 0 to facilitate the subsequent ICP algorithm in calculating the matching relationship; Step S4.4: Use the improved ICP algorithm to calculate the matching relationship between the pixel center and the coordinates of the insulation sleeve center in the map; Step S4.5: Using the ICP algorithm matching results as input feature points, solve for the camera's rotation matrix R and translation vector T to obtain the camera's position and pose information, as shown in the following formula: [R T] = PnP({p i},{q i}) Step S4.6: Calculate the robot's position and orientation information based on the relative positional relationship between the camera and the robot.

6. The image feature point extraction method according to claim 5, characterized in that, Step S4.4 includes: Step S4.4.1: Nearest-neighbor pairing of the source point set src and the target point set dst, finding the nearest point p i The nearest point in the target point set, the nearest-neighbor pairing formula is as follows: In the formula, Q represents a target point set, q j represents the nearest point of each source point pi in the target point set; Step S4.4.2: Record the number of matches for each target point and count each point; Step S4.4.3: If a target point is matched by multiple source points, randomly select a matching result, record the information of other matching results, and then rematch, ignoring the target points that have already been matched during the rematch. Step S4.4.4: After obtaining valid matching pairs, use the least squares method to calculate the rigid transformation of the source point set and the target point set to obtain the rotation matrix R and the translation vector T, as shown in the following formula: Step S4.4.5: Update the source point set coordinates based on the calculated R and T; Step S4.4.6: Calculate the error value ∈ for each iteration iter If the error value is less than the threshold value, abort the iteration, otherwise, return to step S4.4.1, as follows:

7. The image feature point extraction method according to claim 6, characterized in that, In each iteration, if there are duplicate matches, the error is recorded recursively and the optimal matching result is selected. The error values ​​of different matching schemes are recorded recursively, and the matching result with the smallest error is finally selected.

8. The image feature point extraction method according to claim 1, characterized in that, Step S4.6 includes: Step S4.6.1: Calculate the robot's position vector using the following formula: t robot-world = t cam-world -R cam-world · t robot-cam where t cam-world represents the translation vector T of the camera, R cam-world represents the rotation matrix R of the camera, t robot-cam represents the position vector of the camera relative to the robot; Step S4.6.2: Calculate the robot rotation matrix, using the following formula: in, This represents the transpose of the rotation matrix of the camera relative to the robot.

9. The image feature point extraction method according to claim 1, characterized in that, Step S5 includes: Step S5.1: Read the sliding window size parameter and initialize the sliding window; Step S5.2: Input the robot position vector into the sliding window; Step S5.3: Determine whether there is enough data in the sliding window and whether the fluctuation value of the input data exceeds the threshold. If there is insufficient data or the input data is abnormal, end the current step directly and wait for subsequent data; otherwise, proceed to step S5.

4. Step S5.4: Calculate the smoothed position vector using the moving average formula, as follows: In the formula, Let n represent the smoothed position vector, n represent the size of the sliding window, k represent the index of the sliding window, i represent the time step i, and t represent the position vector. k This represents the k-th position vector within the sliding window; Step S5.6: Update the sliding window data and output the optimized positioning results.

10. An image feature point extraction system, characterized in that, include: Module M1: Construct a feature map and obtain the center coordinates of all circular flared openings of the insulation sleeves; Module M2: Image acquisition and preprocessing; Module M3: Target recognition and feature point extraction; Module M4: Feature point matching and position / pose calculation; Module M5: Optimization of positioning results.

Images