Four-degree-of-freedom visual guidance method based on planar redundant target point group

Abstract

The invention relates to a four-degree-of-freedom visual guidance method based on a planar redundant target point group. The method comprises the following steps of: installing a camera for shooting downwards at the tail end of a Z axis of a rectangular coordinate robot, and installing a group of upward circular visual target spot groups on one side of a target; the distribution of all target spots in the target spot group accords with a certain constraint, especially all target spots must be located on the same space plane, and the centers of the target spots coincide with the center of the target; internal parameters of the camera are obtained through calibration, and a target spot template is established by shooting a calibration plate image and a target spot image under the same pose; after the template is established, a single target picture containing a target spot is shot by using a camera under any pose, so that the three-dimensional pose measurement of the target can be realized. The target point group has redundant design, so that the positioning precision is not influenced by a small number of damaged target points; when damaged target spots exist, the number and serial number information of the damaged target spots can be given by the method; according to the method, three-dimensional pose measurement of the target can be rapidly and stably completed, and motion guidance is provided for other mechanical structures.

Description

Technical Field

[0001] The invention relates to a four-degree-of-freedom visual guidance method based on a planar redundant target point group, and belongs to the field of machine vision. Background Art

[0002] Robots are widely used in the industrial field, replacing manual repetitive or heavy labor and greatly improving production efficiency. In some robot application scenarios, the location of the target is uncertain, and high-performance visual guidance methods are the key technology for the successful application of robots in these scenarios.

[0003] Common visual guidance solutions can be divided into 2D and 3D. The 2D solution can only visually locate the target in a scene where the target moves in a plane. The 3D solution can locate the target in a three-dimensional space, and generally uses a 3D sensor or other three-dimensional imaging technology to achieve target positioning. However, general three-dimensional sensors are relatively expensive, such as laser-based three-dimensional scanners; or they occupy a large space, such as binocular cameras; or their accuracy is not ideal, such as TOF-based three-dimensional cameras. The above three-dimensional data processing algorithms generally have a large amount of calculations and require high processor performance.

[0004] Some 3D vision guidance methods can also achieve guidance based on a single camera and customized targets, but these targets require high manufacturing accuracy, and guidance cannot be completed when the target is damaged. Summary of the invention

[0005] The present invention provides a four-degree-of-freedom visual guidance method based on a planar redundant target group. The method can achieve high-precision four-degree-of-freedom visual guidance using a single camera and a low-cost visual target group. When some targets are damaged, the success rate and accuracy of positioning will not be affected, and a reminder can be given for the damaged targets.

[0006] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0007] A four-degree-of-freedom visual guidance method based on a planar redundant target group comprises the following steps:

[0008] 1) Calibrate the camera and obtain the camera's internal parameters;

[0009] 2) Install the camera and the circular visual target group on the end of the rectangular coordinate robot and the positioning target surface respectively;

[0010] 3) Use tool T1 to perform hand-eye calibration on the rectangular coordinate robot and camera to obtain the relative position and posture relationship between the camera coordinate system and the robot coordinate system;

[0011] 4) Perform an action teaching to obtain the position and posture relationship between the tool coordinate system of the end tool T2 and the robot coordinate system;

[0012] 5) Use the visual calibration plate to establish the target group image template;

[0013] 6) During the online operation phase, the rectangular coordinate robot moves to the working position, and the camera takes a single picture containing the target group, extracts the contour of the target group and calculates its three-dimensional pose based on it.

[0014] The following steps are also included:

[0015] 7) If the target group is partially damaged, its three-dimensional pose is normally output based on step 6), and the number and serial number of the damaged parts are identified.

[0016] The circular visual target point group is composed of four targets located in the same spatial plane, and the center of the target point group coincides with the target center. The target point is circular, highly reflective gray in color, and surrounded by a black static area of ​​fixed width. A two-dimensional coordinate system is constructed based on the spatial plane where the target point group is located, with the center point of the target point group as the origin, and the target directly above it is marked as target point 1, and the other targets are marked as targets 2-4 in a clockwise order. The two-dimensional coordinates of targets 1-4 are (0, L), (L, 0), (0, -L), (-L, -L / 2), respectively, where L is the distance from target point 1 to the center of the target point group.

[0017] The step 3) comprises the following steps:

[0018] 3.1) Fix the calibration tool T1 at the end of the robot's Z axis, place the calibration plate on the surface of the positioning target, and make the center of the calibration plate coincide with the center of the positioning target. At this time, the centers of the calibration plate, the target group, and the positioning target coincide;

[0019] 3.2) The calibration plate is fixed in place, and the mobile robot takes multiple images of the calibration plate, and the robot's end tool T1 coordinates are recorded for each shot. During the shooting, the calibration plate is kept stationary;

[0020] 3.3) Substitute multiple sets of dot array calibration plate images and corresponding coordinates into the hand-eye model, and use a nonlinear optimization algorithm to obtain the pose relationship between the camera coordinate system and the tool coordinate system.

[0021] The hand-eye model is:

[0022]

[0023] Among them, camera_H_cal is the transfer matrix from the camera coordinate system to the calibration plate coordinate system, and camera_H_tool1 is the transfer matrix from the camera coordinate system to the tool T1 coordinate system. It is the inverse matrix of the transfer matrix from the robot base coordinate system to the tool T1 coordinate system, and base_H_cal is the transfer matrix from the base coordinate system to the calibration plate coordinate system.

[0024] The step 4) comprises the following steps:

[0025] 4.1) Keep the calibration plate pose unchanged, and obtain the relationship between the calibration plate coordinate system and the base coordinate system based on camera_H_tool1, and then obtain the pose of the calibration plate in the base coordinate system Pcal = (Xcal, Ycal, Zcal, Rcal);

[0026] 4.2) Translate the calibration plate along its z-axis by a distance L t , get the target group pose Pobj = (Xobj, Yobj, Zobj, Robj), where L t is the thickness of the calibration plate;

[0027] 4.3) Remove the calibration plate, move the robot and move tool T2 to the target position, so that tool T2 can complete the action normally, and record the robot posture P at this time T20 =(XT20,YT20,ZT20,RT20);

[0028] 4.4) Since the target point group coincides with the target center, calculate the posture P of tool T2 to the center point of the Z axis end T2 :

[0029]

[0030] Among them, P obj is the target group pose, is the robot posture P T20 The inverse of.

[0031] The relationship between the calibration plate coordinate system and the base coordinate system base_H_cal is obtained based on camera_H_tool1, specifically:

[0032]

[0033] The step 5) comprises the following steps:

[0034] 5.1) Keep the positioning target position unchanged, move the robot to a suitable position, and take a target group image I obj , so that the image contains all the targets of the target group;

[0035] 5.2) Place the calibration plate on the positioning target again so that the centers of the calibration plate, target point group and positioning target coincide with each other, and take a calibration plate image I cal ;

[0036] 5.3) Identify image I cal The three-dimensional pose P of the calibration plate in cal , P cal The target reference pose P is obtained by translating the distance T along the z-axis direction obj , T is the thickness of the calibration plate;

[0037] 5.4) According to the target number sequence, in I obj The target area is selected in turn, and the selected area is adaptively binarized to extract the target center circle area, and the contour of the central circular area is elliptical fitted, and the fitted contour C is model As a template outline;

[0038] 5.5) Set the possible range of changes in the target group template contour, including rotation angle, image scaling, target contrast, and image pyramid level, and then create the target contour template, and set the reference pose of the contour template to P obj .

[0039] The step 6) is specifically as follows:

[0040] The template matching algorithm based on contour is used to search for the target group template in the image. If the target is not found, the output score = 0 indicates that the target positioning fails. If the target is found, let P′ obj Represents the target pose, and the template contour C model Based on P′ obj Projecting onto the image plane is used to demonstrate the matching effect, and outputting a score between 0.5 and 1. Moving the robot to P′ so that the end tool reaches P′ obj To complete the action, P′ is specifically:

[0041] P′=P′ obj ·P T2

[0042] Among them, P′ bj is the target pose, P T2 is the position of tool T2 relative to the center of the end flange.

[0043] The step 7) comprises the following steps:

[0044] 7.1) Based on the position and posture obtained by positioning the target points, project the template contour onto the image containing the target point group;

[0045] 7.2) Using the circle as the core, expand the outline of each projected target point to form a ring area;

[0046] 7.3) Segment the original image within the annular region and extract the edge of the segmented image;

[0047] 7.4) Fit the edge of each target point contour extracted into an ellipse. If the fit cannot be made or the fitting error is greater than a threshold, it indicates that the target point is damaged. If the fit can be made, compare the fitted ellipse with the target point projection and calculate the distance between the contours. If the difference between the fitted ellipse and the projected template contour is greater than a threshold, it also indicates that the target point is damaged.

[0048] 7.5) Count the number and serial number of damaged targets and output them.

[0049] The present invention has the following beneficial effects and advantages:

[0050] 1. The present invention uses only a single camera and a set of low-cost visual targets to achieve four-degree-of-freedom target positioning and guidance.

[0051] 2. The present invention performs hand-eye calibration on a rectangular coordinate robot with four degrees of freedom: X, Y, Z, and R.

[0052] 3. The present invention will not affect the success rate and accuracy of positioning when some target points are damaged, and can remind the number and number of damaged target points. It has the advantages of strong adaptability, fast speed, high accuracy, low cost and easy maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a diagram of the visual target structure;

[0054] Figure 2 is the structure diagram of the visual target group used in this method;

[0055] Figure 3 is a schematic diagram of using tool T1 to contact the origin of the calibration plate;

[0056] Figure 4 It is a flow chart for extracting the contour of the target group and calculating its three-dimensional pose based on it;

[0057] Figure 5 It is a flow chart for detecting and outputting the number and serial number of damaged target points;

[0058] Figure 6 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0060] The present invention relates to a four-degree-of-freedom visual guidance method based on a planar redundant target group, which has the advantages of strong adaptability, high speed, high precision, low cost and easy maintenance.

[0061] like Figure 6 As shown, a four-degree-of-freedom visual guidance method based on a planar redundant target group includes the following steps:

[0062] (1) Calibrate the camera to obtain the camera's internal parameters;

[0063] (2) A camera is mounted on the end of a rectangular coordinate robot, with the camera facing vertically downward, and a set of circular visual target points are mounted on the surface of the positioning target, with the direction facing upward;

[0064] (3) Perform hand-eye calibration on the rectangular coordinate robot and the camera to obtain the position and posture relationship between the camera coordinate system and the robot coordinate system. A needle-shaped tool T1 of known length is used in this process. Tool T1 is only used for this hand-eye calibration process.

[0065] (4) Perform an action teaching to obtain the position relationship between the tool coordinate system of the end tool T2 and the robot coordinate system; the tool T2 is used for operations after visual positioning of the target, such as docking, grasping, and suction;

[0066] (5) Use the visual calibration plate to establish a target group template;

[0067] (6) During the operation phase, the rectangular coordinate robot moves to the working position, and the camera takes a single picture containing the target group, extracts the contour of the target group, and calculates its three-dimensional pose based on it.

[0068] (7) The target group has a redundant design. If the target group in step (6) is partially damaged, its three-dimensional pose can still be output normally, and the number and number of damaged parts can be identified.

[0069] In step (1), multiple images of the calibration plate at different positions and angles in the field of view are taken during calibration. The internal parameters obtained by calibration are mainly the focal length, pixel size, distortion coefficient, etc. of the camera.

[0070] In step (2), a camera is installed at the end of the rectangular coordinate robot, the camera direction is vertically downward, and a group of circular visual target points are installed on the positioning target surface, the direction is upward;

[0071] The rectangular coordinate robot has four degrees of freedom: X, Y, Z, and R, which are translation in the X direction, translation in the Y direction, translation in the Z direction, and rotation around the Z axis. Correspondingly, the end position of the robot is recorded as P = (X, Y, Z, R).

[0072] The visual target is circular, highly reflective gray, and surrounded by a black static area of ​​fixed width; Figure 1 is the visual target image used.

[0073] The local surface of the target to be located is a plane, and the visual target group is installed on the plane. The structure of the target group is as follows: Figure 2 The target distribution in the target group meets the following constraints:

[0074] ① All targets in the target group are located in the same spatial plane;

[0075] ② Construct a two-dimensional coordinate system based on the spatial plane where the target group is located, with the center point of the target group as the origin, the target directly above as point 1, and targets 2-4 arranged clockwise. The two-dimensional coordinates of targets 1-4 are (0, L), (L, 0), (0, -L), (-L, -L / 2);

[0076] ③ The center of the target group coincides with the center of the target;

[0077] Through the above steps, we define the local coordinate system of the target to coincide with the local coordinate system of the target group.

[0078] In step (3), the hand-eye calibration is performed on the rectangular coordinate robot and the camera. The calibration process uses the same dot calibration plate as step (1). The calibration plate is placed on the surface of the positioning target, and the center of the calibration plate is made to coincide with the center of the positioning target. At this time, the centers of the calibration plate, the target group, and the positioning target coincide. The calibration process is:

[0079] The calibration plate is fixed in place, and the mobile robot takes multiple images of the calibration plate, and records the coordinates of the robot's end tool during each shot. During the shooting, the calibration plate cannot move. Multiple sets of circular dot array calibration plate images and corresponding coordinates are substituted into the hand-eye model, and the pose relationship between the camera coordinate system and the tool coordinate system is obtained through a nonlinear optimization algorithm.

[0080] During the hand-eye calibration process, the hand-eye model is:

[0081]

[0082] Among them, tool T1 is a needle-shaped tool with a known length and vertically downwards, which is installed at the center of the robot's Z-axis end. Its structure is as follows: Figure 3 As shown. camera_H_cal is the transfer matrix from the camera coordinate system to the calibration plate coordinate system, camera_H_tool1 is the transfer matrix from the camera coordinate system to the tool T1 coordinate system, It is the inverse matrix of the transfer matrix from the robot base coordinate system to the tool T1 coordinate system, and base_H_cal is the transfer matrix from the base coordinate system to the calibration plate coordinate system.

[0083] In the above steps, since the robot can only perform translation and rotation movements, the Z-direction translation in base_H_cal cannot be accurately solved. Move the robot so that the end of tool T1 touches the origin of the calibration plate, such as Figure 3 As shown, the Z-direction translation vector in base_H_cal is measured accordingly;

[0084] Tool T1 is only used in the calibration phase. After the calibration is completed, tool T1 is removed.

[0085] In step (4), an action teaching is performed to obtain the posture relationship between the tool coordinate system of the end tool T2 and the robot coordinate system; the tool T2 is a tool used by the robot during long-term operation, and its functions may be docking, grasping, sucking, etc. The relationship between the end tool T2 and the robot coordinate system is determined by the following steps:

[0086] ① Keep the calibration plate posture in step (3) unchanged. At this time, the camera_H_tool1 obtained in step (3) can obtain the relationship between the calibration plate coordinate system and the base coordinate system base_H_cal, and the formula is as follows:

[0087]

[0088] From this, we can further know the position P of the calibration plate in the base coordinate system cal =(X cal ,Y cal ,Z cal ,R cal )

[0089] ②Translate the calibration plate along its z-axis by a distance L t The target group pose P can be obtained obj =(X obj ,Y obj ,Z obj ,R obj ), where L t is the thickness of the calibration plate;

[0090] ③ Remove the calibration plate, move the robot and move tool T2 to the target position, so that tool T2 can complete actions normally, such as grasping, sucking, and fitting. Record the robot posture P at this time T20 =(X T20 ,Y T20 ,Z T20 ,R T20 );

[0091] ④ Since the target point group coincides with the target center, the posture P of the tool T2 to the center point of the Z-axis end can be obtained T2 , the method is

[0092]

[0093] Where P obj is the target group pose obtained in step (2), is the robot posture P obtained in step (3) T20 The inverse of

[0094] Step (5) uses the visual calibration plate to establish a target group template, and the process is as follows:

[0095] ① Keep the positioning target posture in steps (3) and (4) unchanged, move the robot to a suitable position, and take a target group image I obj , which needs to contain all targets of the target group;

[0096] ② Place the calibration plate on the positioning target again. The specific posture and requirements are the same as step (3). At this time, the centers of the calibration plate, target group, and positioning target coincide again. Take a calibration plate image I cal ;

[0097] ③Recognition image I cal The three-dimensional pose of the calibration plate in is recorded as P cal The calibration plate pose and the target group pose only have a Z-direction translation, and the translation amount is the thickness T of the calibration plate. cal The reference position P of the target can be obtained by translating the distance T along the z-axis direction. obj ;

[0098] ④In I obj Select the target area in sequence, and the selection order must be consistent with the target number; perform adaptive binarization on the selected area to extract the target center circle area; perform ellipse fitting on the contour of the central circular area, and use the fitted contour C model As a template outline.

[0099] ⑤ Set the possible range of changes in the target group template contour, including rotation angle range, image zoom range, target contrast range, image pyramid level, etc., and create the target contour template accordingly, and set the reference pose of the contour template to P obj ;

[0100] In step (6), the contours of the target group are extracted and their three-dimensional pose is calculated based on them. Figure 4 The flowchart for calculating the three-dimensional pose is shown in Figure 1. At this stage, the rectangular coordinate robot moves to the working position according to the external sensor data, and taking images at this position can ensure that the complete target group is captured.

[0101] The steps of calculating the target pose according to the target group image are as follows: using the contour-based template matching algorithm to search for the target group template in the image, if no target is found, the output score = 0 indicates that the target positioning fails; if the target is found, let P obj 'Represents the target pose, and the template contour C model Based on P obj'Projection to the image plane is used to show the matching effect, and the output value is between 0.5 and 1; the score is used to indicate the matching degree of the contour. The better the contour matching, the closer the score value is to 1.

[0102] Move the robot to P' so that the end tool reaches P obj 'To complete the action, P' is calculated as follows

[0103] P′=P′ obj ·P T2

[0104] Among them, P′ obj is the target pose, P T2 is the position of tool T2 relative to the center of the end flange;

[0105] In step (7), the damaged target points are detected, and the number and serial number of the damaged target points are output. Figure 5 is the flowchart of this step. According to the position and posture obtained by the target positioning, the template contour is projected onto the image containing the target group. The template contour is a circle, and the projected contour is usually an ellipse. The contour of each projected target is expanded with the circle as the core to form a ring area; the original image in the area is segmented, and the edge of the segmented image is extracted. The edge extracted from each target contour is fitted into an ellipse. If it cannot be fitted or the fitting error is too large, it indicates that the target is damaged. If it can be fitted, the fitted ellipse is compared with the target projection, and the distance between the contours is calculated. If the difference between the fitted ellipse and the projected template contour is too large, it also indicates that the target is damaged. The number and number of damaged targets are counted and output.

Claims

1. A four-degree-of-freedom visual guidance method based on a planar redundant target group, characterized in that: The following steps are involved: 1) Calibrate the camera and obtain the camera's internal parameters; 2) Install the camera and the circular visual target group on the end of the rectangular coordinate robot and the positioning target surface respectively; 3) Use tool T1 to perform hand-eye calibration on the rectangular coordinate robot and camera to obtain the relative position and posture relationship between the camera coordinate system and the robot coordinate system; 4) Perform an action teaching to obtain the position and posture relationship between the tool coordinate system of the end tool T2 and the robot coordinate system; 5) Use the visual calibration plate to establish the target group image template; 6) During the online operation phase, the rectangular coordinate robot moves to the working position, and the camera takes a single picture containing the target group, extracts the contour of the target group and calculates its three-dimensional pose based on it.

2. A four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 1, characterized in that: The following steps are also included: 7) If the target group is partially damaged, its three-dimensional pose is normally output based on step 6), and the number and serial number of the damaged parts are identified.

3. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 1, characterized in that: The circular visual target point group is composed of four targets located in the same spatial plane, and the center of the target point group coincides with the target center. The target point is circular, highly reflective gray in color, and surrounded by a black static area of ​​fixed width. A two-dimensional coordinate system is constructed based on the spatial plane where the target point group is located, with the center point of the target point group as the origin, and the target directly above it is marked as target point 1, and the other targets are marked as targets 2-4 in a clockwise order. The two-dimensional coordinates of targets 1-4 are (0, L), (L, 0), (0, -L), (-L, -L / 2), respectively, where L is the distance from target point 1 to the center of the target point group.

4. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 1, characterized in that: The step 3) comprises the following steps: 3.1) Fix the calibration tool T1 at the end of the robot's Z axis, place the calibration plate on the surface of the positioning target, and make the center of the calibration plate coincide with the center of the positioning target. At this time, the centers of the calibration plate, the target group, and the positioning target coincide; 3.2) The calibration plate is fixed in place, and the mobile robot takes multiple images of the calibration plate, and the robot's end tool T1 coordinates are recorded for each shot. During the shooting, the calibration plate is kept stationary; 3.3) Substitute multiple sets of dot array calibration plate images and corresponding coordinates into the hand-eye model, and use a nonlinear optimization algorithm to obtain the pose relationship between the camera coordinate system and the tool coordinate system.

5. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 4, characterized in that: The hand-eye model is: Among them, camera_H_cal is the transfer matrix from the camera coordinate system to the calibration plate coordinate system, and camera_H_tool1 is the transfer matrix from the camera coordinate system to the tool T1 coordinate system. It is the inverse matrix of the transfer matrix from the robot base coordinate system to the tool T1 coordinate system, and base_H_cal is the transfer matrix from the base coordinate system to the calibration plate coordinate system.

6. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 1, characterized in that: The step 4) comprises the following steps: 4.1) Keep the calibration plate pose unchanged, and obtain the relationship between the calibration plate coordinate system and the base coordinate system based on camera_H_tool1, and then obtain the pose of the calibration plate in the base coordinate system Pcal = (Xcal, Ycal, Zcal, Rcal); 4.2) Translate the calibration plate along its z-axis by a distance L t , get the target group pose Pobj = (Xobj, Yobj, Zobj, Robj), where L t is the thickness of the calibration plate; 4.3) Remove the calibration plate, move the robot and move tool T2 to the target position, so that tool T2 can complete the action normally, and record the robot posture P at this time T20 =(XT20,YT20,ZT20,RT20); 4.4) Since the target point group coincides with the target center, calculate the posture P of tool T2 to the center point of the Z axis end T2 : Among them, P obj is the target group pose, is the robot posture P T20 The inverse of.

7. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 6, characterized in that: The relationship between the calibration plate coordinate system and the base coordinate system base_H_cal is obtained based on camera_H_tool1, specifically:

8. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 1, characterized in that: The step 5) comprises the following steps: 5.1) Keep the positioning target position unchanged, move the robot to a suitable position, and take a target group image I obj , so that the image contains all the targets of the target group; 5.2) Place the calibration plate on the positioning target again so that the centers of the calibration plate, target point group and positioning target coincide with each other, and take a calibration plate image I cal ; 5.3) Identify image I cal The three-dimensional pose P of the calibration plate in cal , P cal The target reference pose P is obtained by translating the distance T along the z-axis direction obj , T is the thickness of the calibration plate; 5.4) According to the target number sequence, in I obj The target area is selected in turn, and the selected area is adaptively binarized to extract the target center circle area, and the contour of the central circular area is elliptical fitted, and the fitted contour C is model As a template outline; 5.5) Set the possible range of changes in the target group template contour, including rotation angle, image scaling, target contrast, and image pyramid level, and then create the target contour template, and set the reference pose of the contour template to P obj .

9. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 1, characterized in that: The step 6) is specifically as follows: The template matching algorithm based on contour is used to search for the target group template in the image. If the target is not found, the output score = 0 indicates that the target positioning fails. If the target is found, let P′ obj Represents the target pose, and the template contour C model Based on P′ obj Projecting onto the image plane is used to demonstrate the matching effect, and outputting a score between 0.5 and 1. Moving the robot to P′ so that the end tool reaches P′ obj To complete the action, P′ is specifically: P′=P′ obj ·P T2 Among them, P′ obj is the target pose, P T2 is the position of tool T2 relative to the center of the end flange.

10. The four-degree-of-freedom visual guidance method based on a planar redundant target group according to claim 2, characterized in that: The step 7) comprises the following steps: 7.1) Based on the position and posture obtained by positioning the target points, project the template contour onto the image containing the target point group; 7.2) Using the circle as the core, expand the outline of each projected target point to form a ring area; 7.3) Segment the original image within the annular region and extract the edge of the segmented image; 7.4) Fit the edge of each target point contour extracted into an ellipse. If the fit cannot be made or the fitting error is greater than a threshold, it indicates that the target point is damaged. If the fit can be made, compare the fitted ellipse with the target point projection and calculate the distance between the contours. If the difference between the fitted ellipse and the projected template contour is greater than a threshold, it also indicates that the target point is damaged. 7.5) Count the number and serial number of damaged targets and output them.

Images