A posture recognition and deviation correction system based on image analysis

Abstract

The application discloses a kind of posture recognition and deviation correction system based on image analysis, it is related to image processing technical field, including image inquiry acquisition module, posture response area distinguishes screening module and deviation correction output module;The image inquiry acquisition module is used to collect the image sequence of the tool to be corrected in reference posture, positive micro-posture inquiry posture and reverse micro-posture inquiry posture, the positive micro-posture inquiry posture and reverse micro-posture inquiry posture are respectively after the posture of small amplitude posture change of the tool to be corrected along known direction and opposite direction, the positive and negative micro-posture inquiry image sequence of the tool to be corrected is generated before posture recognition in the application, and using the positive and negative response difference in the image sequence of real contour, false edge formed by reflection, shielding edge, a plurality of posture candidates are distinguished and screened, so as to obtain real posture deviation correction information.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a pose recognition and deviation correction system based on image analysis. Background Technology

[0002] In industrial processing, assembly inspection, welding positioning, precision clamping, and fruit harvesting, vision systems are commonly used to identify the spatial posture of a tool to be calibrated relative to a target object and generate posture deviation correction information based on the recognition results. The tool to be calibrated can be a robotic arm end effector, welding torch, nozzle, cutting tool, suction head, harvesting actuator, or other processing tools requiring posture control. As industrial environments demand increasing positioning accuracy, motion continuity, and adaptability, image-based posture recognition systems need to stably identify the tool's contour, end-effector orientation, and posture deviations under conditions of narrow field of view, partial occlusion, surface reflection, low tool texture, and close-range interference from the target object, and then provide the recognition results to the subsequent posture correction process.

[0003] Existing image analysis-based pose recognition and deviation correction methods typically rely on contours, edges, key points, depth information, or neural network confidence in a single frame image to determine tool pose. When the tool to be corrected has a reflective, transparent, or low-texture surface, and is located near a workpiece, fixture, branch, weld sidewall, or narrow processing area, the real tool edges, specular highlights, false edges formed by reflections, occluded edges, and unstable changing edges in the image may locally overlap. In this case, the vision system may misidentify the false edges formed by reflections as tool body edges, or it may select the wrong candidate among multiple similar-looking pose candidates, thus generating pose correction information opposite to the true deviation direction.

[0004] This problem becomes apparent when the tool to be calibrated is on a reflective surface, a transparent surface, or a low-texture surface, and when close-range target interference and high-precision pose correction are present simultaneously. Simply increasing the number of training samples, increasing the model complexity, introducing a depth camera, or performing conventional de-highlighting on the image are all insufficient to reliably determine whether suspicious edges in the image originate from the tool itself, false edges formed by reflection, or occluded edges. Summary of the Invention

[0005] The purpose of this invention is to provide an image analysis-based posture recognition and deviation correction system to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a posture recognition and deviation correction system based on image analysis, comprising an image inquiry acquisition module, a posture response differentiation and filtering module, and a deviation correction output module; the image inquiry acquisition module is used to acquire image sequences of the tool to be corrected under a reference posture, a forward micro-posture inquiry posture, and a reverse micro-posture inquiry posture, wherein the forward micro-posture inquiry posture and the reverse micro-posture inquiry posture are the postures of the tool to be corrected after undergoing small posture changes along a known direction and the opposite direction, respectively, so that the true contour, false edges formed by reflection, and occlusion edges of the tool to be corrected form comparable response changes in the image; the posture response differentiation and filtering module is used to extract local image features from the image sequence, generate multiple posture candidates, and filter the multiple posture candidates according to the local image features to obtain the true posture recognition result; the deviation correction output module is used to generate image deviation correction information based on the difference between the true posture recognition result and the target posture, and output the image deviation correction information to the posture correction execution end.

[0007] According to the above technical solution, the image interrogation acquisition module includes a reference image acquisition unit, a micro-attitude interrogation triggering unit, and an interrogation image registration unit. The reference image acquisition unit is used to acquire a reference image after the tool to be corrected enters the correction judgment area. The reference image includes the visible outline, visible end, and local image content of the adjacent target object of the tool to be corrected. The correction judgment area is the image acquisition area where the tool to be corrected approaches the target object but has not yet performed attitude correction. The micro-attitude interrogation triggering unit is used to send a positive micro-attitude interrogation signal and a negative micro-attitude interrogation signal to the tool to be corrected, so that the tool to be corrected can form positive and negative image responses within a range without contacting the target object. The interrogation image registration unit is used to register the reference image, the positive micro-attitude interrogation image, and the negative micro-attitude interrogation image to the same image coordinate system, so that the positions of the same local image features in different images can be compared. The posture response differentiation and filtering module includes a local feature extraction unit, a posture candidate generation unit, a response type classification unit, and a candidate posture filtering unit. The local feature extraction unit extracts edge segments, end points, axis segments, and brightness abrupt change regions from the image. The posture candidate generation unit generates multiple posture candidates based on the edge segments, end points, and axis segments. The response type classification unit classifies local image features into tool body synchronization features, reflection response features, occlusion edge features, and unstable change features based on the reversible response value, response amplitude value, and brightness change value under forward and reverse micro-posture interrogation. The candidate posture filtering unit differentiates and filters multiple posture candidates based on the tool body synchronization features and reflection response features to obtain the true posture recognition result. The deviation correction output module includes a target posture comparison unit, a correction direction determination unit, and a correction information output unit. The target posture comparison unit is used to compare the real posture recognition result with the target posture. The correction direction determination unit is used to determine the posture deviation direction and posture deviation magnitude of the tool to be corrected in the image coordinate system. The correction information output unit is used to output image deviation correction information including posture deviation direction, posture deviation magnitude, and confidence level.

[0008] According to the above technical solution, the workflow of the system includes the following steps: S1. Acquire a reference image of the tool to be calibrated, and extract local image features of the tool to be calibrated and its neighboring target objects from the reference image. Generate multiple pose candidates based on the local image features, instead of directly taking the edge with the highest confidence in a single frame image as the true pose. The multiple pose candidates are multiple possible tool poses temporarily retained from a single frame image. They are used to retain multiple pose candidates that are similar in appearance but different in actual pose under conditions of reflection, occlusion and low texture. Subsequent steps will then filter the true pose from them based on changes in image response. S2, perform forward micro-pose interrogation and reverse micro-pose interrogation on the tool to be calibrated, and acquire forward micro-pose interrogation images and reverse micro-pose interrogation images respectively. The forward micro-pose interrogation and reverse micro-pose interrogation are used to make the real tool outline, false edges formed by reflection and occlusion edges produce different responses in the image sequence. This step is not aimed at completing the processing action, but at forming response changes that can be used for image analysis. S3, the reference image, the forward micro-attitude interrogation image and the reverse micro-attitude interrogation image are registered to the same image coordinate system, and the local image features are tracked in response. Based on the changes in displacement direction, displacement amplitude and brightness of the local image features under forward and reverse micro-attitude interrogation, an image response feature combination is formed. The image response feature combination is used to characterize the correspondence between the local image features and the micro-attitude changes of the tool to be corrected. S4. Based on the combination of image response features, the system divides the tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. The system then uses the division results to filter multiple pose candidates. For pose candidates that are similar in appearance in a single frame but have incorrect response patterns, the system reduces their credibility. For pose candidates that match both the tool body synchronization features and reflection response features, the system determines them as the true pose recognition results. S5. The system compares the real pose recognition result with the target pose to generate image deviation correction information. The image deviation correction information includes the pose deviation direction, pose deviation amplitude and confidence level. The system outputs the image deviation correction information to the pose correction execution terminal so that the subsequent correction process can be adjusted based on the real pose after differentiation and screening.

[0009] According to the above technical solution, step S1 includes the following steps: S1-1, The reference image acquisition unit acquires a reference image when the tool to be calibrated is in the calibration judgment area. ,in, This refers to an image acquired before the tool to be calibrated performed micro-pose interrogation; in the reference image In the middle, the local feature extraction unit extracts the first... Local image features ,in, The index representing the local image feature. This refers to image features formed by edge segments, end points, axis segments, or regions of abrupt changes in brightness. S1-2, Local feature extraction unit determines the first Local image features In the reference image Image coordinates ,in, Indicates the first The two-dimensional position of a local image feature in the image coordinate system; when When it is an edge segment, Let these be the coordinates of the center point of the edge segment; when When it is an end point, Let these be the coordinates of the end point; when When it is an axis segment, The coordinates of the midpoint of this axis segment; S1-3, the attitude candidate generation unit generates the first... based on the visible end, visible axis, and visible contour of the tool to be corrected. Candidate postures ,in, Indicates the sequence number of the attitude candidate. This represents a possible pose of the tool to be corrected in the image coordinate system; the pose candidate It should at least include the tool end position, the tool axis direction, and the tool outer contour direction; S1-4, to avoid prematurely determining a unique pose before the reflective edges, occluded edges, and the true tool outline are distinguished, the pose candidate generation unit first performs an initial mapping between each pose candidate and the visible local image features in the reference image, and only calculates the single-frame matching value used to retain the candidate; the pose candidate generation unit calculates the following formula... Candidate postures Single frame matching value ,in Indicates being the first Each pose candidate is interpreted as a set of local image features of the tool ontology features, the From falling into the first Each pose candidate is composed of local image features within the neighborhood of the tool's outer contour, and whose directional difference from the tool's outer contour is less than a preset directional difference range. Represents a set The number of local image features; Indicates the first The local image features and the first The degree of single-frame matching between the contour direction, end position, or axis direction of each pose candidate, wherein the degree of single-frame matching According to the The local image features and the first The image distance, orientation angle, and end proximity relationships between the corresponding contours of each pose candidate are determined, and The value of increases as the image distance decreases, the directional angle decreases, and the proximity of the ends increases; This represents a stable quantity that prevents the denominator from being zero; this formula is used to determine the first... The image is used to determine whether each pose candidate has the basis for entering the subsequent response differentiation and screening process, rather than to directly output the true pose. S1-5, The attitude candidate generation unit retains the single-frame matching value. Greater than the candidate retention threshold Among the pose candidates, The threshold used to retain pose candidates is indicated; the retained pose candidates enter the image response differentiation and filtering process in steps S2 and S3, which ensures that pose multiple solutions caused by reflective surfaces, transparent surfaces and low-texture surfaces are not prematurely deleted in a single frame stage.

[0010] According to the above technical solution, step S2 includes the following steps: S2-1, the micro-attitude interrogation trigger unit sends a positive micro-attitude interrogation signal to the tool to be calibrated, causing the tool to make a small attitude change along the preset interrogation direction, and acquires a positive micro-attitude interrogation image. ,in, This refers to the image acquired after the tool to be calibrated performs a positive micro-pose interrogation; the preset interrogation direction is the direction in which the tool to be calibrated can produce an observable contour displacement in the image; S2-2, the micro-attitude interrogation triggering unit sends a reverse micro-attitude interrogation signal to the tool to be calibrated, causing the tool to undergo a small attitude change in the opposite direction to the preset interrogation direction, and acquires a reverse micro-attitude interrogation image. ,in, This represents the image acquired after the tool to be calibrated performs a reverse micro-pose interrogation; S2-3, Inquire with the image registration unit to obtain the reference image. Forward micro-gesture query image and reverse micro-pose interrogation image Registered to the same image coordinate system, and for the first Local image features Tracking was conducted to obtain the first The displacement vector of a local image feature under positive micro-pose query and the displacement vector under reverse micro-attitude query ,in, express From the baseline image To query the image with a positive micro-pose Image displacement, express From the baseline image To reverse micro-pose query image Image displacement; S2-4, since the true tool contour should produce image displacements with opposite directions and similar amplitudes under forward and reverse micro-pose queries, if the residual amount after adding the two displacement vectors is small, it indicates that the local image feature is more likely to move synchronously with the tool body to be corrected; the local feature extraction unit calculates the following formula... Reversible response value of a local image feature ,in Indicates the first The displacement vector of a local image feature under positive micro-pose query; Indicates the first The displacement vector of a local image feature under reverse micro-pose interrogation; This represents a stable quantity to prevent the denominator from being zero. This formula is used to convert whether the forward and reverse displacements are opposite responses into comparable values. The higher the reversible response value, the more the local image features match the synchronous motion features of the tool body. S2-5, after determining whether the local image features have a reversible response, it is also necessary to determine whether the response has reached an observable level, in order to avoid mistaking the basically static occlusion edges as tool body features; the local feature extraction unit calculates the following formula... Response amplitude value of a local image feature This formula is used to determine whether local image features produce sufficiently observable displacement in micro-pose interrogation, as occluded edges typically have low response amplitude values.

[0011] According to the above technical solution, step S3 includes the following steps: S3-1, Response type division unit according to the first Reversible response value of a local image feature Response amplitude value and brightness change value Forming the first Image response feature combination of local image features ,in, Indicates the first Local image features in positive micro-pose query image and reverse micro-pose interrogation image The degree of local brightness difference in the image. This represents a response description composed of reversible response, displacement amplitude, and brightness change. S3-2, for local image content formed by specular highlights, false edges due to reflection, and refraction of transparent surfaces, the positional response may be unstable, but obvious brightness changes usually appear under forward and reverse micro-pose queries; therefore, in addition to the displacement response, the local brightness change value is further calculated to identify reflection response features that are not suitable as direct tool contours but can reflect changes in tool surface orientation; the local feature extraction unit calculates the following formula... Brightness variation values ​​of local image features ,in, Indicates the first Local image features in positive micro-pose query image Local average brightness in Indicates the first Local image features in the reverse micro-pose interrogation image Local average brightness in Indicates the first Local image features in the reference image The local average brightness is used to measure the difference in brightness of local image features under forward and reverse micro-pose queries. The higher the brightness change value, the more likely the local image feature is to be affected by reflection, refraction or specular changes. S3-3, Response type division unit based on reversible response value Response amplitude value and brightness change value , will the The local image features are divided into tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. The response type division unit is divided in the order of tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. If the same local image feature has already been classified into a previous type, it will not be classified into a subsequent type again. Greater than the reversible threshold and greater than the displacement threshold At that time, the first The local image features were divided into tool ontology synchronization features, among which... This represents the threshold used to determine whether the forward and reverse responses are reversible. This represents the threshold for determining whether a micro-pose query results in a valid displacement; when Greater than the brightness threshold And the first When a local image feature is not classified as a tool ontology synchronization feature, the first... The local image features are divided into reflectance response features, where... This represents the threshold for determining whether local brightness changes significantly with micro-pose changes; when Less than the resting threshold At that time, the first The local image features were divided into occlusion edge features, where... This represents the threshold used to determine whether local image features are essentially unaffected by changes in the micro-pose of the tool being corrected; when the... When a local image feature does not meet all three of the aforementioned conditions, the local image feature is classified as an unstable change feature. S3-4, the response type division unit uses local image features belonging to the tool body synchronization features as the main matching basis for pose candidates, uses local image features belonging to the reflection response features as the auxiliary basis for surface orientation, and excludes local image features belonging to the occlusion edge features and unstable change features from the main matching basis. This process is not a simple deletion of reflective areas, but rather the transformation of reflection response features into auxiliary information for pose differentiation and screening.

[0012] According to the above technical solution, step S4 includes the following steps: S4-1, the candidate pose selection unit selects candidates based on the... Candidate postures The tool axis direction, tool tip position, and positive micro-attitude query direction are determined. The positive micro-attitude query direction is projected onto the image coordinate system to determine the first... The local image features in the first The predicted displacement direction that should occur when an attitude candidate is valid. ,in, Indicates the first When the first pose candidate is valid, the second pose candidate is valid. Each local image feature should follow the image displacement direction generated by the positive micro-pose query of the tool to be corrected; S4-2, since the forward and reverse micro-pose queries form a set of responses with opposite directions, the displacement difference between the two can highlight the main response direction of the local image feature relative to the query direction; the candidate pose selection unit compares the main response direction with the predicted displacement direction under the corresponding pose candidate, and calculates the first... The local image features relative to the first The direction consistency value of each attitude candidate ,in Indicates the first The displacement vector of a local image feature under positive micro-pose query; Indicates the first The displacement vector of a local image feature under reverse micro-pose interrogation; Indicates the first The predicted displacement direction under the nth pose candidate is used to determine whether the main response direction of the local image features conforms to the nth pose candidate. The tool motion direction corresponding to each posture candidate; S4-3, when calculating the overall credibility of pose candidates, the candidate pose selection unit not only examines whether the pose candidate can explain the synchronization features of the tool body, but also examines whether the pose candidate incorrectly includes occlusion edge features within the scope of the tool body; the former is used to improve the credibility of the pose candidate, and the latter is used to reduce the credibility of the pose candidate; the candidate pose selection unit calculates the following formula... Response discriminant value of each pose candidate ,in Indicates being the first Each pose candidate is interpreted as a set of local image features of the tool ontology features; Represents a set The number of local image features; Indicates being the first The set of occlusion edge features that are mistakenly included in the tool body scope for pose candidates. Features that have already been classified as occluded edge features, but whose positions fall into the first category. Each pose candidate is composed of local image features within the neighborhood of the tool's outer contour; Represents a set The number of local image features; Indicates the first The reversible response value of a local image feature; Indicates the first The local image features relative to the first The orientation consistency value of each pose candidate; This represents a stable quantity to prevent the denominator from being zero. The formula evaluates the consistency between the pose candidate and the real tool contour response through the first term and the degree to which the pose candidate erroneously absorbs occluded edges through the second term, thereby transforming pose candidates with similar appearances in a single frame into pose candidates with distinguishable positive and negative responses. S4-4, When determining the true posture recognition result, the candidate posture screening unit uses the reflection response feature to verify whether the change in the surface orientation of the tool is consistent with the true posture recognition result; when the brightness change direction of the reflection response feature is inconsistent with the surface orientation change corresponding to the true posture recognition result, the candidate posture screening unit reduces the response discrimination value of the posture candidate. This processing makes the reflective area no longer regarded as just interference information, but is transformed into auxiliary image information for judging the true orientation of the tool to be corrected. S4-5, The candidate pose selection unit will respond with the discrimination value. Maximum and greater than the distinguishing threshold The pose candidates are determined as the true pose recognition results, where, This represents the threshold used to distinguish attitude candidates based on their responses. If the distinguishing values ​​of the responses of all attitude candidates are not greater than the distinguishing threshold, then... The candidate pose selection unit outputs a re-acquisition command, causing the image interrogation acquisition module to re-execute the baseline image acquisition and micro-pose interrogation image acquisition.

[0013] According to the above technical solution, step S5 includes the following steps: S5-1, The target pose comparison unit will use the real pose recognition result... With target attitude In comparison, among which, This represents the true attitude of the tool to be corrected, obtained after response differentiation and filtering. This indicates the target posture that the tool to be calibrated should achieve at the current working position; S5-2, since the attitude deviation of the tool to be corrected is mainly manifested as the deviation of the tool tip's working position from the target position and the deviation of the tool axis direction from the target direction, the target attitude comparison unit uses both the difference in tip position and the difference in axis angle as the basis for calculating the image attitude deviation, and calculates the image attitude deviation of the tool to be corrected according to the following formula. ,in Indicates the true pose recognition result The tool tip image position; Indicates the target attitude The target image position at the end of the tool; Indicates the true pose recognition result The tool axis angle in the middle; Indicates the target attitude The target angle of the tool axis in the middle; The proportionality coefficient representing the deviation of the end position; This represents the proportionality coefficient of the axis angle deviation. This formula is used to unify positional and orientation deviations into comparable image attitude deviations. and The range of allowable end position error and allowable axis angle error of the tool to be calibrated in the image coordinate system is determined and remains unchanged in the same batch of pose recognition processes; S5-3, To ensure that the processing object of this system is the image analysis result, the correction direction determination unit does not directly limit the mechanical control method of the attitude correction execution end. Instead, it first generates an image deviation correction vector pointing from the true attitude to the target attitude in the image coordinate system. The correction direction determination unit generates the image deviation correction vector according to the following formula. ,in This indicates the position, direction, and magnitude of the tool tip to be corrected in the image coordinate system. This formula indicates the angle direction and magnitude of the tool axis to be corrected in the image coordinate system. This formula is used to give the posture deviation correction direction in the image coordinate system, so as to avoid the correction direction being reversed due to misjudgment of reflection edges. S5-4, to determine whether the true pose recognition result has sufficient discriminative power relative to other pose candidates, the correction information output unit compares the difference in response discrimination values ​​between the selected pose candidate and the second-best pose candidate; the greater the difference, the more fully the multiple pose solutions have been distinguished, and the more reliable the image deviation correction information is; the correction information output unit calculates the reliability of the image deviation correction information according to the following formula. ,in This indicates the candidate pose number that has been determined as the true pose recognition result. Indicates except The attitude candidate number with the highest external response discrimination value. The response discriminant value represents the pose candidate that has been identified as the true pose recognition result; This represents the response discrimination value of the pose candidate with the highest discrimination value besides the true pose recognition result. This formula is used to evaluate the degree of discrimination between the true pose recognition result and the second-best pose candidate. The greater the discrimination value, the more reliable the image deviation correction information. Not greater than At this time, the correction information output unit sets the confidence level U to zero and outputs a re-interrogation signal; S5-5, The correction information output unit outputs the image deviation correction vector. The image pose deviation E and confidence level U are output as image deviation correction information to the pose correction execution end; when the confidence level U is less than the confidence level threshold... At this time, the correction information output unit does not output the execution-level correction amount, but outputs a re-interrogation signal to the image interrogation acquisition module, wherein, This threshold represents the decision threshold used to determine whether image deviation correction information is sufficient for execution-level correction. Through this process, the system will not directly perform pose correction before pose candidates have been sufficiently distinguished, thereby reducing the risk of reverse correction caused by false edges or occluded edges formed by reflections. The candidate retention threshold is... Reversible threshold Displacement threshold Brightness threshold , rest threshold Distinguishing threshold and credibility threshold The values ​​are all determined based on the local feature response range of the tool to be calibrated in the calibration image without reflective occlusion, and remain unchanged during the same batch of pose recognition processes.

[0014] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention retains multiple possible pose candidates, and then through forward and reverse micro-pose interrogation, makes the true tool contour, reflected highlights, false edges formed by reflection, and occluded edges exhibit different motion responses in continuous images. The true tool contour will undergo reversible and synchronous displacement with the micro-pose changes of the tool to be corrected, while the false edges formed by reflection will produce amplified displacement or asymmetric displacement related to changes in surface orientation, and the occluded edges will not remain synchronized with the tool's micro-pose. Based on this, the system can eliminate pose candidates that appear reasonable in appearance but have incorrect response patterns at the image processing level, avoid mistaking reflected edges for tool edges, reduce the risk of reverse judgment in pose correction direction, and improve the stability of pose recognition and the reliability of deviation correction for tools with reflective, transparent, or low-texture surfaces in partially occluded scenarios. When the distinction between the true pose recognition result and the suboptimal pose candidate is insufficient, the system does not directly output the execution-level correction amount, but instead re-acquires the image response information, thereby avoiding reverse correction or over-correction when pose evidence is insufficient. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the present invention; Figure 2 This is a schematic diagram of the overall modular structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] The terms "reflective surface," "transparent surface," or "low-texture surface" in this paper refer to surfaces on which the tool to be calibrated is unlikely to form stable texture features in a single frame image. These include polished metal surfaces, mirror-coated surfaces, transparent material surfaces, and smooth surfaces with little texture. The term "micro-pose inquiry" in this paper refers to the process by which the tool to be calibrated generates small positive and negative pose changes in a known direction without contacting the target object or performing any actual machining actions. The purpose is to ensure that the tool's true outline, false edges formed by reflections, and occluded edges in the image exhibit different image responses. The term "image response" in this paper refers to the changes in displacement direction, displacement amplitude, and brightness of local image features before and after positive and negative micro-pose inquiries.

[0018] Please see Figure 1 and Figure 2 This invention provides a technical solution: a posture recognition and deviation correction system based on image analysis, comprising an image inquiry acquisition module, a posture response differentiation and filtering module, and a deviation correction output module. The image inquiry acquisition module acquires image sequences of the tool to be corrected under a reference posture, a forward micro-posture inquiry posture, and a reverse micro-posture inquiry posture. The forward micro-posture inquiry posture and the reverse micro-posture inquiry posture represent the postures of the tool to be corrected after small posture changes along a known direction and the opposite direction, respectively, allowing the true contour, false edges formed by reflections, and occlusion edges of the tool to be corrected to form comparable response changes in the image. The posture response differentiation and filtering module extracts local image features from the image sequence, generates multiple posture candidates, and filters these candidates based on the local image features to obtain the true posture recognition result. The deviation correction output module generates image deviation correction information based on the difference between the true posture recognition result and the target posture, and outputs the image deviation correction information to the posture correction execution end. The image interrogation acquisition module includes a reference image acquisition unit, a micro-attitude interrogation triggering unit, and an interrogation image registration unit. The reference image acquisition unit acquires a reference image after the tool to be corrected enters the correction judgment area. The reference image contains the visible outline, visible ends, and local image content of the adjacent target object of the tool to be corrected. The correction judgment area is the image acquisition area where the tool to be corrected approaches the target object but has not yet performed attitude correction. The micro-attitude interrogation triggering unit sends positive and negative micro-attitude interrogation signals to the tool to be corrected, enabling the tool to form positive and negative image responses within a range without contacting the target object. The interrogation image registration unit registers the reference image, the positive micro-attitude interrogation image, and the negative micro-attitude interrogation image to the same image coordinate system, allowing the positions of the same local image features in different images to be compared. The pose response differentiation and filtering module includes a local feature extraction unit, a pose candidate generation unit, a response type classification unit, and a candidate pose filtering unit. The local feature extraction unit is used to extract edge segments, end points, axis segments, and brightness change regions in the image. The pose candidate generation unit is used to generate multiple pose candidates based on edge segments, end points, and axis segments. The response type classification unit is used to classify local image features into tool body synchronization features, reflection response features, occlusion edge features, and unstable change features based on the reversible response value, response amplitude value, and brightness change value of local image features under forward and reverse micro-pose queries. The candidate pose filtering unit is used to differentiate and filter multiple pose candidates based on tool body synchronization features and reflection response features to obtain the true pose recognition result. The deviation correction output module includes a target pose comparison unit, a correction direction determination unit, and a correction information output unit. The target pose comparison unit is used to compare the real pose recognition result with the target pose. The correction direction determination unit is used to determine the pose deviation direction and pose deviation magnitude of the tool to be corrected in the image coordinate system. The correction information output unit is used to output image deviation correction information containing the pose deviation direction, pose deviation magnitude, and confidence level. The system's workflow includes the following steps: S1. Acquire a reference image of the tool to be calibrated and extract local image features of the tool to be calibrated and its neighboring target objects from the reference image. Generate multiple pose candidates based on the local image features, instead of directly taking the edge with the highest confidence in a single frame image as the true pose. The multiple pose candidates are multiple possible tool poses temporarily retained from a single frame image. They are used to retain multiple pose candidates that look similar but have different actual poses under conditions of reflection, occlusion and low texture. Subsequent steps will then filter the true poses from these candidates based on changes in image response. S2, perform forward micro-pose interrogation and reverse micro-pose interrogation on the tool to be calibrated, and acquire forward micro-pose interrogation images and reverse micro-pose interrogation images respectively. Forward micro-pose interrogation and reverse micro-pose interrogation are used to make the real tool outline, false edges formed by reflection and occlusion edges produce different responses in the image sequence. This step is not aimed at completing the processing action, but at forming response changes that can be used for image analysis. S3. The reference image, the forward micro-attitude interrogation image, and the reverse micro-attitude interrogation image are registered to the same image coordinate system, and the local image features are tracked in response. Based on the changes in displacement direction, displacement amplitude, and brightness of the local image features under forward and reverse micro-attitude interrogation, an image response feature combination is formed. The image response feature combination is used to characterize the correspondence between the local image features and the micro-attitude changes of the tool to be corrected. S4. Based on the combination of image response features, the system divides the tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. The system then uses the division results to filter multiple pose candidates. For pose candidates that are similar in appearance in a single frame but have incorrect response patterns, the system reduces their credibility. For pose candidates that match both the tool body synchronization features and reflection response features, the system determines them as the true pose recognition results. S5 compares the real pose recognition result with the target pose and generates image deviation correction information. The image deviation correction information includes the pose deviation direction, pose deviation magnitude and confidence level. The system outputs the image deviation correction information to the pose correction execution end so that the subsequent correction process can be adjusted based on the real pose after differentiation and screening. Step S1 includes the following steps: S1-1, The reference image acquisition unit acquires a reference image when the tool to be calibrated is in the calibration judgment area. ,in, This refers to an image acquired before the tool to be calibrated performed micro-pose interrogation; in the reference image In the middle, the local feature extraction unit extracts the first... Local image features ,in, The index representing the local image feature. This refers to image features formed by edge segments, end points, axis segments, or regions of abrupt changes in brightness. S1-2, Local feature extraction unit determines the first Local image features In the reference image Image coordinates ,in, Indicates the first The two-dimensional position of a local image feature in the image coordinate system; when When it is an edge segment, Let these be the coordinates of the center point of the edge segment; when When it is an end point, Let these be the coordinates of the end point; when When it is an axis segment, The coordinates of the midpoint of this axis segment; S1-3, the attitude candidate generation unit generates the first... based on the visible end, visible axis, and visible contour of the tool to be corrected. Candidate postures ,in, Indicates the sequence number of the attitude candidate. This represents a possible pose of the tool to be corrected in the image coordinate system; pose candidate. It should at least include the tool end position, the tool axis direction, and the tool outer contour direction; S1-4, to avoid prematurely determining a unique pose before the reflective edges, occluded edges, and the true tool outline are distinguished, the pose candidate generation unit first performs an initial mapping between each pose candidate and the visible local image features in the reference image, and only calculates the single-frame matching value used to retain the candidate; the pose candidate generation unit calculates the following formula... Candidate postures Single frame matching value ,in Indicates being the first Each pose candidate is interpreted as a set of local image features of the tool ontology. From falling into the first Each pose candidate is composed of local image features within the neighborhood of the tool's outer contour, and whose directional difference from the tool's outer contour is less than a preset directional difference range. Represents a set The number of local image features; Indicates the first The local image features and the first The degree of single-frame matching between the contour direction, end position, or axis direction of each pose candidate; the degree of single-frame matching. According to the The local image features and the first The image distance, orientation angle, and end proximity relationships between the corresponding contours of each pose candidate are determined, and The value of increases as the image distance decreases, the directional angle decreases, and the proximity of the ends increases; This represents a stable quantity that prevents the denominator from being zero; this formula is used to determine the first... The image is used to determine whether each pose candidate has the basis for entering the subsequent response differentiation and screening process, rather than to directly output the true pose. S1-5, The attitude candidate generation unit retains the single-frame matching value. Greater than the candidate retention threshold Among the pose candidates, The threshold used to retain pose candidates is indicated; the retained pose candidates enter the image response differentiation and filtering process in steps S2 and S3, which prevents multiple pose solutions caused by reflective surfaces, transparent surfaces and low-texture surfaces from being prematurely deleted in a single frame stage. When a tool on a reflective, transparent, or low-texture surface approaches a workpiece, fixture, or narrow target area, the clearest edge in the image may not necessarily originate from the tool itself. It could also come from reflections of the workpiece edge on the tool surface, the boundaries of occluded objects, or abrupt changes in brightness in a highlight area. If the system directly selects the pose with the highest confidence in a single frame using conventional methods, it might initially fix erroneous edges as true contours. Even with high computational accuracy, subsequent correction processes would simply amplify the deviation along the incorrect direction. This step preserves multiple possible tool poses in a single frame image, making the pose judgment temporarily comparable, rather than prematurely outputting a single conclusion. The originality of this approach lies in the fact that the system does not attempt to force a final judgment in a single frame image with insufficient information, but intentionally preserves multiple pose solutions and creates conditions for distinguishing these candidate poses through forward and reverse image responses. Conventional image recognition methods typically aim to output the highest confidence result as quickly as possible, while this solution adopts a delayed judgment approach, transforming the single-frame judgment, which is easily misled by reflections and occlusions, into a subsequent verifiable image response judgment.

[0019] Step S2 includes the following steps: S2-1, the micro-attitude interrogation trigger unit sends a positive micro-attitude interrogation signal to the tool to be calibrated, causing the tool to make a small attitude change along the preset interrogation direction, and acquires a positive micro-attitude interrogation image. ,in, This represents the image acquired after the tool to be calibrated performs a positive micro-pose interrogation; the preset interrogation direction is the direction in which the tool to be calibrated can produce an observable contour displacement in the image. S2-2, the micro-attitude interrogation trigger unit sends a reverse micro-attitude interrogation signal to the tool to be calibrated, causing the tool to make a small attitude change in the opposite direction to the preset interrogation direction, and acquires a reverse micro-attitude interrogation image. ,in, This represents the image acquired after the tool to be calibrated performs a reverse micro-pose interrogation; S2-3, Inquire with the image registration unit to obtain the reference image. Forward micro-gesture query image and reverse micro-pose interrogation image Registered to the same image coordinate system, and for the first Local image features Tracking was conducted to obtain the first The displacement vector of a local image feature under positive micro-pose query and the displacement vector under reverse micro-attitude query ,in, express From the baseline image To query the image with a positive micro-pose Image displacement, express From the baseline image To reverse micro-pose query image Image displacement; S2-4, since the true tool contour should produce image displacements with opposite directions and similar amplitudes under forward and reverse micro-pose queries, if the residual amount after adding the two displacement vectors is small, it indicates that the local image feature is more likely to move synchronously with the tool body to be corrected; the local feature extraction unit calculates the following formula... Reversible response value of a local image feature ,in Indicates the first The displacement vector of a local image feature under positive micro-pose query; Indicates the first The displacement vector of a local image feature under reverse micro-pose interrogation; This represents a stable quantity to prevent the denominator from being zero. This formula is used to convert whether the forward and reverse displacements are opposite responses into comparable values. The higher the reversible response value, the more the local image features match the synchronous motion features of the tool body. S2-5, after determining whether the local image features have a reversible response, it is also necessary to determine whether the response has reached an observable level, in order to avoid mistaking the basically static occlusion edges as tool body features; the local feature extraction unit calculates the following formula... Response amplitude value of a local image feature This formula is used to determine whether local image features produce sufficiently observable displacement in micro-pose interrogation, and occluded edges typically have low response amplitude values; This scheme utilizes subtle changes in the orientation of the tool to be corrected to create distinct responses in consecutive images to the real tool outline, false edges formed by reflections, and occluded edges. The real tool outline, being part of the tool itself, typically exhibits synchronous and reversible displacement in its image position during small, forward and reverse orientation changes. False edges formed by reflections are influenced by both the tool's surface orientation and the surrounding environment, often exhibiting significant brightness variations, unstable positional changes, or movement directions different from the real outline. Occluded edges primarily originate from the workpiece, fixture, or target object itself and usually do not move synchronously with the tool's subtle orientation changes. This scheme leverages the different behaviors of these three types of image content in forward and reverse micro-orientation queries, transforming the source of edges, which is difficult to distinguish in a single frame, into a problem that can be solved through image sequence comparison. The originality of this step compared to conventional techniques lies in the fact that conventional methods typically treat reflections and highlights as noise to suppress, or improve recognition capabilities by adding cameras, training samples, or depth information. This scheme does not simply avoid reflections but actively creates a small, reversible image change process with a known direction, allowing reflections, occlusions, and the real outline to reveal their differences during the change. This avoids relying solely on complex models and also avoids completely entrusting pose determination to the unstable appearance of a single frame.

[0020] Step S3 includes the following steps: S3-1, Response type division unit according to the first Reversible response value of a local image feature Response amplitude value and brightness change value Forming the first Image response feature combination of local image features ,in, Indicates the first Local image features in positive micro-pose query image and reverse micro-pose interrogation image The degree of local brightness difference in the image. This represents a response description composed of reversible response, displacement amplitude, and brightness change. S3-2, for local image content formed by specular highlights, false edges due to reflection, and refraction of transparent surfaces, the positional response may be unstable, but obvious brightness changes usually appear under forward and reverse micro-pose queries; therefore, in addition to the displacement response, the local brightness change value is further calculated to identify reflection response features that are not suitable as direct tool contours but can reflect changes in tool surface orientation; the local feature extraction unit calculates the following formula... Brightness variation values ​​of local image features ,in, Indicates the first Local image features in positive micro-pose query image Local average brightness in Indicates the first Local image features in the reverse micro-pose interrogation image Local average brightness in Indicates the first Local image features in the reference image The local average brightness is used to measure the difference in brightness of local image features under forward and reverse micro-pose queries. The higher the brightness change value, the more likely the local image feature is to be affected by reflection, refraction or specular changes. S3-3, Response type division unit based on reversible response value Response amplitude value and brightness change value , will the The local image features are divided into tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. The response type division unit is divided in the order of tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. If the same local image feature has already been classified into a previous type, it will not be classified into a subsequent type again. Greater than the reversible threshold and greater than the displacement threshold At that time, the first The local image features were divided into tool ontology synchronization features, among which... This represents the threshold used to determine whether the forward and reverse responses are reversible. This represents the threshold for determining whether a micro-pose query results in a valid displacement; when Greater than the brightness threshold And the first When a local image feature is not classified as a tool ontology synchronization feature, the first... The local image features are divided into reflectance response features, where... This represents the threshold for determining whether local brightness changes significantly with micro-pose changes; when Less than the resting threshold At that time, the first The local image features were divided into occlusion edge features, where... This represents the threshold used to determine whether local image features are essentially unaffected by changes in the micro-pose of the tool being corrected; when the... When a local image feature does not meet all three of the aforementioned conditions, the local image feature is classified as an unstable change feature. S3-4, the response type division unit uses local image features belonging to the tool body synchronization features as the main matching basis for pose candidates, uses local image features belonging to the reflection response features as the auxiliary basis for surface orientation, and excludes local image features belonging to the occlusion edge features and unstable change features from the main matching basis. This processing is not simply deleting reflective areas, but transforming the reflection response features into auxiliary information for pose differentiation and filtering. Conventional methods typically involve simply deleting reflective areas or inputting all detected features into the model for scoring. This approach, however, doesn't simply discard reflective information; instead, it retains the reflection response as a separate type of auxiliary information. Thus, the reflection phenomenon, which would normally interfere with pose recognition, is transformed into an auxiliary criterion for judging the reasonableness of the tool's surface orientation. The difference lies in that this approach neither uses the reflection response features as the main contour nor simply deletes them; instead, it uses them in subsequent candidate selection to verify whether the tool's surface orientation corresponding to a particular pose candidate matches the actual image changes. This improves the reliability of reflective tool pose determination without adding extra hardware.

[0021] Step S4 includes the following steps: S4-1, the candidate pose selection unit selects candidates based on the... Candidate postures The tool axis direction, tool tip position, and positive micro-attitude query direction are determined. The positive micro-attitude query direction is projected onto the image coordinate system to determine the first... The local image features in the first The predicted displacement direction that should occur when an attitude candidate is valid. ,in, Indicates the first When the first pose candidate is valid, the second pose candidate is valid. Each local image feature should follow the image displacement direction generated by the positive micro-pose query of the tool to be corrected; S4-2, since the forward and reverse micro-pose queries form a set of responses with opposite directions, the displacement difference between the two can highlight the main response direction of the local image feature relative to the query direction; the candidate pose selection unit compares the main response direction with the predicted displacement direction under the corresponding pose candidate, and calculates the first... The local image features relative to the first The direction consistency value of each attitude candidate ,in Indicates the first The displacement vector of a local image feature under positive micro-pose query; Indicates the first The displacement vector of a local image feature under reverse micro-pose interrogation; Indicates the first The predicted displacement direction under the nth pose candidate is used to determine whether the main response direction of the local image features conforms to the nth pose candidate. The tool motion direction corresponding to each posture candidate; S4-3, when calculating the overall credibility of pose candidates, the candidate pose selection unit not only examines whether the pose candidate can explain the synchronization features of the tool body, but also examines whether the pose candidate incorrectly includes occlusion edge features within the scope of the tool body; the former is used to improve the credibility of the pose candidate, and the latter is used to reduce the credibility of the pose candidate; the candidate pose selection unit calculates the following formula... Response discriminant value of each pose candidate ,in Indicates being the first Each pose candidate is interpreted as a set of local image features of the tool ontology features; Represents a set The number of local image features; Indicates being the first The set of occlusion edge features that are mistakenly included in the tool body scope for pose candidates. Features that have already been classified as occluded edge features, but whose positions fall into the first category. Each pose candidate is composed of local image features within the neighborhood of the tool's outer contour; Represents a set The number of local image features; Indicates the first The reversible response value of a local image feature; Indicates the first The local image features relative to the first The orientation consistency value of each pose candidate; This represents a stable quantity to prevent the denominator from being zero. The formula evaluates the consistency between the pose candidate and the real tool contour response through the first term and the degree to which the pose candidate erroneously absorbs occluded edges through the second term, thereby transforming pose candidates with similar appearances in a single frame into pose candidates with distinguishable positive and negative responses. S4-4, When determining the true posture recognition result, the candidate posture selection unit uses the reflection response features to verify whether the change in the orientation of the tool surface is consistent with the true posture recognition result. When the brightness change direction of the reflection response features is inconsistent with the surface orientation change corresponding to the true posture recognition result, the candidate posture selection unit reduces the response discrimination value of the posture candidate. This processing makes the reflective area no longer regarded as just interference information, but is transformed into auxiliary image information for judging the true orientation of the tool to be corrected. S4-5, The candidate pose selection unit will respond with the discrimination value. Maximum and greater than the distinguishing threshold The pose candidates are determined as the true pose recognition results, where, This represents the threshold used to distinguish attitude candidates based on their responses. If the distinguishing values ​​of the responses of all attitude candidates are not greater than the distinguishing threshold, then... The candidate pose selection unit outputs a re-acquisition command, causing the image interrogation acquisition module to re-execute the baseline image acquisition and micro-pose interrogation image acquisition. The working principle of this step is that if a pose candidate is a true pose, then the corresponding tool contour position, tool axis direction, and tool end position should be able to explain the motion direction of the synchronized features of the tool body, and should not incorrectly include local features already identified as occluded edges within the scope of the tool body. Conversely, if a pose candidate is only supported by false edges formed by reflections or occluded edges, even if it coincides with some edges in a single frame image, it is difficult to maintain consistency in forward and reverse response changes. This scheme distinguishes multiple pose candidates with similar appearances in a single frame precisely through this response relationship.

[0022] The originality of this step compared to conventional techniques lies in the fact that conventional pose recognition typically uses the degree of geometric overlap in a single or multiple frames of images, the confidence level of key points, or the model output score as the judgment criteria. In contrast, this scheme uses whether the candidate pose can explain the image response after controlled micro-pose changes as the selection criterion. In other words, this scheme does not simply increase the complexity of the recognition model, but changes the type of information on which pose judgment is based.

[0023] Step S5 includes the following steps: S5-1, The target pose comparison unit will use the real pose recognition result... With target attitude In comparison, among which, This represents the true attitude of the tool to be corrected, obtained after response differentiation and filtering. This indicates the target posture that the tool to be calibrated should achieve at the current working position; S5-2, since the attitude deviation of the tool to be corrected is mainly manifested as the deviation of the tool tip's working position from the target position and the deviation of the tool axis direction from the target direction, the target attitude comparison unit uses both the difference in tip position and the difference in axis angle as the basis for calculating the image attitude deviation, and calculates the image attitude deviation of the tool to be corrected according to the following formula. ,in Indicates the true pose recognition result The tool tip image position; Indicates the target attitude The target image position at the end of the tool; Indicates the true pose recognition result The tool axis angle in the middle; Indicates the target attitude The target angle of the tool axis in the middle; The proportionality coefficient representing the deviation of the end position; This represents the proportionality coefficient of the axis angle deviation. This formula is used to unify positional and orientation deviations into comparable image attitude deviations. and The range of allowable end position error and allowable axis angle error of the tool to be calibrated in the image coordinate system is determined and remains unchanged in the same batch of pose recognition processes; S5-3, To ensure that the processing object of this system is the image analysis result, the correction direction determination unit does not directly limit the mechanical control method of the attitude correction execution end. Instead, it first generates an image deviation correction vector pointing from the true attitude to the target attitude in the image coordinate system. The correction direction determination unit generates the image deviation correction vector according to the following formula. ,in This indicates the position, direction, and magnitude of the tool tip to be corrected in the image coordinate system. This formula indicates the angle direction and magnitude of the tool axis to be corrected in the image coordinate system. This formula is used to give the posture deviation correction direction in the image coordinate system, so as to avoid the correction direction being reversed due to misjudgment of reflection edges. S5-4, to determine whether the true pose recognition result has sufficient discriminative power relative to other pose candidates, the correction information output unit compares the difference in response discrimination values ​​between the selected pose candidate and the second-best pose candidate; the greater the difference, the more fully the multiple pose solutions have been distinguished, and the more reliable the image deviation correction information is; the correction information output unit calculates the reliability of the image deviation correction information according to the following formula. ,in This indicates the candidate pose number that has been determined as the true pose recognition result. Indicates except The attitude candidate number with the highest external response discrimination value. The response discriminant value represents the pose candidate that has been identified as the true pose recognition result; This represents the response discrimination value of the pose candidate with the highest discrimination value besides the true pose recognition result. This formula is used to evaluate the degree of discrimination between the true pose recognition result and the second-best pose candidate. The greater the discrimination value, the more reliable the image deviation correction information. Not greater than At this time, the correction information output unit sets the confidence level U to zero and outputs a re-interrogation signal; S5-5, The correction information output unit outputs the image deviation correction vector. The image pose deviation E and confidence level U are output as image deviation correction information to the pose correction execution end; when the confidence level U is less than the confidence level threshold... At this time, the correction information output unit does not output the execution-level correction amount, but outputs a re-interrogation signal to the image interrogation acquisition module, wherein, This threshold determines whether image deviation correction information is sufficient to warrant execution-level correction. This process prevents the system from directly performing pose correction before pose candidates are sufficiently distinguished, thereby reducing the risk of reverse correction caused by false edges or occluded edges formed by reflections. (Candidate retention threshold) Reversible threshold Displacement threshold Brightness threshold , rest threshold Distinguishing threshold and credibility threshold The values ​​are all determined based on the local feature response range of the tool to be calibrated in the calibration image without reflective occlusion, and remain unchanged during the same batch of pose recognition processes.

[0024] The most innovative aspect of this implementation is that the system does not simply attribute issues like glare, occlusion, and low texture to poor image quality, nor does it address these problems by simply increasing the complexity of the recognition model. Instead, it changes the basis for pose determination. The system first retains multiple possible poses, then uses forward and reverse micro-pose queries to make the real tool contour, false edges formed by reflections, and occluded edges exhibit different response patterns. Subsequently, it distinguishes pose candidates based on these response patterns. In this way, the source of edges that could not be reliably determined in a single frame image is transformed into a response relationship that can be compared in continuous images.

[0025] If conventional techniques are used, those skilled in the art would typically consider increasing the number of training samples, introducing depth images, enhancing edge detection, filtering out highlight areas, or increasing the camera's field of view. These methods either rely on more hardware and data or still treat reflections as interference to be removed, making it difficult to address the problem of highly similar reflected edges and true contours in a single frame. The difference in this approach is that it actively induces small, reversible, and directional image changes in the tool to be corrected, allowing image features from different sources to self-distinguish through the patterns of change. Based on this differentiation, pose deviation correction information is then output. This processing method is not a simple replacement of conventional image recognition algorithms, but rather a redesign of the way discriminative information is obtained in pose recognition.

[0026] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0027] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A posture recognition and deviation correction system based on image analysis, characterized in that: The system includes an image inquiry acquisition module, a posture response discrimination and filtering module, and a deviation correction output module. The image inquiry acquisition module acquires image sequences of the tool to be corrected in a baseline posture, a forward micro-posture inquiry posture, and a reverse micro-posture inquiry posture. The forward and reverse micro-posture inquiry postures represent the postures of the tool after small posture changes along a known direction and the opposite direction, respectively, allowing for comparable response changes in the image of the tool's true contour, false edges formed by reflections, and occlusion edges. The posture response discrimination and filtering module extracts local image features from the image sequences, generates multiple posture candidates, and filters these candidates based on the local image features to obtain a true posture recognition result. The deviation correction output module generates image deviation correction information based on the difference between the true posture recognition result and the target posture, and outputs this information to the posture correction execution end.

2. The pose recognition and deviation correction system based on image analysis according to claim 1, characterized in that: The image interrogation acquisition module includes a reference image acquisition unit, a micro-attitude interrogation triggering unit, and an interrogation image registration unit. The reference image acquisition unit is used to acquire a reference image after the tool to be corrected enters the correction judgment area. The micro-attitude interrogation triggering unit is used to send a positive micro-attitude interrogation signal and a negative micro-attitude interrogation signal to the tool to be corrected, so that the tool to be corrected can form positive and negative image responses within a range without contacting the target object. The interrogation image registration unit is used to register the reference image, the positive micro-attitude interrogation image, and the negative micro-attitude interrogation image to the same image coordinate system, so that the positions of the same local image features in different images can be compared. The attitude response differentiation and filtering module includes a local feature extraction unit, an attitude candidate generation unit, a response type classification unit, and a candidate attitude filtering unit. The local feature extraction unit is used to extract edge segments, end points, axis segments, and brightness change regions in the image. The attitude candidate generation unit is used to generate multiple attitude candidates based on the edge segments, end points, and axis segments. The response type classification unit is used to classify local image features into tool body synchronization features, reflection response features, occlusion edge features, and unstable change features based on the reversible response value, response amplitude value, and brightness change value of local image features under forward and reverse micro-attitude queries. The candidate pose filtering unit is used to distinguish and filter multiple pose candidates based on the tool body synchronization features and reflection response features to obtain the real pose recognition result. The deviation correction output module includes a target posture comparison unit, a correction direction determination unit, and a correction information output unit. The target posture comparison unit is used to compare the real posture recognition result with the target posture. The correction direction determination unit is used to determine the posture deviation direction and posture deviation magnitude of the tool to be corrected in the image coordinate system. The correction information output unit is used to output image deviation correction information including posture deviation direction, posture deviation magnitude, and confidence level.

3. The pose recognition and deviation correction system based on image analysis according to claim 2, characterized in that: The system's workflow includes the following steps: S1. Acquire a reference image of the tool to be calibrated, and extract local image features of the tool to be calibrated and its neighboring target objects in the reference image. Generate multiple pose candidates based on the local image features. The multiple pose candidates are multiple possible tool poses temporarily retained based on a single frame image. They are used to retain multiple pose candidates that are similar in appearance but different in actual poses under conditions of reflection, occlusion and low texture. S2, perform forward micro-pose interrogation and reverse micro-pose interrogation on the tool to be calibrated, and acquire forward micro-pose interrogation images and reverse micro-pose interrogation images respectively. The forward micro-pose interrogation and reverse micro-pose interrogation are used to make the real tool outline, the false edge formed by reflection and the occlusion edge produce different responses in the image sequence. S3, register the reference image, the forward micro-attitude interrogation image and the reverse micro-attitude interrogation image to the same image coordinate system, and perform response tracking on local image features. Based on the changes in displacement direction, displacement amplitude and brightness of local image features under forward and reverse micro-attitude interrogation, form a combination of image response features. S4. Based on the combination of image response features, the system divides the tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. The system then uses the division results to filter multiple pose candidates. For pose candidates that are similar in appearance in a single frame but have incorrect response patterns, the system reduces their credibility. For pose candidates that match both the tool body synchronization features and reflection response features, the system determines them as the true pose recognition results. S5. The system compares the real pose recognition result with the target pose to generate image deviation correction information. The image deviation correction information includes the pose deviation direction, pose deviation amplitude and confidence level. The system outputs the image deviation correction information to the pose correction execution terminal so that the subsequent correction process can be adjusted based on the real pose after differentiation and screening.

4. The pose recognition and deviation correction system based on image analysis according to claim 3, characterized in that: Step S1 includes the following steps: S1-1, The reference image acquisition unit acquires a reference image when the tool to be calibrated is in the calibration judgment area. ,in, This refers to an image acquired before the tool to be calibrated performed micro-pose interrogation; in the reference image In the middle, the local feature extraction unit extracts the first... Local image features ,in, The index representing the local image feature. This refers to image features formed by edge segments, end points, axis segments, or regions of abrupt changes in brightness. S1-2, Local feature extraction unit determines the first Local image features In the reference image Image coordinates ,in, Indicates the first The two-dimensional position of a local image feature in the image coordinate system; when When it is an edge segment, Let these be the coordinates of the center point of the edge segment; when When it is an end point, Let these be the coordinates of the end point; when When it is an axis segment, The coordinates of the midpoint of this axis segment; S1-3, the attitude candidate generation unit generates the first... based on the visible end, visible axis, and visible contour of the tool to be corrected. Candidate postures ,in, Indicates the sequence number of the attitude candidate. This represents a possible pose of the tool to be corrected in the image coordinate system; the pose candidate It should at least include the tool end position, the tool axis direction, and the tool outer contour direction; S1-4, the attitude candidate generation unit calculates the following formula: Candidate postures Single frame matching value ,in Indicates being the first Each pose candidate is interpreted as a set of local image features of the tool ontology features, the From falling into the first Each pose candidate is composed of local image features within the neighborhood of the tool's outer contour, and whose directional difference from the tool's outer contour is less than a preset directional difference range. Represents a set The number of local image features; Indicates the first The local image features and the first The degree of single-frame matching between the contour direction, end position, or axis direction of each pose candidate; S1-5, The attitude candidate generation unit retains the single-frame matching value. Greater than the candidate retention threshold The pose candidates are selected; the retained pose candidates enter the image response differentiation and filtering process in steps S2 and S3.

5. The pose recognition and deviation correction system based on image analysis according to claim 4, characterized in that: Step S2 includes the following steps: S2-1, the micro-attitude interrogation trigger unit sends a positive micro-attitude interrogation signal to the tool to be calibrated, causing the tool to make a small attitude change along the preset interrogation direction, and acquires a positive micro-attitude interrogation image. ; S2-2, the micro-attitude interrogation triggering unit sends a reverse micro-attitude interrogation signal to the tool to be calibrated, causing the tool to undergo a small attitude change in the opposite direction to the preset interrogation direction, and acquires a reverse micro-attitude interrogation image. ; S2-3, Inquire with the image registration unit to obtain the reference image. Forward micro-gesture query image and reverse micro-pose query image Registered to the same image coordinate system, and for the first Local image features Tracking was conducted to obtain the first The displacement vector of a local image feature under positive micro-pose query and the displacement vector under reverse micro-attitude query ; S2-4, the local feature extraction unit calculates the following formula: Reversible response value of a local image feature , This represents a stable quantity that prevents the denominator from being zero. S2-5, the local feature extraction unit calculates the following formula: Response amplitude value of a local image feature .

6. The pose recognition and deviation correction system based on image analysis according to claim 5, characterized in that: Step S3 includes the following steps: S3-1, Response type division unit according to the first Reversible response value of a local image feature Response amplitude value and brightness change value Forming the first Image response feature combination of local image features ,in, Indicates the first Local image features in positive micro-pose query image and reverse micro-pose query image The degree of local brightness difference in the image. This represents a response description composed of reversible response, displacement amplitude, and brightness change. S3-2, the local feature extraction unit calculates the first according to the following formula. Brightness variation values ​​of local image features ,in, Indicates the first Local image features in positive micro-pose query image Local average brightness in Indicates the first Local image features in the reverse micro-pose interrogation image Local average brightness in Indicates the first Local image features in the reference image Local average brightness; This represents a stable quantity that prevents the denominator from being zero. S3-3, Response type division unit based on reversible response value Response amplitude value and brightness change value , will the The local image features are divided into tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. The response type division unit is divided in the order of tool body synchronization features, reflection response features, occlusion edge features, and unstable change features. If the same local image feature has already been classified into a previous type, it will not be classified into a subsequent type again. Greater than the reversible threshold and greater than the displacement threshold At that time, the first Local image features are divided into tool ontology synchronization features, when Greater than the brightness threshold And the When a local image feature is not classified as a tool ontology synchronization feature, the first... A local image feature is divided into reflection response features, when Less than the resting threshold At that time, the first The local image features are divided into occlusion edge features; when the... When a local image feature does not satisfy all three of the aforementioned conditions, the local image feature is classified as an unstable change feature. S3-4, the response type division unit uses local image features belonging to the tool body synchronization features as the main matching basis for pose candidates, uses local image features belonging to the reflection response features as the auxiliary basis for surface orientation, and excludes local image features belonging to the occlusion edge features and unstable change features from the main matching basis.

7. The pose recognition and deviation correction system based on image analysis according to claim 6, characterized in that: Step S4 includes the following steps: S4-1, the candidate pose selection unit selects candidates based on the... Candidate postures The tool axis direction, tool tip position, and positive micro-attitude query direction are determined. The positive micro-attitude query direction is projected onto the image coordinate system to determine the first... The local image features in the first The predicted displacement direction that should occur when an attitude candidate is valid. ; S4-2, the candidate attitude selection unit compares the main response direction with the predicted displacement direction under the corresponding attitude candidate, and calculates the first... The local image features relative to the first The direction consistency value of each attitude candidate ,in Indicates the first The displacement vector of a local image feature under positive micro-pose query; Indicates the first The displacement vector of a local image feature under reverse micro-pose interrogation; Indicates the first Predicted displacement direction under one attitude candidate; S4-3, the candidate attitude selection unit calculates the following formula: Response discriminant value of each pose candidate ,in Indicates being the first Each pose candidate is interpreted as a set of local image features of the tool ontology features; Represents a set The number of local image features; Indicates being the first The set of occlusion edge features that are mistakenly included in the tool body scope for pose candidates. Features that have already been classified as occluded edge features, but whose positions fall into the first category. Each pose candidate is composed of local image features within the neighborhood of the tool's outer contour; Represents a set The number of local image features; Indicates the first The reversible response value of a local image feature; Indicates the first The local image features relative to the first The orientation consistency value of each pose candidate; S4-4, When determining the true posture recognition result, the candidate posture screening unit uses the reflection response feature to verify whether the change in the orientation of the tool surface is consistent with the true posture recognition result. When the direction of brightness change of the reflection response feature is inconsistent with the surface orientation change corresponding to the true pose recognition result, the candidate pose selection unit reduces the response discrimination value of the pose candidate. S4-5, The candidate pose selection unit will respond with the discrimination value. Maximum and greater than the distinguishing threshold The pose candidates are determined as the true pose recognition results if the discrimination value of the responses of all pose candidates is not greater than the discrimination threshold. The candidate pose selection unit outputs a re-acquisition command, causing the image interrogation acquisition module to re-execute the baseline image acquisition and micro-pose interrogation image acquisition.

8. The pose recognition and deviation correction system based on image analysis according to claim 7, characterized in that: Step S5 includes the following steps: S5-1, The target pose comparison unit will use the real pose recognition result... With target attitude In comparison, among which, This represents the true attitude of the tool to be corrected, obtained after response differentiation and filtering. This indicates the target posture that the tool to be calibrated should achieve at the current working position; S5-2, the target attitude comparison unit uses both end position differences and axis angle differences as the basis for calculating image attitude deviation, and calculates the image attitude deviation of the tool to be corrected according to the following formula. ,in Indicates the true pose recognition result The tool tip image position; Indicates the target attitude The target image position at the end of the tool; Indicates the true pose recognition result Tool axis angle; Indicates the target attitude The target angle of the tool axis in the middle; The proportionality coefficient representing the deviation of the end position; The proportionality factor representing the angular deviation of the axis; S5-3, The correction direction determination unit generates the image deviation correction vector according to the following formula. ,in This indicates the position, direction, and magnitude of the tool tip to be corrected in the image coordinate system. This indicates the angle direction and magnitude of the axis of the tool to be corrected in the image coordinate system that need to be corrected. S5-4, The correction information output unit calculates the reliability of the image deviation correction information according to the following formula. ,in This indicates the candidate pose number that has been determined as the true pose recognition result. Indicates except The attitude candidate number with the highest external response discrimination value. The response discriminant value represents the pose candidate that has been identified as the true pose recognition result; This represents the response discrimination value of the pose candidate with the highest response discrimination value besides the true pose recognition result. Not greater than At this time, the correction information output unit sets the confidence level U to zero and outputs a re-interrogation signal; S5-5, The correction information output unit outputs the image deviation correction vector. The image pose deviation E and confidence level U are output as image deviation correction information to the pose correction execution end; when the confidence level U is less than the confidence level threshold... At this time, the correction information output unit does not output the execution-level correction amount, but outputs a re-interrogation signal to the image interrogation acquisition module, wherein, This represents the threshold used to determine whether image deviation correction information is sufficient to warrant execution-level correction.

Images