A method and related device for detecting the position of a linkage.

By employing dual-reference calibration and continuous motion trajectory detection, the accuracy problem of linkage motion positioning detection is solved, enabling stable positioning judgment of linkage motion and avoiding misjudgments. This technology is suitable for industrial machine vision and unattended inspection.

CN123118508APending Publication Date: 2026-11-17HUANENG SHANTOU HAIMEN POWER GENERATION CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611296704.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-25
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately detect whether a linkage has fully moved to the target limit position, leading to misjudgments and an inability to distinguish between stable positioning, brief passage through the positioning zone, and being stuck in the middle position. Furthermore, camera vibrations and deviations in the inspection robot contribute to inaccurate detection.

Method used

A dual-reference calibration method is adopted, which uses a fixed reference for the equipment and key points of the linkage. The linkage motion is detected by image correction and continuous motion trajectory. Combined with the state quantity trajectory, it is determined whether the linkage meets the conditions for stable position holding, thus avoiding misjudgment.

Benefits of technology

It improves the accuracy of linkage motion positioning detection, reduces the impact of camera offset and robot deviation, provides quantifiable detection results, and sets an undeterminable state in case of occlusion to facilitate re-acquisition or manual verification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118508A_ABST
    Figure CN123118508A_ABST
Patent Text Reader

Abstract

This invention discloses a method and related apparatus for detecting the position of a linkage, comprising: reading a detection template corresponding to the linkage to be detected; acquiring an image sequence of the linkage to be detected; identifying a first fixed reference point and a second fixed reference point in the image sequence, and correcting the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point; identifying the fixed pivot point and the movable endpoint of the linkage in the corrected image, and calculating the state variables characterizing the position of the linkage; recording the trajectory of the continuous state variables after the start of the action; determining the linkage position result based on the direction of motion, speed of motion, mechanical allowable range, and whether the linkage enters the position area and meets the position stability holding condition according to the trajectory of the continuous state variables. This method and related apparatus can simultaneously detect the linkage action using a fixed reference device, key points of the linkage, and the continuous action trajectory.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of industrial machine vision and unattended inspection technology, and relates to a method and related device for detecting the position of a linkage. Background Technology

[0002] Power plant sites contain numerous valves, dampers, airlocks, and actuators whose actions are transmitted via linkages. After receiving an operating command, the equipment needs to verify that the linkage has moved from its initial position to the target limit position. If the linkage is not in place, valves may not be fully opened or closed, or dampers may not be switched to their correct positions, thus affecting the operation of subsequent equipment.

[0003] Existing technologies typically employ manual inspection, single-frame image classification, bounding box detection, or single-frame key point recognition. Manual inspection results rely on human experience and are discontinuous; single-frame image classification can only provide a category result of whether the object is in position or not, lacking information on angle, displacement, and movement process; bounding boxes cannot accurately represent the pivot and moving end points of slender connecting rods; while single-frame key point recognition can calculate angles, it cannot distinguish between stable positioning, brief passage through the positioning area, being stuck in the middle position, and retraction after positioning.

[0004] Furthermore, camera vibration, robot parking position deviation, and changes in shooting angle can cause an overall shift in the position of the linkage in the image. If a fixed positioning area is directly used in an uncorrected image, the shooting deviation can easily be misinterpreted as mechanical displacement. The linkage may also be temporarily obstructed by pipelines, personnel, or other structures; a single frame recognition failure should not directly lead to a conclusion of equipment malfunction. Therefore, a detection method that simultaneously utilizes a fixed equipment reference, key linkage points, and continuous motion trajectories is needed. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and related device for detecting the position of a linkage. This method and related device can simultaneously detect the linkage action using a fixed equipment reference, key points of the linkage, and continuous motion trajectory.

[0006] To achieve the above objectives, this invention discloses a method for detecting the position of a linkage action, comprising: Read the detection template corresponding to the link to be tested. The detection template includes at least a first fixed reference point and a second fixed reference point located on the equipment body, a link fixed pivot point, a link movable end point, an initial area, a positioning area, a movement direction, a movement time limit, and positioning stability holding conditions. Acquire image sequences of the link to be detected; Identify a first fixed reference point and a second fixed reference point in the image sequence, and correct the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point; Identify the fixed pivot point and the movable endpoint of the link in the corrected image, and calculate the state variables characterizing the position of the link. Record the trajectory of continuous state variables after the start of the action; Based on the motion direction, motion speed, mechanical allowable range, and whether the continuous state quantity trajectory enters the positioning area and meets the positioning stability holding condition, the linkage action positioning result is determined.

[0007] Furthermore, the process of correcting the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point is as follows: Based on the positional relationship between the first and second fixed reference points in the current image and their corresponding fixed reference points in the detection template, translation transformation, rotation and scaling transformation, affine transformation, or perspective transformation are calculated to correct the current image to the template coordinate system of the detection template.

[0008] Furthermore, when both the first and second fixed reference points are successfully identified, a dual-reference point complete correction is performed; when only one fixed reference point is identified, a downgrade correction is performed by combining the device edge direction, historical attitude, or inertial measurement information; when the confidence level of the fixed reference point is lower than a preset threshold, the corresponding image frame is marked as an invalid frame.

[0009] Furthermore, the fixed pivot point and the movable endpoint of the link are obtained through key point detection, line segment detection, template matching, target detection and skeleton extraction or a fusion of the above methods; when the movable endpoint is temporarily occluded, the position of the movable endpoint is predicted based on the position of the previous moment, the length of the link and the direction of movement.

[0010] Furthermore, when the link is rotating, the state quantity is the link angle θ = atan2(y1-y0, x1-x0), where (x0, y0) are the coordinates of the fixed pivot point of the link, and (x1, y1) are the coordinates of the moving endpoint of the link; when the link is moving in a straight line or approximately in a straight line, the state quantity is the projected displacement of the moving endpoint along the direction of motion of a unit vector.

[0011] Furthermore, the linkage action completion result includes action completion, jamming, retraction, timeout failure to complete, and indeterminate; when the rate of change of the state quantity before entering the completion area is continuously lower than the stagnation threshold, it is determined to be jamming; when the state quantity approaches or enters the completion area and the reverse change along the initial direction exceeds the reverse change threshold, it is determined to be retraction; when the action time limit is exceeded and the completion area is still not entered, it is determined to be timeout failure to complete.

[0012] Furthermore, the stable positioning conditions include: the active endpoint or state quantity is located within the positioning area for N consecutive frames or a duration T, and the fluctuation of the state quantity within the N consecutive frames or a duration T is less than a stability threshold; when the fixed reference point or linkage key point fails to be identified continuously for more than a preset time, the output cannot be determined.

[0013] This invention discloses a linkage action positioning detection system, comprising: The image input module is used to read the detection template corresponding to the link to be detected. The detection template includes at least a first fixed reference point and a second fixed reference point located on the device body, a link fixed pivot point, a link movable end point, an initial area, a positioning area, a movement direction, a movement time limit, and positioning stability holding conditions. The template configuration module is used to acquire image sequences of the connecting rod to be detected and to identify the first and second fixed reference points in the image sequence.

[0014] A dual-reference point correction module is used to correct the current image to the template coordinate system of the detection template based on a first fixed reference point and a second fixed reference point; The key point recognition module is used to identify the fixed pivot point and the movable end point of the link in the corrected image, and to calculate the state quantity characterizing the position of the link. The trajectory constraint module is used to record the trajectory of continuous state variables after the start of an action; The positioning and abnormal output module is used to determine the positioning result of the linkage action based on the motion direction, motion speed, mechanical allowable range, and whether it enters the positioning area and meets the positioning stability holding condition of the continuous state quantity trajectory.

[0015] The present invention discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the linkage action positioning detection method.

[0016] The present invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the linkage action positioning detection method.

[0017] The present invention has the following beneficial effects: In practical operation, the linkage motion positioning detection method and related device described in this invention can reduce the impact of camera offset and inspection robot parking deviation on the positioning area judgment through dual-reference correction; the angle or displacement is calculated based on the fixed pivot point and the moving endpoint, and the detection results have quantifiable basis; the continuous trajectory can distinguish between normal positioning, jamming, retreat, and timeout; after entering the positioning area, the stability maintenance condition must be met to avoid misjudging a brief passage as positioning; and an undeterminable state is set for occlusion and recognition failure, which facilitates re-collection or manual verification. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a system structure diagram of the present invention; Figure 2 This is a schematic diagram of the linkage detection template configuration of the present invention; Figure 3 This is a schematic diagram of the dual-reference calibration of the present invention; Figure 4 This is a flowchart of the linkage state quantity trajectory and positioning determination process of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0024] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0025] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0027] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0028] Example 1 The linkage action completion detection method of the present invention includes: Read the detection template corresponding to the link to be tested. The detection template includes at least a first fixed reference point and a second fixed reference point located on the equipment body, a link fixed pivot point, a link movable end point, an initial area, a positioning area, a movement direction, a movement time limit, and positioning stability holding conditions. Acquire image sequences of the link to be detected; Identify a first fixed reference point and a second fixed reference point in the image sequence, and correct the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point; Identify the fixed pivot point and the movable endpoint of the link in the corrected image, and calculate the state variables characterizing the position of the link. Record the trajectory of continuous state variables after the start of the action; Based on the motion direction, motion speed, mechanical allowable range, and whether the continuous state quantity trajectory enters the positioning area and meets the positioning stability holding condition, the linkage action positioning result is determined.

[0029] In this embodiment, the detection template further includes a fixed reference point identification area, a link length range, a mechanically permissible area at the moving end point, a start threshold, a reverse change threshold, a stagnation threshold, and a key point confidence threshold.

[0030] In this embodiment, the process of correcting the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point is as follows: Based on the positional relationship between the first and second fixed reference points in the current image and their corresponding fixed reference points in the detection template, translation transformation, rotation and scaling transformation, affine transformation, or perspective transformation are calculated to correct the current image to the template coordinate system of the detection template.

[0031] In this embodiment, when both the first fixed reference point and the second fixed reference point are successfully identified, a dual-reference point complete correction is performed; when only one fixed reference point is identified, a downgrade correction is performed by combining the device edge direction, historical attitude, or inertial measurement information; when the confidence level of the fixed reference point is lower than a preset threshold, the corresponding image frame is marked as an invalid frame.

[0032] In this embodiment, the fixed pivot point and the movable endpoint of the link are obtained by key point detection, line segment detection, template matching, target detection and skeleton extraction or a fusion of the above methods; when the movable endpoint is temporarily occluded, the position of the movable endpoint is predicted based on the position of the previous moment, the length of the link and the direction of movement.

[0033] In this embodiment, when the link is rotating, the state quantity is the link angle θ = atan2(y1-y0, x1-x0), where (x0, y0) are the coordinates of the fixed axis of rotation of the link, and (x1, y1) are the coordinates of the moving end point of the link; when the link is moving in a straight line or approximately in a straight line, the state quantity is the projected displacement of the moving end point along the direction of movement of a unit vector.

[0034] In this embodiment, when the change of the state quantity relative to the initial state quantity within a consecutive preset number of frames exceeds the start threshold, the linkage action is determined to start and the start time is recorded.

[0035] In this embodiment, the linkage action result includes action in place, jamming, retraction, timeout failure to reach the position, and indeterminate; when the rate of change of the state quantity before entering the position area is continuously lower than the stagnation threshold, it is determined to be jamming; when the state quantity approaches or enters the position area and the reverse change along the initial direction exceeds the reverse change threshold, it is determined to be retraction; when the action time limit is exceeded and the position area is still not reached, it is determined to be timeout failure to reach the position.

[0036] In this embodiment, the stable positioning conditions include: the active endpoint or state quantity is located within the positioning area for N consecutive frames or a duration T, and the fluctuation of the state quantity within the N consecutive frames or a duration T is less than a stability threshold; when the fixed reference point or link key point fails to be identified continuously for more than a preset time, the output cannot be determined.

[0037] Example 2 The linkage action completion detection method of the present invention includes the following steps: Step S1: Create a link detection template. (For example...) Figure 2 As shown, select a first fixed reference point A and a second fixed reference point B on the equipment body that do not change with the movement of the linkage, such as a bolt, nameplate corner, bracket intersection, artificial mark point or stable natural texture point; configure the linkage fixed pivot point P0 and the active end point P1; set the initial area R1, the position area R2, the direction of movement, the mechanically permissible area, the maximum permissible time of movement and the position stability holding time.

[0038] The detection template can also save the fixed reference point identification area, the nominal length of the link, the allowable trajectory range of the moving endpoint, the start threshold, the reverse change threshold, the stagnation threshold, the stability threshold, and the confidence threshold of each key point. The positioning area R2 represents the rectangular or polygonal area of ​​the moving endpoint, the angle range, the displacement range, or the distance range from the moving endpoint to the limit line.

[0039] Step S2, Image Acquisition and Action Triggering. If the system can receive action signals from the DCS, PLC, or inspection task system, it will begin acquiring and analyzing image sequences after the action command. If there is no external signal, it will continuously monitor the linkage status variables. When the change in the status variable relative to the initial value within a preset number of consecutive frames exceeds the start threshold, the action will be determined to start and the start time will be recorded.

[0040] Step S3, dual-reference calibration. (e.g.) Figure 3 As shown, the first fixed reference point A and the second fixed reference point B in the current image are identified respectively. Based on the positional relationship between the first fixed reference point A and the second fixed reference point B and the corresponding points in the template, the translation, rotation, scaling, affine or perspective transformation of the current image to the template coordinate system is calculated, and the current image is mapped to the template coordinate system. Alternatively, the initial area R1, the positioning area R2 and the mechanically permissible area in the template can be mapped to the current image.

[0041] When both the first fixed reference point A and the second fixed reference point B are successfully identified, a complete correction is performed using the positions, connecting directions, and distances between the two fixed reference points A and B. When only one fixed reference point is identified, a downgrade correction can be performed by combining the device edge direction, the attitude at the previous moment, or inertial measurement information. When the confidence levels of both the first fixed reference point A and the second fixed reference point B are below the threshold, the corresponding frame is marked as an invalid frame and is not included in the positioning determination.

[0042] Step S4, Linkage Key Point Identification. Identify the fixed pivot point P0 and the moving endpoint P1 in the corrected image. Identification methods can include key point detection models, line segment detection, template matching, skeleton extraction after target detection, or a fusion of multiple methods. For multi-link mechanisms, a fixed pivot point and a moving endpoint can be set for each link, and consistency checks can be performed on the state variables of each link.

[0043] When the active endpoint P1 is briefly occluded, the position of P1 can be predicted based on the position of the previous frame, the link length, the direction of motion, and the rate of change of state variables, and the confidence of the predicted frame will be reduced. If the duration of occlusion exceeds a set threshold, or if the positional relationship between P0 and P1 does not conform to the constraints of link length and mechanical allowable area, the output cannot determine or identify the anomaly.

[0044] Step S5, State Variable Calculation. For a link moving about a fixed axis, calculate the link's angle θ = atan2(y1-y0, x1-x0), where (x0, y0) are the coordinates of P0 and (x1, y1) are the coordinates of P1. For a link moving in a straight line or approximately in a straight line, calculate the projected displacement s = (P1-Pstart)·u, where Pstart is the initial position of the moving endpoint and u is the unit vector of the motion direction. Use θ or s as the state variable q.

[0045] Step S6: Form a continuous state trajectory. After the action begins, record Q={q(t1),q(t2),…,q(tn)} in chronological order, and simultaneously record the baseline confidence, keypoint confidence, and effective image state at each time point. Median filtering or Kalman filtering can be applied to Q to reduce keypoint jitter while preserving the true action trend.

[0046] Step S7, trajectory constraints. For example... Figure 4 As shown, the system determines whether the state variables are changing in the overall direction towards the target direction based on the direction of motion in the template; if continuous reverse changes exceed the reverse change threshold, it is judged as a reversal. The motion speed is calculated based on adjacent state variables; if the system has not yet entered the positioning area and the speed is continuously lower than the stagnation threshold, it is judged as jamming or intermediate stagnation. If P1 exceeds the mechanically permissible area or the linkage length is abnormal, it is judged as a recognition abnormality or mechanism abnormality.

[0047] Step S8, Position Determination. When the active endpoint P1 or state variable q enters the position region R2, the system does not immediately output "positioned". The system continues to observe for N consecutive frames or a duration T; if P1 or q remains within the position region during this time, and the difference between the maximum and minimum values ​​of the state variable is less than the stability threshold, then the action is determined to be in place. This stability constraint can prevent the linkage from being mistakenly judged as a successful action when it briefly passes through the position region.

[0048] If the maximum allowed time for the action is exceeded and the target area is not reached, the output will show "timeout not reached"; if the target area is between the initial area and the target area for a long time, the output will show "stuck"; if the target area is approached or entered and then returns to the initial direction, the output will show "reverse"; if the fixed reference point or key point continues to fail to be identified, the output will show "cannot be determined" and request re-acquisition or manual review.

[0049] Step S9, Result Output. The system outputs the device name, inspection point, action start time, real-time angle or displacement, target arrival area, status trajectory curve, judgment result, cause of abnormality, representative screenshots, and short video. The results can be correlated with action commands, DCS or PLC feedback signals to form a traceable action confirmation record.

[0050] Example 3 The linkage action positioning detection system of the present invention includes: The image input module is used to read the detection template corresponding to the link to be detected. The detection template includes at least a first fixed reference point and a second fixed reference point located on the device body, a link fixed pivot point, a link movable end point, an initial area, a positioning area, a movement direction, a movement time limit, and positioning stability holding conditions. The template configuration module is used to acquire image sequences of the connecting rod to be detected and to identify the first and second fixed reference points in the image sequence.

[0051] A dual-reference point correction module is used to correct the current image to the template coordinate system of the detection template based on a first fixed reference point and a second fixed reference point; The key point recognition module is used to identify the fixed pivot point and the movable end point of the link in the corrected image, and to calculate the state quantity characterizing the position of the link. The trajectory constraint module is used to record the trajectory of continuous state variables after the start of an action; The positioning and abnormal output module is used to determine the positioning result of the linkage action based on the motion direction, motion speed, mechanical allowable range, and whether it enters the positioning area and meets the positioning stability holding condition of the continuous state quantity trajectory.

[0052] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in each embodiment of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0053] Example 4 A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a linkage motion positioning detection method. For example, the method includes: reading a detection template corresponding to a linkage to be detected, the detection template including at least a first fixed reference point and a second fixed reference point located on the device body, a linkage fixed pivot point, a linkage movable end point, an initial region, a positioning region, a motion direction, a motion time limit, and positioning stability holding conditions; acquiring an image sequence of the linkage to be detected, and identifying the first fixed reference point and the second fixed reference point in the image sequence; correcting the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point; identifying the linkage fixed pivot point and the linkage movable end point in the corrected image, and calculating state quantities characterizing the linkage position; recording a continuous state quantity trajectory after the motion begins; and determining the linkage motion positioning result based on the motion direction, motion speed, mechanical allowable range, and whether the continuous state quantity trajectory enters the positioning region and satisfies the positioning stability holding conditions. The memory may include main memory, such as high-speed random access memory (RAM), or non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus, which may be an industry-standard architecture bus, a peripheral component interconnection standard bus, or an extended industry-standard architecture bus. The bus can be categorized as an address bus, data bus, or control bus. The memory stores programs; specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0054] Example 5 A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a linkage motion positioning detection method. For example, the method includes: reading a detection template corresponding to a linkage to be detected, the detection template including at least a first fixed reference point and a second fixed reference point located on the device body, a linkage fixed pivot point, a linkage movable endpoint, an initial region, a positioning region, a motion direction, a motion time limit, and positioning stability holding conditions; acquiring an image sequence of the linkage to be detected, and identifying the first fixed reference point and the second fixed reference point in the image sequence; correcting the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point; identifying the linkage fixed pivot point and the linkage movable endpoint in the corrected image, and calculating state quantities characterizing the linkage position; recording a continuous state quantity trajectory after the motion begins; and determining the linkage motion positioning result based on the motion direction, motion speed, mechanical allowable range, and whether the continuous state quantity trajectory enters the positioning region and satisfies the positioning stability holding conditions. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0055] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and disclosure of the invention. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0060] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

[0061] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for detecting the position of a linkage, characterized in that, include: Read the detection template corresponding to the link to be tested. The detection template includes at least a first fixed reference point and a second fixed reference point located on the equipment body, a link fixed pivot point, a link movable end point, an initial area, a positioning area, a movement direction, a movement time limit, and positioning stability holding conditions. Acquire image sequences of the link to be detected; Identify a first fixed reference point and a second fixed reference point in the image sequence, and correct the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point; Identify the fixed pivot point and the movable endpoint of the link in the corrected image, and calculate the state variables characterizing the position of the link. Record the trajectory of continuous state variables after the start of the action; Based on the motion direction, motion speed, mechanical allowable range, and whether the continuous state quantity trajectory enters the positioning area and meets the positioning stability holding condition, the linkage action positioning result is determined.

2. The linkage action completion detection method according to claim 1, characterized in that, The process of correcting the current image to the template coordinate system of the detection template based on the first fixed reference point and the second fixed reference point is as follows: Based on the positional relationship between the first and second fixed reference points in the current image and their corresponding fixed reference points in the detection template, translation transformation, rotation and scaling transformation, affine transformation, or perspective transformation are calculated to correct the current image to the template coordinate system of the detection template.

3. The linkage action positioning detection method according to claim 1, characterized in that, When both the first and second fixed reference points are successfully identified, a dual-reference point complete correction is performed. When only one fixed reference point is identified, a downgrade correction is performed by combining the device edge direction, historical attitude, or inertial measurement information. When the confidence level of a fixed reference point is lower than a preset threshold, the corresponding image frame is marked as an invalid frame.

4. The linkage action completion detection method according to claim 1, characterized in that, The fixed pivot point and the movable endpoint of the link are obtained by key point detection, line segment detection, template matching, target detection and skeleton extraction or a fusion of the above methods; when the movable endpoint is temporarily occluded, the position of the movable endpoint is predicted based on the position of the previous moment, the length of the link and the direction of movement.

5. The linkage action completion detection method according to claim 1, characterized in that, When the link is rotating, the state quantity is the link angle θ = atan2(y1-y0, x1-x0), where (x0, y0) are the coordinates of the fixed axis of rotation of the link, and (x1, y1) are the coordinates of the moving end point of the link; when the link is moving in a straight line or approximately in a straight line, the state quantity is the projected displacement of the moving end point along the direction of motion of a unit vector.

6. The linkage action completion detection method according to claim 1, characterized in that, The linkage action results include action in place, jamming, retraction, timeout failure to reach the desired position, and indeterminate. When the rate of change of the state quantity before entering the desired position area is continuously lower than the stagnation threshold, it is determined to be jamming. When the state quantity approaches or enters the desired position area and the reverse change along the initial direction exceeds the reverse change threshold, it is determined to be retraction. When the action time limit is exceeded and the desired position area is still not reached, it is determined to be timeout failure to reach the desired position.

7. The linkage action positioning detection method according to claim 1, characterized in that, The conditions for maintaining stable positioning include: the active endpoint or state quantity is located within the positioning area for N consecutive frames or a duration T, and the fluctuation of the state quantity within the N consecutive frames or a duration T is less than a stability threshold; when the fixed reference point or key link point fails to be identified continuously for more than a preset time, the output cannot be determined.

8. A linkage action positioning detection system, characterized in that, include: The image input module is used to read the detection template corresponding to the link to be detected. The detection template includes at least a first fixed reference point and a second fixed reference point located on the device body, a link fixed pivot point, a link movable end point, an initial area, a positioning area, a movement direction, a movement time limit, and positioning stability holding conditions. The template configuration module is used to acquire image sequences of the link to be detected and identify the first and second fixed reference points in the image sequence. A dual-reference point correction module is used to correct the current image to the template coordinate system of the detection template based on a first fixed reference point and a second fixed reference point; The key point recognition module is used to identify the fixed pivot point and the movable end point of the link in the corrected image, and to calculate the state quantity characterizing the position of the link. The trajectory constraint module is used to record the trajectory of continuous state variables after the start of an action; The positioning and abnormal output module is used to determine the positioning result of the linkage action based on the motion direction, motion speed, mechanical allowable range, and whether it enters the positioning area and meets the positioning stability holding condition of the continuous state quantity trajectory.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the linkage action positioning detection method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the linkage action positioning detection method as described in any one of claims 1-7.