A mode recognition-based AI precise identification and grabbing system and method for picking and replacing hooks

Abstract

The present application relates to the technical field of image state recognition, in particular to a kind of based on pattern recognition's picking and replacing hook AI accurate identification and capture system and method, in the present application, the outline change of hook assembly in image sequence frame, boundary difference and gray dynamic are carried out node construction, and combined with edge displacement accumulation analysis, while through graph neural network, cosine value between nodes and coordinate difference are compared jointly, construct path jump sequence, to enhance the response sensitivity of state mutation, under the condition of complex background interference or local occlusion, still can stably extract the key path of morphological evolution, effectively improve the anti-interference and fault tolerance of spatial path recognition, further statistical modeling is carried out on state rate mutation point by hidden Markov model, by matching standard state mode, paragraph merging and invalid segment rejection operation are carried out to abnormal point section, construct state label sequence, realize the accurate division of high confidence, multi-section continuous state.

Description

Technical Field

[0001] This invention relates to the field of image state recognition technology, and in particular to a pattern recognition-based AI-powered precise recognition and grasping system and method for picking and grabbing hooks. Background Technology

[0002] The field of state recognition technology focuses on identifying, judging, and classifying the specific state of target objects or environments through computational models, image processing algorithms, and sensor data. It is widely used in industrial automation, intelligent manufacturing, medical monitoring, and robot vision systems. By combining image segmentation, target detection, and state reasoning models, it extracts the spatial location, morphological features, and physical state of the target and uses this as a basis for subsequent control decisions.

[0003] A pattern recognition-based AI-powered precision identification and grasping system for hook removal and re-hooking is an automated system that integrates artificial intelligence visual recognition and mechanical execution technology. It acquires hook image information through an image acquisition device, uses a pattern recognition algorithm to determine the current state, and drives the end effector to complete the corresponding hook removal or re-hooking operation. This improves the recognition accuracy and grasping stability, reduces the risk of misoperation, and is suitable for application scenarios where the hook is in a complex state, irregularly distributed, or where the working space is limited.

[0004] Existing technologies have shortcomings in recognition accuracy and continuous state tracking. They cannot establish a continuous model of state evolution over time. When the hook body is slightly deformed or obstructed, the static discrimination mechanism cannot distinguish between normal operation and error states, resulting in misjudgment. The recognition mechanism lacks joint analysis of motion path and posture angle, making it difficult to accurately grasp the interaction between state and posture. When performing hook removal operations, if the hook body rotates or rolls slightly, the impact of angle changes on the gripping point is easily missed. As a result, the actuator applies force in an inappropriate direction, producing angular deviation, which leads to hook body falling off or mechanical damage. This significantly reduces recognition efficiency and operational accuracy, and restricts the automation level and operational safety of the overall system. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a pattern recognition-based AI-powered precise identification and grasping system and method for picking and grabbing hooks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A pattern recognition-based AI-powered precise identification and grasping system for de-hooking and re-hooking includes:

[0007] Dynamic node construction module: It acquires a set of image sequence frames during the hook removal and re-hook operation through the camera, performs hook component contour mapping, boundary difference calculation, regional grayscale change measurement and edge displacement accumulation processing, and establishes a sequence node structure;

[0008] State path parsing module: Based on the sequence node structure, a graph neural network is used to jointly compare the node cosine value and coordinate difference, extract the path weight branch, count the state vector jump amplitude and combine them into a sequence to obtain key morphological features;

[0009] State label determination module: Based on the key morphological features, a hidden Markov model is used to screen state rate mutation points, perform standard value matching and classification on state change segments, merge duplicate label segments and remove invalid segments, and establish a polymorphic label sequence.

[0010] Attitude angle estimation module: Based on the polymorphic label sequence, calculate the offset and angle trend of the regional center point, extract the continuous change values ​​of pitch, yaw and roll angles according to the direction channel, and obtain the attitude rotation angle vector set;

[0011] Spatial state fusion module: Based on the attitude rotation angle vector set and polymorphic label sequence, bind the state and attitude combination of each frame, calculate the difference range of repeated combinations and generate joint annotations, and establish a fusion annotation result set.

[0012] As a further aspect of the present invention, the dynamic node construction module includes:

[0013] Image contour mapping submodule: It acquires a set of image sequence frames during the hook removal and re-hook operation through the camera, performs gray-scale change point localization processing in the edge region of the hook component image, performs alignment and trajectory connectivity verification of the same component contour boundary points between adjacent frames, calculates the pixel movement path of boundary points with the same number in consecutive frames and records the corresponding frame index information, and establishes a contour correspondence mapping table.

[0014] Boundary parameter extraction submodule: Based on the contour mapping table, the coordinate difference of the boundary points is obtained and the position change amplitude is calculated. The mean value of gray pixels in each frame within the mapping area is calculated and subtracted from the result of the previous frame. Then, the boundary curvature change and displacement amplitude are superimposed and analyzed to generate displacement change feature group.

[0015] State sequence generation submodule: Based on the displacement change feature group, the feature point values ​​of each frame are combined in time sequence and uniformly transformed into a state vector structure. All state vectors are bound by frame index and reorganized into data blocks in the form of frame sequence. The continuous jump points and directional trend information between frames are marked, and a sequence node structure is established.

[0016] As a further aspect of the present invention, the state path parsing module includes:

[0017] Point association judgment submodule: Based on the sequence node structure, a graph neural network is used to perform numerical cosine comparison between the state vectors of any two nodes and extract the included angle value. The distance change between the coordinates of the node centers is weighted and combined with the included angle value and compared with a preset threshold. Node pairs that meet the connection conditions are retained and a continuous connection mapping relationship is formed to obtain a graph structure connection relationship table.

[0018] Path weight extraction submodule: Based on the graph structure connection relationship table, the change amplitude of state vector between adjacent nodes in all connection paths is statistically analyzed and superimposed to obtain the path weight value. The length and weight product of all paths are calculated and sorted to extract the path with the maximum value. The node index in the branch of the maximum path is completely extracted to obtain the set of adaptive path weights.

[0019] The state segment construction submodule extracts abrupt change points in state values ​​between path node indices based on the adapted path weight set and filters them according to the amplitude threshold. It aggregates the sequence data on both sides of the jump point to form a state change segment, performs continuous segment merging on the segment direction change trend, and obtains key morphological features.

[0020] As a further aspect of the present invention, the graph neural network is configured according to the formula:

[0021] ;

[0022] in: This represents the multidimensional weighted matching value between node i and node j. This represents the state vector of node i after encoding by the graph neural network. This represents the state vector of node j after encoding by the graph neural network. The L2 norm of the state vector of node i. The L2 norm of the state vector of node j. This represents a small constant to prevent division by zero errors. This represents the center coordinate vector of node i in the image space. This represents the center coordinate vector of node j in the image space. This represents the Euclidean distance between node i and node j. The weighting coefficients for the spatial distance components are represented. This represents the absolute value of the difference between the mean rates of displacement change of node i and node j across multiple historical frames. This represents the weighting coefficient of the dynamic difference component. This represents the timestamp index of node i. This represents the timestamp index of node j. This represents the maximum timestamp index in the current input frame sequence. This represents the weighting coefficient for the time decay component.

[0023] As a further aspect of the present invention, the path weight value is composed of the cumulative value of the change amplitude of the state vector between adjacent nodes in the connecting path, and is used to measure the intensity of state change on the entire path.

[0024] The directional change trend is processed by merging consecutive segments. The directional change angle is obtained by calculating the direction vector of the line connecting the center coordinates between the start and end points of each segment, and the included angle threshold is used to determine whether they belong to the same trend interval.

[0025] As a further aspect of the present invention, the status tag determination module includes:

[0026] The mutation point extraction submodule calculates the rate of change of the inter-frame difference of each state vector based on the key morphological features, performs local peak search on the rate values ​​of consecutive frames and performs adjacent difference amplitude filtering, extracts the direction of change and marks the position of the mutation frame after sorting the high amplitude points by index, and obtains the rate mutation candidate group.

[0027] Standard value matching submodule: Based on the rate mutation candidate group, calculate the difference between the state value of each mutation frame and the value of each dimension in the standard label template, determine the difference result and the upper and lower boundary range at the same time, label and encode the points that meet the conditions of all dimension ranges and record the corresponding frame index, and generate a label index structure set;

[0028] The tag sequence construction submodule: Based on the tag index structure set, a hidden Markov model is used to extract the difference between adjacent frames in the continuous tag index and determine the connectivity of segments. The same type of continuous segments are merged and a frame segment identification table is constructed. For segments with a frame distance less than a set threshold, the tag inheritance is merged and zero-label segments are removed to establish a polymorphic tag sequence.

[0029] As a further aspect of the present invention, the hidden Markov model is defined according to the formula:

[0030] ;

[0031] in: Indicates at frame time Below, the tag status is The optimal path probability value, Indicates the time in the previous frame In the middle, the tag status is The optimal path probability value, Indicates from label state Transition to tag state The state transition probability, Indicates state transition from to The compatibility weighting factor in paragraph structure, Indicates the label status In the In the class label pattern, the stability-based weight adjustment coefficient Indicates the label status frame time Observation value The probability of observation, Indicates the label status In the current frame Time series confidence correction factor This represents the total number of states in the set of hidden states labeled. Indicates in frame The label index difference observation features extracted from it.

[0032] As a further aspect of the present invention, the attitude angle calculation module includes:

[0033] Center point determination submodule: Based on the polymorphic label sequence, extract the bounding box of the target region image in each frame, average the coordinates of the upper left and lower right corners of the bounding box and calculate the center of the rectangle, perform Euclidean distance calculation on the coordinates of the rectangle center and the coordinates of the image center point and combine them into a two-dimensional offset vector, and arrange all offset vectors in the order of frame index to obtain a spatial offset vector set;

[0034] Angle trend derivation submodule: Based on the spatial offset vector set, calculate the angle between continuous vectors in groups of three frames, count the absolute value of the angle between the first and last vectors of each group and judge the upward and downward trends, bind the trend information with the corresponding frame index, mark the angle direction trend category of each frame, and generate the direction angle label group.

[0035] Angle channel extraction submodule: Based on the direction angle marking group, all frames are divided into pitch channel, yaw channel and roll channel according to trend category. The amplitude change value of continuous angle trend segment in each channel is extracted and combined into floating segment. After channel number appending processing of each segment, an angle sequence is generated to obtain the attitude rotation angle vector set.

[0036] As a further aspect of the present invention, the spatial state fusion module includes:

[0037] State and attitude binding submodule: Based on the attitude rotation angle vector set and polymorphic label sequence, the state label and angle vector number under each frame index are aligned. By filtering the confidence ranking value and selecting the matching frame number, the label number and the three-axis angle value are concatenated and bound, and a binding index mapping matrix is ​​established to generate frame-level binding label pairs.

[0038] Combination difference calculation submodule: Based on the frame-level bound tag pair, perform numerical combination extraction of all three-axis angle vectors under the same state category, calculate the difference between the maximum and minimum amplitudes of each angle sequence and generate fluctuation interval values, count the frequency of angle changes in each state combination and delineate stable and unstable range segments, and generate attitude difference score set;

[0039] The structure result generation submodule: Based on the attitude difference score set, it performs frame sequence restoration operation on the binding information of each state label and score segment, performs joint sorting of state channel and angle channel on the combined data sequence, fills the state code and angle number of each frame with dual channels and marks them into the frame index column, and establishes a fusion annotation result set.

[0040] A pattern recognition-based AI-powered method for precise identification and capture of hook removal techniques, wherein the method is executed based on the aforementioned pattern recognition-based AI-powered method for precise identification and capture of hook removal techniques, and includes the following steps:

[0041] S1: Based on the set of image sequence frames captured by the camera, extract the edge contour value, boundary gray value, regional gray average value and edge coordinate offset of the hook component in each frame, arrange the contour point set according to the time sequence, calculate the matching position coordinate difference and gray value change of the edge contour points between adjacent frames, superimpose the edge contour offset trajectory and record the corresponding frame number, establish a node index matrix and associate it with the corresponding edge data structure, and establish a sequence node structure;

[0042] S2: Based on the sequence node structure, extract the node state vector and calculate the Euclidean distance between the cosine angle of the edge morphological features and the center coordinates. After weighting the two, filter the node pairs below the threshold to generate a connection index table, construct a path set and count the path jump amplitude, filter the maximum path and extract the frame number of the mutation point, merge the direction segments according to the similarity of the angle, use a graph neural network to construct the connection structure and extract the jump path, obtain the key morphological feature index and classify it into the same group, perform group judgment on the angle of the direction vector between nodes in the same group and merge similar direction segments, and use a graph neural network to construct the structure of the connection relationship between nodes and extract the jump path to obtain the key morphological features;

[0043] S3: Based on the key morphological features, calculate the sequence of node position change rates in each segment and obtain the jump rate value. Compare and match the jump rate value with the set standard state rate reference value segment by segment. Identify the frame number sequence corresponding to the matching segment and mark the state category index. Perform a duplicate segment merging operation on the state category index. After removing segments with unclear classification, use a hidden Markov model to judge the stability of the jump rate and optimize the multi-segment state sequence to obtain a polymorphic label sequence.

[0044] S4: Based on the polymorphic tag sequence, extract the center coordinate point of the corresponding region of each frame and calculate the center point displacement vector value between adjacent frames. Extract the angle between adjacent vectors according to the frame sequence and record the angle trend change value of each directional channel. Separate the pitch angle change sequence, yaw angle change sequence and roll angle change sequence according to the time sequence. Perform continuous segment judgment and directional trend classification of the angle sequence respectively to obtain the attitude rotation angle vector set.

[0045] S5: Based on the attitude rotation angle vector set and polymorphic label sequence, extract the combination value of state label and three-axis attitude angle for each frame, statistically analyze the repeated state and attitude combinations and calculate the range of three-axis angle difference between combinations, index and merge the combination segments with difference range lower than the set tolerance and generate a continuous segment label index table, and establish a fusion annotation result set.

[0046] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0047] 1. In this invention, nodes are constructed by the contour changes, boundary differences and grayscale dynamics of the hook component in the image sequence frame, and combined with edge displacement accumulation analysis, the continuous evolution trajectory of the target state can be captured in a scene without a static reference, so as to realize the complete modeling of the operation action chain;

[0048] 2. In this invention, a graph neural network is used to jointly compare the cosine values ​​and coordinate differences between nodes to construct a path transition sequence, thereby enhancing the response sensitivity to sudden changes in state. Even under complex background interference or local occlusion conditions, it can still stably extract the key path of morphological evolution, effectively improving the anti-interference and fault tolerance of spatial path recognition.

[0049] 3. In this invention, statistical modeling is further performed at the state rate mutation point by using a hidden Markov model. By matching standard state patterns, segment merging and invalid segment removal operations are performed on abnormal segments to construct a state label sequence, thereby achieving accurate division of high-confidence, multi-segment continuous states and improving recognition accuracy, response speed and system reliability. Attached Figure Description

[0050] Figure 1 This is a system flowchart of an AI-based precision identification and grasping system for descrambling and reassembling hooks based on pattern recognition, according to the present invention.

[0051] Figure 2 This is a schematic diagram of the system framework of the AI-based precise identification and grasping system for descrambling and reassembling hooks based on pattern recognition according to the present invention;

[0052] Figure 3 This is a schematic diagram illustrating the steps of a pattern recognition-based AI-powered method for precise identification and grasping of hooks and clasps according to the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] Please see Figure 1 This invention provides a technical solution: a pattern recognition-based AI-powered precise identification and grasping system for debonding hooks, comprising:

[0055] Dynamic node construction module: It acquires a set of image sequence frames during the hook removal and re-hook operation through the camera, performs hook component contour mapping, boundary difference calculation, regional grayscale change measurement and edge displacement accumulation processing, and establishes a sequence node structure;

[0056] State path parsing module: Based on the sequence node structure, it uses a graph neural network to jointly compare the node cosine value and coordinate difference, extract path weight branches, count the jump amplitude of the state vector and combine them into a sequence to obtain key morphological features;

[0057] State label determination module: Based on key morphological features, a hidden Markov model is used to screen state rate mutation points, perform standard value matching and classification on state change segments, merge duplicate label segments and remove invalid segments, and establish a polymorphic label sequence.

[0058] Attitude angle estimation module: Based on the polymorphic label sequence, calculate the offset and angle trend of the regional center point, extract the continuous change values ​​of pitch, yaw and roll angles according to the direction channel, and obtain the attitude rotation angle vector set;

[0059] Spatial state fusion module: Based on the attitude rotation angle vector set and polymorphic label sequence, bind the state and attitude combination of each frame, calculate the difference range of repeated combinations and generate joint annotations, and establish a fusion annotation result set.

[0060] Please see Figure 2 The dynamic node construction module includes:

[0061] Image contour mapping submodule: It acquires a set of image sequence frames during the hook removal and re-hook operation through the camera, performs gray-scale change point localization processing in the edge region of the hook component image, performs alignment and trajectory connectivity verification of the same component contour boundary points between adjacent frames, calculates the pixel movement path of boundary points with the same number in consecutive frames and records the corresponding frame index information, and establishes a contour correspondence mapping table.

[0062] Boundary parameter extraction submodule: Based on the contour correspondence mapping table, the coordinate difference of the boundary point is obtained and the position change amplitude is calculated. The mean value of gray-scale pixels in each frame within the mapping area is calculated and subtracted from the result of the previous frame. Then, the boundary curvature change and displacement amplitude are superimposed and analyzed to generate displacement change feature group.

[0063] State sequence generation submodule: Based on displacement change feature group, the feature point values ​​of each frame are combined in time sequence and uniformly transformed into state vector structure. All state vectors are bound by frame index and reorganized into data blocks in the form of frame sequence. The continuous jump points and directional trend information between frames are marked, and the sequence node structure is established.

[0064] Image contour mapping submodule: Based on the set of image sequence frames acquired by the camera during the hook removal and reattachment operation, the Canny edge detection method is used to perform gradient magnitude calculation on the image matrix and set a low threshold of 50 and a high threshold of 150 to extract edge pixels in gray-level change regions. The detected edge pixels are assigned unique point indexes through a label mapping function to locate gray-level change points in the edge regions of the hook component image. The SIFT feature point matching method is used to call the scale space extreme value detection function on the edge contour point set in adjacent frames, select extreme points in the DoG image and use a 128-dimensional local descriptor for point feature matching. After matching, the Euclidean distance between the point pairs is calculated and a threshold of 0.8 times the minimum distance value is set for matching and filtering. The alignment of the same component contour boundary points between adjacent frames and the trajectory connectivity verification are performed. The pixel coordinate change path of the boundary points with the same number in consecutive frames is calculated. The corresponding index information in each frame is extracted and written into the trajectory dictionary structure to establish a contour correspondence mapping table.

[0065] Boundary parameter extraction submodule: Based on the contour mapping table, the module uses a combination of the pointPolygonTest and curvature functions in OpenCV to perform position curvature quantization calculation on the boundary point set. It extracts the coordinate difference of the same point number between adjacent frames in the X and Y axes and performs square root processing on the sum of square differences to obtain the position change amplitude value. It uses the gray-level mean difference function mean_difference(x, y) to statistically analyze the mean of gray-level pixels in each frame within the mapping area and performs difference processing on the results of adjacent frames. It calls the boundary curvature change accumulation function accumulateCurvature(c1, c2) to perform linear accumulation processing on the boundary curvature change between the current frame and the previous frame. Then, it merges the product of the accumulated curvature value and the coordinate difference into a single feature term to form a parameter term set and generate a displacement change feature group.

[0066] The state sequence generation submodule, based on displacement change feature groups, uses the state vector generation method transform_to_state_vector(V, t) to sequentially merge the feature point values ​​of each frame according to the frame index t. It combines each feature parameter into a fixed-dimensional state vector structure, binds all state vectors to the corresponding frame numbers using a frame index mapping table, and reassembles them into data blocks in a two-dimensional matrix format. It calls the jump detection function jump_detection(v, n) to perform first-order interpolation processing on the state vectors of adjacent frames and marks frames with differences exceeding the threshold n as jump points. It performs a comparison of the gradient directions of the state vectors before and after the jump points, records the continuous labels of the direction changes and writes them into the trend dictionary structure, and establishes the sequence node structure.

[0067] Please see Figure 2 The state path resolution module includes:

[0068] Point association judgment submodule: Based on the sequence node structure, a graph neural network is used to perform numerical cosine comparison between the state vectors of any two nodes and extract the included angle value. The distance change between the coordinates of the node centers is weighted and combined with the included angle value and compared with a preset threshold. Node pairs that meet the connection conditions are retained and a continuous connection mapping relationship is formed to obtain a graph structure connection relationship table.

[0069] Path weight extraction submodule: Based on the graph structure connection relationship table, it performs statistical analysis on the change magnitude of state vectors between adjacent nodes in all connection paths and superimposes them to obtain the path weight value. It performs length and weight product calculation on all paths and sorts them to extract the path with the maximum value. It fully extracts the node index in the branch of the maximum path to obtain the set of adapted path weights.

[0070] The state segment construction submodule extracts the state value mutation points between path node indices based on the adapted path weight set and filters them according to the amplitude threshold. It aggregates the sequence data on both sides of the jump point to form state change segments, performs continuous segment merging on the segment direction change trend, and obtains key morphological features.

[0071] The point association judgment submodule, based on the sequence node structure, uses the graph neural network model GCNConv function to perform input matrix construction operations on the state vector of each node, constructing a node feature matrix X and a graph connection edge table Edge_index. The state vector matrix X consists of a three-dimensional vector composed of the edge curvature value, contour gray value, and boundary position difference of each node. Cosine similarity is calculated for the state vectors of each pair of nodes, and the included angle value is extracted. The torch.nn.functional.cosine_similarity function is called with the dimension set to dim=1 for batch processing to obtain the included angle values ​​between all node pairs. The torch.dist function is called to solve the Euclidean distance for the node center coordinates. The above distance values ​​and the corresponding included angle values ​​are linearly weighted and combined, with the weight ratios set as included angle weight 0.6 and coordinate distance weight 0.4. The torch.where function is called to extract node pairs with weighted combination values ​​less than a set threshold, forming a structure pair index group, constructing a new edge set matrix, and generating a graph structure connection table.

[0072] The path weight extraction submodule, based on a graph structure connection table, uses the network path traversal function `networkx.all_simple_paths` with a maximum path length of 10. It extracts the node index sequence from all paths, uses the `torch.sub` and `torch.norm` functions to calculate the difference in state vectors between adjacent nodes in a path to extract the magnitude of change, and calls the `torch.sum` function to accumulate the changes sequentially along the paths, generating the total state change for a single path. It records the number of nodes in all paths, executes the `torch.mul` function to multiply the number of nodes by the total change to generate a path length weight product, and calls the `torch.topk` function on all path products to obtain the path with the maximum value and the set of node indices, generating a set of adapted path weights.

[0073] The state segment construction submodule, based on the adapted path weight set, uses the jump point filtering function numpy.diff combined with a threshold filtering strategy to extract the inter-frame difference of state values ​​in the path node index sequence and sets the jump threshold to the mean of the state vector amplitude plus twice the standard deviation. For all jump point indices, it extracts the sequence data of three frames before and after the jump point and constructs continuous index segments. The vector direction cosine angle calculation function scipy.spatial.distance.cosine is used to calculate the vector angle of the direction change trend in the segment and sets adjacent segments with an angle less than 15 degrees to be merged into one segment, generating key morphological features.

[0074] Please see Figure 2 Graph neural networks, according to the formula:

[0075] ;

[0076] in: This represents the multidimensional weighted matching value between node i and node j. This represents the state vector of node i after encoding by the graph neural network. This represents the state vector of node j after encoding by the graph neural network. The L2 norm of the state vector of node i. The L2 norm of the state vector of node j. This represents a small constant to prevent division by zero errors. This represents the center coordinate vector of node i in the image space. This represents the center coordinate vector of node j in the image space. This represents the Euclidean distance between node i and node j. The weighting coefficients for the spatial distance components are represented. This represents the absolute value of the difference between the mean rates of displacement change of node i and node j across multiple historical frames. This represents the weighting coefficient of the dynamic difference component. This represents the timestamp index of node i. This represents the timestamp index of node j. This represents the maximum timestamp index in the current input frame sequence. Represents the weighting coefficients for the time decay component;

[0077] Execution process: First, structural feature encoding is performed on candidate nodes in the input image to generate node state vectors. , Then, their modulus lengths were calculated respectively. , And combined with small constants The cosine angle is calculated to avoid division-by-zero instability. Then, the state variable angle value of the node pair is obtained through the inverse cosine function to represent the similarity. Finally, the center coordinates of node 1 and node 2 in the image are extracted. , Calculate its Euclidean distance as a spatial distribution feature, and multiply it by a weighting coefficient. The first weighted component is formed, and the movement path of the node in consecutive frames is further extracted. Its displacement change rate is fitted, and the absolute difference is calculated after taking the mean. As a measure of dynamic behavioral variability and multiplied by weights Generate the second weighted component, and then obtain the timestamps of nodes i and j. , Combined global maximum timestamp Perform normalization calculations to obtain the time decay coefficient and multiply by A third weighted component is formed, and the three weighted results are finally added to the included angle value to form a complete multidimensional node pair correlation value. It compares the result with a threshold to determine whether to build a node connection relationship, and then generates a graph structure connection relationship table for subsequent crawling path planning and target location.

[0078] Please see Figure 2 The path weight value is composed of the cumulative value of the change amplitude of the state vector between adjacent nodes in the connecting path, and is used to measure the intensity of state changes on the entire path.

[0079] The directional change trend is processed by merging continuous segments. The directional change angle is obtained by calculating the direction vector of the line connecting the center coordinates between the start and end points of each segment, and the angle threshold is used to determine whether they belong to the same trend interval.

[0080] Please see Figure 2 The status label determination module includes:

[0081] The mutation point extraction submodule calculates the rate of change of the inter-frame difference of each state vector based on key morphological features, performs local peak search on the rate values ​​of consecutive frames and performs adjacent difference amplitude filtering, extracts the direction of change and marks the position of the mutation frame after sorting the high amplitude points by index, and obtains the rate mutation candidate group.

[0082] Standard value matching submodule: Based on the rate mutation candidate group, calculate the difference between the state value of each mutation frame and the value of each dimension in the standard label template, judge the difference result and the upper and lower boundary range at the same time, label and encode the points that meet the conditions of all dimensions and record the corresponding frame index, and generate a label index structure set;

[0083] The tag sequence construction submodule is based on the tag index structure set and uses a hidden Markov model to extract the difference between adjacent frames in the continuous tag index and determine the connectivity of segments. It performs a similar merging operation on continuous segments and constructs a frame segment identifier table. For segments with a frame distance less than a set threshold, it performs tag inheritance merging and removes zero-label segments to establish a polymorphic tag sequence.

[0084] The mutation point extraction submodule, based on key morphological features, uses the time series difference rate calculation method numpy.diff combined with scipy.signal.find_peaks to extract the rate of change of the difference results of each state vector in the frame sequence. It uses the diff function to calculate the difference between adjacent frame state values ​​along axis 0 and divides it by the previous frame value to obtain the difference rate of change sequence. It then calls the find_peaks function, setting the height threshold to the mean plus 1.5 times the standard deviation and the distance threshold to the interval of 3 frames, to perform local peak search. For all the searched local extreme points, it calls the numpy.abs function to extract the amplitude value and performs adjacent difference judgment, filtering out peak points with adjacent amplitude differences less than 0.05. After sorting the retained high amplitude points in ascending order by frame index, it extracts the Euclidean distance directional offset between the state vectors of the previous and next frames and calls the numpy.sign function to calibrate the directional trend. It records the frame index and directional value of the mutation location and obtains the rate mutation candidate group.

[0085] Standard value matching submodule: Based on rate mutation candidate groups, it uses the multidimensional difference judgment function pandas.DataFrame.between combined with the vector difference function torch.sub to perform dimension-corresponding difference calculation on the state vector in each mutation frame and the baseline vector of each dimension in the standard label template. For each dimension, the torch.sub function is called to obtain the difference between the mutation vector and the template vector and convert it into DataFrame format. The between function is called to set the upper and lower bound thresholds to the template vector value ±10%. For index points that simultaneously meet the range conditions in all dimensions, the valid label positions are marked by Boolean filtering. The filtered frame numbers are processed by LabelEncoder to encode the labels and record the correspondence between the frame numbers and the encodings, generating a label index structure set.

[0086] The label sequence construction submodule, based on the label index structure set, adopts the combined fit and predict method of the HiddenMarkovModel class, initializes the number of states to 5, and calls the fit function to perform state transition probability learning for all frame numbers in the label index. For the trained model, the predict function is called to predict the state of the input label sequence. Segments with consecutive identical state values ​​in the prediction results are divided by connectivity index. For each segment, the difference between the start and end frames is calculated and segments with a spacing of less than 5 frames are inherited and merged. Then, segments with unlabeled state values ​​of 0 are removed through index removal to eliminate invalid labels, thus establishing a polymorphic label sequence.

[0087] Please see Figure 2 Hidden Markov Model, according to the formula:

[0088] ;

[0089] in: Indicates at frame time Below, the tag status is The optimal path probability value, Indicates the time in the previous frame In the middle, the tag status is The optimal path probability value, Indicates from label state Transition to tag state The state transition probability, Indicates state transition from to The compatibility weighting factor in paragraph structure. Indicates the label status In the In the class label pattern, the stability-based weight adjustment coefficient Indicates the label status frame time Observation value The probability of observation, Indicates the label status In the current frame Time series confidence correction factor This represents the total number of states in the set of hidden states labeled. Indicates in frame The observed features of the tag index difference extracted from it;

[0090] Execution process: First, extract the tag index of each frame from consecutive video frames to form a frame sequence tag observation set. By calculating the state of the previous frame Optimal path probability As a basic score for path continuation, combined with state transition probability Assessment from state Transition to the current candidate state The possibility of structural consistency is considered, and structural compatibility factors are extracted from the paragraph structure diagram. Strengthen logically continuous tag paragraph paths, and then calculate tag states through a tag pattern recognition model. In the Stability weights in class tag pattern Prioritize retaining tag paths with high tag frequency and stable duration, then utilize state. For the current observation value Matching probability Conduct an observation consistency assessment, and finally combine the time frame location. Calculate the time series confidence factor The state probabilities in later frames are dynamically adjusted. After weighting all factors, the maximum path probability is selected from all preceding states to obtain the final result. Based on this, a label state sequence is constructed to form a polymorphic label sequence with continuity, inheritance and segmentation recognition capabilities, which is used to guide the generation of grasping strategies and improve the label recognition accuracy in the process of multi-target decoupling.

[0091] Please see Figure 2 The attitude angle calculation module includes:

[0092] Center point determination submodule: Based on the polymorphic label sequence, extract the bounding box of the target region image in each frame, average the coordinates of the upper left and lower right corners of the bounding box and calculate the center of the rectangle, perform Euclidean distance calculation on the coordinates of the rectangle center and the coordinates of the image center point and combine them into a two-dimensional offset vector, and arrange all offset vectors in frame index order to obtain a spatial offset vector set;

[0093] Angle trend derivation submodule: Based on the spatial offset vector set, it calculates the angle between continuous vectors in groups of three frames, counts the absolute value of the angle between the first and last vectors of each group and judges the upward and downward trends, binds the trend information with the corresponding frame index, marks the angle direction trend category of each frame, and generates direction angle label groups.

[0094] Angle channel extraction submodule: Based on the direction angle marker group, all frames are divided into pitch channel, yaw channel and roll channel according to trend category. The amplitude change value of continuous angle trend segment in each channel is extracted and combined into floating segment. After channel number appending processing of each segment, an angle sequence is generated to obtain the attitude rotation angle vector set.

[0095] Center point determination submodule: Based on polymorphic label sequences, it uses a combination of OpenCV's cv2.boundingRect and coordinate mean functions to extract contours from the 2D image of the target region in each frame. It calls the cv2.findContours function, sets the search mode to RETR_EXTERNAL and the approximation method to CHAIN_APPROX_SIMPLE, to obtain the maximum bounding contour. It then applies the cv2.boundingRect function to the contour to obtain the coordinates of the top left and bottom right corners. The X and Y coordinates of these two points are then averaged using the numpy.mean function to obtain the coordinates of the rectangle's center point. The center point of each frame is set as the highest point in the image's width and height. The numpy.sqrt and numpy.square functions are used to calculate the sum of squares and the square root of the X and Y coordinate differences between the center point and the image center point, generating a single-frame 2D offset vector. Finally, the offset vectors of all frames are combined in ascending order of frame index to generate a spatial offset vector set.

[0096] The angle trend derivation submodule: Based on the spatial offset vector set, it uses the directional angle function scipy.spatial.distance.cosine combined with the sliding window grouping method to perform a sliding window segmentation of three frames as a group on the offset vector sequence arranged by index. It extracts the first and last offset vectors of each group, calls the cosine function to calculate the directional angle value, takes the arccos result, multiplies it by 180 and divides it by π to get the angle value, calls the numpy.diff function to extract the first-order rate of change for all angle values, uses the numpy.sign function to determine whether the direction of the rate of change is positive, negative or zero, sets mapping label values ​​for different directional categories, and calls pandas.DataFrame to bind the trend category with the corresponding frame index one by one to generate directional angle label groups;

[0097] The angle channel extraction submodule, based on the directional angle label group, uses the trend grouping function pandas.groupby and the amplitude classification processing function numpy.ptp to group the trend information of all frames according to the trend category field. The grouped frame segments are mapped to three numbered channels to represent pitch, yaw, and roll categories, respectively. The directional angle value of each segment in each channel is extracted, and the amplitude change value is obtained by calling the numpy.ptp function to calculate the difference between the maximum and minimum values ​​of the segment. The amplitude value and trend label are merged into a floating segment structure. The numpy.concatenate function is called to add numbering information to each channel segment and integrate them into a unified angle sequence structure to generate an attitude rotation angle vector set.

[0098] Please see Figure 2 The spatial state fusion module includes:

[0099] State and attitude binding submodule: Based on the attitude rotation angle vector set and polymorphic label sequence, it performs alignment processing of state label and angle vector number under each frame index, selects matching frame number by filtering the range of confidence sort value, performs label number and three-axis angle value splicing binding and establishes binding index mapping matrix to generate frame-level binding label pair;

[0100] Combined difference calculation submodule: Based on frame-level bound tag pairs, it performs combined extraction of all three-axis angle vector values ​​under the same state category. It calculates the difference between the maximum and minimum amplitudes of each angle sequence and generates fluctuation range values. It counts the frequency of angle changes in each state combination and delineates stable and unstable range segments to generate an attitude difference score set.

[0101] The structural result generation submodule: Based on the attitude difference scoring set, it performs frame sequence restoration operation on the binding information of each state label and scoring segment, performs joint sorting of state channel and angle channel on the combined data sequence, fills the state code and angle number of each frame with dual channels and marks them into the frame index column, and establishes a fusion annotation result set.

[0102] The state and pose binding submodule, based on the pose rotation angle vector set and polymorphic label sequence, uses the matrix index alignment function pandas.merge and the confidence filtering algorithm torch.topk to perform primary key merging operations on the state labels and angle vector numbers under each frame index and generate a pairing matrix. All matching records are sorted according to the confidence values ​​output by the model from which the three-axis angle vectors originate, with the retention interval set to the top 30%. The torch.topk function is called to select the frame number within the top 100%, and the state label number of the corresponding frame number is concatenated with the X, Y, and Z values ​​of the three-axis angle vectors using the torch.cat function in the row direction, merging them into a six-dimensional state-pose vector. A data matrix is ​​then built using pandas.DataFrame with the frame number as the index and the merged vector as the content, generating frame-level binding label pairs.

[0103] The combined difference calculation submodule, based on frame-level bound label pairs, uses the grouping operation function pandas.groupby and the amplitude difference calculation function numpy.ptp to perform grouping operations on combined data with the same state number in all records. It extracts the X, Y, and Z axis angle sequences in each group, calls the numpy.ptp function to calculate the difference between the maximum and minimum values ​​of each axis, merges the amplitude values ​​of the three axes into a single fluctuation interval item, calls the numpy.histogram function to divide the frequency intervals into 5° intervals, marks intervals with frequencies greater than the average frequency as stable segments, and the rest as unstable segments. It combines the state number with the classification value of each interval to generate a new scoring field, constructs a state-angle fluctuation mapping dictionary, and generates an attitude difference scoring set.

[0104] The structure result generation submodule, based on the attitude difference score set, uses a combination of the index restoration and channel number sorting functions pandas.merge and numpy.lexsort to restore each state number and score segment field according to the frame sequence number and state label field, constructing a frame sequence data frame. The combined data table is then processed by calling the numpy.lexsort function, setting the first key to the state channel number and the second key to the angle channel number. For each row of the sorted results, the numpy.append function is called to fill the state label number and angle direction number into each frame record. Finally, the DataFrame.insert function is called to create an output data table with the frame index as the column header, establishing the fused annotation result set.

[0105] Please see Figure 3 A pattern recognition-based AI-powered method for precise identification and capture of hooks, executed based on the aforementioned pattern recognition-based AI-powered method for precise identification and capture of hooks, includes the following steps:

[0106] S1: Based on the set of image sequence frames captured by the camera, extract the edge contour value, boundary gray value, regional gray average value and edge coordinate offset of the hook component in each frame, arrange the contour point set according to the time sequence, calculate the matching position coordinate difference and gray value change of the edge contour points between adjacent frames, superimpose the edge contour offset trajectory and record the corresponding frame number, establish a node index matrix and associate it with the corresponding edge data structure, and establish a sequence node structure;

[0107] S2: Based on the sequence node structure, extract the node state vector and calculate the Euclidean distance between the cosine angle of the edge morphological features and the center coordinates. After weighting the two, select node pairs below the threshold to generate a connection index table, construct a path set and count the path jump amplitude, select the maximum path and extract the frame number of the mutation point, merge the direction segments according to the similarity of the angle, use a graph neural network to construct the connection structure and extract the jump path, obtain the key morphological feature index and classify it into the same group, perform group judgment on the angle of the direction vector between nodes in the same group and merge similar direction segments, and use a graph neural network to construct the structure of the connection relationship between nodes and extract the jump path to obtain key morphological features;

[0108] S3: Based on key morphological features, calculate the sequence of node position change rates in each segment and obtain the jump rate value. Compare and match the jump rate value with the set standard state rate reference value segment by segment. Identify the frame number sequence corresponding to the matching segment and mark the state category index. Perform a duplicate segment merging operation on the state category index. After removing segments with unclear classification, use a hidden Markov model to judge the stability of the jump rate and optimize the multi-segment state sequence to obtain a polymorphic label sequence.

[0109] S4: Based on the polymorphic label sequence, extract the center coordinate point of the corresponding region of each frame and calculate the center point displacement vector value between adjacent frames. Extract the angle between adjacent vectors according to the frame sequence and record the angle trend change value of each directional channel. Separate the pitch angle change sequence, yaw angle change sequence and roll angle change sequence according to the time series. Perform continuous segment judgment and directional trend classification of the angle sequence respectively to obtain the attitude rotation angle vector set.

[0110] S5: Based on the attitude rotation angle vector set and polymorphic label sequence, extract the combination value of state label and three-axis attitude angle for each frame, statistically analyze the repeated state and attitude combinations and calculate the range of three-axis angle difference between combinations, index and merge the combination segments with difference range lower than the set tolerance and generate a continuous segment label index table, and establish a fusion annotation result set.

[0111] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A pattern recognition-based AI-powered precise identification and grasping system for debonding hooks, characterized in that, The system includes: Dynamic node construction module: It acquires a set of image sequence frames during the hook removal and re-hook operation through the camera, performs hook component contour mapping, boundary difference calculation, regional grayscale change measurement and edge displacement accumulation processing, and establishes a sequence node structure; State path parsing module: Based on the sequence node structure, a graph neural network is used to jointly compare the node cosine value and coordinate difference, extract the path weight branch, count the state vector jump amplitude and combine them into a sequence to obtain key morphological features; The state path parsing module includes: Point association judgment submodule: Based on the sequence node structure, a graph neural network is used to perform numerical cosine comparison between the state vectors of any two nodes and extract the included angle value. The distance change between the coordinates of the node centers is weighted and combined with the included angle value and compared with a preset threshold. Node pairs that meet the connection conditions are retained and a continuous connection mapping relationship is formed to obtain a graph structure connection relationship table. Path weight extraction submodule: Based on the graph structure connection relationship table, the change amplitude of state vector between adjacent nodes in all connection paths is statistically analyzed and superimposed to obtain the path weight value. The length and weight product of all paths are calculated and sorted to extract the path with the maximum value. The node index in the branch of the maximum path is completely extracted to obtain the set of adaptive path weights. State segment construction submodule: Based on the adapted path weight set, extract the state value mutation points between path node indices and filter them according to the amplitude threshold. Aggregate the sequence data on both sides of the jump point to form state change segments. Perform continuous segment merging processing on the segment direction change trend to obtain key morphological features. State label determination module: Based on the key morphological features, a hidden Markov model is used to screen state rate mutation points, perform standard value matching and classification on state change segments, merge duplicate label segments and remove invalid segments, and establish a polymorphic label sequence. Attitude angle estimation module: Based on the polymorphic label sequence, calculate the offset and angle trend of the regional center point, extract the continuous change values ​​of pitch, yaw and roll angles according to the direction channel, and obtain the attitude rotation angle vector set; Spatial state fusion module: Based on the attitude rotation angle vector set and polymorphic label sequence, bind the state and attitude combination of each frame, calculate the difference range of repeated combinations and generate joint annotations, and establish a fusion annotation result set; The graph neural network is defined according to the formula: ； in: This represents the multidimensional weighted matching value between node i and node j. This represents the state vector of node i after encoding by the graph neural network. This represents the state vector of node j after encoding by the graph neural network. The L2 norm of the state vector of node i. The L2 norm of the state vector of node j. This represents a small constant to prevent division by zero errors. This represents the center coordinate vector of node i in the image space. This represents the center coordinate vector of node j in the image space. This represents the Euclidean distance between node i and node j. The weighting coefficients for the spatial distance components are represented. This represents the absolute value of the difference between the mean rates of displacement change of node i and node j across multiple historical frames. This represents the weighting coefficient of the dynamic difference component. This represents the timestamp index of node i. This represents the timestamp index of node j. This represents the maximum timestamp index in the current input frame sequence. Represents the weighting coefficients for the time decay component; Execution process: First, structural feature encoding is performed on candidate nodes in the input image to generate node state vectors. , Then, their modulus lengths were calculated respectively. , And combined with small constants The cosine angle is calculated to avoid division-by-zero instability. Then, the state variable angle value of the node pair is obtained through the inverse cosine function to represent the similarity. Finally, the center coordinates of node 1 and node 2 in the image are extracted. , Calculate its Euclidean distance as a spatial distribution feature, and multiply it by a weighting coefficient. The first weighted component is formed, and the movement path of the node in consecutive frames is further extracted. Its displacement change rate is fitted, and the absolute difference is calculated after taking the mean. As a measure of dynamic behavioral variability and multiplied by weights Generate the second weighted component, and then obtain the timestamps of nodes i and j. , Combined global maximum timestamp Perform normalization calculations to obtain the time decay coefficient and multiply by A third weighted component is formed, and the three weighted results are finally added to the included angle value to form a complete multidimensional node pair correlation value. It compares the result with a threshold to determine whether to build a node connection relationship, and then generates a graph structure connection relationship table for subsequent crawling path planning and target location. The Hidden Markov Model is defined according to the formula: ； in: Indicates at frame time Below, the tag status is The optimal path probability value, Indicates the time in the previous frame In the middle, the tag status is The optimal path probability value, Indicates from label state Transition to tag state The state transition probability, Indicates state transition from to The compatibility weighting factor in paragraph structure. Indicates the label status In the In the class label pattern, the stability-based weight adjustment coefficient Indicates the label status frame time Observation value The probability of observation, Indicates the label status In the current frame Time series confidence correction factor This represents the total number of states in the set of hidden states labeled. Indicates in frame The observed features of the tag index difference extracted from it; Execution process: First, extract the tag index of each frame from consecutive video frames to form a frame sequence tag observation set. By calculating the state of the previous frame Optimal path probability As a basic score for path continuation, combined with state transition probability Assessment from state Transition to the current candidate state The possibility of structural consistency is considered, and structural compatibility factors are extracted from the paragraph structure diagram. Strengthen logically continuous tag paragraph paths, and then calculate tag states through a tag pattern recognition model. In the Stability weights in class tag pattern Prioritize retaining tag paths with high tag frequency and stable duration, then utilize state. For the current observation value Matching probability Conduct an observation consistency assessment, and finally combine the time frame location. Calculate the time series confidence factor The state probabilities in later frames are dynamically adjusted. After weighting all factors, the maximum path probability is selected from all preceding states to obtain the final result. Based on this, a label state sequence is constructed to form a polymorphic label sequence with continuity, inheritance and segmentation recognition capabilities, which is used to guide the generation of grasping strategies and improve the label recognition accuracy in the process of multi-target decoupling.

2. The AI-based precise identification and grasping system for debonding hooks based on pattern recognition according to claim 1, characterized in that, The dynamic node construction module includes: Image contour mapping submodule: It acquires a set of image sequence frames during the hook removal and re-hook operation through the camera, performs gray-scale change point localization processing in the edge region of the hook component image, performs alignment and trajectory connectivity verification of the same component contour boundary points between adjacent frames, calculates the pixel movement path of boundary points with the same number in consecutive frames and records the corresponding frame index information, and establishes a contour correspondence mapping table. Boundary parameter extraction submodule: Based on the contour mapping table, the coordinate difference of the boundary points is obtained and the position change amplitude is calculated. The mean value of gray pixels in each frame within the mapping area is calculated and subtracted from the result of the previous frame. Then, the boundary curvature change and displacement amplitude are superimposed and analyzed to generate displacement change feature group. State sequence generation submodule: Based on the displacement change feature group, the feature point values ​​of each frame are combined in time sequence and uniformly transformed into a state vector structure. All state vectors are bound by frame index and reorganized into data blocks in the form of frame sequence. The continuous jump points and directional trend information between frames are marked, and a sequence node structure is established.

3. The AI-based precise identification and grasping system for debonding hooks based on pattern recognition according to claim 1, characterized in that, The path weight value is composed of the cumulative value of the change amplitude of the state vector between adjacent nodes in the connecting path, and is used to measure the intensity of state change on the entire path. The directional change trend is processed by merging consecutive segments. The directional change angle is obtained by calculating the direction vector of the line connecting the center coordinates between the start and end points of each segment, and the included angle threshold is used to determine whether they belong to the same trend interval.

4. The AI-based precise identification and grasping system for debonding hooks based on pattern recognition according to claim 1, characterized in that, The status label determination module includes: The mutation point extraction submodule calculates the rate of change of the inter-frame difference of each state vector based on the key morphological features, performs local peak search on the rate values ​​of consecutive frames and performs adjacent difference amplitude filtering, extracts the direction of change and marks the position of the mutation frame after sorting the high amplitude points by index, and obtains the rate mutation candidate group. Standard value matching submodule: Based on the rate mutation candidate group, calculate the difference between the state value of each mutation frame and the value of each dimension in the standard label template, determine the difference result and the upper and lower boundary range at the same time, label and encode the points that meet the conditions of all dimension ranges and record the corresponding frame index, and generate a label index structure set; The tag sequence construction submodule: Based on the tag index structure set, a hidden Markov model is used to extract the difference between adjacent frames in the continuous tag index and determine the connectivity of segments. The same type of continuous segments are merged and a frame segment identification table is constructed. For segments with a frame distance less than a set threshold, the tag inheritance is merged and zero-label segments are removed to establish a polymorphic tag sequence.

5. The AI-based precise identification and grasping system for debonding hooks based on pattern recognition according to claim 1, characterized in that, The attitude angle calculation module includes: Center point determination submodule: Based on the polymorphic label sequence, extract the bounding box of the target region image in each frame, average the coordinates of the upper left and lower right corners of the bounding box and calculate the center of the rectangle, perform Euclidean distance calculation on the coordinates of the rectangle center and the coordinates of the image center point and combine them into a two-dimensional offset vector, and arrange all offset vectors in the order of frame index to obtain a spatial offset vector set; Angle trend derivation submodule: Based on the spatial offset vector set, calculate the angle between continuous vectors in groups of three frames, count the absolute value of the angle between the first and last vectors of each group and judge the upward and downward trends, bind the trend information with the corresponding frame index, mark the angle direction trend category of each frame, and generate the direction angle label group. Angle channel extraction submodule: Based on the direction angle marking group, all frames are divided into pitch channel, yaw channel and roll channel according to trend category. The amplitude change value of continuous angle trend segment in each channel is extracted and combined into floating segment. After channel number appending processing of each segment, an angle sequence is generated to obtain the attitude rotation angle vector set.

6. The AI-based precise identification and grasping system for debonding hooks based on pattern recognition according to claim 1, characterized in that, The spatial state fusion module includes: State and attitude binding submodule: Based on the attitude rotation angle vector set and polymorphic label sequence, the state label and angle vector number under each frame index are aligned. By filtering the confidence ranking value and selecting the matching frame number, the label number and the three-axis angle value are concatenated and bound, and a binding index mapping matrix is ​​established to generate frame-level binding label pairs. Combination difference calculation submodule: Based on the frame-level bound tag pair, perform numerical combination extraction of all three-axis angle vectors under the same state category, calculate the difference between the maximum and minimum amplitudes of each angle sequence and generate fluctuation interval values, count the frequency of angle changes in each state combination and delineate stable and unstable range segments, and generate attitude difference score set; The structure result generation submodule: Based on the attitude difference score set, it performs frame sequence restoration operation on the binding information of each state label and score segment, performs joint sorting of state channel and angle channel on the combined data sequence, fills the state code and angle number of each frame with dual channels and marks them into the frame index column, and establishes a fusion annotation result set.

7. A pattern recognition-based AI-powered method for precise identification and grasping of hooks and pins, characterized in that, The execution of the pattern recognition-based AI-based precise identification and grasping system for hook removal according to any one of claims 1-6 includes the following steps: S1: Based on the set of image sequence frames captured by the camera, extract the edge contour value, boundary gray value, regional gray average value and edge coordinate offset of the hook component in each frame, arrange the contour point set according to the time sequence, calculate the matching position coordinate difference and gray value change of the edge contour points between adjacent frames, superimpose the edge contour offset trajectory and record the corresponding frame number, establish a node index matrix and associate it with the corresponding edge data structure, and establish a sequence node structure; S2: Based on the sequence node structure, extract the node state vector and calculate the Euclidean distance between the cosine angle of the edge morphological features and the center coordinates. After weighting the two, filter the node pairs below the threshold to generate a connection index table, construct a path set and count the path jump amplitude, filter the maximum path and extract the frame number of the mutation point, merge the direction segments according to the similarity of the angle, use a graph neural network to construct the connection structure and extract the jump path, obtain the key morphological feature index and classify it into the same group, perform group judgment on the angle of the direction vector between nodes in the same group and merge similar direction segments, and use a graph neural network to construct the structure of the connection relationship between nodes and extract the jump path to obtain the key morphological features; S3: Based on the key morphological features, calculate the sequence of node position change rates in each segment and obtain the jump rate value. Compare and match the jump rate value with the set standard state rate reference value segment by segment. Identify the frame number sequence corresponding to the matching segment and mark the state category index. Perform a duplicate segment merging operation on the state category index. After removing segments with unclear classification, use a hidden Markov model to judge the stability of the jump rate and optimize the multi-segment state sequence to obtain a polymorphic label sequence. S4: Based on the polymorphic tag sequence, extract the center coordinate point of the corresponding region of each frame and calculate the center point displacement vector value between adjacent frames. Extract the angle between adjacent vectors according to the frame sequence and record the angle trend change value of each directional channel. Separate the pitch angle change sequence, yaw angle change sequence and roll angle change sequence according to the time sequence. Perform continuous segment judgment and directional trend classification of the angle sequence respectively to obtain the attitude rotation angle vector set. S5: Based on the attitude rotation angle vector set and polymorphic label sequence, extract the combination value of state label and three-axis attitude angle for each frame, statistically analyze the repeated state and attitude combinations and calculate the range of three-axis angle difference between combinations, index and merge the combination segments with difference range lower than the set tolerance and generate a continuous segment label index table, and establish a fusion annotation result set.

Images