Pattern recognition-based unhooking and rehooking AI accurate recognition grabbing system and method

Abstract

The invention relates to the technical field of image state recognition, in particular to an unhooking and rehooking AI accurate recognition grabbing system and method based on pattern recognition, in the system, node construction is conducted through contour changes, boundary difference values and gray level dynamic states of a hook assembly in an image sequence frame, edge displacement accumulation analysis is combined, meanwhile, through a graph neural network, an image sequence frame is obtained, and the image sequence frame is obtained. Cosine values and coordinate difference values between nodes are subjected to combined comparison, and a path hopping sequence is constructed, so that the response sensitivity to state abrupt change is enhanced, a key path of morphological evolution can still be stably extracted under the condition of complex background interference or local shielding, the anti-interference performance and fault tolerance of space path identification are effectively improved, and the space path identification accuracy is improved. Statistical modeling is further carried out on state rate sudden change points through a hidden Markov model, paragraph merging and invalid fragment removing operation are carried out on abnormal point segments by matching a standard state mode, a state label sequence is constructed, and accurate division of high-confidence and multi-segment continuous states is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image state recognition, in particular to a hook picking and replacing AI accurate recognition and grabbing system and method based on pattern recognition. BACKGROUND

[0002] The state recognition technology field focuses on identifying, judging and classifying the specific state of target objects or environment through computational models, image processing algorithms, sensor data, etc., and is widely used in industrial automation, intelligent manufacturing, medical monitoring and robot vision systems. Combined with image segmentation, target detection, state inference models, the spatial position, morphological features and physical state of the target are extracted, and this is used as the basis for subsequent control decisions.

[0003] A hook picking and replacing AI accurate recognition and grabbing system based on pattern recognition is an automated system that combines artificial intelligence vision recognition and mechanical execution technology. It acquires hook body image information through an image acquisition device, uses pattern recognition algorithms to determine the current state, and drives the end execution mechanism to complete the corresponding hook picking or replacing operation. This improves recognition accuracy and grabbing stability, reduces the risk of misoperation, and is suitable for application scenarios where the hook body state is complex, irregularly distributed, or the working space is limited.

[0004] The existing technology has shortcomings in recognition accuracy and continuous state tracking. It cannot establish a continuous model of state evolution over time. When the hook body state has slight deformation or occlusion interference, the static discrimination mechanism cannot distinguish between normal operation and error state, causing misjudgment. The recognition mechanism lacks joint analysis of action path and attitude angle, making it difficult to accurately grasp the interaction between state and attitude. When performing hook picking operations, the influence of angle changes on the grabbing point is easily missed when the hook body rotates or rolls slightly. The execution mechanism therefore exerts force in an inappropriate direction, resulting in an angle deviation that causes the hook body to fall off or mechanical damage, significantly reducing recognition efficiency and operation accuracy, and restricting the overall system's automation level and operational safety. SUMMARY

[0005] The purpose of the present application is to solve the shortcomings of the prior art and to provide a hook picking and replacing AI accurate recognition and grabbing system and method based on pattern recognition.

[0006] To achieve the above purpose, the present application adopts the following technical solution: a hook picking and replacing AI accurate recognition and grabbing system based on pattern recognition comprises:

[0007] Dynamic node construction module: acquire a set of image sequence frames during the hook picking and replacing operation process through a camera, perform hook assembly contour mapping, boundary difference calculation, regional gray change calculation and edge displacement accumulation processing, and establish a sequence node structure body;

[0008] State path analysis module: based on the sequence node structure body, using graph neural network, joint comparison of node cosine value and coordinate difference value, extracting path weight branch, statistical state vector jump amplitude and combining into sequence, obtaining key morphological features;

[0009] State label determination module: based on the key morphological features, using hidden Markov model, state rate mutation point screening, candidate paragraph execution standard value matching and classification, merging repeated label segments and eliminating invalid segments, establishing multi-state label sequence;

[0010] Attitude angle calculation module: based on the multi-state label sequence, region center point offset and angle trend calculation, pitch, yaw and roll angle continuous change value extraction according to direction channel, obtaining attitude rotation angle vector set;

[0011] Space state fusion module: based on the attitude rotation angle vector set and multi-state label sequence, binding each frame state and attitude combination, calculating difference range of repeated combination and generating joint label, establishing fusion label result set.

[0012] As a further scheme of the present application, the dynamic node construction module comprises:

[0013] Image contour mapping sub-module: through the camera, the image sequence frame set in the hooking operation process is obtained, the edge region gray mutation point positioning processing in the hook component image is carried out, and the adjacent frame alignment and trajectory connection verification of the same component contour boundary point position are carried out, the pixel movement path of the same numbered boundary point in the continuous frame is calculated and the corresponding frame index information is recorded, and the contour corresponding mapping table is established;

[0014] Boundary parameter extraction sub-module: based on the contour corresponding mapping table, the boundary point corresponding coordinate difference value is obtained and the position change amplitude value is calculated, the average value of the gray pixels in the mapping area of each frame is calculated and the difference with the previous frame result is processed, then the boundary curvature change amount and displacement amplitude are superimposed and analyzed, and the displacement change feature group is generated;

[0015] State sequence generation sub-module: based on the displacement change feature group, the feature point values of each frame are combined according to the time sequence order and uniformly converted into state vector structure, all state vectors are bound with frame index and recombined into frame sequence form data block, the continuous jump point and direction trend information between frames are calibrated, and the sequence node structure body is established.

[0016] As a further scheme of the present application, the state path analysis module comprises:

[0017] Point association judgment submodule: based on the sequence node structure body, using graph neural network, comparing the numerical cosine between the state vectors of any two nodes and extracting the included angle value, weighting the distance change between the node center coordinates and the included angle value, comparing with the preset threshold, retaining the node pairs meeting the connection condition and forming a continuous connection mapping relationship, and obtaining a graph structure connection relationship table;

[0018] Path weight extraction submodule: based on the graph structure connection relationship table, the state vector change amplitude between adjacent nodes in all connection paths is counted and superimposed to obtain a path weight value, the length and weight product of all paths are calculated and sorted to extract the maximum value path, and the node index in the maximum path branch is completely extracted to obtain an adaptive path weight set;

[0019] State segment construction submodule: based on the adaptive path weight set, the state value mutation point between path node indexes is extracted and screened according to the amplitude threshold, the sequence data on both sides of the jump point is aggregated to form a state change segment, and the continuous segment merging processing is performed on the segment direction change trend to obtain key morphological features.

[0020] As a further scheme of the application, the graph neural network is according to the formula:

[0021]

[0022] Wherein: θ i,j represents the multi-dimensional weighted matching value between node i and node j, represents the state vector of node i after graph neural network encoding, represents the state vector of node j after graph neural network encoding, represents the two-norm of the state vector of node i, represents the two-norm of the state vector of node j, and ∈ represents a small constant to prevent division by zero error, represents the center coordinate vector of node i in the image space, represents the center coordinate vector of node j in the image space, represents the Euclidean distance between node i and node j, λ1 represents the weighting coefficient of the space distance component, and Δd i,j represents the absolute value of the difference between the average displacement rate of node i and node j in the historical multiple frames, λ2 represents the weighting coefficient of the dynamic difference component, and t i represents the timestamp index of node i, t j represents the timestamp index of node j, T represents the maximum timestamp index in the current input frame sequence, and λ3 represents the weighting coefficient of the time decay component.

[0023] As a further scheme of the present application, the path weight value is composed of the cumulative value of the state vector variation amplitude between adjacent nodes in the connection path, and is used to measure the state variation intensity on the whole path.

[0024] The direction change trend is subjected to continuous segment merging processing, the direction vector of the center coordinate connecting line between the starting point and the ending point of each segment is calculated, the direction change angle is obtained, and whether it belongs to the same trend interval is determined by using the included angle threshold.

[0025] As a further scheme of the present application, the state label determination module comprises:

[0026] The mutation point extraction submodule: based on the key morphological features, the rate difference change rate between frames is calculated, the local peak value of the continuous frame rate value is searched, the adjacent difference amplitude is filtered, the high-amplitude points are sorted by index, the change direction is extracted, the mutation frame position is labeled, and the rate mutation candidate group is obtained;

[0027] The standard value matching submodule: based on the rate mutation candidate group, the difference between each mutation frame state value and the numerical value in the standard label template is calculated, the difference result is determined with the upper and lower boundary range, the points meeting all dimension range conditions are labeled with label codes and the corresponding frame index is recorded, and the label index structure set is generated;

[0028] The label sequence construction submodule: based on the label index structure set, the hidden Markov model is used to extract the difference between adjacent frames in the continuous label index and judge the paragraph connectivity, the same type merging operation is performed on the continuous segments, the frame segment identification table is constructed, the segments with frame distance less than the set threshold are merged and removed, and the multi-state label sequence is established.

[0029] As a further scheme of the present application, the hidden Markov model is according to the formula:

[0030]

[0031] Wherein: δ t (m) represents the optimal path probability value of the label state m at frame time t, δ t-1 (n) represents the optimal path probability value of the label state n at the last frame time t-1, α nm represents the state transition probability from the label state n to the label state m, κ nm represents the compatibility weight factor of state transition from n to m in the paragraph structure, represents the weight adjustment coefficient of the label state n in the rth label mode based on stability, β m (z t ) represents the label state m to the observation value zt the observation probability of the state m m (t) represents the time sequence confidence correction factor of the label state m in the current frame t N represents the total state quantity in the label hidden state set, z t represents the label index difference value observation feature extracted in the frame t.

[0032] As a further scheme of the present application, the posture angle calculation module comprises:

[0033] A center point determination submodule: based on the multi-state label sequence, a bounding box of each frame of target region image is extracted, the upper left and lower right corner coordinates of the bounding box are averaged and the center of the rectangle is calculated, the Euclidean distance between the center coordinates of the rectangle and the center coordinates of the image is calculated and combined into a two-dimensional offset vector, all offset vectors are arranged in order of frame index to obtain a set of spatial offset vectors;

[0034] An included angle trend derivation submodule: based on the set of spatial offset vectors, the included angle of three frames as a group of continuous vectors is calculated, the direction change between the first and last vectors of each group is statistically calculated and the rising and falling trends are judged, and the trend information is bound with the corresponding frame index, the angle direction trend category to which each frame belongs is labeled, and a direction included angle label group is generated.

[0035] An angle channel extraction submodule: based on the direction included angle label group, all frames are divided into pitch channel, yaw channel and roll channel according to the trend category, the amplitude change value of the continuous included angle trend segment in each category is extracted and combined into a floating section, and the angle sequence is generated after the channel number is added to each section to obtain a set of posture rotation angle vectors.

[0036] As a further scheme of the present application, the spatial state fusion module comprises:

[0037] A state posture binding submodule: based on the set of posture rotation angle vectors and the multi-state label sequence, the state label and the angle vector number under each frame index are aligned, the matching frame sequence number is selected by range screening of the confidence sorting value, the label number and the three-axis angle value are spliced and bound, and a binding index mapping matrix is established to generate a frame-level binding label pair.

[0038] A combined difference calculation submodule: based on the frame-level binding label pair, the three-axis angle vector numerical combination extraction processing under the same state category is performed, the fluctuation interval value is generated by performing maximum amplitude and minimum amplitude difference calculation on each group of angle sequences, the angle change frequency in each state combination is counted and the stable and unstable range segments are divided to generate a posture difference score set.

[0039] The structure result generation sub-module: based on the posture difference score set, the frame sequence restoration operation of each state label and score segment binding information is performed, the state channel and angle channel joint sorting is performed on the combined data sequence, the double channel filling and labeling of each frame state coding and angle numbering are performed, and the fusion labeling result set is established.

[0040] A pattern recognition-based AI precise identification and grabbing method for hook picking, which is based on the above-mentioned pattern recognition-based AI precise identification and grabbing system for hook picking, comprising the following steps:

[0041] S1: Based on the image sequence frame set collected by the camera, the edge contour value, boundary gray value, regional gray mean value and edge coordinate offset of the hook assembly in each frame are extracted, the contour point set is arranged in time sequence, the matching position coordinate difference and gray value change between adjacent frames are calculated, the edge contour offset trajectory is superimposed and the corresponding frame number is recorded, the node index matrix is established and the corresponding edge data structure is associated, and the sequence node structure body is established;

[0042] S2: Based on the sequence node structure body, the node state vector is extracted and the cosine angle of the edge shape feature and the Euclidean distance between the center coordinates are calculated, the two are weighted and then the nodes below the threshold are screened to generate a connection index table, the path set is constructed and the path jump amplitude is counted, the maximum path is screened and the mutation point frame number is extracted, the direction segment is merged according to the angle similarity, the connection structure is constructed by using the graph neural network and the jump path is extracted, the key morphological feature index is obtained and classified into the same group, the direction vector angle between the nodes in the same group is judged and the similar direction segment is merged, the structure construction and jump path extraction of the connection relationship between the nodes are performed by the graph neural network, and the key morphological feature is obtained;

[0043] S3: Based on the key morphological feature, the node position change rate sequence in each segment is calculated and the jump rate value is obtained, the jump rate value is compared with the set standard state rate reference value segment by segment, the matching segment corresponding frame number sequence is identified and the state category index is marked, the state category index is subjected to a repeated segment merging operation, after eliminating the paragraphs with unclear classification, the jump rate stability is judged and the multi-segment state sequence is optimized by the hidden Markov model, and the multi-state label sequence is obtained.

[0044] S4: Based on the multi-state label sequence, the center coordinate point of each frame corresponding region is extracted and the center point displacement vector value between adjacent frames is calculated, the angle between adjacent vectors is extracted according to the frame sequence and the angle trend change value of each direction channel is recorded, the pitch angle change sequence, the yaw angle change sequence and the roll angle change sequence are separated according to the time sequence, the continuous segment judgment and direction trend classification of the angle sequence are performed respectively, and the posture rotation angle vector set is obtained.

[0045] S5: Based on the posture rotation angle vector set and the polymorphic label sequence, extract the combination value of each frame state label and the three-axis posture angle, count the repeated state and posture combinations and calculate the three-axis angle difference range between the combinations, perform index merging processing on the combination segments whose difference range is lower than the set tolerance and generate a continuous segment label index table to establish a fusion annotation result set.

[0046] Compared with the prior art, the advantages and positive effects of the present invention are:

[0047] 1. This method constructs nodes by analyzing the contour changes, boundary differences, and grayscale dynamics of the hook assembly in image sequence frames. Combined with edge displacement accumulation analysis, it can capture the continuous evolution trajectory of the target state in scenes without a static reference, achieving complete modeling of the operation action chain.

[0048] 2. In this invention, a graph neural network is used to jointly compare the cosine values ​​and coordinate differences between nodes to construct a path jump sequence, thereby enhancing the response sensitivity to sudden changes in state. This allows for stable extraction of key paths of morphological evolution even under complex background interference or partial occlusion conditions, effectively improving the interference resistance and fault tolerance of spatial path recognition.

[0049] 3. In the present invention, a hidden Markov model is used to further perform statistical modeling on the state rate mutation points. By matching the standard state pattern, segment merging and invalid segment removal operations are performed on the abnormal point segments to construct a state label sequence, thereby achieving high-confidence and accurate division of multiple continuous states, thereby improving recognition accuracy, response speed and system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a system flow chart of the AI ​​precise recognition and grasping system for picking and re-hooking based on pattern recognition of the present invention;

[0051] Figure 2 This is a schematic diagram of the system framework of the AI ​​precise recognition and grasping system for picking and re-hooking based on pattern recognition of the present invention;

[0052] Figure 3 This is a schematic diagram of the method steps of the AI ​​precise recognition and grabbing method for picking and re-hooking based on pattern recognition of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] See also Figure 1The application provides a technical solution: an AI precise identification and grabbing system based on pattern recognition of picking and replacing hooks comprises:

[0055] A dynamic node construction module: a set of image sequence frames in the picking and replacing hook operation process are obtained through a camera, a hook component contour mapping is performed, a boundary difference value is calculated, a regional gray scale change is calculated and measured, and an edge displacement accumulation is processed, and a sequence node structure body is established;

[0056] A state path analysis module: based on the sequence node structure body, a graph neural network is used to perform joint comparison of node cosine values and coordinate difference values, extract path weight branches, count state vector jump amplitudes and combine them into sequences, and obtain key morphological features;

[0057] A state label determination module: based on the key morphological features, an hidden Markov model is used to perform state rate mutation point screening, perform standard value matching and classification on candidate paragraphs, merge repeated label segments and remove invalid segments, and establish a multi-state label sequence;

[0058] An attitude angle calculation module: based on the multi-state label sequence, a region center point offset and a clamping angle trend are calculated, a continuous change value of pitch, yaw and roll angles is extracted according to a direction channel, and an attitude rotation angle vector set is obtained;

[0059] A space state fusion module: based on the attitude rotation angle vector set and the multi-state label sequence, each frame state and attitude combination is bound, a difference range is calculated and a joint label is generated for repeated combinations, and a fusion label result set is established.

[0060] Please refer to Figure 2 The dynamic node construction module comprises:

[0061] An image contour mapping submodule: a set of image sequence frames in the picking and replacing hook operation process are obtained through a camera, an edge region gray scale mutation point in a hook component image is located and processed, and adjacent frame alignment of the same component contour boundary points and trajectory connectivity verification are performed, a pixel movement path of the same numbered boundary points in consecutive frames is calculated and corresponding frame index information is recorded, and a contour corresponding mapping table is established;

[0062] A boundary parameter extraction submodule: based on the contour corresponding mapping table, a boundary point corresponding coordinate difference is obtained and a position change amplitude value is calculated, a mapping region average value of each frame gray scale pixel point is calculated and a difference processing is performed with a previous frame result, a boundary curvature change amount and a displacement amplitude are superimposed and analyzed, and a displacement change feature group is generated;

[0063] State sequence generation submodule: based on displacement change feature group, the feature point value of each frame is combined in time sequence order and uniformly converted into state vector structure, all state vectors are bound by frame index and reorganized into data block in form of frame sequence, the continuous jump point and direction trend information between frames are calibrated, and the sequence node structure body is established;

[0064] Image contour mapping submodule: based on the image sequence frame set obtained by the camera during the operation of the hook, the Canny edge detection method is used to perform gradient amplitude calculation operation on the image matrix and set the low threshold to 50 and the high threshold to 150, the edge pixel points of the gray mutation area are extracted, the edge pixel points detected are assigned a unique point index by a label mapping function, the edge region gray mutation point positioning processing in the hook component image is performed, the SIFT feature point matching method is used to call the scale space extreme value detection function for the edge contour point set in adjacent frames, the extreme points in the DoG image are selected and the 128-dimensional local descriptor is used for point feature matching, the Euclidean distance calculation of the matched point pairs is performed and the threshold is set to 0.8 times the minimum distance value for matching screening, the same component contour boundary point alignment and trajectory connectivity verification between adjacent frames are performed, the pixel coordinate change path of the same numbered boundary points in consecutive frames is calculated, the corresponding index information in each frame is extracted and written into the trajectory dictionary structure, and the contour corresponding mapping table is established;

[0065] Boundary parameter extraction submodule: based on the contour corresponding mapping table, the pointPolygonTest function and curvature function combination method in OpenCV are used to perform position curvature quantization calculation on the boundary point set, the coordinate difference of the same point number in X axis and Y axis direction between adjacent frames is extracted and the position change amplitude value is obtained by square difference sum processing, the mean_difference(x, y) function is used to calculate the mean difference of the gray pixel points in the mapping area, the results of adjacent frames are processed, the accumulateCurvature(c1, c2) function is called to perform linear accumulation processing on the boundary curvature change value of the current frame and the previous frame, and the accumulated curvature value and the coordinate difference value are multiplied to form a single feature item, a parameter item group set is formed, and a displacement change feature group is generated;

[0066] State sequence generation submodule: based on displacement change feature group, using state vector generation method transform_to_state_vector(V, t), the feature point value of each frame is sequentially merged according to frame index t, and each feature parameter is combined into a fixed dimension state vector structure. Using frame index mapping table to bind all state vectors to corresponding frame number and reorganize into two-dimensional matrix format data block, calling jump detection function jump_detection(v, n) to perform first-order difference processing on adjacent frame state vectors and mark the frame whose difference value exceeds threshold n as jump point. Compare the gradient direction of the state vector of the jump point before and after the sequence, record the continuous label of the direction change and write it into the trend dictionary structure, and establish the sequence node structure body.

[0067] Please refer to Figure 2 , the state path analysis module includes:

[0068] Point association judgment submodule: based on sequence node structure body, using graph neural network, performing numerical cosine comparison between state vectors of any two nodes and extracting angle value, weighting combining distance change between node center coordinates and angle value and comparing with preset threshold, retaining node pairs meeting connection conditions and forming continuous connection mapping relationship, obtaining graph structure connection relationship table;

[0069] Path weight extraction submodule: based on graph structure connection relationship table, performing state vector change amplitude statistics between adjacent nodes in all connection paths and superimposing to obtain path weight value, performing length and weight product calculation on all paths and sorting to extract maximum value path, extracting node index in maximum path branch completely, obtaining adaptive path weight set;

[0070] State segment construction submodule: based on adaptive path weight set, extracting state value mutation points between path node indexes and screening according to amplitude threshold, aggregating sequence data on both sides of jump point to form state change segment, performing continuous segment merging processing on segment direction change trend, and obtaining key morphological features;

[0071] Point association judgment submodule: based on the sequence node structure, using the graph neural network model GCNConv function, the input matrix construction operation is performed on each node state vector, the node feature matrix X and the graph connection edge table Edge_index are constructed, the state vector matrix X is composed of three-dimensional vector composed of edge curvature value, contour gray value, boundary position difference value of each node, the cosine similarity calculation is performed on each pair of node state vectors and the included angle value is extracted, the torch.nn.functional.cosine_similarity function is called and the dimension is set as dim=1 for batch processing, the included angle value between all node pairs is obtained, the euclidean distance is solved by calling the torch.dist function for node center coordinates, the above distance value and the corresponding included angle value are executed linearly weighted combination, the weight proportion is set as included angle weight 0.6, coordinate distance weight 0.4, and the torch.where function is called to extract the node pairs with weighted combination value less than the set threshold, form the structure pair index group, construct the new edge set matrix, and generate the graph structure connection relationship table;

[0072] Path weight extraction submodule: based on the graph structure connection relationship table, using the network path traversal function networkx.all_simple_paths, setting the maximum path length as 10, extracting the node index sequence in all paths, using the path adjacent node state vector difference calculation function torch.sub and torch.norm to extract the change amplitude, and calling the torch.sum function to execute accumulation according to the path order, generating the state change total amount of single path, recording the number of nodes of all paths, executing the torch.mul function to multiply the number of nodes and the change total amount to generate the path length weight product, calling the torch.topk function to get the maximum value path and node index set for all path products, generating the adaptive path weight set;

[0073] State fragment construction submodule: based on the adaptive path weight set, using the jump point screening function numpy.diff combined with threshold filtering strategy, extracting the interframe difference value of state value in path node index sequence and setting the jump threshold as the mean value of state vector amplitude plus twice standard deviation, extracting the jump point before and after three frame sequence data for all jump point indexes and constructing continuous index segment, using the vector direction cosine included angle calculation function scipy.spatial.distance.cosine to calculate the direction change trend in the segment and setting the adjacent segment with included angle less than 15 degrees as one segment, generating the key morphological feature.

[0074] Please refer to Figure 2 , graph neural network, according to the formula:

[0075]

[0076] wherein: θ i,j represents the multi-dimensional weighted matching value between node i and node j, represents the state vector of node i after encoding by the graph neural network, represents the state vector of node j after encoding by the graph neural network, represents the two-norm of the state vector of node i, represents the two-norm of the state vector of node j, and ∈ represents a small constant to prevent division by zero error, represents the center coordinate vector of node i in the image space, represents the center coordinate vector of node j in the image space, represents the Euclidean distance between node i and node j, λ1 represents the weighting coefficient of the spatial distance component, and Δd i,j represents the absolute value of the difference between the average displacement rate of node i and node j in the historical multi-frames, λ2 represents the weighting coefficient of the dynamic difference component, and t i represents the timestamp index of node i, t j represents the timestamp index of node j, T represents the maximum timestamp index in the current input frame sequence, and λ3 represents the weighting coefficient of the time decay component;

[0077] Execution process: First, the structural feature of the candidate nodes in the input image is encoded to generate the node state vector Subsequently, the module length of each node is calculated and the cosine angle calculation is performed in combination with a small amount of ∈ to avoid division by zero instability, and then the state vector angle value of the node pair is obtained by the inverse cosine function to represent the semantic similarity. Then the center coordinates of node 1 and node j in the image are extracted The Euclidean distance is calculated as a spatial distribution feature, multiplied by the weight coefficient λ1 to form the first weighted component, the moving path of the node in the continuous frame is further extracted, the displacement rate is fitted, and the absolute difference Δd i,j is calculated as a dynamic behavior difference measure and multiplied by the weight λ2 to generate the second weighted component. Then the timestamps t i , t j of nodes i and j are obtained, and the time decay coefficient is calculated by normalized calculation with the global maximum timestamp T and multiplied by λ3 to form the third weighted component. Finally, the three weighted results are added to the angle value to form the complete multi-dimensional node pair association value θ i,j , which is compared with the threshold value to determine whether to construct the node connection relationship, and then the graph structure connection relationship table is generated for subsequent path planning and target object positioning.

[0078] Please refer to Figure 2, a path weight value composed of a cumulative value of state vector variation amplitudes between adjacent nodes in a connection path, used to measure the state variation intensity on the whole path;

[0079] The continuous segment merging processing is performed on the direction change trend, a direction vector of a line connecting the center coordinates between the starting point and the ending point of each segment is calculated, a direction change angle is obtained, and whether it belongs to the same trend interval is determined by using an included angle threshold.

[0080] Please refer to Figure 2 The state label determination module comprises:

[0081] The mutation point extraction submodule: based on the key morphological features, the rate of difference value change between frames is calculated, the local peak value of the continuous frame rate value is searched, and the adjacent difference amplitude is filtered, the high-amplitude points are sorted by index, the change direction is extracted, and the mutation frame position is labeled, and the rate mutation candidate group is obtained;

[0082] The standard value matching submodule: based on the rate mutation candidate group, the difference value between each mutation frame state value and the numerical value of each dimension in the standard label template is calculated, the difference value result is determined with the upper and lower boundary range, the points meeting all dimension range conditions are labeled with label codes and the corresponding frame index is recorded, and the label index structure set is generated;

[0083] The label sequence construction submodule: based on the label index structure set, the hidden Markov model is used to extract the difference value between adjacent frames in the continuous label index and judge the paragraph connectivity, the same type merging operation is performed on the continuous segment, the frame segment identification table is constructed, the segments with a frame distance less than a set threshold are merged and removed, and the multi-state label sequence is established;

[0084] The mutation point extraction submodule: based on the key morphological features, the time series difference rate calculation method numpy.diff is combined with scipy.signal.find_peaks, the difference value of each state vector in the frame sequence is extracted, the diff function is used to calculate the state value difference between adjacent frames in the 0-axis direction and divide the previous frame value to obtain the difference value change rate sequence, the find_peaks function is called and the height threshold is set to the mean value plus 1.5 times the standard deviation, the distance threshold is set to 3 frame intervals to perform local peak value search, the numpy.abs function is called to extract the amplitude value of all local extreme points obtained by searching and perform adjacent difference judgment, and the peak points with an adjacent amplitude difference less than 0.05 are filtered out, the high-amplitude points are sorted in ascending order by frame index, the Euclidean distance direction offset between the previous frame and the next frame state vector is extracted, and the numpy.sign function is called to label the direction trend, the frame number index and direction value of the mutation position are recorded, and the rate mutation candidate group is obtained;

[0085] The standard value matching sub-module: based on the rate jump candidate group, a multi-dimensional difference judgment function pandas.DataFrame.between is combined with a vector difference function torch.sub to perform a dimensional corresponding difference calculation on the state vector in each jump frame and the reference vector in each dimension of the standard label template, call the torch.sub function to calculate the difference between the jump vector and the template vector, and convert it to a DataFrame format, set the upper and lower threshold to ±10% of the template vector value by calling the between function, mark the valid label position by Boolean screening method for the index point that meets the range condition of all dimensions at the same time, call LabelEncoder for label encoding processing after screening the frame number, and record the corresponding relationship between the frame number and the encoding, generate a set of label index structures;

[0086] The label sequence construction sub-module: based on the set of label index structures, a combination method of fit and predict in the Hidden Markov Model class is used, the state number is initialized to 5, and the observation dimension is the state label coding dimension, a time sequence is constructed for all frame numbers in the label index, and the fit function is called to learn the state transition probability, the predict function of the trained model is called to output the state prediction of the input label sequence, the paragraphs with the same state value in the prediction result are divided according to the connectivity index, the start and end frame difference of each paragraph is calculated and the paragraphs with a distance less than 5 frames are merged, and the paragraphs with a state value of 0 are removed by index removal operation. Establish a multi-state label sequence.

[0087] Please refer to Figure 2 , Hidden Markov Model, according to the formula:

[0088]

[0089] Where: δ t (m) represents the optimal path probability value of the label state m at frame time t, δ t-1 (n) represents the optimal path probability value of the label state n at the previous frame time t-1, α nm represents the state transition probability from label state n to label state m, κ nm represents the compatibility weight factor of state transition from n to m in the paragraph structure, represents the weight adjustment coefficient of label state n in the rth label mode based on stability, β m (z t ) represents the observation probability of label state m to observation value z t at frame time t, μ m(t) represents the timing confidence correction factor of the label state m on the current frame t, N represents the total number of label hidden state sets, z t represents the label index difference value observation feature extracted in frame t;

[0090] Execution process: first, the label index of each frame is extracted from the continuous video frame to form the frame sequence label observation set z t , by calculating the optimal path probability δ t -1(n) of the last frame state n as the basis score of path continuation, combined with the state transition probability α nm to evaluate the possibility of transitioning from state n to the current candidate state m, while performing structural consistency test on the paragraph structure graph to extract the structural compatibility factor κ nm to strengthen the logically continuous label paragraph path, and then calculate the stability weight of the label state n under the rth label pattern by the label pattern recognition model , the label path with high label frequency and stable duration is preferentially retained, then the matching probability β t of the state m to the current observation value z m (z t ) is used for observation consistency evaluation, and finally the timing confidence factor μ m (t) is calculated combined with the time frame position t, the state probability in the later frame is dynamically adjusted, all factors are weighted to select the maximum path probability in all previous states, and finally δ t (m) is obtained and is used to construct the label state sequence backward, forming a multi-state label sequence with continuity, inheritance and segmentation recognition ability, which is used to guide the generation of the grabbing strategy and improve the label recognition accuracy in the multi-target unhooking process.

[0091] Please refer to Figure 2 , the attitude angle calculation module comprises:

[0092] Center point determination submodule: based on the multi-state label sequence, the bounding box of each frame target region image is extracted, the average of the left upper and right lower corner coordinates of the bounding box is calculated, and the Euclidean distance between the rectangular center coordinates and the image center point coordinates is calculated and combined into a two-dimensional offset vector. All offset vectors are arranged in order of frame index to obtain a spatial offset vector set;

[0093] Angle trend derivation submodule: based on the spatial offset vector set, the angle between three consecutive vectors is calculated, the absolute value of the angle between the first and last vectors of each group is counted, and the rising and falling trend is judged, and the trend information is bound with the corresponding frame index. Label each frame with the angle direction trend category to generate a direction angle label group;

[0094] Angle channel extraction submodule: based on the direction angle label group, all frames are divided into pitch channel, yaw channel and roll channel according to the trend category, the amplitude change value of each channel is extracted, combined into a floating section, and the angle sequence is generated after the channel number is added to the processing, and the attitude rotation angle vector set is obtained;

[0095] Center point determination submodule: based on the multi-state label sequence, the contour extraction is performed on the two-dimensional image of the target area in each frame by combining the cv2.boundingRect and coordinate mean function in OpenCV, the cv2.findContours function is called to set the retrieval mode as RETR_EXTERNAL and the approximation method as CHAIN_APPROX_SIMPLE to obtain the maximum circumscribed contour, the cv2.boundingRect function is applied to the contour to obtain the left upper corner and right lower corner coordinate values, the X coordinate and Y coordinate of the two points are respectively called numpy.mean function to calculate the mean value, and the rectangular center point coordinates are obtained, the center point of each frame image is set as the image width and height center point position, the numpy.sqrt and numpy.square functions are called to perform square sum and square root calculation on the X and Y coordinate difference between the center point and the image center point, and the single-frame two-dimensional offset vector is generated, and the offset vectors of all frames are combined in ascending order of frame index to generate the spatial offset vector set;

[0096] Angle trend derivation submodule: based on the spatial offset vector set, the direction angle cosine function scipy.spatial.distance.cosine is combined with the sliding window grouping processing method to perform three-frame sliding window cutting on the offset vector sequence arranged according to the index, the first and last two offset vectors of each group are extracted, the cosine function is called to calculate the direction angle value and take the arccos result, then multiply by 180 and divide by π to get the angle value, the numpy.diff function is called to extract the first-order change rate of all angle values, the numpy.sign function is used to judge the change rate direction as positive, negative or zero, the mapping label value is set for different direction categories, and the trend category and corresponding frame index are bound one by one by calling pandas.DataFrame to generate the direction angle label group;

[0097] Angle channel extraction submodule: based on the direction angle label group, using the trend grouping function pandas.groupby and the amplitude classification processing function numpy.ptp, grouping the trend information of all frames according to the trend category field, mapping the grouped frame segments to three numbered channels representing pitch, yaw and roll categories respectively, extracting the direction angle value of each paragraph in each channel and calling the numpy.ptp function to calculate the maximum and minimum value difference of the segment to obtain the amplitude change value, merging the amplitude value and the trend label into a floating segment structure, calling the numpy.concatenate function to add numbered information to each channel segment and integrate into a unified angle sequence structure, generating a posture rotation angle vector set.

[0098] Please refer to Figure 2 , the spatial state fusion module comprises:

[0099] State posture binding submodule: based on the posture rotation angle vector set and the multi-state label sequence, the state label and angle vector number are aligned under each frame index, the matching frame number is selected by range filtering the confidence order value, the label number and three-axis angle value are spliced and bound, and a binding index mapping matrix is established, generating a frame-level binding label pair;

[0100] Combined difference calculation submodule: based on the frame-level binding label pair, all three-axis angle vector numerical combination extraction processing under the same state category is performed, the maximum amplitude and minimum amplitude difference calculation is performed on each angle sequence to generate the fluctuation interval value, the angle change frequency in each state combination is counted and the stable and unstable range is divided, and a posture difference score set is generated;

[0101] Structure result generation submodule: based on the posture difference score set, the frame sequence restoration operation of each state label and score segment binding information is performed, the combined data sequence is executed state channel and angle channel joint sorting, each frame state code and angle number is filled and marked into the frame index column, and the fusion annotation result set is established;

[0102] State pose binding submodule: based on the pose rotation angle vector set and the multi-state label sequence, using the matrix index alignment function pandas.merge and the confidence filtering algorithm torch.topk, the primary key merging operation is performed on the state label and angle vector number under each frame index, and a matching matrix is generated, and the confidence values of all matching records are sorted according to the three-axis angle vector source model output, and the reserved interval range is set to top30%, the torch.topk function is called to select the frame number in the top percentage segment, the state label number corresponding to the frame number and the three-axis angle vector X, Y, Z value are called torch.cat function to splice in the row direction, and merged into a state-pose six-dimensional vector, and a pandas.DataFrame index is established as the frame number and the merged vector content Data matrix, generate frame-level binding label pairs;

[0103] Combined difference calculation submodule: based on the frame-level binding label pair, using the grouping operation function pandas.groupby and the amplitude difference calculation function numpy.ptp, the grouping operation is performed on the combined data with the same state number in all records, the X, Y, Z three-axis angle sequence in each group is extracted, and the numpy.ptp function is called to calculate the maximum and minimum difference of each axis respectively, the amplitude values of the three axes are combined into a single fluctuation interval item, and the numpy.histogram function is called to divide the frequency interval by 5°, the interval with a frequency greater than the average frequency is marked as a stable segment, and the rest is a non-stable segment. The state number and the respective interval classification value are combined to generate a new score field, and a state-angle fluctuation mapping dictionary is constructed to generate a pose difference score set;

[0104] Structure result generation submodule: based on the pose difference score set, using the index restoration and channel number sorting function pandas.merge and the combination of numpy.lexsort, the state number and score segment field are restored according to the frame sequence and state label field, and a frame sequence data frame is constructed. Call the numpy.lexsort function on the combined data table to set the first key as the state channel number and the second key as the angle channel number, and call the numpy.append function for each row of the sorted result to fill the state label number and the angle direction number into each frame record respectively. Call the DataFrame.insert function to create an output data table with the frame index as the column label to establish a fusion annotation result set.

[0105] Please refer to Figure 3 An AI precise recognition and grabbing method based on pattern recognition of the hook picking system, the AI precise recognition and grabbing method based on pattern recognition of the hook picking system is executed based on the above-mentioned AI precise recognition and grabbing system based on pattern recognition of the hook picking system, comprising the following steps:

[0106] S1: Based on the image sequence frame set captured by the camera, the edge contour value, boundary grayscale value, regional grayscale mean and edge coordinate offset of the hook component in each frame are extracted, the contour point set is arranged in time series, and the matching position coordinate difference and grayscale value change of the edge contour points between adjacent frames are calculated. The edge contour offset trajectory is superimposed and the corresponding frame number is recorded. The node index matrix is ​​established and associated with the corresponding edge data structure to establish a sequence node structure.

[0107] S2: Based on the sequence node structure, the node state vector is extracted and the Euclidean distance between the cosine angle of the edge morphological feature and the center coordinate is calculated. The two are weighted and the node pairs below the threshold are filtered to generate a connection index table. The path set is constructed and the path jump amplitude is counted. The maximum path is filtered and the frame number of the mutation point is extracted. The direction segments are merged according to the angle similarity. The graph neural network is used to construct the connection structure and extract the jump path. The key morphological feature index is obtained and classified into the same group. The direction vector angle between the nodes in the same group is grouped and the similar direction segments are merged. The graph neural network is used to construct the structure of the connection relationship between the nodes and extract the jump path to obtain the key morphological features.

[0108] S3: Based on key morphological features, the node position change rate sequence in each segment is calculated and the jump rate value is obtained. The jump rate value is compared and matched with the set standard state rate reference value segment by segment. The frame number sequence corresponding to the matching segment is identified and the state category index is marked. The repeated segment merge operation is performed on the state category index. After removing the unclear classification segments, the jump rate stability is judged and the multi-segment state sequence is optimized through the hidden Markov model to obtain the polymorphic label sequence.

[0109] S4: Based on the polymorphic label sequence, the center coordinate point of the corresponding area of ​​each frame is extracted and the displacement vector value of the center point between adjacent frames is calculated. The angle between adjacent vectors is extracted according to the frame sequence and the angle trend change value of each direction channel is recorded. The pitch angle change sequence, yaw angle change sequence, and roll angle change sequence are separated according to the time series. The continuous segment judgment and direction trend classification of the angle sequence are performed respectively to obtain the attitude rotation angle vector set;

[0110] S5: Based on the posture rotation angle vector set and the polymorphic label sequence, the combined value of each frame state label and the three-axis posture angle is extracted. The repeated state and posture combinations are counted and the three-axis angle difference range between the combinations is calculated. The combination segments whose difference range is lower than the set tolerance are indexed and merged to generate a continuous segment label index table to establish a fusion annotation result set.

[0111] The above merely describes the preferred embodiments of the present application, and is not intended to limit the present application in other forms. Any skilled person in the art can modify or change the disclosed technical content into equivalent embodiments with equivalent changes, and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical solution content of the present application, still falls within the protection scope of the present application.

Claims

1. A pattern recognition-based AI precise recognition and grabbing system for retrieving hooks, characterized in that: The system comprises: Dynamic node construction module: The camera is used to obtain a set of image sequence frames during the hook removal and re-hook operation, and the hook component contour mapping, boundary difference calculation, regional grayscale change measurement and edge displacement accumulation processing are performed to establish a sequence node structure; State path parsing module: Based on the sequence node structure, a graph neural network is used to perform a joint comparison between node cosine values ​​and coordinate differences, extract path weight branches, count the state vector jump amplitudes and combine them into a sequence to obtain key morphological features; State label determination module: Based on the key morphological features, the hidden Markov model is used to screen state rate mutation points, perform standard value matching and classification on candidate segments, merge repeated label segments and eliminate invalid segments to establish a polymorphic label sequence; Attitude angle estimation module: Based on the polymorphic tag sequence, it calculates the regional center point offset and angle trend, extracts the continuously changing values ​​of pitch, yaw, and roll angles according to the direction channel, and obtains the attitude rotation angle vector set; Spatial state fusion module: Based on the posture rotation angle vector set and the polymorphic label sequence, bind each frame state and posture combination, count the difference range of repeated combinations and generate joint annotations, and establish a fusion annotation result set.

2. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 1 is characterized in that: The dynamic node construction module includes: Image contour mapping submodule: This module uses a camera to capture a set of image sequence frames during the hook removal and re-hooking operation, locates grayscale mutation points in the edge area of ​​the hook component image, aligns the boundary points of the same component contours between adjacent frames, and verifies the trajectory connectivity. It calculates the pixel movement paths of the same-numbered boundary points in consecutive frames, records the corresponding frame index information, and establishes a contour corresponding mapping table. Boundary parameter extraction submodule: Based on the contour mapping table, the coordinate difference of the boundary points is obtained and the position change amplitude is calculated. The mean value of the grayscale pixels in each frame in the mapping area is calculated and subtracted from the result of the previous frame. The boundary curvature change and displacement amplitude are then superimposed and analyzed to generate a displacement change feature group. State sequence generation submodule: Based on the displacement change feature group, the feature point values ​​under each frame are combined in time series order and uniformly converted into a state vector structure. All state vectors are bound to frame indexes and reorganized into data blocks in the form of frame sequences. The continuous jump points and directional trend information between frames are calibrated to establish a sequence node structure.

3. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 1 is characterized in that: The state path parsing module includes: Point association judgment submodule: Based on the sequence node structure, a graph neural network is used to perform a numerical cosine comparison between the state vectors of any two nodes and extract the angle value. The distance change between the node center coordinates and the angle value are weighted and combined and compared with a preset threshold. The node pairs that meet the connection conditions are retained and a continuous connection mapping relationship is formed to obtain a graph structure connection relationship table; Path weight extraction submodule: Based on the graph structure connection relationship table, the state vector change amplitude between adjacent nodes in all connection paths is counted and superimposed to obtain the path weight value. The length and weight product of all paths are calculated and sorted to extract the maximum value path. The node index in the maximum path branch is fully extracted to obtain the adaptive path weight set. State fragment construction submodule: Based on the adaptation path weight set, the state value mutation points between the path node indexes are extracted and filtered according to the amplitude threshold, the sequence data on both sides of the jump point are aggregated to form state change fragments, and the continuous segment merging processing is performed on the fragment direction change trend to obtain key morphological features.

4. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 3 is characterized in that: The graph neural network is based on the formula: Where: θ i,j represents the multi-dimensional weighted matching value between node i and node j, represents the state vector of node i after the graph neural network is encoded, represents the state vector of node j after graph neural network encoding, represents the two-norm of the state vector of node i, represents the two-norm of the state vector of node j, ∈ represents a small constant to prevent division by zero errors, represents the center coordinate vector of node i in the image space, represents the center coordinate vector of node j in the image space, represents the Euclidean distance between node i and node j, λ1 represents the weighting coefficient of the spatial distance component, Δd i,j represents the absolute value of the difference between the mean displacement change rates of node i and node j in the historical multi-frame, λ2 represents the weighting coefficient of the dynamic difference component, t i Represents the timestamp index of node i, t j represents the timestamp index of node j, T represents the maximum timestamp index in the current input frame sequence, and λ3 represents the weighting coefficient of the time decay component.

5. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 3 is characterized in that: The path weight value is composed of the cumulative value of the change amplitude of the state vector between adjacent nodes in the connection path, which is used to measure the intensity of the state change on the entire path; The direction change trend is processed by merging continuous segments. The direction change angle is obtained by calculating the direction vector of the center coordinate connecting the starting point and the ending point of each segment, and the angle threshold is used to determine whether it belongs to the same trend interval.

6. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 1 is characterized in that: The status label determination module includes: Mutation point extraction submodule: Based on the key morphological features, it calculates the difference change rate of each state vector frame, searches for local peaks in the continuous frame rate values ​​and performs adjacent difference amplitude screening. After sorting the high amplitude points by index, it extracts the change direction and marks the mutation frame position to obtain the rate mutation candidate group; Standard value matching submodule: Based on the rate mutation candidate group, the difference between each mutation frame state value and the value of each dimension in the standard label template is calculated, the difference result and the upper and lower boundary ranges are judged simultaneously, and the points that meet the conditions of all dimension ranges are annotated with labels and the corresponding frame index is recorded to generate a label index structure set; Label sequence construction submodule: Based on the label index structure set, a hidden Markov model is used to extract the difference between adjacent frames in the continuous label index and determine the connectivity of paragraphs. The same type of merging operation is performed on continuous segments and a frame segment identification table is constructed. For segments with a frame distance less than a set threshold, label inheritance merging is performed and zero-label paragraphs are removed to establish a polymorphic label sequence.

7. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 6 is characterized in that: The hidden Markov model is based on the formula: Where: t (m) represents the optimal path probability value with the tag state m at frame time t, δ t-1 (n) represents the optimal path probability value of the tag state n at the previous frame time t-1, α nm represents the state transition probability from label state n to label state m, κ nm Represents the compatibility weight factor of the state transition from n to m in the paragraph structure, represents the weight adjustment coefficient of label state n in the rth label mode based on stability, β m (z t ) represents the observed value z at the frame time t for the label state m t The observation probability, μ m (t) represents the temporal confidence correction factor of the label state m at the current frame t, N represents the total number of states in the label hidden state set, z t represents the label index difference observation feature extracted in frame t.

8. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 1 is characterized in that: The attitude angle calculation module includes: Center point determination submodule: Based on the polymorphic tag sequence, extract the bounding box of the target area image in each frame, average the coordinate values ​​of the upper left and lower right corners of the bounding box and calculate the center of the rectangle, perform Euclidean distance calculation on the coordinates of the rectangle center and the coordinates of the image center point, and combine them into a two-dimensional offset vector. All offset vectors are arranged in frame index order to obtain a spatial offset vector set; Angle trend derivation submodule: Based on the spatial offset vector set, it calculates the angle of continuous vectors in groups of three frames, calculates the absolute value of the angle between the direction changes of the first and last vectors of each group, and determines the upward and downward trends. It also binds the trend information to the corresponding frame index, labels the angle direction trend category of each frame, and generates a direction angle tag group; Angle channel extraction submodule: Based on the direction angle tag group, all frames are divided into pitch channel, yaw channel, and roll channel according to trend categories. The amplitude change values ​​of the continuous angle trend segments in each channel are extracted and combined into floating segments. After additional channel number processing is performed on each segment, an angle sequence is generated to obtain the attitude rotation angle vector set.

9. The AI ​​precise recognition and grabbing system for retrieving hooks based on pattern recognition according to claim 1 is characterized in that: The spatial state fusion module includes: State and posture binding submodule: Based on the posture rotation angle vector set and the polymorphic label sequence, the state label and the angle vector number are aligned under each frame index. By filtering the confidence ranking value and selecting the matching frame sequence, the label number and the three-axis angle value are spliced ​​and bound, and a binding index mapping matrix is ​​established to generate a frame-level binding label pair. Combination difference calculation submodule: Based on the frame-level binding label pairs, it extracts and processes the numerical combinations of all three-axis angle vectors under the same state category. By calculating the maximum and minimum amplitude differences of each angle sequence and generating a fluctuation interval value, it counts the frequency of angle changes in each state combination and delineates stable and unstable range segments to generate a posture difference score set. Structural result generation submodule: Based on the posture difference score set, a frame sequence restoration operation is performed on the binding information of each state label and score segment, the state channel and angle channel are jointly sorted on the combined data sequence, the state code and angle number of each frame are dual-channel filled and marked into the frame index column, and a fusion annotation result set is established.

10. A method for accurately identifying and grabbing hooks based on pattern recognition AI, characterized in that: The AI ​​precise recognition and grasping system for picking and re-hooking based on pattern recognition according to any one of claims 1 to 9 is implemented, comprising the following steps: S1: Based on the image sequence frame set captured by the camera, the edge contour value, boundary grayscale value, regional grayscale mean and edge coordinate offset of the hook component in each frame are extracted, the contour point set is arranged in time series, and the matching position coordinate difference and grayscale value change of the edge contour points between adjacent frames are calculated. The edge contour offset trajectory is superimposed and the corresponding frame number is recorded. The node index matrix is ​​established and associated with the corresponding edge data structure to establish a sequence node structure. S2: Based on the sequence node structure, extract the node state vector and calculate the Euclidean distance between the cosine angle of the edge morphological feature and the center coordinate. After weighting the two, filter out the node pairs below the threshold to generate a connection index table, build a path set and count the path jump amplitude, filter the maximum path and extract the mutation point frame number, merge the direction segments according to the angle similarity, use the graph neural network to build the connection structure and extract the jump path, obtain the key morphological feature index and classify them into the same group, perform group judgment on the direction vector angle between nodes in the same group and merge similar direction segments, use the graph neural network to construct the structure of the connection relationship between nodes and extract the jump path, and obtain the key morphological features; S3: Based on the key morphological features, the node position change rate sequence in each segment is calculated and the jump rate value is obtained. The jump rate value is compared and matched with the set standard state rate reference value segment by segment, the frame number sequence corresponding to the matching segment is identified and the state category index is marked. The repeated segment merging operation is performed on the state category index. After removing the unclear classification segments, the jump rate stability is judged and the multi-segment state sequence is optimized through the hidden Markov model to obtain a polymorphic label sequence. S4: Based on the polymorphic tag sequence, extract the center coordinate point of the corresponding area of ​​each frame and calculate the center point displacement vector value between adjacent frames, extract the angle between adjacent vectors according to the frame sequence and record the angle trend change value of each direction channel, separate the pitch angle change sequence, yaw angle change sequence and roll angle change sequence according to the time series, perform continuous segment judgment and direction trend classification of the angle sequence respectively, and obtain the attitude rotation angle vector set; S5: Based on the posture rotation angle vector set and the polymorphic label sequence, extract the combination value of each frame state label and the three-axis posture angle, count the repeated state and posture combinations and calculate the three-axis angle difference range between the combinations, perform index merging processing on the combination segments whose difference range is lower than the set tolerance and generate a continuous segment label index table to establish a fusion annotation result set.

Images