Gesture recognition method of automatic cloth cutting machine based on deep vision

Abstract

The application relates to the technical field of intelligent cloth cutting, and discloses an automatic cloth cutting machine gesture recognition method based on deep vision. The method acquires deep vision data to be recognized, identifies a gesture operation environment and a gesture type according to the deep vision data, and collects environment characteristic data; gesture recognition model parameters are set based on the gesture operation environment and the type, and the relative positions of key gestures in a model are located; a trajectory prediction network of the key gestures is constructed based on the relative positions, gesture response rules are set in combination with the trajectory prediction network and the relative positions; gesture operation states are sensed in real time through the gesture recognition model and the response rules, and environment change trends are analyzed based on the deep vision data; an abnormality sensing mode is set in combination with the operation states and the environment change trends, gesture abnormal conditions are identified based on the mode; a gesture adjustment mechanism is set according to the abnormal conditions and the trajectory prediction network, and finally, gesture recognition results are output according to the abnormality sensing mode, the adjustment mechanism and the response rules.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent cloth cutting, in particular to a gesture recognition method for an automatic cloth cutting machine based on deep vision. BACKGROUND

[0002] In the automatic production process of the textile and garment industry, the automatic cloth cutting machine as a key equipment, its operation convenience and intelligent level directly affect the production efficiency and product quality. The operation of the traditional automatic cloth cutting machine mostly depends on physical keys, touch panels or pre-set programs, and the operator needs to control the equipment operation through specific mechanical contact or fixed instruction input. This operation mode has obvious limitations. When facing diversified cloth types, complex cutting patterns or temporary adjustment of production requirements, the operator needs to frequently manually switch parameters or modify programs, which not only increases the operation burden, but also easily causes cutting deviation due to operation errors, affecting the production progress.

[0003] With the development of machine vision and artificial intelligence technology, gesture recognition technology has been gradually introduced into the field of industrial equipment control, and its non-contact operation characteristics provide the possibility for improving equipment interaction efficiency. However, when applying gesture recognition technology to the automatic cloth cutting scene, the existing scheme faces many challenges. The cloth cutting workshop environment is complex, with unstable lighting conditions, diverse cloth textures, and machine vibration interference factors. The traditional gesture recognition method based on RGB images is easily affected by light changes and is difficult to accurately extract gesture features. At the same time, the gesture actions of the operator vary under different cloth thicknesses and cutting speeds, and the existing model lacks dynamic perception ability to the operating environment, resulting in large fluctuations in gesture recognition accuracy.

[0004] The operation gesture of the automatic cloth cutting machine often contains continuous trajectory and instantaneous instruction. The existing method is not accurate enough in positioning the key gesture, and the trajectory prediction is lagging, which cannot respond to the operator's intention in time. In actual production, if the gesture recognition is misjudged or missed, it may cause abnormal operation of the equipment, such as cutting path deviation, cutter misstart, etc., which not only affects the cloth cutting quality, but also has safety hazards. The existing control mode of the cloth cutting machine has the problems of difficult operation of emergency braking and inconvenient manual keys, which is the main factor causing danger. The traditional key is a positive key type, and people in danger need a "simple and accessible" control mode, so it is necessary and urgent to add a gesture recognition module to the cloth cutting machine for control. At the same time, the existing technology lacks effective perception and processing mechanism for abnormal situations in the operation process, such as blurred gestures and environmental mutations, which further limits the reliable application of gesture recognition technology in automatic cloth cutting machines. Therefore, there is an urgent need for a gesture recognition method that can adapt to complex environments, accurately recognize gesture intentions and has abnormal processing capability to meet the intelligent operation needs of automatic cloth cutting machines. SUMMARY

[0005] The application aims to provide a gesture recognition method for an automatic cloth cutting machine based on depth vision to solve the problems in the background art.

[0006] To achieve the above-mentioned purpose, the application provides a gesture recognition method for an automatic cloth cutting machine based on depth vision, which comprises the following steps:

[0007] Obtaining depth vision data to be recognized, and recognizing a gesture operation environment and a gesture type based on the depth vision data, and collecting environment feature data of the gesture operation environment;

[0008] Based on the gesture operation environment and the gesture type, setting model parameters of a gesture recognition model, and positioning a relative position of a key gesture in the gesture recognition model;

[0009] Based on the relative position, constructing a trajectory prediction network of the key gesture;

[0010] According to the trajectory prediction network and the relative position, setting a gesture response rule;

[0011] Combining the gesture recognition model and the gesture response rule, real-time sensing a gesture operation state; based on the depth vision data, analyzing an environment change trend of the gesture operation environment;

[0012] Combining the gesture operation state and the environment change trend, setting an abnormality sensing mode;

[0013] Based on the abnormality sensing mode, recognizing a gesture abnormality situation;

[0014] According to the gesture abnormality situation and the trajectory prediction network, setting a gesture adjustment mechanism;

[0015] According to the abnormality sensing mode, the gesture adjustment mechanism and the gesture response rule, outputting a gesture recognition result.

[0016] Preferably, the gesture operation environment and the gesture type are recognized based on the depth vision data, which comprises the following steps: based on the depth vision data, extracting background features of the gesture operation environment;

[0017] According to the background features, dispatching operation scene information and a gesture type of the gesture operation environment;

[0018] Based on the operation scene information and the gesture type, adaptively configuring model parameters of the gesture recognition model to obtain a configured gesture recognition model;

[0019] Combining the operation scene information and the configured gesture recognition model, determining environment feature data of the gesture operation environment.

[0020] Preferably, the constructing a trajectory prediction network of the key gesture based on the relative position comprises: calculating a relative position of adjacent gestures of the key gesture based on the relative position;

[0021] identifying a non-key gesture on the relative position of the adjacent gestures;

[0022] analyzing a characteristic behavior of the non-key gesture relative to the key gesture;

[0023] extracting an operation behavior feature of the key gesture, and analyzing an operation influence of the operation behavior feature on the non-key gesture according to the characteristic behavior;

[0024] constructing a trajectory prediction network of the key gesture according to the characteristic behavior and the operation influence.

[0025] Preferably, the setting a gesture response rule according to the trajectory prediction network and the relative position comprises: monitoring an operation gesture of a gesture sequence in real time, and identifying an abnormal gesture in the operation gesture;

[0026] analyzing a sequence influence of the abnormal gesture on the operation gesture according to the trajectory prediction network;

[0027] scheduling operation association data of the gesture sequence and the abnormal gesture, identifying an abnormal reason of the abnormal gesture based on the operation association data;

[0028] setting a gesture compensation mechanism of the abnormal gesture according to the sequence influence and the abnormal reason;

[0029] identifying an operation trajectory of the gesture sequence, setting a homing path of the abnormal gesture based on the operation trajectory and the gesture compensation mechanism;

[0030] setting the gesture response rule in combination with the relative position and the homing path.

[0031] Preferably, the analyzing an environmental change trend of the gesture operation environment based on the depth visual data comprises: performing attribute classification on the depth visual data to obtain classified environmental data;

[0032] extracting a key environmental feature of the classified environmental data, and collecting historical environmental data of the gesture operation environment based on the key environmental feature;

[0033] identifying operation interference data and light change data in the historical environmental data;

[0034] analyzing an operation interference mode and a gesture behavior mode of the gesture operation environment according to the operation interference data;

[0035] analyzing a visibility level of the gesture operation environment based on the light change data;

[0036] Combine the operation interference mode, gesture behavior mode and visibility level, analyze the environmental change trend of the gesture operation environment.

[0037] Preferably, the gesture operation state and the environmental change trend are combined to set an abnormal perception mode, including: identifying surrounding interference gestures of the gesture sequence based on the gesture operation state;

[0038] Analyze the interaction relationship between the surrounding interference gestures and the gesture sequence; identify potential risk factors of the gesture sequence according to the environmental change trend and the interaction relationship, and identify the risk level of the potential risk factors;

[0039] According to the gesture operation state and the risk level, set the risk response mode of the gesture sequence;

[0040] Combine the risk level and the risk response mode to set the abnormal perception mode.

[0041] Preferably, the abnormal perception mode is used to identify the gesture abnormal situation, including: collecting historical operation data of the gesture sequence based on the abnormal perception mode; identifying normal operation parameters of the gesture sequence according to the historical operation data;

[0042] Identify the current operation state of the gesture sequence, and identify the current behavior deviation of the gesture sequence in combination with the current operation state and the historical operation data;

[0043] Based on the normal operation parameters, set the abnormal parameter threshold of the gesture sequence;

[0044] According to the abnormal parameter threshold and the current behavior deviation, identify the gesture abnormal situation.

[0045] Preferably, the gesture adjustment mechanism is set according to the gesture abnormal situation and the trajectory prediction network, including: positioning the abnormal operation gesture of the gesture sequence according to the gesture abnormal situation;

[0046] Analyze the abnormal type of the abnormal operation gesture, and set the abnormal response layer of the abnormal operation gesture based on the abnormal type;

[0047] According to the trajectory prediction network, identify the adjacent gesture trajectory of the abnormal operation gesture;

[0048] Based on the adjacent gesture trajectory, set the adjustment path of the abnormal operation gesture;

[0049] Combine the abnormal response layer and the adjustment path to set the gesture adjustment mechanism.

[0050] Preferably, the output gesture recognition result includes: identifying the operation abnormality of the gesture sequence according to the abnormal perception mode;

[0051] Based on the operation exception, the trigger condition of the gesture adjustment mechanism is set, and the risk avoidance mode of the gesture sequence is set;

[0052] According to the trigger condition and the risk avoidance mode, the gesture sequence is recognized and adjusted to obtain adjusted gesture data;

[0053] Based on the adjusted gesture data and the gesture response rule, a gesture recognition result is output.

[0054] Preferably, the method further comprises: based on the gesture recognition result, estimating the overall stability of the gesture sequence; generating a warning gesture sequence according to the overall stability; and temporarily adjusting the warning gesture sequence.

[0055] Compared with the prior art, the present application has the following advantages:

[0056] The gesture recognition method of the automatic cloth cutting machine based on depth vision provided by the present application acquires gesture operation environment and gesture type through depth vision data. Compared with the traditional recognition method relying on RGB images, the depth data can effectively avoid the interference caused by changes in illumination and differences in fabric texture, more accurately capture the three-dimensional features and environmental information of gestures, and make the gesture recognition stable in a complex cloth cutting workshop environment. After collecting environmental feature data, the gesture recognition model parameters are set based on the specific operation environment and gesture type, realizing dynamic adjustment of the model parameters and avoiding the problem of insufficient adaptability of the fixed parameter model under different working conditions, so that the model can better match the actual operation scene and improve the pertinence of recognition.

[0057] The relative position of the key gesture in the model is determined, laying a foundation for subsequent trajectory prediction, so that the trajectory prediction can focus on the motion path of the key gesture and reduce the interference of irrelevant information. The key gesture trajectory prediction network constructed based on the relative position can perceive the motion trend of the gesture in advance, making the gesture response more forward-looking and avoiding operation delay caused by response lag. The gesture response rule is set in combination with the trajectory prediction network and the relative position, so that the device responds to different gestures more in line with the intention of the operator, enhancing the smoothness of human-computer interaction.

[0058] The gesture operation state is perceived in real time through the gesture recognition model and the response rule, the gesture changes in the operation process can be dynamically tracked, and the accurate capture of each operation step is ensured. At the same time, based on the depth vision data, the trend of environmental change is continuously analyzed, the subtle changes in the environment during the cloth cutting process, such as changes in the stacking height of the cloth and changes in the running state of the device, can be perceived in time, providing a basis for subsequent abnormal perception. The abnormal perception mode is set in combination with the gesture operation state and the trend of environmental change, so that the system can comprehensively judge whether there is an abnormality in the operation process, such as gesture blur and operation trajectory deviation from the regular, and active perception of abnormal conditions is realized.

[0059] After recognizing the gesture abnormal situation based on the abnormal perception mode, the gesture adjustment mechanism is set according to the abnormal situation and the trajectory prediction network, the recognition result can be dynamically corrected, the influence of the abnormal situation on the recognition accuracy is reduced, and it is ensured that the device can still maintain stable operation when encountering a sudden situation. Finally, the gesture recognition result is output according to the abnormal perception mode, the gesture adjustment mechanism and the gesture response rule, and the information such as environment perception, trajectory prediction, abnormal processing and the like is integrated, so that the output recognition result is more comprehensive and accurate, the intelligent operation level of the automatic cloth cutting machine is effectively improved, and the diversified and high-precision cutting demand in textile production is met. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 A working principle diagram of the gesture recognition method of the automatic cloth cutting machine based on deep vision is provided.

[0061] Figure 2 A flowchart for gesture operation environment and gesture type recognition is provided.

[0062] Figure 3 A flowchart for key gesture trajectory prediction network construction is provided.

[0063] Figure 4 A flowchart for gesture operation environment change trend analysis is provided. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0065] Please refer to Figure 1 The present application provides a gesture recognition method of an automatic cloth cutting machine based on deep vision, which comprises:

[0066] The gesture recognition method of the automatic cloth cutting machine based on depth vision is implemented as follows: the depth vision data containing the gestures of the operator and the surrounding environment are acquired through a depth vision sensor. Based on the depth vision data, the specific environment (such as the light condition, the background object, and the state of the workbench) in which the current gesture operation is located and the type of the gesture (such as line drawing, cutting, and pausing) being performed by the operator are identified; at the same time, the specific environmental feature data of the gesture operation environment are collected. According to the identified gesture operation environment and gesture type, the key parameters of the gesture recognition model are dynamically set, such as adjusting the sensitivity of feature extraction, the classification threshold, or the network structure parameters. Using the configured model, the relative position coordinates of the key gestures (such as the cutting starting point and the direction turning point) in the current sequence in the model processing space are located. Based on the relative position information of the key gestures, a trajectory prediction network is constructed, which is used to predict the possible moving path and trend of the key gestures. Combined with the constructed trajectory prediction network and the relative position information of the key gestures, gesture response rules are set, which define the specific response actions (such as starting cutting, adjusting speed, and stopping) that the system should make to different gestures and their predicted trajectories. Integrating the gesture recognition model and the gesture response rules, the system can real-time perceive and analyze the gesture operation state (such as gesture type, position, speed, and continuity) of the operator. At the same time, based on the continuously acquired depth vision data, the change trend of the gesture operation environment (such as the gradual darkening of the light and the moving of the background object close to the operator) is analyzed. Fusing the real-time perceived gesture operation state and the change trend of the environment, an abnormal perception mode is set, which includes the conditions and thresholds for detecting the deviation from the normal operation. Using this abnormal perception mode, various gesture abnormal situations (such as gesture interruption, sudden acceleration, and exceeding the safety area) that may occur are real-time monitored and identified. According to the identified gesture abnormal situations and their specific types, and combined with the prediction information provided by the trajectory prediction network, a corresponding gesture adjustment mechanism is set, which aims to correct the abnormality or guide the gesture back to the expected path. Finally, by comprehensively using the abnormal perception mode, the gesture adjustment mechanism, and the pre-set gesture response rules, the gesture sequence of the operator is real-time processed, adjusted, and judged, and the final gesture recognition result is output, which directly drives the execution mechanism (such as the cutting head) of the automatic cloth cutting machine to perform corresponding actions.

[0067] Example 1: see Figure 2After acquiring the depth visual data, the system first processes the data to extract the background features of the gesture operating environment. The background features include the three-dimensional depth distribution of the workbench surface, the contour geometric information of the background objects, the distance parameters of the static obstacles, the spatial orientation and intensity gradient of the main light source, and other physical attributes. These features are extracted through depth image segmentation algorithms and point cloud spatial clustering techniques, such as using region growing method to segment the workbench surface, identifying the edge of the background object through curvature analysis, and calculating the light source influence range with the aid of light sensor auxiliary data. According to the extracted background features, the system calls the pre-set environment scene database for matching and scheduling, and outputs the operating scene information classification of the current gesture operating environment (such as "high-reflective metal table surface environment" "multiple hanging cloth interference environment") and the preliminary identification of gesture type (such as "straight line cutting gesture" "Z-shaped fold line gesture").

[0068] The operating scene information includes the typical interference mode library, the historical recognition accuracy statistics, and the recommended processing strategy index of the environment category. The gesture type determination is based on the spatiotemporal feature analysis of the depth image sequence by convolutional neural network, and outputs the preliminary classification label and confidence of the gesture. Based on the scene classification and gesture type label obtained by scheduling, the system performs dynamic configuration of the gesture recognition model parameters: for "high-reflective environment", enable the depth data reflection suppression module and adjust the edge detection threshold to the high value interval of the pre-set range; for "Z-shaped fold line gesture", load the high-precision trajectory tracking sub-model, and set the trajectory smoothing coefficient to low sensitivity mode. The parameter configuration process is realized by querying the pre-defined configuration rule matrix, which establishes the mapping relationship between scene type, gesture category and model parameter adjustment item. After completing the parameter configuration, the optimized gesture recognition model instance is output.

[0069] Finally, combined with the environmental attribute description (such as interference object density value, light source stability rating) in the operating scene information and the calculation requirements of the optimized model, the environment feature data set is generated. The data set includes quantitative indicators: environmental noise coefficient calculated by the signal-to-noise ratio of the depth image, effective operating area boundary coordinate set based on point cloud density analysis, light interference probability value calculated according to light fluctuation history data, etc. These data are stored in a structured format for subsequent module calls.

[0070] Embodiment 2: see Figure 3 After determining the relative position coordinates of the key gesture in the model space, the system takes the coordinates as the reference point to calculate the relative position offset of the gesture center point in the adjacent time sequence frame, generating a sequence of adjacent gesture relative positions containing distance difference and angle difference. Identify the non-key gesture entities corresponding to each position point in the sequence, which are usually represented as preparation actions before the key gesture operation or subsequent continuous fine-tuning actions.

[0071] The characteristic behavior patterns of non-critical gestures relative to critical gestures are analyzed: the direction distribution histogram of the motion vector of the non-critical gesture is calculated, the distance change rate curve of the non-critical gesture relative to the critical gesture is counted, and the Fourier descriptor difference sequence of the gesture shape contour is extracted. The core operation features of the critical gesture are synchronously extracted: including the gesture motion smoothness index based on Kalman filtering, the size change rate of the gesture bounding box, the key point displacement acceleration spectrum, etc. By establishing the time domain correlation model of the non-critical gesture characteristic behavior (such as acceleration mutation) and the critical gesture operation feature (such as trajectory curvature change), the operation influence strength and delay effect of the critical gesture on the non-critical gesture are analyzed. For example, the transfer function of the critical gesture acceleration and the non-critical gesture displacement deviation is constructed, and the influence weight coefficient is calculated. The trajectory prediction model based on the spatio-temporal graph convolution network is constructed by comprehensively analyzing the characteristic behavior pattern and the operation influence quantization result. The model takes the critical gesture position as the input node, the non-critical gesture feature as the adjacent node, learns the motion propagation law between the nodes through the graph convolution layer, and outputs the trajectory probability distribution graph of the critical gesture in the next N frames.

[0072] During system operation, the gesture sequence stream is monitored in real time, and the abnormal gesture frame is identified by comparing the shape difference degree and the motion continuity threshold of the current gesture with the preset template. The theoretical trajectory of the abnormal frame when it is not disturbed is simulated by using the trajectory prediction network, the offset vector of the theoretical trajectory and the actual trajectory is calculated, and the cumulative error influence value of the abnormal frame on the subsequent gesture sequence is evaluated. The similar abnormal cases (index keys include abnormal type code and environment feature hash value) in the historical database are called, and the operation associated data when the abnormality occurs is analyzed, such as vibration sensor reading, light mutation timestamp, and operator hand biomechanics feature. Based on the associated data analysis, the abnormal cause is analyzed, and an abnormal cause set including shaking interference, device response delay, and environmental shielding is established. According to the trajectory offset vector and the abnormal reason label, a hierarchical compensation mechanism is set: for low-level position offset, B-spline interpolation is used to correct the trajectory; for high-level shape abnormality, a virtual gesture generation module is enabled to replace the abnormal frame. At the same time, the complete operation trajectory function of the current gesture sequence is extracted, and the homing path of the abnormal frame is calculated combined with the compensation mechanism type: for example, the target position is defined as the nearest point on the predicted trajectory line, and the Bezier curve path that satisfies the acceleration constraint is planned. Finally, the spatial relationship between the key gesture position coordinates and the homing path is integrated, and the gesture response rule library is updated: entries such as "when type B abnormality is detected and the homing path length < L, enable dynamic speed adjustment strategy" are added.

[0073] Embodiment 3: see Figure 4Multi-channel attribute classification is performed on continuously input depth vision data streams: a static background layer is separated using a background subtraction algorithm, dynamic interference objects are detected using optical flow, illumination intensity channels are divided using HSV color space analysis, a data quality mask is generated based on depth map integrity detection, and a structured environment dataset after classification is output. Key environmental feature vectors are extracted from the classified data: the average moving speed of dynamic objects. With directional dispersion Sliding window variance of light intensity Area percentage of regions lacking depth data The gradient sharpness value at the edge of the working area etc. Among them, the environmental trend assessment index The calculation formula is:

[0074] ;

[0075] in: These are the normalized weight coefficients for each feature. The illumination reference constant is used. A time window is obtained by retrieving historical environmental data from a feature vector database. An internal environmental state sequence. Identifying operational disturbance patterns within the sequence: extracting the dominant frequency from the vibration acceleration spectrum. and harmonic energy Analyze the spatiotemporal distribution density of dynamic objects. Identifying characteristics of light changes: Calculating the attenuation slope of the moving average of illumination. Detect the duration of flicker events and interval period By analyzing the time-frequency characteristics of operational interference data, an interference mode classification model is established: identifying periodic interference (satisfying...) and Random pulse interference ( And pulse width < ) and other patterns. Among them: For threshold frequency, For total energy, The threshold for the proportion of harmonic energy. Minimum density threshold, The pulse time threshold is used. Synchronous analysis of historical hand gesture behavior characteristics under various interference modes is performed: the percentage reduction in hand gesture speed in a vibration environment is statistically analyzed. Calculate the amplitude of trajectory jitter when the catkins cause interference. The distribution function. The current visibility level is calculated based on light variation data. :

[0076] ;

[0077] in: Light attenuation factor, Assess the current depth data quality. This is the baseline quality value. It incorporates characteristics of the overall interference mode (such as periodic interference intensity). ), gesture behavior pattern parameters (such as velocity inhibition coefficient) ), visibility level and its changing trends An environmental change trend assessment matrix is ​​constructed. This matrix outputs a multi-dimensional trend description: the probability of occurrence of disturbance patterns. Markov chain predictions, visibility decay rate Environmental stability index The differential changes, etc. Based on the environmental trend analysis results, the radius of the current operation gesture sequence is identified through a gesture space clustering algorithm. A set of surrounding interfering gestures is identified. An interaction model between the interfering gestures and the main gesture sequence is established: the penetration depth of the interfering gestures into the workspace warning zone is calculated. Analyze the projection overlap rate of the occlusion of the main gesture. Assessing mutual information entropy of deep data confusion Combined with environmental change trend parameters (such as when...) (At the time), identify a set of potential risk factors: including the risk level of gesture occlusion. (and Positive correlation), probability of identifying incorrect risk (and and Negative correlation), equipment collision risk index (and (positive correlation) and visibility decay rate threshold Classify and quantify various risks: Set... Level 3 threshold , probability interval Etc. Based on the main gesture operation state parameters (such as the complexity of the current gesture type). Speed ​​of movement ) and risk level, and set risk response strategies: when and Dual-camera redundant recognition is enabled at this time; when Reduce the equipment speed to its original value. The final anomaly detection pattern rule table is generated: defining when... and The system triggers a three-level monitoring strategy, which includes specific measures such as increasing the sampling frequency to 60Hz and enabling dedicated convolutional kernels for anomaly detection.

[0078] Example 4: Based on the anomaly detection mode configuration, the system collects historical operation datasets of the current gesture sequence, including past... The gesture parameters for each similar operation are recorded. See Table 1 for the historical data storage format and example.

[0079] Table 1: Historical Data Storage Formats and Examples

[0080] By statistically analyzing historical data, a baseline for normal operating parameters was established: the speed range of straight-line cutting gestures. mm / s, acceleration standard deviation threshold m / s², allowable radius of position offset mm, lower limit of morphological matching Real-time monitoring of the current gesture sequence status: within a time window. Internal calculation of instantaneous velocity Moving average, acceleration Real-time differential value, median velocity The current morphological matching degree is obtained through a template matching algorithm. Calculate the position offset based on the key point coordinates Compare the current state parameters with historical benchmarks: calculate the speed deviation. acceleration wave entropy Position offset index Morphological differences Based on preset normal parameter thresholds (speed anomaly thresholds) Position offset threshold ), Determine abnormal status: when The time stamp is marked as an overspeeding anomaly. When marked as an offset anomaly, The time stamp is used to identify morphological anomalies. After detecting anomalies, the time period of the abnormal gesture is located (e.g., a gesture at the turning point with a timestamp of 1560ms), and the depth image sequence and motion parameters within that time period are extracted. Anomaly type characteristics are analyzed: the acceleration gradient of the sudden change in acceleration is calculated for overspeed anomalies. The rate of change of trajectory curvature for anomaly analysis Set the strain level according to the anomaly type: Level 1 strain (immediate interruption) is applicable to... Sudden overvelocity in m / s³, secondary strain (smooth correction) is applicable. The offset is abnormal. The trajectory prediction function of the trajectory prediction network is called to obtain the predicted trajectory of adjacent frames (e.g., 1540ms and 1580ms) of the abnormal gesture. The theoretical location at the time of the anomaly is calculated based on this function. Generate adjustment path: Define path function

[0081] ;

[0082] in: Current position For the regression rate parameter, This is the initial time point. Based on the combined strain levels and adjustment path parameters, a gesture adjustment mechanism is constructed: the first-level strain mechanism calls and executes the emergency braking protocol. The secondary strain mechanism injects path correction while maintaining equipment operation. .in, This indicates the target position that the gesture (or the device execution component related to the gesture) should be at at time t after path planning. This indicates the current actual position of the gesture (or the device execution component associated with the gesture) at time t.

[0083] Example 5: The system continuously runs the monitoring rules in the anomaly perception mode. By comparing the current gesture parameters with preset threshold conditions in real time (such as speed exceeding the limit for three consecutive frames, or position offset continuously increasing), it identifies the occurrence of abnormal operation events. When an abnormal event is detected, based on the anomaly type code and severity level (refer to the grading standard in Example 4), the trigger conditions for the gesture adjustment mechanism are set: an immediate trigger flag is set for level 1 anomalies, and a delayed trigger time is set for level 2 anomalies. ms, set composite trigger conditions for level three anomalies (such as when the device load is high). (triggered at time), where Set the load threshold. Simultaneously set the risk avoidance strategy: define a safe distance for tool retraction for cutting gestures. mm, setting the self-test time window after the device is paused. ms, the planned emergency rollback path is in the opposite direction to the normal of the current trajectory. The adjustment mechanism is dynamically activated based on trigger conditions: in During the delay period, the rate of change of abnormal parameters is continuously monitored. If the rate of change exceeds the limit, an early trigger is initiated. After triggering, the gesture adjustment algorithm defined in this embodiment is executed to correct the data stream during the abnormal period in real time. Simultaneously, the activation conditions of the risk avoidance strategy are evaluated: when the intensity of mechanical vibration is detected... ( When the vibration intensity threshold is reached and the anomaly level is Level 1, the current operation sequence is interrupted and a tool retraction action is executed. The processed gesture data stream is output: for position offset anomalies, interpolated and corrected trajectory coordinates are output; for morphological anomalies, virtual gesture replacement frames are output. Based on the corrected data stream, a preset gesture response rule library is applied: when a corrected "cutting complete" gesture is recognized, a tool lifting command is sent; when an "emergency stop" gesture feature is detected, the equipment braking protocol is activated. The final recognition result is output to the actuator control bus. After completing a single gesture sequence recognition, the system calculates the overall sequence stability index based on the position correction amount. variance Number of speed adjustments Variation range of morphological matching degree Calculate the stability score using parameters such as:

[0084] ;

[0085] when ( When the stability score threshold is reached, a warning gesture sequence description is generated: predicting the types and probabilities of abnormal gestures that may occur within the next 5 frames, and marking the spatial coordinate range of high-risk areas. Temporary adjustments are made to the warning sequence: a high-sensitivity recognition model is pre-loaded, the device movement speed is preset to 80% of the baseline value, and a safe buffer distance is reserved in the control system. mm, and send a warning signal with the code "W03" to the operation interface.

[0086] Example 6: In the actual production scenario of an automatic fabric cutting machine, operators need to flexibly adjust the machine's operating speed according to the fabric material and the complexity of the cutting pattern. The method of this invention can achieve non-contact speed control based on gestures. First, a depth camera is used to collect depth vision data above the fabric cutting machine's worktable. This data includes the three-dimensional spatial coordinates of the operator's hand, the movement trajectory of the finger joints, as well as the depth distribution of the worktable surface, the height information of the fabric stack, the light intensity of the surrounding environment, and the slight background displacement caused by equipment vibration. Based on this depth vision data, the background features of the gesture operation environment are first extracted: for example, it is identified that the current workbench is covered with thin cotton fabric, stacked to a height of about 3 cm, and the surrounding environment is workshop LED lighting with no obvious natural light interference, but the operation of the nearby sewing machine produces slight periodic vibrations. The vibration frequency is captured by the displacement change of the background point of the workbench in the depth data. At the same time, the gesture type is identified. The speed control gestures preset by the operator include: keeping the palm horizontal and moving up and down (moving up corresponds to acceleration, moving down corresponds to deceleration), clenching the fist (corresponding to pausing speed), and spreading the five fingers (corresponding to restoring the default speed). At this time, through the spatiotemporal feature analysis of the depth data, it is initially determined that the operator is initiating an acceleration gesture of moving the palm horizontally upward, and environmental feature data is collected, including the thickness parameters of the fabric (calculated by the distance difference between the fabric surface and the workbench surface in the depth data), the real-time light intensity value of the LED light (determined by the grayscale of the depth image), and the amplitude and frequency data of the sewing machine vibration.

[0087] Based on the aforementioned gesture operation environment (thin cotton fabric, LED lighting, slight periodic vibration) and gesture type (palm moving upwards to accelerate), the parameters of the gesture recognition model were set as follows: Considering that the cutting speed of thin cotton fabric should be kept from being too fast to avoid wrinkling the fabric, the sensitivity of the model to the "palm moving upwards to accelerate" gesture was adjusted to a medium level, that is, the acceleration command is only triggered when the palm moves upwards to a certain threshold. At the same time, the influence of vibration interference on model recognition was reduced by filtering background displacement signals in the depth data whose frequency is consistent with the sewing machine vibration to reduce false recognition of gestures. Then, the relative position of the key gesture in the model was located. The three-dimensional coordinates of the center of the palm when the operator initiates the gesture are used as the initial reference point. The vertical distance change of the center of the palm relative to the initial reference point in each frame of the image during the palm's upward movement was recorded. For example, in the first frame, the center of the palm is 0.5 cm above the initial reference point, and in the second frame, it is 1 cm above the initial reference point. These relative position data are directly related to the subsequent speed adjustment range.

[0088] A trajectory prediction network is constructed based on the relative positions of key gestures: First, the relative positions of adjacent gestures to the key gesture of palm raising are calculated. For example, a brief preparatory action of placing the palm horizontally before the palm raises is identified; this action is a non-key gesture. The characteristic behavior of this non-key gesture relative to the key gesture is analyzed. During the preparatory action, the palm remains horizontal, the fingers are naturally extended, the duration is about 0.8 seconds, and the distance between the center of the palm and the initial reference point fluctuates by no more than 0.2 cm. The operational behavior features of the key gesture (palm raising) are extracted, including the average speed of the palm raising (0.3 cm per frame), the palm position, and the relative positions of the palm and the relative positions of the hands. The stability of the edges (no obvious wobbling) and the degree of bending of the finger joints (maintaining an extended state, without curling) were analyzed, and the impact of these features on the operation of non-critical gestures was examined. It was found that the better the horizontal stability of the palm during the preparatory action, the smoother the upward trajectory of the subsequent critical gestures, and the higher the accuracy of speed control. Based on these characteristic behaviors and operational influences, a trajectory prediction network was constructed. This network can predict the height that the palm may reach in the next 5 frames based on the relative position data of the palm in the first 3 frames of the upward movement. For example, it can predict that the palm will continue to move upward to 3 cm above the initial reference point, providing a predictive basis for adjusting the speed of the fabric cutting machine in advance.

[0089] Based on the trajectory prediction network and the relative positions of key gestures, gesture response rules are set: Real-time monitoring of the hand gesture sequence is performed. If a slight left-right swaying of the hand occurs during the upward movement of the palm (possibly due to operator fatigue or slight contact with the fabric), the trajectory prediction network analyzes the impact of this abnormal gesture on the sequence of hand gestures. It predicts that if the speed is adjusted to follow the swaying, it may cause frequent fluctuations in the cutting machine speed of 0.2 meters per minute. The operational correlation data between this abnormal gesture and normal upward movement gestures is analyzed. For example, in historical data, similar swaying often occurs after 30 minutes of continuous operation or when the fabric edge bulges, thus identifying the cause of the abnormality as hand fatigue. Based on the sequence impact (speed fluctuation) and the cause of the abnormality... (Hand fatigue) Set up a gesture compensation mechanism: When a left-right sway of less than 0.4 cm is detected, maintain the current speed temporarily and do not immediately adjust to follow the sway; at the same time, identify the operation trajectory of the gesture sequence and find that the palm still has a tendency to continue to move upward after the sway. Based on this trajectory and the compensation mechanism, set a return path. That is, if the palm returns to the upward trajectory after the sway, the speed of the fabric cutting machine continues to be adjusted according to the upward sway; combining the relative position of key gestures (such as the distance of the current height of the palm relative to the initial reference point) and the return path, clarify the response rules: for every 1 cm the palm moves upward, the speed of the fabric cutting machine increases by 0.5 m / min. When the sway amplitude is ≤0.4 cm, the speed is not adjusted. When the palm returns to the upward trajectory after the sway, the speed continues to increase according to the amplitude.

[0090] Combining gesture recognition models and response rules, the system perceives the gesture operation status in real time: when the operator's palm moves from the initial reference point to 2 cm and remains stable, the model identifies the current operation status as "stable acceleration to 1 m / min" (2 cm × 0.5 m / min·cm). At the same time, based on depth vision data analysis, the system analyzes the trend of environmental changes: continuously collected depth data shows that the light intensity of the workshop LED lights decreased by 10% within 5 minutes (possibly due to lamp aging), the fabric stack height decreased by 0.5 cm due to the progress of cutting, and the vibration frequency of the sewing machine did not change, but the amplitude increased slightly (from 0.1 mm to 0.15 mm). These environmental changes may affect the clarity of subsequent gesture recognition. For example, a decrease in light intensity may reduce the accuracy of depth data extraction from the edge of the palm, and an increase in amplitude may increase the probability of misidentification of false gestures.

[0091] An anomaly detection mode is set up by combining the gesture operation state (stable acceleration) and environmental change trends (decreasing light intensity, increased amplitude): Based on the current stable acceleration operation state, surrounding interfering gestures are identified, such as the slight tremor of the palm edge in the depth data due to increased amplitude, which may be misidentified as a small upward or downward movement; the interaction between this interfering gesture and the normal speed control gesture is analyzed: the movement amplitude of the interfering gesture is only 0.1-0.2 cm, and the duration is ≤0.3 seconds, which is significantly different from the uniform and stable movement of the normal upward movement gesture (amplitude ≥0.3 cm / frame, duration ≥0.5 seconds); according to the environmental change trend (increased amplitude leads to enhanced interference) and the interaction relationship (small amplitude and short duration of interfering gesture), the potential risk factor is identified as "increased amplitude may lead to misidentification of speed control gestures, causing unnecessary speed fine-tuning", with a risk level of mild; based on the current stable operation state and mild risk level, a risk response mode is set: when a hand movement with a movement amplitude <0.3 cm and a duration <0.5 seconds is detected, speed adjustment is not triggered and it is regarded as an interference signal.

[0092] Anomaly detection based on pattern recognition: In an environment with increased amplitude, depth data shows a slight upward movement of 0.25 cm for 0.4 seconds. Combined with historical operation data (when the amplitude is normal, the operator's hand movement amplitude during stable acceleration is ≥0.3 cm and the duration is ≥0.5 seconds), the current behavioral deviation is identified as "not conforming to the movement characteristics of normal acceleration gestures". Based on the normal operation parameters in the historical data (acceleration gesture movement amplitude ≥0.3 cm and duration ≥0.5 seconds), anomaly parameter thresholds are set: hand movements with a movement amplitude <0.3 cm or a duration <0.5 seconds are judged as abnormal. Based on this threshold and the current behavioral deviation of 0.25 cm and 0.4 seconds, this slight upward movement is identified as an abnormal gesture, and the cause of the abnormality is a slight hand tremor caused by the increased amplitude of the sewing machine.

[0093] Based on the abnormal gesture and the trajectory prediction network, a gesture adjustment mechanism is set up: the abnormal gesture is identified as a "minor upward movement of 0.25 cm and 0.4 seconds", and the abnormality type is "false acceleration gesture caused by environmental vibration". Based on this abnormality type, an abnormality strain layer is set up: for false acceleration gestures, no acceleration command is executed, and the current speed is maintained; the trajectory prediction network identifies the adjacent gesture trajectories of the abnormal gesture, and it is found that before the abnormality, the palm was steadily moving upward to 2 cm, and within 0.5 seconds after the abnormality, the palm returned to 2 cm and continued to move upward steadily. Based on this adjacent trajectory, an adjustment path is set up: maintain the speed of 1 m / min corresponding to the current 2 cm height, and when the palm returns to the normal upward movement trajectory (such as continuing to move upward to 2.5 cm), the speed is increased to 1.25 m / min according to the response rules; combining the abnormality strain layer (not executing false acceleration) and the adjustment path (adjusting after returning to the trajectory), the gesture adjustment mechanism is completed.

[0094] Based on the anomaly perception mode, gesture adjustment mechanism, and response rules, the gesture recognition result is output: The anomaly perception mode identifies the slight upward movement as a false gesture caused by vibration; the adjustment mechanism stipulates that false acceleration should not be performed and adjustments should only be made after the trajectory is reverted; the response rules clearly define the correspondence between the upward movement amplitude and speed. Therefore, the final output is "maintain the current cutting machine speed of 1 meter / minute, and adjust according to the amplitude after the palm continues to move steadily upward." The cutting machine maintains the current speed according to this result. Subsequently, the operator's palm moves steadily upward to 2.5 centimeters, and the depth vision data captures this normal gesture. The model increases the speed to 1.25 meters / minute according to the response rules. At the same time, the environmental change trend shows that the light intensity continues to decrease slowly. The anomaly perception mode automatically increases the monitoring intensity of "palm edge recognition accuracy." When the light intensity decreases to the point that it affects the extraction of the palm contour, the adjustment mechanism will further optimize the filtering parameters of the depth data to ensure the accuracy of gesture recognition. This achieves stable and precise control of the cutting machine's operating speed through gestures, adapting to different cutting conditions.

[0095] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0096] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A gesture recognition method for a deep-vision-based automatic cloth cutting machine, characterized in that, The method comprises: acquiring depth vision data to be identified, and identifying a gesture operation environment and a gesture type based on the depth vision data; collecting environment feature data of the gesture operation environment; Based on the gesture operation environment and the gesture type, set the model parameters of the gesture recognition model, and locate the relative position of the key gesture in the gesture recognition model; Based on the relative position, construct a trajectory prediction network of the key gesture; According to the trajectory prediction network and the relative position, set the gesture response rule; Combine the gesture recognition model and the gesture response rule to realize real-time sensing of the gesture operation state; based on the depth vision data, analyze the environment change trend of the gesture operation environment; Combine the gesture operation state and the environment change trend to set the abnormal perception mode; Based on the abnormal perception mode, identify the gesture abnormal situation; According to the gesture abnormal situation and the trajectory prediction network, set the gesture adjustment mechanism; According to the abnormal perception mode, the gesture adjustment mechanism and the gesture response rule, output the gesture recognition result.

2. The gesture recognition method of claim 1, wherein, The method comprises: Based on the depth vision data, extract the background features of the gesture operation environment; According to the background features, dispatch the operation scene information and gesture type of the gesture operation environment; Based on the operation scene information and gesture type, adaptively configure the model parameters of the gesture recognition model to obtain the configured gesture recognition model; Combine the operation scene information and the configured gesture recognition model to determine the environment feature data of the gesture operation environment.

3. The gesture recognition method of claim 1, wherein the gesture recognition method comprises: The method comprises: Based on the relative position, calculate the relative position of the adjacent gesture of the key gesture; Identify the non-key gesture in the relative position of the adjacent gesture; Analyze the characteristic behavior of the non-key gesture relative to the key gesture; Extract the operation behavior features of the key gesture, and analyze the operation influence of the operation behavior features on the non-key gesture according to the characteristic behavior; According to the characteristic behavior and the operation influence, construct the trajectory prediction network of the key gesture.

4. The gesture recognition method of claim 1, wherein, The method comprises: Real-time monitor the operation gesture of the gesture sequence, and identify the abnormal gesture in the operation gesture; According to the trajectory prediction network, analyze the sequence influence of the abnormal gesture on the operation gesture; Dispatch the operation association data of the gesture sequence and the abnormal gesture, identify the abnormal reason of the abnormal gesture based on the operation association data; According to the sequence influence and the abnormal reason, set the gesture compensation mechanism of the abnormal gesture; Identify the operation trajectory of the gesture sequence, and set the homing path of the abnormal gesture based on the operation trajectory and the gesture compensation mechanism; Combine the relative position and the homing path to set the gesture response rule.

5. The gesture recognition method of claim 1, wherein, The method comprises: Attribute classification is performed on the depth vision data to obtain classified environment data; extracting key environmental features of the classified environmental data, collecting historical environmental data of the gesture operation environment based on the key environmental features; identifying operation interference data and light change data in the historical environmental data; analyzing operation interference patterns and gesture behavior patterns of the gesture operation environment according to the operation interference data; analyzing visibility levels of the gesture operation environment based on the light change data; combining the operation interference patterns, gesture behavior patterns and visibility levels to analyze environmental change trends of the gesture operation environment.

6. The gesture recognition method of claim 1, wherein, The combination of the gesture operation state and the environmental change trend sets an abnormal perception mode, including: Based on the gesture operation state, identify the surrounding interference gestures of the gesture sequence; analyze the interaction relationship between the surrounding interference gestures and the gesture sequence; according to the environmental change trend and the interaction relationship, identify the potential risk factors of the gesture sequence, and identify the risk level of the potential risk factors; According to the gesture operation state and the risk level, set the risk response mode of the gesture sequence; Combine the risk level and the risk response mode to set the abnormal perception mode.

7. The gesture recognition method of claim 1, wherein the gesture recognition method further comprises: determining a distance between the user and the cutting device; and determining a gesture of the user based on the determined distance. The abnormal perception mode based on the gesture sequence includes: Based on the abnormal perception mode, collect historical operation data of the gesture sequence; according to the historical operation data, identify the normal operation parameters of the gesture sequence; Identify the current operation state of the gesture sequence, and identify the current behavior deviation of the gesture sequence in combination with the current operation state and the historical operation data; Based on the normal operation parameters, set the abnormal parameter threshold of the gesture sequence; According to the abnormal parameter threshold and the current behavior deviation, identify the gesture abnormal situation.

8. The gesture recognition method of claim 1, wherein, The gesture adjustment mechanism based on the gesture abnormal situation and the trajectory prediction network includes: According to the gesture abnormal situation, locate the abnormal operation gesture of the gesture sequence; analyze the abnormal type of the abnormal operation gesture, and set the abnormal response layer of the abnormal operation gesture based on the abnormal type; According to the trajectory prediction network, identify the adjacent gesture trajectory of the abnormal operation gesture; Based on the adjacent gesture trajectory, set the adjustment path of the abnormal operation gesture; Combine the abnormal response layer and the adjustment path to set the gesture adjustment mechanism.

9. The gesture recognition method of claim 1, wherein, The output gesture recognition result includes: According to the abnormal perception mode, identify the operation abnormality of the gesture sequence; Based on the operation abnormality, set the trigger condition of the gesture adjustment mechanism, and set the risk avoidance mode of the gesture sequence; According to the trigger condition and the risk avoidance mode, identify and adjust the gesture sequence to obtain adjusted gesture data; Based on the adjusted gesture data and the gesture response rule, output the gesture recognition result.

10. The gesture recognition method of claim 1, wherein, The method further includes: based on the gesture recognition result, estimating the overall stability of the gesture sequence; according to the overall stability, generating a warning gesture sequence; temporarily adjusting the warning gesture sequence.

Images