Automatic fabric cutting machine gesture recognition method based on depth vision

Abstract

The invention relates to the technical field of intelligent cloth cutting, and discloses an automatic cloth cutting machine gesture recognition method based on depth vision. The method comprises the following steps: acquiring to-be-recognized depth vision data, recognizing a gesture operation environment and a gesture type according to the depth vision data, and collecting environment feature data; setting gesture recognition model parameters based on the gesture operation environment and type, and positioning the relative position of the key gesture in the model; constructing a trajectory prediction network of the key gesture based on the relative position, and setting a gesture response rule in combination with the trajectory prediction network and the relative position; sensing a gesture operation state in real time through a gesture recognition model and a response rule, and analyzing an environment change trend based on depth vision data; setting an exception sensing mode in combination with the operation state and the environment change trend, and recognizing a gesture exception condition based on the mode; setting a gesture adjustment mechanism according to the abnormal condition and the trajectory prediction network, and finally outputting a gesture recognition result according to an abnormal sensing mode, the adjustment mechanism and a response rule.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent cloth cutting, in particular to a gesture recognition method for an automatic cloth cutting machine based on deep vision. BACKGROUND

[0002] In the automatic production process of the textile and garment industry, the automatic cloth cutting machine as a key equipment, its operation convenience and intelligent level directly affect the production efficiency and product quality. The operation of the traditional automatic cloth cutting machine mostly depends on physical keys, touch panels or pre-set programs, and the operator needs to control the equipment operation through specific mechanical contact or fixed instruction input. This operation mode has obvious limitations. When facing diversified cloth types, complex cutting patterns or temporary adjustment of production requirements, the operator needs to frequently manually switch parameters or modify programs, which not only increases the operation burden, but also easily causes cutting deviation due to operation errors, affecting the production progress.

[0003] With the development of machine vision and artificial intelligence technology, gesture recognition technology has been gradually introduced into the field of industrial equipment control, and its non-contact operation characteristics provide the possibility for improving equipment interaction efficiency. However, when applying gesture recognition technology to the automatic cloth cutting scene, the existing scheme faces many challenges. The environment in the cloth cutting workshop is complex, with unstable lighting conditions, diverse cloth textures, and machine vibration interference factors. The traditional gesture recognition method based on RGB images is easily affected by light changes and is difficult to accurately extract gesture features. At the same time, the gesture actions of the operator vary under different cloth thicknesses and cutting speeds, and the existing model lacks dynamic perception ability to the operating environment, resulting in large fluctuations in gesture recognition accuracy.

[0004] The operation gesture of the automatic cloth cutting machine often contains continuous trajectory and instantaneous instruction. The existing method is not accurate enough in positioning the key gesture, and the trajectory prediction is lagging, which cannot respond to the operator's intention in time. In actual production, if the gesture recognition is misjudged or missed, it may cause abnormal operation of the equipment, such as cutting path deviation, cutter misstart, etc., which not only affects the cloth cutting quality, but also has safety hazards. The existing control mode of the cloth cutting machine has the problems of difficult operation of emergency braking and inconvenient manual keys, which is the main factor causing danger. The traditional key is a positive key type, and people in danger need a "simple and accessible" control mode, so it is necessary and urgent to add a gesture recognition module to the cloth cutting machine for control. At the same time, the existing technology lacks effective perception and processing mechanism for abnormal situations in the operation process, such as blurred gestures and environmental mutations, which further limits the reliable application of gesture recognition technology in automatic cloth cutting machines. Therefore, there is an urgent need for a gesture recognition method that can adapt to complex environments, accurately recognize gesture intentions and has abnormal processing capability to meet the needs of intelligent operation of automatic cloth cutting machines. SUMMARY

[0005] The purpose of the present invention is to provide a gesture recognition method for an automatic cloth cutting machine based on depth vision to solve the problems raised in the above background technology.

[0006] To achieve the above objectives, the present invention provides a gesture recognition method for an automatic cloth cutting machine based on depth vision, the method comprising: Acquire depth vision data to be recognized, identify a gesture operation environment and a gesture type based on the depth vision data, and collect environmental feature data of the gesture operation environment; Based on the gesture operation environment and gesture type, setting model parameters of a gesture recognition model, and locating relative positions of key gestures in the gesture recognition model; Based on the relative positions, construct a trajectory prediction network for key gestures; Setting gesture response rules based on the trajectory prediction network and relative positions; Combining the gesture recognition model and gesture response rules, real-time perception of gesture operation status; based on the depth vision data, analyzing the environmental change trend of the gesture operation environment; Combine gesture operation status and environmental change trends to set abnormal perception mode; Based on the abnormality perception pattern, identifying abnormal gesture conditions; Setting a gesture adjustment mechanism based on the gesture abnormality and trajectory prediction network; According to the abnormal perception mode, gesture adjustment mechanism and gesture response rules, a gesture recognition result is output.

[0007] Preferably, the identifying the gesture operation environment and the gesture type based on the depth vision data includes: extracting background features of the gesture operation environment based on the depth vision data; Dispatching operation scene information and gesture types of the gesture operation environment according to the background features; Adaptively configuring model parameters of the gesture recognition model based on the operation scenario information and the gesture type to obtain a configured gesture recognition model; The environmental feature data of the gesture operation environment is determined by combining the operation scenario information and the configured gesture recognition model.

[0008] Preferably, the constructing of a trajectory prediction network for a key gesture based on the relative positions includes: calculating relative positions of adjacent gestures of the key gesture based on the relative positions; Identifying non-critical gestures at relative positions of the adjacent gestures; analyzing characteristic behaviors of the non-critical gestures relative to the critical gestures; extracting an operation behavior feature of a key gesture, and analyzing an operation influence of the operation behavior feature on a non-key gesture according to the feature behavior; constructing a trajectory prediction network of the key gesture according to the feature behavior and the operation influence.

[0009] Preferably, the setting of the gesture response rule according to the trajectory prediction network and the relative position comprises: monitoring an operation gesture of a gesture sequence in real time, and identifying an abnormal gesture in the operation gesture; analyzing a sequence influence of the abnormal gesture on the operation gesture according to the trajectory prediction network; scheduling operation association data of the gesture sequence and the abnormal gesture, identifying an abnormal reason of the abnormal gesture based on the operation association data; setting a gesture compensation mechanism of the abnormal gesture according to the sequence influence and the abnormal reason; identifying an operation trajectory of the gesture sequence, setting a homing path of the abnormal gesture based on the operation trajectory and the gesture compensation mechanism; setting the gesture response rule in combination with the relative position and the homing path.

[0010] Preferably, the analyzing of the environmental change trend of the gesture operation environment based on the depth visual data comprises: performing attribute classification on the depth visual data to obtain classified environmental data; extracting a key environmental feature of the classified environmental data, and collecting historical environmental data of the gesture operation environment based on the key environmental feature; identifying operation interference data and light change data in the historical environmental data; analyzing an operation interference mode and a gesture behavior mode of the gesture operation environment according to the operation interference data; analyzing a visibility level of the gesture operation environment based on the light change data; combining the operation interference mode, the gesture behavior mode and the visibility level to analyze the environmental change trend of the gesture operation environment.

[0011] Preferably, the setting of the abnormal perception mode in combination with the gesture operation state and the environmental change trend comprises: identifying a surrounding interference gesture of the gesture sequence based on the gesture operation state; analyzing an interaction relationship between the surrounding interference gesture and the gesture sequence, identifying a potential risk factor of the gesture sequence and a risk level of the potential risk factor according to the environmental change trend and the interaction relationship; setting a risk response mode of the gesture sequence according to the gesture operation state and the risk level; combining the risk level and the risk response mode to set the abnormal perception mode.

[0012] Preferably, the gesture abnormality is identified based on the abnormal perception mode, including: collecting historical operation data of the gesture sequence based on the abnormal perception mode; identifying normal operation parameters of the gesture sequence according to the historical operation data; identifying a current operation state of the gesture sequence, and identifying a current behavior deviation of the gesture sequence in combination with the current operation state and the historical operation data; setting an abnormal parameter threshold of the gesture sequence based on the normal operation parameters; identifying the gesture abnormality according to the abnormal parameter threshold and the current behavior deviation.

[0013] Preferably, the gesture adjustment mechanism is set according to the gesture abnormality and the trajectory prediction network, including: locating an abnormal operation gesture of the gesture sequence according to the gesture abnormality; analyzing an abnormal type of the abnormal operation gesture, and setting an abnormal strain layer of the abnormal operation gesture based on the abnormal type; identifying adjacent gesture trajectories of the abnormal operation gesture according to the trajectory prediction network; setting an adjustment path of the abnormal operation gesture based on the adjacent gesture trajectories; setting the gesture adjustment mechanism in combination with the abnormal strain layer and the adjustment path.

[0014] Preferably, the gesture recognition result is output, including: identifying an operation abnormality of the gesture sequence according to the abnormal perception mode; setting a trigger condition of the gesture adjustment mechanism based on the operation abnormality, and setting a risk-avoiding mode of the gesture sequence; identifying and adjusting the gesture sequence according to the trigger condition and the risk-avoiding mode, to obtain adjusted gesture data; outputting the gesture recognition result based on the adjusted gesture data and a gesture response rule.

[0015] Preferably, the method further includes: estimating overall stability of the gesture sequence based on the gesture recognition result; generating a warning gesture sequence according to the overall stability; and performing a temporary adjustment operation on the warning gesture sequence.

[0016] Compared with the prior art, the present application has the following advantages: The gesture recognition method of the automatic cloth cutting machine based on deep vision provided by the application acquires gesture operation environment and gesture type through deep vision data, compared with the traditional recognition mode relying on RGB image, the deep data can effectively avoid the interference brought by light change and cloth texture difference, more accurately capture the three-dimensional features and environment information of the gesture, so that the gesture recognition remains stable in the complex cloth cutting workshop environment. After collecting the environment characteristic data, the gesture recognition model parameters are set based on the specific operation environment and gesture type, realizing the dynamic adjustment of the model parameters, avoiding the problem of insufficient adaptability of the fixed parameter model under different working conditions, so that the model can better match the actual operation scene, improving the pertinence of recognition. The relative position of the key gesture in the model is located, laying a foundation for subsequent trajectory prediction, so that the trajectory prediction can focus on the motion path of the key gesture, reducing the interference of irrelevant information. The key gesture trajectory prediction network constructed based on the relative position can perceive the motion trend of the gesture in advance, making the gesture response more forward-looking, and avoiding the operation delay caused by response lag. Combined with the trajectory prediction network and the relative position, the gesture response rule is set, so that the response of the device to different gestures is more in line with the intention of the operator, enhancing the smoothness of human-computer interaction. Through the gesture recognition model and the response rule, the gesture operation state is perceived in real time, the gesture change in the operation process can be dynamically tracked, and the accurate capture of each operation step is ensured. At the same time, based on the deep vision data, the environment change trend is continuously analyzed, the subtle changes in the environment during the cloth cutting process can be perceived in time, such as the change of cloth stacking height, the change of equipment running state, etc., which provides the basis for subsequent abnormal perception. Combined with the gesture operation state and the environment change trend, the abnormal perception mode is set, so that the system can comprehensively judge whether there is an abnormality in the operation process, such as gesture blur, operation trajectory deviation, etc., realizing the active perception of abnormal conditions. After identifying the gesture abnormal condition based on the abnormal perception mode, the gesture adjustment mechanism is set according to the abnormal condition and the trajectory prediction network, which can dynamically correct the recognition result, reduce the influence of abnormal conditions on the recognition accuracy, and ensure that the device can still operate stably when encountering sudden conditions. Finally, the gesture recognition result is output according to the abnormal perception mode, the gesture adjustment mechanism and the gesture response rule, integrating information such as environment perception, trajectory prediction, abnormal handling, etc., so that the output recognition result is more comprehensive and accurate, effectively improving the intelligent operation level of the automatic cloth cutting machine, meeting the diversified and high-precision cutting demand in textile production. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The working principle diagram of the gesture recognition method of the automatic cloth cutting machine based on deep vision described in the application; Figure 2 The flowchart of gesture operation environment and gesture type recognition; Figure 3 Flow chart for key gesture trajectory prediction network construction; Figure 4 Flow chart for gesture operation environment change trend analysis. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0019] Please refer to Figure 1 The present application provides a gesture recognition method for an automatic cloth cutting machine based on deep vision, which comprises the following steps: The gesture recognition method of the automatic cloth cutting machine based on depth vision is implemented as follows: the depth vision data containing the gestures of the operator and the surrounding environment are acquired through a depth vision sensor. Based on the depth vision data, the specific environment (such as the light condition, the background object, and the state of the workbench) in which the current gesture operation is located and the type of the gesture (such as line drawing, cutting, and pausing) being performed by the operator are identified; at the same time, the specific environmental feature data of the gesture operation environment are collected. According to the identified gesture operation environment and gesture type, the key parameters of the gesture recognition model are dynamically set, such as adjusting the sensitivity of feature extraction, the classification threshold, or the network structure parameters. Using the configured model, the relative position coordinates of the key gestures (such as the cutting starting point and the direction turning point) in the current sequence in the model processing space are located. Based on the relative position information of the key gestures, a trajectory prediction network is constructed, which is used to predict the possible moving path and trend of the key gestures. Combined with the constructed trajectory prediction network and the relative position information of the key gestures, gesture response rules are set, which define the specific response actions (such as starting cutting, adjusting speed, and stopping) that the system should make to different gestures and their predicted trajectories. Integrating the gesture recognition model and the gesture response rules, the system can real-time perceive and analyze the gesture operation state (such as gesture type, position, speed, and continuity) of the operator. At the same time, based on the continuously acquired depth vision data, the change trend of the gesture operation environment (such as the gradual darkening of the light and the moving of the background object close to the operator) is analyzed. Fusing the real-time perceived gesture operation state and the change trend of the environment, an abnormal perception mode is set, which includes the conditions and thresholds for detecting the deviation from the normal operation. Using this abnormal perception mode, various gesture abnormal situations (such as gesture interruption, sudden acceleration, and exceeding the safety area) that may occur are real-time monitored and identified. According to the identified gesture abnormal situations and their specific types, and combined with the prediction information provided by the trajectory prediction network, a corresponding gesture adjustment mechanism is set, which aims to correct the abnormality or guide the gesture back to the expected path. Finally, by comprehensively using the abnormal perception mode, the gesture adjustment mechanism, and the pre-set gesture response rules, the gesture sequence of the operator is real-time processed, adjusted, and judged, and the final gesture recognition result is output, which directly drives the execution mechanism (such as the cutting head) of the automatic cloth cutting machine to perform corresponding actions.

[0020] Example 1: see Figure 2After acquiring the depth visual data, the system first processes the data to extract the background features of the gesture operating environment. The background features include the three-dimensional depth distribution of the workbench surface, the contour geometric information of the background objects, the distance parameters of the static obstacles, the spatial orientation and intensity gradient of the main light source, and other physical attributes. These features are extracted through depth image segmentation algorithms and point cloud spatial clustering techniques, such as using region growing method to segment the workbench surface, identifying the edge of the background object through curvature analysis, and calculating the light source influence range with the aid of light sensor auxiliary data. According to the extracted background features, the system calls the pre-set environment scene database for matching and scheduling, and outputs the operating scene information classification of the current gesture operating environment (such as "high-reflective metal table surface environment" "multiple hanging cloth interference environment") and the preliminary identification of gesture type (such as "straight line cutting gesture" "Z-shaped fold line gesture").

[0021] The operating scene information includes the typical interference mode library under this environment category, the historical recognition accuracy statistics, and the recommended processing strategy index, etc. The gesture type determination is based on the spatiotemporal feature analysis of the depth image sequence by convolutional neural network, and outputs the preliminary classification label and confidence of the gesture. Based on the scene classification and gesture type label obtained by scheduling, the system performs dynamic configuration of the gesture recognition model parameters: for "high-reflective environment", enable the depth data reflection suppression module and adjust the edge detection threshold to the high value interval of the pre-set range; for "Z-shaped fold line gesture", load the high-precision trajectory tracking sub-model, and set the trajectory smoothing coefficient to low sensitivity mode. The parameter configuration process is realized by querying the pre-defined configuration rule matrix, which establishes the mapping relationship between scene type, gesture category and model parameter adjustment item. After completing the parameter configuration, the optimized gesture recognition model instance is output.

[0022] Finally, combined with the environmental attribute description in the operating scene information (such as the interference object density value, the light source stability rating) and the calculation requirements of the optimized model, the environmental feature data set is generated. This data set includes quantitative indicators: environmental noise coefficient calculated by computing the signal-to-noise ratio of the depth image, effective operating area boundary coordinate set based on point cloud density analysis, light interference probability value calculated according to light fluctuation historical data, etc. These data are stored in a structured format for subsequent module calling.

[0023] Embodiment 2: see Figure 3 After determining the relative position coordinates of the key gesture in the model space, the system takes the coordinates as the reference point to calculate the relative position offset of the gesture center point in the adjacent time sequence frame, generating a sequence of adjacent gesture relative positions containing distance difference and angle difference. Identify the non-key gesture entities corresponding to each position point in the sequence, which are usually the preparatory actions before the key gesture operation or the continuous fine-tuning actions after the key gesture operation.

[0024] The characteristic behavior patterns of non-critical gestures relative to critical gestures are analyzed: the direction distribution histogram of the motion vector of the non-critical gesture is calculated, the distance change rate curve of the non-critical gesture relative to the critical gesture is counted, and the Fourier descriptor difference sequence of the gesture shape contour is extracted. The core operation features of the critical gesture are synchronously extracted: including the gesture motion smoothness index based on Kalman filtering, the size change rate of the gesture bounding box, the key point displacement acceleration spectrum, etc. By establishing the time domain correlation model of the non-critical gesture characteristic behavior (such as acceleration mutation) and the critical gesture operation feature (such as trajectory curvature change), the operation influence strength and delay effect of the critical gesture on the non-critical gesture are analyzed. For example, the transfer function of the critical gesture acceleration and the non-critical gesture displacement deviation is constructed, and the influence weight coefficient is calculated. The trajectory prediction model based on the spatio-temporal graph convolution network is constructed by comprehensively analyzing the characteristic behavior pattern and the operation influence quantization result. The model takes the critical gesture position as the input node, the non-critical gesture feature as the adjacent node, learns the motion propagation law between the nodes through the graph convolution layer, and outputs the trajectory probability distribution graph of the critical gesture in the next N frames.

[0025] During system operation, the gesture sequence stream is monitored in real time, and the abnormal gesture frame is identified by comparing the shape difference degree and motion continuity threshold of the current gesture with the preset template. The theoretical trajectory of the abnormal frame when it is not disturbed is simulated by using the trajectory prediction network, the offset vector of the theoretical trajectory and the actual trajectory is calculated, and the cumulative error influence value of the abnormal frame on the subsequent gesture sequence is evaluated. The similar abnormal cases (index keys include abnormal type code and environment feature hash value) in the historical database are called, and the operation associated data when the abnormality occurs is analyzed, such as vibration sensor reading, light mutation timestamp, and operator hand biomechanics feature. Based on the associated data analysis, the abnormal cause is analyzed, and an abnormal cause set including shaking interference, device response delay, and environmental shielding is established. According to the trajectory offset vector and the abnormal reason label, a hierarchical compensation mechanism is set: for low-level position offset, B-spline interpolation is used to correct the trajectory; for high-level shape abnormality, a virtual gesture generation module is enabled to replace the abnormal frame. At the same time, the complete operation trajectory function of the current gesture sequence is extracted, and the homing path of the abnormal frame is calculated combined with the compensation mechanism type: for example, the target position is defined as the nearest point on the predicted trajectory line, and the Bezier curve path that satisfies the acceleration constraint is planned. Finally, the spatial relationship between the key gesture position coordinates and the homing path is integrated, and the gesture response rule library is updated: entries such as "when type B abnormality is detected and the homing path length < L, enable dynamic speed adjustment strategy" are added.

[0026] Embodiment 3: see Figure 4Multi-channel attribute classification on continuous input depth vision data stream: static background layer separation by background subtraction algorithm, dynamic interference object detection by optical flow method, light intensity channel division by HSV color space analysis, data quality mask generation based on depth map integrity detection, output structured environment dataset after classification. Key environment feature vector extraction from classification data: dynamic object moving speed average , direction dispersion , light intensity sliding window variance , depth data missing area proportion , work area edge gradient sharpness , etc. The calculation formula of environment trend evaluation index is: ; Wherein: is the normalized weight coefficient of each feature, is the light reference constant. Based on the feature vector, the historical environment database is retrieved to obtain the environment state sequence within the time window . Identify the operation interference feature pattern in the sequence: extract the main frequency and harmonic energy in the vibration acceleration spectrum, analyze the spatio-temporal distribution density of dynamic objects . Identify the light change feature: calculate the decay slope of the light sliding average , detect the duration and interval period of the stroboscopic event. By analyzing the time-frequency characteristics of the operation interference data, an interference pattern classification model is established: identify periodic interference (satisfying and ), random pulse interference (satisfying and pulse width < 0.5 ). Wherein: is the threshold frequency, is the total energy, is the harmonic energy proportion threshold, is the minimum density threshold, is the pulse time threshold. Synchronous analysis of historical gesture behavior characteristics under each interference mode: statistics of gesture speed reduction proportion in vibration environment , calculation of distribution function of trajectory jitter amplitude under flying dust interference. Calculate the current visibility level based on the light change data: ; Wherein: is the light decay factor, is the current depth data quality score,​ is the baseline quality value. Comprehensive interference pattern characteristics (such as periodic interference intensity ), gesture behavior pattern parameters (such as speed suppression coefficient ), visibility level and its changing trends , build an environmental change trend assessment matrix. The matrix outputs a multi-dimensional trend description: the probability of interference pattern occurrence Markov chain prediction value, visibility decay rate , Environmental Stability Index Based on the results of environmental trend analysis, the gesture space clustering algorithm is used to identify the radius of the current operation gesture sequence. The set of surrounding interference gestures within the workspace. Establish an interactive relationship model between the interference gestures and the main gesture sequence: Calculate the penetration depth of the interference gestures into the warning zone of the workspace , analyze the projection overlap rate of its occluded main gesture , evaluate the mutual information entropy of deep data confusion . Combined with environmental change trend parameters (such as ), identify potential risk factor sets: including gesture occlusion risk level (and positive correlation), identification error risk probability (and and negative correlation), equipment collision risk index (and positive correlation) and the threshold value of the rate of visual attenuation . Classify and quantify each risk: set The third level threshold , The probability interval Etc. According to the main gesture operation state parameters (such as the complexity of the current gesture type , movement speed ) and risk level, set risk response strategy: and Enable dual camera redundant recognition when Reduce the device's speed to its original value Finally, the abnormal perception pattern rule table is generated: define when and The three-level monitoring strategy is triggered when an abnormality occurs, including increasing the sampling frequency to 60Hz, enabling the convolution kernel dedicated to anomaly detection, and other specific measures.

[0027] Example 4: Based on the abnormal perception mode configuration, the system collects the historical operation data set of the current gesture sequence, including the past Gesture parameter records of similar operations. History data storage format and example, see table 1.

[0028] Table 1: History data storage format and example

[0029] Establish normal operation parameter benchmark by statistical analysis of history data: straight cutting gesture speed range mm / s, acceleration standard deviation threshold m / s², position offset allowed radius mm, morphology matching degree lower limit Real-time monitoring of current gesture sequence state: calculate moving average of instantaneous speed , real-time differential value of acceleration , median speed in time window Get current morphology matching degree , calculate position offset based on key point coordinates Compare current state parameters with history benchmark: calculate speed deviation , acceleration fluctuation entropy , position offset index , morphology difference degree Determine abnormal state based on preset normal parameter threshold (speed abnormal threshold , position offset threshold ): mark as overspeed abnormality when , mark as offset abnormality when , mark as morphology abnormality when After detecting abnormal situation, locate abnormal operation gesture occurrence period (such as turning point gesture with time stamp 1560ms), extract depth image sequence and motion parameters in this period. Analyze abnormal type characteristics: calculate acceleration mutation gradient for overspeed abnormality, analyze trajectory curvature change rate for offset abnormality. Set strain level according to abnormal type: first level strain (immediate interruption) is suitable for sudden overspeed of m / s³, second level strain (smooth correction) is suitable for offset abnormality of Call trajectory prediction network to get predicted trajectory function of abnormal gesture adjacent frames (such as 1540ms and 1580ms) Calculate theoretical position at abnormal moment based on the function Generate adjustment path: define path function ; Wherein: is current position, is the regression rate parameter, The initial time point is . The comprehensive response level and adjustment path parameters are used to build a gesture adjustment mechanism: the first-level response mechanism calls the emergency braking protocol and executes The secondary strain mechanism injects path correction while maintaining equipment operation .in, Indicates the target position of the gesture (or the device execution component related to the gesture) at time t after path planning. Represents the current actual position of the gesture (or the device execution component associated with the gesture) at time t.

[0030] Example 5: The system continuously runs the monitoring rules in the abnormal perception mode, and identifies the occurrence of abnormal operation events by comparing the current gesture parameters with the preset threshold conditions (such as speed exceeding the limit for 3 consecutive frames or position offset continuously increasing). When an abnormal event is detected, the trigger conditions of the gesture adjustment mechanism are set based on the abnormality type code and severity level (refer to the classification standards of Example 4): an immediate trigger flag is set for level 1 abnormalities, and a delayed trigger time is set for level 2 abnormalities. ms, set compound trigger conditions for the three-level abnormality (such as when the device load ), where The load threshold. Set the risk avoidance strategy simultaneously: define the tool retraction safety distance for cutting gestures mm, set the self-test time window after the device is paused ms, the planned emergency retreat path is the normal opposite direction of the current trajectory. The adjustment mechanism is dynamically activated according to the triggering conditions: During the delay period, the abnormal parameter change rate is continuously monitored. If the change rate exceeds the limit, it is triggered in advance. After the trigger, the gesture adjustment algorithm defined in this embodiment is executed to make real-time corrections to the data flow during the abnormal period. At the same time, the activation conditions of the risk avoidance strategy are evaluated: when the mechanical vibration intensity is detected ( is the vibration intensity threshold) and the abnormality level is level one, the current operation sequence is interrupted and the tool retraction action is executed. Output the gesture data stream processed by the adjustment mechanism: output the trajectory coordinates after interpolation correction for position offset abnormalities, and output the virtual gesture replacement frame for morphological abnormalities. Based on the corrected data stream, the preset gesture response rule library is applied: when the corrected "cutting completed" gesture is recognized, the tool lifting command is sent; when the "emergency stop" gesture feature is detected, the device braking protocol is activated. Output the final recognition result to the actuator control bus. After completing the recognition of a single gesture sequence, the system calculates the overall stability index of the sequence: based on the position correction amount Variance , speed adjustment times , morphological matching fluctuation range Equal parameters, calculate stability score: ; When ( is the stability score threshold value), generate a pre-warning gesture sequence description: predict the abnormal gesture types that may occur in the next 5 frames and the occurrence probability, and mark the spatial coordinate range of the high-risk area. Perform temporary adjustment on the pre-warning sequence: load the high-sensitivity recognition model in advance, set the device movement speed to 80% of the baseline value, reserve a safety buffer distance of 20 mm in the control system, and send a pre-warning signal with code "W03" to the operation interface.

[0031] In the actual production scene of the automatic cloth cutting machine, the operator needs to flexibly adjust the device running speed according to the cloth material and the complexity of the cutting pattern, and the method of the present application can realize non-contact speed control based on gestures. First, use a depth camera to collect depth vision data above the workbench of the cloth cutting machine. This data covers the three-dimensional spatial coordinates of the operator's hands, the movement trajectory of the finger joints, the depth distribution of the workbench surface, the height information of the cloth stack, the light intensity of the surrounding environment, and the background micro-displacement caused by device vibration. Based on these depth vision data, the background features of the gesture operation environment are first extracted: for example, it is identified that the current workbench surface is covered with thin cotton cloth, the stack height is about 3 cm, the surrounding environment is a workshop LED lighting, there is no obvious natural light interference, but the adjacent sewing machine produces slight periodic vibration, and the vibration frequency is captured through the displacement change of the background points in the depth data; At the same time, identify the gesture type, the operator's pre-set speed control gestures include: palm kept horizontal up and down movement (up corresponds to acceleration, down corresponds to deceleration), clenched fist (corresponds to pause speed), open five fingers (corresponds to restore default speed), at this time through the analysis of the spatiotemporal characteristics of the depth data, it is preliminarily determined that the operator is initiating the acceleration gesture of the palm horizontal up movement, and the environmental feature data is collected, including the thickness parameter of the cloth (calculated by the distance difference between the cloth surface and the workbench surface in the depth data), the real-time light intensity value of the LED lamp (judged by the gray scale of the depth image), and the amplitude and frequency data of the sewing machine vibration.

[0032] ​Based on the gesture operation environment (thin cotton fabric, LED lighting, and slight periodic vibration) and gesture type (palm-up acceleration) identified above, the parameters of the gesture recognition model were set. Considering the need to avoid wrinkling due to excessive speed when cutting thin cotton fabric, the model's sensitivity to the "palm-up acceleration" gesture was adjusted to a medium level. That is, the acceleration command was triggered only when the palm's upward movement amplitude reached a certain threshold. At the same time, the impact of vibration interference on model recognition was reduced. By filtering out background displacement signals in the depth data with frequencies consistent with sewing machine vibration, the false recognition of false gestures was reduced. The relative position of the key gestures in the model was then located. The three-dimensional coordinates of the center of the operator's palm when initiating the gesture were used as the initial reference point. The vertical distance change of the palm center relative to the initial reference point was recorded in each frame during the palm-up movement. For example, in the first frame, the palm center was 0.5 cm above the initial reference point, and in the second frame, it was 1 cm. This relative position data was directly related to the subsequent speed adjustment amplitude.

[0033] Construct a trajectory prediction network based on the relative positions of key gestures: first calculate the relative positions of adjacent gestures of the key gesture of palm-up, for example, recognize that there is a short preparatory action of placing the palm horizontally before the palm moves up, which is a non-key gesture; analyze the characteristic behavior of this non-key gesture relative to the key gesture, the palm remains horizontal during the preparatory action, the fingers are naturally extended, the duration is about 0.8 seconds, and the distance between the center of the palm and the initial reference point fluctuates by no more than 0.2 cm; extract the operational behavior characteristics of the key gesture (palm-up), including the average speed of the palm-up (0.3 cm per frame), the palm The stability of the edge (no obvious shaking) and the degree of bending of the finger joints (maintaining an extended state, not curled up) are analyzed, and the impact of these features on the operation of non-critical gestures is analyzed. It is found that the better the horizontal stability of the palm in the preparatory action, the smoother the upward trajectory of the subsequent critical gesture and the higher the accuracy of speed control. According to these characteristic behaviors and operational impacts, a trajectory prediction network is constructed. Based on the relative position data of the palm in the first 3 frames of movement, the network can predict the height that the palm may reach in the next 5 frames. For example, it is predicted that the palm will continue to move up to 3 cm above the initial reference point, providing a predictive basis for adjusting the speed of the cloth cutting machine in advance.

[0034] Set gesture response rules according to the trajectory prediction network and the relative position of the key gesture: Monitor the operating gesture of the gesture sequence in real time. If a slight left-right shaking of the palm suddenly appears during the upward movement of the palm (which may be caused by the operator's hand fatigue or the slight touch of the cloth), analyze the influence of the abnormal gesture on the sequence of the operating gesture through the trajectory prediction network, and predict that if the shaking continues to adjust the speed, it may cause the cutting machine speed to fluctuate frequently by 0.2 meters / minute. Schedule the operating association data of the abnormal gesture and the normal upward gesture, such as similar shaking in the historical data occurring after the operator has been continuously operating for 30 minutes or when the edge of the cloth is raised, thereby identifying the abnormal reason as hand fatigue. According to the sequence influence (speed fluctuation) and the abnormal reason (hand fatigue), set a gesture compensation mechanism: when a left-right shaking with an amplitude less than 0.4 centimeters is detected, the current speed is temporarily maintained and not immediately adjusted according to the shaking. At the same time, identify the operating trajectory of the gesture sequence and find that the palm still has a tendency to continue to move upward after shaking. Based on the trajectory and the compensation mechanism, set a homing path, i.e., if the palm returns to the upward trajectory after shaking, the cutting machine speed continues to adjust according to the upward amplitude. Combine the relative position of the key gesture (such as the current height of the palm relative to the initial reference point) and the homing path to clearly define the response rule: the cutting machine speed increases by 0.5 meters / minute for every 1 centimeter of upward movement of the palm, the speed is not adjusted when the shaking amplitude is ≤0.4 centimeters, and the speed continues to increase according to the amplitude if the palm returns to the upward trajectory after shaking.

[0035] Combine the gesture recognition model and the response rule to perceive the gesture operation state in real time: When the operator's palm moves upward from the initial reference point to 2 centimeters and remains stable, the model identifies the current operation state as "stable acceleration to 1 meter / minute" (2 centimeters x 0.5 meters / minute per centimeter). At the same time, analyze the trend of environmental changes based on the depth vision data: The continuously collected depth data shows that the light intensity value of the workshop LED lamp decreases by 10% within 5 minutes (which may be caused by the aging of the lamp tube), the stacking height of the cloth decreases by 0.5 centimeters due to the advancement of the cutting progress, the vibration frequency of the sewing machine does not change, but the amplitude slightly increases (from 0.1 millimeters to 0.15 millimeters). These environmental changes may affect the clarity of subsequent gesture recognition, such as the decrease in light intensity may reduce the accuracy of depth data extraction at the edge of the palm, and the increase in amplitude may increase the false recognition probability of false gestures.

[0036] Set the abnormal perception mode according to the gesture operation state (stable acceleration) and the environmental change trend (light intensity decreases, amplitude increases): based on the current stable acceleration operation state, identify the surrounding interference gestures, such as the small tremor of the palm edge in the depth data due to the increase in amplitude, which may be misidentified as a small upward or downward movement; analyze the interaction between this interference gesture and the normal speed control gesture: the moving amplitude of the interference gesture is only 0.1-0.2 cm, and the duration is ≤0.3 seconds, which is significantly different from the uniform and stable movement of the normal upward gesture (amplitude ≥0.3 cm / frame, duration ≥0.5 seconds); according to the environmental change trend (increased amplitude leading to increased interference) and the interaction relationship (small amplitude and short duration of interference gesture), identify the potential risk factor as "increased amplitude may lead to misidentification of speed control gestures, causing unnecessary speed adjustment", and the risk level is mild; according to the current stable operation state and the mild risk level, set the risk response mode: when a hand movement with a moving amplitude <0.3 cm and a duration <0.5 seconds is detected, do not trigger speed adjustment, and consider it as an interference signal.

[0037] Identify gesture abnormality based on abnormal perception mode: in the environment with increased amplitude, the depth data shows that the palm has a small upward movement of 0.25 cm for 0.4 seconds, combined with historical operation data (when the amplitude is normal, the moving amplitude of the operator's stable acceleration gesture is all ≥0.3 cm, and the duration is ≥0.5 seconds), it is identified that the current behavior deviation is "not consistent with the moving characteristics of the normal acceleration gesture"; based on the normal operation parameters in the historical data (acceleration gesture moving amplitude ≥0.3 cm, duration ≥0.5 seconds), set the abnormal parameter threshold: hand movement with moving amplitude <0.3 cm or duration <0.5 seconds is considered abnormal; according to the threshold and the current behavior deviation of 0.25 cm and 0.4 seconds, it is identified that the small upward movement belongs to the gesture abnormality, and the abnormal reason is the slight hand tremor caused by the increased amplitude of the sewing machine.

[0038] According to the gesture abnormal situation and the trajectory prediction network, a gesture adjustment mechanism is set: the abnormal operation gesture is "0.25 cm, 0.4 s of slight upward movement", and the abnormal type is "false acceleration gesture caused by environmental vibration"; based on the abnormal type, an abnormal strain layer is set: for the false acceleration gesture, the acceleration instruction is not executed, and the current speed is maintained; the adjacent gesture trajectory of the abnormal gesture is identified through the trajectory prediction network, it is found that the palm is stably upward to 2 cm before the abnormality, and the palm returns to 2 cm within 0.5 s after the abnormality and continues to stably move upward, and an adjustment path is set based on the adjacent trajectory: the speed corresponding to the current 2 cm height is maintained at 1 m / min, and when the palm returns to the normal upward trajectory (such as continuing to move upward to 2.5 cm), the speed is increased to 1.25 m / min according to the response rule; the abnormal strain layer (false acceleration is not executed) and the adjustment path (adjustment after returning to the trajectory) are combined to complete the setting of the gesture adjustment mechanism.

[0039] According to the abnormal perception mode, the gesture adjustment mechanism and the response rule, a gesture recognition result is output: the abnormal perception mode identifies that the slight upward movement is a false gesture caused by vibration, the adjustment mechanism stipulates that false acceleration is not executed and adjustment is made after returning to the trajectory, and the response rule clearly defines the corresponding relationship between the upward movement amplitude and the speed, so the final output is "maintain the current cutting machine speed of 1 m / min, and adjust the amplitude after the palm continues to stably move upward", and the cutting machine runs at the current speed according to the result. The subsequent operator stably moves the palm upward to 2.5 cm, and the depth vision data captures this normal gesture, the model increases the speed to 1.25 m / min according to the response rule, and the environmental change trend shows that the light intensity continues to slowly decrease, the abnormal perception mode automatically increases the monitoring intensity of the "palm edge recognition accuracy", and when the light intensity decreases to affect the palm contour extraction, the adjustment mechanism further optimizes the filtering parameters of the depth data to ensure the accuracy of gesture recognition, thereby realizing stable and accurate control of the running speed of the cutting machine through gestures, and adapting to different cutting working condition requirements.

[0040] It should be noted that, in the present text, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device.

[0041] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A gesture recognition method for an automatic cloth cutting machine based on depth vision, characterized in that: The method comprises: acquiring depth vision data to be identified, identifying a gesture operation environment and a gesture type based on the depth vision data, and collecting environmental feature data of the gesture operation environment; Based on the gesture operation environment and gesture type, setting model parameters of a gesture recognition model, and locating relative positions of key gestures in the gesture recognition model; Based on the relative positions, construct a trajectory prediction network for key gestures; Setting gesture response rules based on the trajectory prediction network and relative positions; Combining the gesture recognition model and gesture response rules, real-time perception of gesture operation status; based on the depth vision data, analyzing the environmental change trend of the gesture operation environment; Combine gesture operation status and environmental change trends to set abnormal perception mode; Based on the abnormality perception pattern, identifying abnormal gesture conditions; Setting a gesture adjustment mechanism based on the gesture abnormality and trajectory prediction network; According to the abnormal perception mode, gesture adjustment mechanism and gesture response rules, a gesture recognition result is output.

2. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The identifying the gesture operation environment and the gesture type based on the depth vision data includes: extracting background features of the gesture operation environment based on the depth vision data; Dispatching operation scene information and gesture types of the gesture operation environment according to the background features; Adaptively configuring model parameters of the gesture recognition model based on the operation scenario information and the gesture type to obtain a configured gesture recognition model; The environmental feature data of the gesture operation environment is determined by combining the operation scenario information and the configured gesture recognition model.

3. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The step of constructing a trajectory prediction network for key gestures based on the relative positions includes: Based on the relative positions, calculating the relative positions of adjacent gestures of the key gesture; Identifying non-critical gestures at relative positions of the adjacent gestures; Analyzing characteristic behaviors of the non-critical gestures relative to the critical gestures; Extracting the operational behavior features of the key gestures, and analyzing the impact of the operational behavior features on the operations of non-key gestures based on the characteristic behaviors; Based on the characteristic behaviors and operation impacts, a trajectory prediction network for key gestures is constructed.

4. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, wherein: The step of setting a gesture response rule based on the trajectory prediction network and the relative position includes: monitoring the operation gestures of the gesture sequence in real time, and identifying abnormal gestures in the operation gestures; analyzing, based on the trajectory prediction network, the influence of the abnormal gesture on the sequence of the operation gesture; Dispatching operation association data of the gesture sequence and the abnormal gesture, and identifying an abnormal cause of the abnormal gesture based on the operation association data; Setting a gesture compensation mechanism for abnormal gestures based on the sequence impact and abnormal causes; Identify the operation trajectory of the gesture sequence, and set a homing path for abnormal gestures based on the operation trajectory and gesture compensation mechanism; In combination with the relative position and the homing path, a gesture response rule is set.

5. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The analyzing, based on the depth vision data, an environmental change trend of the gesture operation environment includes: Performing attribute classification on the depth vision data to obtain classified environment data; Extracting key environmental features of the classified environmental data, and collecting historical environmental data of the gesture operation environment based on the key environmental features; identifying operation interference data and light change data in the historical environmental data; analyzing an operation interference pattern and a gesture behavior pattern of a gesture operation environment according to the operation interference data; Analyzing the visibility level of the gesture operation environment based on the light change data; Combined with the operation interference pattern, gesture behavior pattern and visibility level, the environmental change trend of the gesture operation environment is analyzed.

6. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The abnormality perception mode is set by combining the gesture operation status and the environmental change trend, including: Based on the gesture operation state, identify the surrounding interfering gestures of the gesture sequence; Analyze the interactive relationship between the surrounding interference gestures and the gesture sequence; identify potential risk factors of the gesture sequence based on the environmental change trend and the interactive relationship, and identify the risk level of the potential risk factors; Set the risk response mode of the gesture sequence according to the gesture operation status and risk level; Set the abnormal perception mode based on the risk level and risk response method.

7. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The identifying abnormal gesture situation based on the abnormal perception pattern includes: Based on the abnormal perception mode, collecting historical operation data of the gesture sequence; and identifying normal operation parameters of the gesture sequence according to the historical operation data; Identify the current operation state of the gesture sequence, and identify the current behavior deviation of the gesture sequence by combining the current operation state and historical operation data; Based on the normal operating parameters, setting abnormal parameter thresholds for the gesture sequence; Identify abnormal gesture situations based on the abnormal parameter threshold and the current behavior deviation.

8. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The step of setting a gesture adjustment mechanism according to the abnormal gesture situation and the trajectory prediction network includes: According to the abnormal gesture situation, locate the abnormal operation gesture of the gesture sequence; Analyzing the abnormality type of the abnormal operation gesture, and setting an abnormal response layer for the abnormal operation gesture based on the abnormality type; identifying adjacent gesture trajectories of the abnormal operation gesture according to the trajectory prediction network; Setting an adjustment path for the abnormal operation gesture based on the adjacent gesture trajectories; Combine the abnormal response layer and adjustment path to set up the gesture adjustment mechanism.

9. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, wherein: The output gesture recognition result includes: Identify operational anomalies in gesture sequences based on anomaly perception patterns; Based on operational anomalies, set the trigger conditions for the gesture adjustment mechanism and set the risk avoidance method for the gesture sequence; According to the triggering conditions and the risk avoidance method, the gesture sequence is recognized and adjusted to obtain the adjusted gesture data; Based on the adjusted gesture data and gesture response rules, the gesture recognition result is output.

10. The method for gesture recognition of an automatic cloth cutting machine based on depth vision according to claim 1, characterized in that: The method further includes: estimating the overall stability of the gesture sequence based on the gesture recognition result; generating a warning gesture sequence according to the overall stability; and performing a temporary adjustment operation on the warning gesture sequence.

Images