A flexible control method for manufacturing process of machining center based on motion capture

By introducing a combination of gesture controllers and visual analysis chips in the machining center, and generating and confirming temporary control instructions, the flexible control problem of the machining center under diversified tasks is solved, and an efficient and safe machining process is achieved.

CN118927013BActive Publication Date: 2025-05-20WUHAN VOCATIONAL COLLEGE OF SOFTWARE & ENG (WUHAN OPEN UNIV)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411259915.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2025-05-20
Estimated Expiration
2044-09-10

AI Technical Summary

Technical Problem

The existing machining center intelligent control system is insufficient in the face of diverse machining tasks and process changes, making it difficult to achieve flexible gesture control.

Method used

The gesture controller is used to detect gesture instructions through the gesture switch module, combine the visual analysis chip and the data analysis chip to generate temporary control instructions, and confirm and fine-tune them through the touch display, and finally execute it by the CNC system.

Benefits of technology

It realizes efficient and flexible gesture control in the machining center environment, avoids misidentification and misoperation, and improves the flexibility and safety of the processing process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118927013B_ABST
    Figure CN118927013B_ABST
Patent Text Reader

Abstract

The present invention discloses a flexible control method for a machining center manufacturing process based on motion capture, and relates to the technical field of machining center program control. A motion picture capture module of a gesture controller first collects an operator's gesture picture, and a laser sensor module then senses gesture scale data; a gesture instruction analysis module parses to obtain gesture picture data and gesture scale data, and a visual analysis chip performs a matching degree analysis on the gesture picture data based on a standard gesture feature database stored in a memory. If there is a matching degree greater than a set matching degree threshold, corresponding control instruction template data stored in the memory is extracted based on the standard gesture feature, and the corresponding control instruction template data is transmitted to an instruction generation chip; when the operator observes the machining process through a transparent window of the machining center, the operator can realize flexible adjustment control of the machining process without leaving the workpiece, thereby providing a more reliable solution for the flexible control of the machining center manufacturing process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of program control of machining centers, and more particularly to a flexible control method for the manufacturing process of machining centers based on motion capture. Background Art

[0002] The intelligent control of machining centers refers to the use of advanced control theories, computer technologies, sensor technologies, and artificial intelligence algorithms to achieve the automation, precision, and high efficiency of the machining process. The following is the current situation and its defects of the existing intelligent control of machining centers:

[0003] Machining centers generally adopt numerical control systems (CNC) to achieve the automation of the machining process. The intelligent programming system can automatically generate machining programs based on CAD models, reducing the workload of manual programming. The intelligent control highly depends on the quality and quantity of data. Insufficient or low-quality data will affect the system performance. In the face of diverse machining tasks and constantly changing process requirements, the adaptability of the intelligent system needs to be improved.

[0004] Gesture control technology is an advanced human-computer interaction method that allows operators to control the execution actions of objects through hand movements, improving the intuitiveness and efficiency of operations.

[0005] Therefore, how to combine gesture control technology with the numerical control system so that operators can use gestures to intervene and control the machining process more delicately and flexibly during the observation of workpieces is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] To solve the technical problem of combining gesture control technology with the numerical control system, the present invention provides a flexible control method for the manufacturing process of machining centers based on motion capture. The following technical solutions are adopted:

[0007] A flexible control method for the manufacturing process of machining centers based on motion capture. The gesture switch module of the gesture controller detects the front gesture instruction switch signal. If there is a gesture intervention instruction signal, the gesture controller starts to identify and analyze the gesture control signal by the following steps:

[0008] Step 1, the action screen capture module of the gesture controller first captures the operator's gesture screen, and then the laser sensor module senses the gesture scale data;

[0009] Step 2, pack and send the gesture screen data and gesture scale data to the gesture instruction analysis module combined with a vision analysis chip and a data analysis chip;

[0010] Step 3: The gesture instruction analysis module parses to obtain gesture screen data and gesture scale data. The visual analysis chip performs a matching degree analysis on the gesture screen data based on the standard gesture feature database stored in the memory. If there is a matching degree greater than the set matching degree threshold, it extracts the corresponding control instruction template data stored in the memory based on the standard gesture features and transmits the corresponding control instruction template data to the instruction generation chip;

[0011] Step 4: The data analysis chip parses the gesture scale data, analyzes the control scale data contained in the gesture scale data, and transmits the control scale data to the instruction generation chip;

[0012] Step 5: The instruction generation chip combines the control scale data with the corresponding control instruction template data to generate a temporary control instruction for controlling the numerical control system;

[0013] Step 6: The temporary control instruction displays the corresponding text description of the temporary control instruction on the gesture instruction touch display, and provides an option to modify the scale data and an instruction confirmation item;

[0014] Step 7: The operator confirms the operation instruction through the instruction confirmation item on the gesture instruction touch display, or confirms the temporary control instruction after fine-tuning through the option to modify the scale data;

[0015] Step 8: The numerical control system replaces the corresponding part of the existing processing control instruction with the temporary control instruction and executes it.

[0016] By adopting the above technical solution, a gesture switch module is used to perform on-off control on the entire gesture detection to avoid misoperation. Specifically, it can be a gesture instruction switch mode, a press switch mode, or an induction switch mode;

[0017] When the gesture switch module detects a front gesture instruction switch signal, it activates the action screen capture module and the laser sensor module. The action screen capture module first captures the operator's gesture screen, and then the laser sensor module senses the gesture scale data;

[0018] The operator's gesture screen is for interactive operation items. For example, the operation items can be feed speed, program selection control, adjustment of feed amount, etc.; the gesture scale data can directly adjust the control value, solving the problem that specific values cannot be accurately adjusted by separate gesture control. For example, the specific value of the feed amount can be directly adjusted through the gesture scale data. Here, gesture recognition is achieved by using a laser sensor, avoiding the problem of misrecognition caused by visual gesture recognition;

[0019] Among them, the recognition of the gesture screen is achieved by the visual analysis chip performing a matching degree analysis on the gesture screen data based on the standard gesture feature database stored in the memory;

[0020] The gesture scale data uses a data analysis chip to analyze the trigger data of the laser sensor, so as to efficiently identify the corresponding operability and operation values of gestures, and better adapt to the noisy application environment of the machining center;

[0021] It can efficiently generate temporary control instructions. Before executing the temporary control instructions, it is also necessary to display the operation instructions to be confirmed through a touch display, and at the same time provide an option to modify the scale data, so as to achieve more flexible control and avoid production accidents caused by directly executing due to misidentification.

[0022] The above-mentioned gesture switch module, action screen capture module, laser sensor module and touch display, these interactive components should be installed on one side of the transparent window of the machining center. When the operator observes the machining process through the transparent window of the machining center, it can realize flexible adjustment control of the machining process through gesture control without basically leaving the workpiece in sight, providing a more reliable solution for the flexible control of the manufacturing process of the machining center.

[0023] Optionally, the laser sensor module includes a plurality of laser sensors arranged in an array and a liquid crystal digital display. The plurality of laser sensors are arranged at equal intervals, and the liquid crystal digital display is respectively communicatively connected to the numerical control system, the visual analysis chip, the data analysis chip and the plurality of laser sensors; the installation position of the action screen capture module is on one side of the laser sensor module.

[0024] By adopting the above technical solution, in the actual operation process, gesture control is generally used for fine adjustment, so the control scale is small. For example, a 2×2 laser sensor matrix can be used.

[0025] The occlusion of the two laser sensors in the first row is used to identify addition and subtraction, and the occlusion of the two laser sensors in the second row is used to realize the selection of digits. The liquid crystal digital display can interact with the visual analysis chip and the data analysis chip to obtain the operation items corresponding to the gesture operation, and then interact with the numerical control system to obtain the value corresponding to the current item; the operator can realize the addition and subtraction of the value by occluding the two laser sensors in the first row, and the occlusion of the two laser sensors in the second row is used to realize the selection of digits, so as to accurately modify the value by gesture.

[0026] Optionally, the gesture switch module includes two contact sensors and a single-chip microcomputer, and the single-chip microcomputer is respectively communicatively connected to the two contact sensors.

[0027] Optionally, the installation distance between the two contact sensors is greater than 100 mm. When the operator's hand swings from in front of any one contact sensor to another contact sensor within a set time, the single-chip microcomputer outputs a signal to start detecting the gesture screen to the action screen capture module and the laser sensor module, and the action screen capture module and the laser sensor module are turned on.

[0028] By adopting the above technical solution, in order to avoid misoperation as much as possible and prevent the action screen capture module and the laser sensor module from being in a compliant working state for a long time, a gesture switch module is used to control the switch. Specifically, a control mode using two contact sensors can be adopted. When a hand quickly sweeps in front of the sensing heads of the two contact sensors, for example, if the single-chip microcomputer receives the signals of the two contact sensors within 0.5 seconds, it will communicate and output a signal to start detecting the gesture screen to the action screen capture module and the laser sensor module.

[0029] Optionally, in step 1, when the action screen capture module captures the operator's gesture screen, it records the corresponding timestamp when capturing each frame of the screen and adds the timestamp data to the beginning position of the frame data.

[0030] Optionally, in step 3, during the matching degree analysis by the vision analysis chip, if there is no matching degree greater than the set matching degree threshold, the standard gesture feature data with the highest matching degree is compared with the gesture screen data in time segments. If there is at least two continuous time stamps forming a time segment where the matching degree of the gesture screen data and the standard gesture feature data in the corresponding time segment is greater than the set matching degree threshold, it is determined that the matching is successful, and the standard gesture feature data will transmit the corresponding control instruction data to the instruction generation chip.

[0031] By adopting the above technical solution, since the environment of the machining center is relatively harsh, environments such as water vapor and smoke may affect gesture recognition. Here, when the action screen capture module captures the gesture screen, the timestamp data is synchronously added to the beginning position of the frame data;

[0032] When the vision analysis chip performs the matching degree analysis, if there is no matching degree greater than the set matching degree threshold, it does not directly determine that the recognition fails. Instead, it can determine the operator's gesture based on whether there is at least two continuous time stamps forming a time segment where the matching degree of the gesture screen data and the standard gesture feature data in the corresponding time segment is greater than the set matching degree threshold, thereby improving the robustness of gesture recognition.

[0033] Optionally, in step 3, the vision analysis chip uses the following method to perform the matching degree analysis:

[0034] Step a, extract the key features composed of the contour and key points from the captured gesture screen;

[0035] Step b, standardize the extracted key features;

[0036] Step c, match the standardized key features with the standard gesture feature database stored in the memory;

[0037] Step d: Calculate the Euclidean distances between multiple key feature vectors of the captured gesture features and the standard multiple key feature vectors in the database respectively, and calculate the matching degree based on the Euclidean distances.

[0038] Optionally, in step d, the formula for calculating the Euclidean distance is:

[0039]

[0040] where A and B represent the key feature vectors of the captured gesture features and the standard multiple key feature vectors in the database respectively, n is the dimension of the feature vectors, A i and B i respectively represent the values of A and B in the i-th dimension;

[0041] Calculate the Euclidean distances between all pairs of key feature vectors, and then use the following formula to calculate the average value:

[0042]

[0043] where d(A i , B i ) is the Euclidean distance between the i-th pair of key feature vectors A i and B i , and w is the number of key feature vectors.

[0044] By adopting the above technical solution, the Euclidean distance itself is a distance metric, which directly gives the straight-line distance between two points in space. In the calculation of the matching degree, the Euclidean distance is usually used as a dissimilarity metric, that is, the smaller the distance, the more similar the two points (or feature vectors).

[0045] Optionally, use the following formula to calculate the matching degree X between the captured gesture picture and the standard gesture picture;

[0046]

[0047] By adopting the above technical solution, to convert the Euclidean distance into a matching degree, since the larger the Euclidean distance, the more dissimilar it is, we can convert it into a matching degree through a formula. The range of the matching degree X is between [0, 1], where 1 represents a perfect match and 0 represents a complete mismatch.

[0048] In summary, the present invention includes at least one of the following beneficial technical effects:

[0049] The present invention can provide a flexible control method for the manufacturing process of a machining center based on motion capture. A gesture switch module is used to control the on / off of the entire gesture detection to avoid misoperations. The gesture screen of the operator is for interactive operation items. For example, the operation items can be feed speed, program selection control, adjustment of feed amount, etc.; the gesture scale data can directly adjust the control value, solving the problem that specific values cannot be accurately adjusted by separate gesture control. For example, the specific value of the feed amount can be directly adjusted through the gesture scale data. Here, gesture recognition is implemented using a laser sensor, avoiding the problem of misrecognition caused by visual gesture recognition; among them, the recognition of the gesture screen is achieved by a vision analysis chip analyzing the matching degree of the gesture screen data based on the standard gesture feature database stored in the memory; and the gesture scale data uses a data analysis chip to analyze the trigger data of the laser sensor, so as to efficiently realize the recognition of gesture corresponding operability and operation value, and be more suitable for the noisy application environment of the machining center; it can efficiently generate temporary control instructions, and before executing the temporary control instructions, it is also necessary to display the operation instructions to be confirmed through a touch display, and at the same time provide an option to modify the scale data, thus realizing more flexible control and avoiding production accidents caused by directly executing due to misrecognition; when the operator observes the machining process through the transparent window of the machining center, the flexible adjustment control of the machining process can be realized by gesture control without the operator's line of sight leaving the workpiece basically, providing a more reliable solution for the flexible control of the machining center manufacturing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 FIG. is a schematic diagram of the electrical component connection principle of a flexible control method for the manufacturing process of a machining center based on motion capture according to the present invention;

[0051] Figure 2 FIG. is a schematic diagram of the electrical component connection principle adopted by a flexible control method for the manufacturing process of a machining center based on motion capture according to the present invention.

[0052] DESCRIPTION OF THE REFERENCE NUMERALS: 1. Gesture controller; 11. Gesture switch module; 111. Contact sensor; 112. Single-chip microcomputer; 12. Action screen capture module; 131. Vision analysis chip; 132. Data analysis chip; 133. Memory; 134. Instruction generation chip; 14. Laser sensor module; 141. Laser sensor; 142. Liquid crystal digital display; 2. Touch display; 100. Numerical control system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The present invention will be further described in detail below with reference to the accompanying drawings.

[0054] An embodiment of the present invention discloses a flexible control method for the manufacturing process of a machining center based on motion capture.

[0055] Reference Figure 1 and Figure 2 , Embodiment 1, a flexible control method for a manufacturing process of a machining center based on motion capture. The gesture switch module 11 of the gesture controller 1 detects the front gesture instruction switch signal. If there is a gesture intervention instruction signal, the gesture controller 1 starts to identify and analyze the gesture control signal by the following steps:

[0056] Step 1, the action screen capture module 12 of the gesture controller 1 first collects the gesture screen of the operator, and the laser sensor module 14 then senses the gesture scale data;

[0057] Step 2, pack and send the gesture screen data and the gesture scale data to the gesture instruction analysis module combined with the vision analysis chip 131 and the data analysis chip 132;

[0058] Step 3, the gesture instruction analysis module parses to obtain the gesture screen data and the gesture scale data. The vision analysis chip 131 performs a matching degree analysis on the gesture screen data based on the standard gesture feature database stored in the memory 133. If there is a matching degree greater than the set matching degree threshold, extract the corresponding control instruction template data stored in the memory 133 based on the standard gesture features, and transmit the corresponding control instruction template data to the instruction generation chip 134;

[0059] Step 4, the data analysis chip 132 analyzes the gesture scale data, analyzes the control scale data included in the gesture scale data, and transmits the control scale data to the instruction generation chip 134;

[0060] Step 5, the instruction generation chip 134 combines the control scale data with the corresponding control instruction template data to generate a temporary control instruction for controlling the numerical control system 100;

[0061] Step 6, the numerical control system 100 displays the text description corresponding to the temporary control instruction on the gesture instruction touch display 2, and provides an option to modify the scale data and an instruction confirmation item;

[0062] Step 7, the operator confirms the operation instruction on the gesture instruction touch display 2 through the instruction confirmation item, or confirms the temporary control instruction after fine-tuning through the option to modify the scale data;

[0063] Step 8, the numerical control system 100 replaces the corresponding part of the existing machining control instruction with the temporary control instruction and executes it.

[0064] The gesture switch module 11 is used to perform switch control on the entire gesture detection to avoid misoperation. Specifically, it can be a gesture instruction switch mode, a press switch mode, or an induction switch mode;

[0065] After the gesture switch module 11 detects the front gesture instruction switch signal, the action picture capture module 12 and the laser sensor module 14 are activated. The action picture capture module 12 first captures the gesture picture of the operator, and then the laser sensor module 14 senses the gesture scale data.

[0066] The gesture picture of the operator is for interactive operation items. For example, the operation items can be feed speed, program selection control, adjustment of feed amount, etc. The gesture scale data can directly adjust the control value, solving the problem that separate gesture control cannot accurately adjust specific values. For example, the specific value of the feed amount can be directly adjusted through the gesture scale data. Here, gesture recognition is implemented using a laser sensor, avoiding the problem of misrecognition caused by visual gesture recognition.

[0067] Among them, the recognition of the gesture picture is realized by the vision analysis chip 131 analyzing the matching degree of the gesture picture data based on the standard gesture feature database stored in the memory 133.

[0068] The gesture scale data is analyzed by the data analysis chip 132 for the trigger data of the laser sensor, so as to efficiently realize the recognition of the operability and operation value corresponding to the gesture, and be more adaptable to the noisy application environment of the machining center.

[0069] It can efficiently generate a temporary control instruction, and before executing the temporary control instruction, it is also necessary to display the operation instruction to be confirmed through the touch display 2, and at the same time provide an option to modify the scale data, thus realizing more flexible control and avoiding production accidents caused by directly executing due to misrecognition.

[0070] The above-mentioned interactive components such as the gesture switch module 11, the action picture capture module 12, the laser sensor module 14, and the touch display 2 should be installed on one side of the transparent window of the machining center. When the operator observes the machining process through the transparent window of the machining center, without basically leaving the machining part, flexible adjustment control of the machining process can be realized through gesture control, providing a more reliable solution for the flexible control of the machining center manufacturing process.

[0071] Embodiment 2: The laser sensor module 14 includes a plurality of laser sensors 141 arranged in an array and a liquid crystal digital display 142. The plurality of laser sensors 141 are arranged at equal intervals. The liquid crystal digital display 142 is respectively communicatively connected to the numerical control system 100, the vision analysis chip 131, the data analysis chip 132, and the plurality of laser sensors 141. The installation position of the action picture capture module 12 is on one side of the laser sensor module 14.

[0072] In actual operation, gesture control is generally used for fine-tuning, so the scale of control is relatively small. For example, a two-by-two laser sensor 141 array can be used. The shielding of the two laser sensors 141 in the first row is used to identify addition and subtraction, and the shielding of the two laser sensors 141 in the second row is used to select the number of digits. The LCD digital display 142 can interact with the visual analysis chip 131 and the data analysis chip 132 to interact with the operation items corresponding to the gesture operation, and then interact with the numerical control system 100 to interact with the numerical control system 100. The operator can add or subtract the value by shielding the two laser sensors 141 in the first row, and the shielding of the two laser sensors 141 in the second row is used to select the number of digits, thereby realizing the precise gesture modification of the value.

[0073] Embodiment 3, the gesture switch module 11 includes two contact sensors 111 and a single-chip microcomputer 112, and the single-chip microcomputer 112 is respectively connected to the two contact sensors 111 for communication.

[0074] Example 4, the installation distance between the two touch sensors is greater than 100mm. When the operator's hand swings from the front of any touch sensor 111 to the other touch sensor 111 within a set time, the single-chip computer 112 communicates and outputs a signal to start detecting the gesture screen to the motion screen capture module 12 and the laser sensor module 14, and the motion screen capture module 12 and the laser sensor module 14 are turned on.

[0075] In order to avoid misoperation as much as possible and to prevent the motion picture capture module 12 and the laser sensor module 14 from being in a working state for a long time, the gesture switch module 11 is used to control the switch. Specifically, the control mode of two contact sensors 111 can be used. The hand is quickly swept in front of the sensor heads of the two contact sensors 111. For example, if the single-chip computer 112 receives the signals of the two contact sensors 111 within 0.5 seconds, it will communicate with the motion picture capture module 12 and the laser sensor module 14 to output a signal to start detecting the gesture picture.

[0076] Example 5, in step 1, the action picture capture module 12 collects the operator's gesture picture and records the corresponding timestamp when capturing each frame, and adds the timestamp data to the beginning of the frame data.

[0077] Example 6, in step 3, during the matching degree analysis performed by the visual analysis chip 131, if there is no matching degree greater than the set matching degree threshold, the standard gesture feature data with the highest matching degree is taken to compare with the gesture screen data in different time periods. If there are gesture screen data with at least two consecutive timestamps forming a time period and the matching degree of the time period corresponding to the standard gesture feature data is greater than the set matching degree threshold, it is determined that the match is successful, and the control instruction data corresponding to the standard gesture feature data is transmitted to the instruction generation chip 134.

[0078] Since the environment of the machining center is relatively harsh, environments such as water vapor and smoke may affect gesture recognition. Here, when the action screen capture module 12 captures a gesture screen, timestamp data is synchronously added to the beginning position of the frame data;

[0079] When the vision analysis chip 131 performs matching degree analysis, if there is no matching degree greater than the set matching degree threshold, it does not directly determine that the recognition fails. Instead, it can determine the operator's gesture based on whether there is at least two consecutive timestamp - composed time - segment gesture screen data whose matching degree with the corresponding time - segment of the standard gesture feature data is greater than the set matching degree threshold, thereby improving the robustness of gesture recognition.

[0080] Example 7, in step 3, the vision analysis chip 131 performs matching degree analysis using the following method:

[0081] Step a, extract the key features composed of contours and key points from the captured gesture screen;

[0082] Step b, standardize the extracted key features;

[0083] Step c, match the standardized key features with the standard gesture feature database stored in the memory 133;

[0084] Step d, calculate the Euclidean distances between multiple key feature vectors of the captured gesture features and multiple standard key feature vectors in the database respectively, and perform matching degree calculation based on the Euclidean distances.

[0085] Example 8, in step d, the formula for calculating the Euclidean distance is:

[0086]

[0087] where A and B represent the key feature vectors of the captured gesture features and multiple standard key feature vectors in the database respectively, n is the dimension of the feature vector, \(A_{i}\) i and \(B_{i}\) i represent the values of A and B in the \(i\) - th dimension respectively;

[0088] Calculate the Euclidean distances between all pairs of key feature vectors, and then use the following formula to find the average value:

[0089]

[0090] where \(d(A\) i , \(B\) i ) is the Euclidean distance between the \(i\) - th pair of key feature vectors \(A\) i and \(B\) i , and w is the number of key feature vectors.

[0091] The Euclidean distance itself is a distance metric that directly gives the straight-line distance between two points in space. In the calculation of the matching degree, the Euclidean distance is usually used as a dissimilarity metric, that is, the smaller the distance, the more similar the two points (or feature vectors).

[0092] Example 9: The matching degree X between the captured gesture screen and the standard gesture screen is calculated using the following formula;

[0093]

[0094] To convert the Euclidean distance into a matching degree, since the larger the Euclidean distance, the more dissimilar it is, we can convert it into a matching degree through a formula. The range of the matching degree X is between [0, 1], where 1 represents a perfect match and 0 represents a complete mismatch.

[0095] The above are all preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A flexible control method for manufacturing process of machining center based on motion capture, characterized in that: The gesture switch module (11) of the gesture controller (1) detects the front gesture command switch signal. If there is a gesture intervention command signal, the gesture controller (1) uses the following steps to start identifying and analyzing the gesture control signal: Step 1, the motion picture capture module (12) of the gesture controller (1) first collects the operator's gesture picture, and the laser sensor module (14) then senses the gesture scale data; Step 2, packaging the gesture picture data and the gesture scale data and sending them to a gesture command analysis module based on a combination of a visual analysis chip (131) and a data analysis chip (132); Step 3: The gesture instruction analysis module parses the gesture screen data and the gesture scale data, and the visual analysis chip (131) performs a matching degree analysis on the gesture screen data based on the standard gesture feature database stored in the memory (133). If the matching degree is greater than a set matching degree threshold, the corresponding control instruction template data stored in the memory (133) is extracted based on the standard gesture feature, and the corresponding control instruction template data is transmitted to the instruction generation chip (134); Step 4, the data analysis chip (132) parses the gesture scale data, analyzes the control scale data contained in the gesture scale data, and transmits the control scale data to the instruction generation chip (134); Step 5, the instruction generation chip (134) combines the control scale data with the corresponding control instruction template data to generate a temporary control instruction for controlling the numerical control system (100); Step 6: The temporary control instruction displays a text description corresponding to the temporary control instruction on the gesture instruction touch display (2), and provides an option to modify the scale data and an instruction confirmation item; Step 7, the operator confirms the operation command through the command confirmation item on the gesture command touch display (2), or confirms the temporary control command after fine-tuning through the option of modifying the scale data; Step 8, the numerical control system (100) replaces the corresponding part of the existing machining control instruction with the temporary control instruction and executes it; The laser sensor module (14) comprises a plurality of laser sensors (141) and a liquid crystal digital display (142) arranged in an array, the plurality of laser sensors (141) being arranged at equal intervals, and the liquid crystal digital display (142) being respectively connected in communication with the numerical control system (100), the visual analysis chip (131), the data analysis chip (132) and the plurality of laser sensors (141); and the installation position of the motion picture capture module (12) is located on one side of the laser sensor module (14).

2. The flexible control method of a machining center manufacturing process based on motion capture according to claim 1 is characterized in that: The gesture switch module (11) comprises two contact sensors (111) and a single-chip microcomputer (112), wherein the single-chip microcomputer (112) is communicatively connected to the two contact sensors (111) respectively.

3. The flexible control method of a machining center manufacturing process based on motion capture according to claim 2 is characterized in that: The installation interval between the two contact sensors is greater than 100 mm. When the operator's hand swings from the front of any contact sensor (111) to the other contact sensor (111) within a set time, the single-chip computer (112) communicates and outputs a signal to start detecting the gesture image to the motion image capture module (12) and the laser sensor module (14), and the motion image capture module (12) and the laser sensor module (14) are turned on.

4. The flexible control method of a machining center manufacturing process based on motion capture according to claim 3 is characterized in that: In step 1, the action picture capturing module (12) collects the operator's gesture picture and records the corresponding timestamp when capturing each frame, and adds the timestamp data to the beginning position of the frame data.

5. The flexible control method of a machining center manufacturing process based on motion capture according to claim 4 is characterized in that: In step 3, during the matching degree analysis performed by the visual analysis chip (131), if there is no matching degree greater than the set matching degree threshold, the standard gesture feature data with the highest matching degree is taken and compared with the gesture screen data in different time periods. If there are gesture screen data consisting of a time period with at least two consecutive timestamps and a matching degree greater than the set matching degree threshold between the time period corresponding to the standard gesture feature data, the match is determined to be successful, and the control instruction data corresponding to the standard gesture feature data is transmitted to the instruction generation chip (134).

6. The method for flexible control of a machining center manufacturing process based on motion capture according to claim 5 is characterized in that: In step 3, the visual analysis chip (131) performs matching analysis using the following method: Step a, extracting key features consisting of contours and key points from the captured gesture image; Step b, standardizing the extracted key features; Step c, matching the standardized key features with a standard gesture feature database stored in a memory (133); Step d: respectively calculating the Euclidean distances between the multiple key feature vectors of the captured gesture features and the multiple standard key feature vectors in the database, and performing matching degree calculation based on the Euclidean distances.

7. The method for flexible control of a machining center manufacturing process based on motion capture according to claim 6 is characterized in that: In step d, the formula for calculating the Euclidean distance is: Where A and B represent the key feature vector of the captured gesture features and the standard multiple key feature vectors in the database, n is the dimension of the feature vector, A i and B i Realize the values ​​of A and B in the i-th dimension respectively; The Euclidean distances between all pairs of key feature vectors are calculated and then averaged using the following formula: Where d(A i , B i ) is the i-th key feature vector A i and B i , w is the number of key eigenvectors.

8. The method for flexible control of a machining center manufacturing process based on motion capture according to claim 7 is characterized in that: The matching degree X between the captured gesture image and the standard gesture image is calculated using the following formula;

Citation Information

Patent Citations

  • Visual industrial controller and method based on PLC

    CN116430795A

  • Non-contact three-dimensional model man-machine interaction method based on machine vision and gesture recognition

    CN118466805A