INTUITIVE AND ADAPTIVE GESTURE ORIENTATION CONTROL SYSTEM AND METHOD FOR ROBOTS

Abstract

The invention relates to an adaptive and real-time robot control system and method which includes a processor unit (600) that detects the orientation of at least one hand image processed by artificial intelligence and controls the movement of robots (100) with at least one degree of freedom that have the potential to move in Cartesian space, with the help of a kalman filter. Figure 1

Description

1 TARIFF INTUITIVE AND ADAPTIVE GESTURE ORIENTATION CONTROL FOR ROBOTS SYSTEM AND METHOD TECHNICAL FIELD The invention allows for the creation of at least one robot capable of processing information in Cartesian space, with a single hand gesture. control based on the hand movement orientation / inclinedness obtained from the image. It relates to a method and system that enables this. 10 PREVIOUS TECHNIQUE The rapid advancement of robotics and automation technologies means that robotic systems are becoming more... This enables its use in a wide range of applications. However, this development, 15 more intuitive and user-friendly, especially in controlling robotic arms This increases the need for control mechanisms. Traditional control methods, commands It is limited to tools such as controllers and programming interfaces, and often requires specialized skills. This requires hand movements to control robotic arms. It can be used. However, this method is compatible with existing solutions. 20 Its work is often difficult and requires precision in complex tasks. It is unable to provide responsiveness. Nowadays, hand control is required for robotic arms. angular encoder sensors and cameras for recognizing movements However, these methods are used. However, these methods have high potential, especially in complex environments. It remains limited in achieving accuracy. Furthermore, it fails to meet user expectations. insufficient to deliver a level of real-time performance to meet the demand. They are able to stay. In this context, many existing solutions rely on asynchronous or delayed control methods. This focuses on robotic manipulators providing real-time and fluid 30-degree operation. This restricts their actions. Some approaches, however, focus on simply identifying the manipulator instead of the actual manipulator. It focuses on controlling a robotic vehicle (for example, a robotic hand), and this situation This significantly reduces movement flexibility. Therefore, both sensitivity and 2 by meeting the responsiveness requirements, robotic manipulators in real time Methods and systems are needed to ensure control. The current patent application, CN114393571A, describes how to control the movements of a robotic arm using a hand. It is related to the system that enables control through gestures. Accordingly, hand gestures Detecting and controlling the robotic arm asynchronously or synchronously 5 is provided. In conclusion, all the problems mentioned above require an innovation in the relevant technical field. This has made it necessary to do it. A BRIEF DESCRIPTION OF THE INVENTION The present invention eliminates the aforementioned disadvantages and the related technical It is about a method and system aimed at bringing new advantages to the field. One aim of the invention is to control robots in real time in Cartesian space. The goal is to develop a method and system that enables this. Another purpose of the invention is to easily integrate it into different robotic arms, thus improving robotics. The goal is to develop a method and system that enables the control of the arms. 20 All the purposes mentioned above and those that will emerge from the detailed explanation below. The present invention aims to create at least one that can perform operations in Cartesian space. a robot that enables the capture of at least one hand movement image of a user. image capture unit, hand 25 taken from the aforementioned image capture unit a robot that can be controlled based on its movement and visual cues in a system containing a processor unit, by the mentioned processor unit It relates to a method for carrying out an action. Accordingly, i- a first-hand gesture of the user referring to the image capture unit taking the image, 30 ii- an input hand gesture image that has been trained with various hand gesture images. When received as, a hand movement's position information in Cartesian space is output as hand movement captured from an image capture unit to an AI model that provides this information. 3 The Cartesian hand gesture mentioned is applied by using the image as input. Receiving initial position information in space as output, iii- the first robotic movement corresponding to the first position information mentioned. determining the information, iv- According to the first robotic motion information mentioned, there is a 5 that enables the robot's movement. starting the engine, v- obtained by applying the first position information to a mathematical model Predicting a second-hand move of the user mentioned in the output, vi- Adjusting the motor angle according to the aforementioned second-hand movement estimation information. It includes the steps of the process. Thus, based on the user's hand movements, 10 more precise and intuitive control of the robot's manipulator is provided. A feature of a possible design of the invention is that the motor angle is the aforementioned second-hand After the step of adjusting the movement according to the predicted information, steps i-iv are 15. It is a repetition based on the user's second hand movement. Another possible configuration feature of the invention is the image capture unit. The process involves applying a pre-processing operation to the captured first-hand motion image. It includes step 20. Another possible configuration of the invention features the first position information. It involves the step of saving the data to the memory unit. Another possible configuration of the invention features 25 resulting from hand movements. by detecting sudden changes and controlling movement transitions. It includes the process step of carrying out the execution. The invention involves at least one robot capable of processing in Cartesian space and at least one user. an image capture unit that enables the capture of a hand movement image, 30 According to the hand movement image obtained from the aforementioned image capture unit a system that includes a processor unit that enables the control of the robot in question Its characteristic is that the mentioned processor unit is like the one in one of the previous requests. The method must be configured in such a way that the steps are carried out. 4 Another possible configuration of the invention involves a processor unit. The robot's operating system includes the control unit. BRIEF DESCRIPTION OF THE FIGURE 5 Figure 1 shows representative views of the robot. Figure 2 shows a representative image of the robot controlled according to the first hand movement. The appearance is given. 10 Figure 3 shows other representative views of robot movement. Figure 4 provides a representative view of the system's operating scenario. DETAILED DESCRIPTION OF THE INVENTION This detailed explanation of the invention only serves to better understand the subject. This is explained with examples that will not create any limiting effect. Referring to Figure 1, the invention involves at least one robot (100) capable of performing operations in Cartesian space. a method and system that enables control based on hand movement image (10) It is related to. The robot in question is a manipulator, and the movement of that manipulator is also related to it. It may include a motor that provides power. As is well known in technology, the manipulator, 25 robots (100) designed to perform a specific movement or operation It is a mechanical unit, usually with a lever mechanism. It controls the movement of the manipulator. The motor that provides this is the manipulator, which converts electrical energy into mechanical motion. It is a drive unit that enables linear or rotary motion. Accordingly, the method and system capture the same heat of the hand movement in the hand movement image. The robot is enabled to do it. 30 In a possible configuration of the invention, the robot (100) is a well-known Cartesian in the field. robot, an articulated manipulator, a SCARA robot, a Delta robot, an industrial robot, a serial robots, collaborative robots (working side-by-side with humans in a shared workspace) robots that can interact and work safely (at least one) There may be one. The system (10) enables the capture of at least one hand gesture image of a user. It includes an image capture unit (200). The invention has a possible 5 The image capture unit mentioned in its structure (200) is a camera It can include. The camera can be a stereo camera. This makes it expensive and... System (10) costs are reduced by reducing the need for special equipment. is being reduced. The system (10) includes a processor unit (300). The invention is possible. The processor unit mentioned in its structure (300) is a CPU, a GPU, a microprocessor etc. The processor unit (300) receives the hand movement from the image capture unit (200) 15 According to its image, it enables the control of the mentioned robot (100). For this purpose, the processor unit (300) and the image capture unit (200) are mentioned. It enables the capture of a first-hand image of the user's hand movement. The invention a preprocessing operation on the first hand movement image in a possible configuration It is applicable. 20 The processor unit (300) is a hand gesture that has been trained with various hand gesture images. the position of a hand movement in Cartesian space when it takes its image as input from the image capture unit to an artificial intelligence model that outputs this information. (200) by applying the first hand movement image taken as input 25 The first hand movement mentioned resulted in the first position information in Cartesian space. This ensures that it is taken as the first position in real time. The information is obtained as output. The first step in a possible construction of the invention. Position information can be information about the direction of hand movement. A possible invention... The artificial intelligence model mentioned in its structure is a deep learning-based object-oriented 30 It can be a sensing model. This significantly improves the accuracy of hand movement detection. by increasing the system effectively in (10) complex and dynamic environments Its operation is ensured. 6 As is well known in technical terms, the aforementioned Cartesian space is a coordinate system. using, usually three, methods for defining the positions of points or objects. It is a three-dimensional space. Accordingly, horizontal, vertical and depth can be represented using the x, y and z axes. The information is being determined. In particular, the z-axis, the depth dimension of hand movement. It can express its position (forward or backward). Accordingly, the first 5 Position information refers to the position of the user's hand movement in Cartesian space. This can contain three-dimensional information consisting of x, y, and z coordinates. In this way, hand movement can be experienced not only on a two-dimensional plane, but also in a three-dimensional volume. It also enables perception within. Therefore, the positional information obtained is handed down. It allows for a complete and precise understanding of its movement. Also, 10 users can directly control the robot (100) in a 3D environment and The robot (100) is enabled to perform precise tasks. Thus, the expert a user-friendly system for those who do not have robot (100) control is simplified (10) is provided. In addition, thanks to Cartesian space, complex It simplifies the execution of tasks and is 15% more efficient compared to joint space control. An intuitive and user-friendly control method and system (10) is provided. The system (10) includes a memory unit (400). The invention is possible. The first position information mentioned in its structure is sent to the memory unit mentioned (400) is recorded. In a possible configuration of the invention, the memory unit (400) stores the data 20 memories that store information permanently and also store information temporarily when needed. It may include a combination of these. Processor unit (300), first robotic movement corresponding to first position information. the determination of its information and the motor's 25 according to the first robotic movement information mentioned. This enables the motor to be operated using primary robotic motion information. It is operated in real time. In a possible configuration of the invention. first robotic movement information, the shape of the hand movement in the first hand movement image, Position information may be available. In a possible configuration of the invention, the hand gesture mentioned would allow the user to control the robot 30 (100) includes physical gestures performed to control. For example, When the user moves their hand horizontally to the right, this movement It is perceived as position information along the positive x-axis in Cartesian space. and the manipulator is made to move to the right. Similarly, 7 When the user raises their hand vertically upwards, this movement is positive. It is interpreted as position information on the axis and the manipulator's upward movement The movement is performed. Also, when the user pushes their hand forward, this movement occurs. It is perceived as depth information along the positive z-axis in Cartesian space. and the manipulator is extended forward. A possible 5 of the invention Its structure includes hand movements, which involve predefined hand gestures. The processor unit (300) converts the first position information into a mathematical model. The output obtained through its implementation refers to a second-hand movement of the user. It enables prediction. The aforementioned mathematical model is a Kalman 10 It is a filter. As is well known in technology, the Kalman Filter is used for real-time control. It is a state estimation algorithm used in systems. The measured current of the system... considering the conditions, measurement accuracy and system model It performs updates. This allows for the best possible representation of the system's actual state. It provides the prediction. 15 The processor unit (300) is the motor according to the mentioned second hand movement prediction information. It allows for the adjustment of the angle. The aforementioned motor angle is well known in technology. to what position or angle the rotating part of the motor brings the manipulator This angle indicates the direction of movement and position of the manipulator. It is used for checking. By using a Kalman filter, the output is first-hand information based on the first position. Directional information / orientation information is obtained from the movement. In this way, the hand movement It is determined in which direction the user tends to move. For example, a user 25 If the hand is moving to the right, the Kalman filter interprets the movement from position information. first hand movement along the positive x-axis by determining that it is to the right It extracts a corresponding directional information. The processor unit (300) takes this directional information based on the motor angle required for the manipulator to move to the right It enables adjustment. In this way, the 30 between the movements of the robot (100) Transitions occur faster and more smoothly, improving the system's response time. shortens and the robot (100) is synchronized with the user's hand movements. It enables the user to control the manipulator. The hand movements are realistic. 8 Timely processing and prediction reduces the risk of unwanted robotic actions, thus minimizing human error. It enhances operational safety during robot interactions. In a possible configuration of the invention, the directional information obtained from hand movement remains. Passing through the filter prediction algorithm, a packet of 5 exposures in Cartesian space (Position and orientation data) are transformed. Then this information is used in inverse kinematics. This is converted into motor position angles, and the motors use this movement. It provides. In a possible configuration of the invention, 10 hand movements can be filtered using a Kalman filter. sudden changes resulting from these changes are detected and the transitions between movements are controlled. This is ensured by applying the Kalman filter to the previous motion data. Based on this, it predicts the next position of movement. This process is sudden. It reduces the impact of sharp movements caused by changes and minimizes the movements. This ensures that it is performed in a more balanced way. Therefore, movement 15 The transitions are carried out in a controlled and systematic manner. Thus, sudden handshakes are avoided. By saturating and smoothing its movements, the robot's stability is increased. and environmental safety is protected. In a possible configuration of the invention, the motor is 20 according to the second hand movement prediction information. After the angle adjustment process step, the processor unit (300), image capture obtaining a second hand movement image from the unit (200) of the users It provides. After the aforementioned second hand gesture was captured, various hand gestures were observed. 25 A hand, trained with images of hand movements, takes an image of a hand movement as input. an artificial intelligence that outputs information about the position of a motion in Cartesian space second hand movement image taken from the image capture unit (200) of the model The second hand movement mentioned is implemented as an input in Cartesian space. The second position information is received as output. 30 The processor unit (300) corresponds to the second robotic position information mentioned. determining the motion information and according to the second robotic motion information mentioned It enables the engine to start. Then the processor unit (300), second 9 from the output obtained by applying position information to the mathematical model the prediction of a third-hand movement of the user in question and the engine the angle is adjusted according to the aforementioned third hand movement prediction information Thus, the hand required for the robot (100) to perform a task is provided. The movements are followed sequentially and fluidly, with 5 corresponding movements for each hand gesture. robot movement (500) is optimized in real time and robot (100) The manipulator adapts to the user's natural movements, making it more precise and It can be effectively controlled. As is well known in the technical field, the Robot Operating System (ROS) is a software framework consisting of 10 communication between hardware and software components in robotic applications It standardizes and ensures compatibility. The invention is possible. In its structure, the robot operating system mentioned in the system (10) includes the robot's hardware and It ensures compatibility by standardizing communication between software components. ROS is a modular and flexible platform for creating robot control systems. 15 by providing, the aforementioned motion control system can be easily used with different manipulators. This makes it possible to integrate and adapt it. In this way, control and movement capabilities of the robot (100) used in the system (10) is being optimized and the usability of the system in (10) different robotic applications is being increased. 20 In a possible configuration of the invention, the system (10) allows users to individually By learning these movements, users can adapt to them. Additionally, users... can create special motions in Cartesian space or the system (10) itself They can adapt it according to their preferences. Thus, personalized control experiences are created. 25 And over time, adaptations are made to new trends. In the system (10) processor units (300), the following steps are to be performed It provides. i- image capture unit (200) mentioned user a first hand 30 capturing the image of the movement, ii- an input hand gesture image that has been trained with various hand gesture images. When taken as such, it extracted the position information of a hand movement in Cartesian space. from the image capture unit to an artificial intelligence model (200) By applying the captured hand movement image as input, the aforementioned hand... the output of the first position information of its motion in Cartesian space taking, iii- the first robotic movement corresponding to the first position information mentioned. determining the information, 5 iv- Operating the motor according to the first robotic motion information mentioned, v- obtained by applying the first position information to a mathematical model The output obtained suggests a prediction of the user's second-hand transaction. being done, vi- motor angle is 10 according to the aforementioned second hand movement estimation information. adjustment In a possible configuration of the invention, the processor unit (300) is the motor angle. After the aforementioned second-hand movement prediction information is adjusted, steps i-iv It ensures that the steps are repeated according to the user's second hand movement. 15 In a possible configuration of the invention, the processor unit (300) includes a ROS control unit. It includes each step, starting with obtaining the first position information from the artificial intelligence model. Hand gesture information received from the image is sent directly to the ROS control unit. It can be transmitted. The mentioned ROS control unit has a Kalman filter, current gesture 20. after multiplying its location by a certain confidence score, it is added to the mathematical model. It transmits the data. The Kalman filter calculates the new state of the model and the state. It provides updates. Optimized gesture information, true Cartesian. The coordinates are translated. Then, the inverse kinematic block drives the robot's motors. It solves the angle values. These target angles are transmitted to the motors, allowing the robot to reach the desired 25° angles. movement is obtained. The ROS control unit writes target angles to the robot motors and controls the robot's actions accordingly. reading current status information (position, speed, torque / force, etc.) together The Kalman filter provides 30 new robot status and hand gesture status information together. It continues to generate new Cartesian position information by evaluating it. Also... Kalman filters saturate sudden hand movements in robot and perimeter safety applications. It softens and smooths. In addition, the target exceeds the safe maximum limits. The values ​​are checked by the ROS control unit and these values ​​are safe. 11 It is guaranteed that the values ​​will remain within the range. Therefore, the invention is a possible... from the step of obtaining the first position information as output with artificial intelligence in its structure The subsequent steps of the procedure can be performed via the ROS control unit. Thanks to the method steps and system (10) described above, robots (100) 5 It is ensured that it is monitored in real time. The scope of protection of the invention is set out in the attached claims and is strictly adhered to. The detailed explanation cannot be limited to the examples given. Because in technology... A specialist, without deviating from the main theme of the invention, will do what is described above. 10 It is clear that similar structures can emerge in light of this. 12 REFERENCE NUMBERS GIVEN IN THE FIGURE System 100 Robot 200 Image capture units 5 300 Processor units 400 memory units 500 Robot movements

Claims

13 REQUESTS 1. The invention is capable of producing at least one robot (100) and one robot capable of operating in Cartesian space. a system that captures at least one image of the user's hand movement image capture unit (200), 5 of the mentioned image capture units (200) According to the hand movement image taken, the mentioned robot (100) control (10) in a system that includes a processor unit (300) that enables it to be done a method to be performed by the mentioned processor unit (300) and its characteristic is; i- image capture unit (200) mentioned by the user a first hand 10 capturing the image of the movement, ii- a hand gesture image that has been trained with various hand gesture images the position of a hand movement in Cartesian space when taken as input an AI model that captures images and outputs this information. The hand movement image taken from unit (200) is input as 15 with its implementation, the aforementioned hand movement is the first in Cartesian space. Obtaining position information as output, iii- the first robotic movement corresponding to the first position information mentioned. determining the information, iv- According to the first robotic motion information mentioned, the robot's movement is 20 starting a motor that provides v- obtained by applying the first position information to a mathematical model The output obtained suggests a prediction of the user's second-hand transaction. being done, vi- motor angle is 25 according to the aforementioned second hand movement estimation information. adjustment It includes the steps of the process.

2. It is a method according to Claim 1, and its characteristic is; the engine angle mentioned is second-hand. After the step of adjusting the movement according to the predicted information, steps i-iv are 30. It is a repetition based on the user's second hand movement. 14 3. It is a method according to claim 1 and its feature is; from the image capture unit (200) Applying a preprocessing step to the captured first-hand movement image. It includes the process step.

4. It is a method according to claim 1, and its characteristic is that the first position information is stored in a memory 5 It includes the step of registering to unit (400).

5. A method according to claim 1, characterized by its ability to absorb sudden impulses resulting from hand movements. by detecting changes and controlling movement transitions It includes the process step of carrying out the operation. 10 6. The invention is capable of producing at least one robot (100) and one that can operate in Cartesian space. a system that captures at least one image of the user's hand movement image capture unit (200), 15 of the mentioned image capture unit (200) According to the hand movement image taken, the mentioned robot (100) control a system (10) which includes a processor unit (300) that enables it to be done its feature is that the mentioned processor unit (300) is like in one of the previous requests. a method is configured in such a way that its steps are carried out.

7. It is a system according to claim 6, and its characteristic is that the mentioned processing unit is a robot. It includes the operating system control unit.

Images