Trajectory Generation Method and Robotic Arm Photography System

Abstract

This invention provides a trajectory generation method and a robotic arm photography system, relating to the field of film and television shooting technology. The trajectory generation method includes the following steps: determining multiple effective points, including camera pose information, focus information, and focus distance; setting the order and trajectory planning parameters for each effective point; and calculating the planned trajectory based on the multiple effective points, their order, and the trajectory planning parameters. The robotic arm photography system can execute the above trajectory generation method and includes a photography module, a robotic arm, a user input device, and a control module. The photography module includes a camera and a manual focus unit, a rangefinder unit, and an autofocus unit connected to the camera; the end effector of the robotic arm is connected to the photography module; and the control module is communicatively connected to the photography module, the robotic arm, and the user input device. This trajectory generation method can obtain a stable and highly repeatable planned trajectory, thereby reducing the time-consuming cost of repeated shooting.

Description

Technical Field

[0001] This invention relates to the field of film and television shooting technology, and more specifically, to a method for generating a planned trajectory and a robotic arm photography system. Background Technology

[0002] Currently, short video shooting is mainly carried out by manually operating the camera system, and most of them use the same template for camera movement and special effects. However, the existing shooting system has poor repeatability and cannot guarantee that the motion trajectory of the video shot under the same template is completely consistent, which leads to high time cost and low shooting efficiency of manual shooting. Summary of the Invention

[0003] The present invention aims to provide a method for generating a planned trajectory and a robotic arm photography system to solve the technical problems of poor repeatability in existing shooting systems, which cannot guarantee that the motion trajectory of the film shot under the same template is completely consistent, resulting in high time and cost and low shooting efficiency of manual shooting.

[0004] To address the above problems, this invention provides a method for generating a planned trajectory, comprising the following steps:

[0005] Multiple valid points are determined, including the camera's pose information, focus information, and focus distance;

[0006] Set the sorting and trajectory planning parameters for each of the effective points;

[0007] The planned trajectory is calculated based on multiple valid points, their order, and trajectory planning parameters.

[0008] Optionally, the step of determining multiple valid points, wherein the valid points include camera pose information, focus information, and focus distance, includes:

[0009] Control the robotic arm to enter zero-force drag mode;

[0010] Drag and adjust the camera's pose, focus ring parameters, focus ring parameters, and focus distance to capture the desired image at the desired pose point. Use the camera's pose information at the desired pose point as the camera's pose information in the effective point, the focus ring parameters and focus ring parameters as the camera's focus information in the effective point, and the focus distance as the camera's focus distance in the effective point.

[0011] Optionally, after calculating the planned trajectory, the calculated planned trajectory is used as the current planned trajectory, and an online adjustment mode is entered. The online adjustment mode includes: controlling the robotic arm to drive the camera to move and shoot according to the current planned trajectory, adjusting the effective points and their order and trajectory planning parameters for subsequent shooting, and calculating the subsequent planned trajectory based on the adjusted effective points and their order and trajectory planning parameters.

[0012] The subsequent planned trajectory is used as the current planned trajectory again, and the online adjustment mode is entered again; this process is repeated until the motion shooting is completed, and the actual motion shooting trajectory of the camera is used as the final planned trajectory.

[0013] Optionally, after calculating the subsequent planned trajectory, it is first determined whether the movement and shooting of the robotic arm and camera according to the subsequent planned trajectory is within the safe movement and shooting range;

[0014] If so, the subsequent planned trajectory will be used as the current planned trajectory again, and the online adjustment mode will be entered again.

[0015] If not, continue shooting along the original planned trajectory.

[0016] Optionally, the trajectory generation method further includes the following steps:

[0017] During the process of the robotic arm driving the camera module to move, it is determined whether the force on the robotic arm is greater than a preset load threshold.

[0018] If so, it is determined that the robotic arm or the camera module has collided, and the robotic arm is shut down.

[0019] If not, it is determined that the robotic arm and the photography module did not collide, and the robotic arm is controlled to continue its current movement.

[0020] Optionally, the acceleration of the planned trajectory changes continuously.

[0021] The present invention also provides a robotic arm photography system capable of executing the above-described trajectory generation method, the robotic arm photography system comprising:

[0022] robotic arm;

[0023] The photography module includes a camera for shooting, and the camera has a manual focus function and an automatic focus function according to the adjustment command; the photography module is connected to the end of the robotic arm and can adjust its posture and move to shoot according to a planned trajectory under the drive of the robotic arm.

[0024] User input devices are used for setting the effective point sorting, inputting trajectory planning parameters, and viewing captured images; and,

[0025] The control module is communicatively connected to the photography module, the robotic arm, and the user input device, and is used to control the photography module and the robotic arm.

[0026] Optionally, the control module includes a storage unit, a calculation unit, and a control unit. The storage unit stores the camera's pose information, focus information, focus distance, and sorting and trajectory planning parameters input by the user input device. The calculation unit calculates the planned trajectory based on the information stored in the storage unit. The control unit issues adjustment commands to the camera module and the robotic arm based on the planned trajectory.

[0027] Optionally, the robotic arm is equipped with a force sensor, which is communicatively connected to the control module to detect the force on the robotic arm and feed it back to the control module; the control module is used to make judgments based on the received force feedback and control the start and stop of the robotic arm.

[0028] The trajectory generation method provided by this invention can offline set the pose information, focus information, focus distance, sorting, and trajectory planning parameters of multiple effective points in a robotic arm photography system. Then, it calculates a smooth transition trajectory that satisfies the relevant parameters of each effective point. The planned trajectory features stable and smooth focus tracking without soft impacts, allowing the robotic arm photography system to repeatedly shoot according to the planned trajectory to obtain the desired continuous footage. It ensures the repeatability and stability of multiple shooting sessions, thereby improving the convenience of video shooting and editing. While ensuring video shooting quality, it effectively reduces time costs and increases shooting efficiency. Furthermore, by setting different trajectory planning parameters, it can obtain a variety of planned trajectories, such as elliptical arc connections, straight line connections, and spline curve connections, which can be selected according to actual needs, thus improving its practicality.

[0029] The robotic arm photography system provided in this application can not only execute the above-mentioned planning trajectory generation method to calculate a stable and highly repeatable planning trajectory, but also perform motion shooting according to the planning trajectory. Specifically, after obtaining the planning trajectory, the robotic arm photography system can be started to perform motion shooting according to the planning trajectory. The control unit of the control module sends corresponding adjustment commands to the robotic arm and the autofocus unit according to the relevant parameters of the planning trajectory stored in the storage unit. Under the adjustment commands, the robotic arm drives the photography module to move and perform pose adjustment. Under the adjustment commands, the autofocus unit adjusts the focus ring data, focusing ring data, and focus distance of the camera, so that the robotic arm and the photography module can stably pass through each effective point along the planning trajectory in sequence and capture the desired shooting scene at the effective point, thereby obtaining the desired shooting video. When repeated shooting is required, it is only necessary to control the robotic arm photography system to perform motion shooting again according to the planning trajectory. The operation is convenient, the operation is stable and the repeatability is high, thereby improving the convenience of video shooting and editing. While ensuring the video shooting quality, it effectively reduces its time cost and improves its shooting efficiency. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0031] Figure 1 An isometric view of the robotic arm photography system provided by this invention;

[0032] Figure 2 for Figure 1 A magnified view of part A in the image;

[0033] Figure 3 A partial first-view schematic diagram of the camera module connected to the robotic arm in the robotic arm photography system provided by the present invention, after the camera has been removed;

[0034] Figure 4 A partial second-view diagram showing the connection of the camera module, which is disconnected from the camera, to the robotic arm in the robotic arm photography system provided by the present invention.

[0035] Figure 5 A schematic diagram of the communication connections of each module in the robotic arm photography system provided by the present invention;

[0036] Figure 6 This is a first flowchart illustrating the planning trajectory generation method provided in an embodiment of the present invention;

[0037] Figure 7This is a schematic diagram of the second process of the planning trajectory generation method provided in an embodiment of the present invention;

[0038] Figure 8 This is a schematic diagram of the online adjustment process after calculating the planned trajectory in the planned trajectory generation method provided in the embodiment of the present invention.

[0039] Figure 9 This is a partial flowchart of the trajectory generation method provided in an embodiment of the present invention.

[0040] Explanation of reference numerals in the attached figures:

[0041] 10-Photography module; 20-Robotic arm; 30-User input device; 40-Control module; 41-Storage unit; 42-Computing unit; 43-Control unit; 44-Auxiliary controller; 50-Moving base; 100-Camera; 110-Focus ring; 120-Focus ring; 200-Manual focusing unit; 210-First knob; 220-Second knob; 230-Button; 300-Range measuring unit; 400-Autofocus unit; 410-Focus motor; 411-Adjusting clamp; 420-Drive gear; 500-Frame; 510-Handle; 520-Base; 600-Adjusting base; 610-First adjusting base; 620-Second adjusting base; 630-Follow-up base; 640-Mounting arm; 650-Second locking element. Detailed Implementation

[0042] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0043] This embodiment provides a robotic arm photography system, such as Figure 1 and Figure 5As shown, the system includes: a photography module 10, a robotic arm 20, a user input device 30, and a control module 40. The photography module 10 includes a camera 100 and a manual focus unit 200, a rangefinder unit 300, and an autofocus unit 400 connected to the camera 100. The manual focus unit 200 is used to manually focus the camera 100 to enable manual focus functionality. The rangefinder unit 300 is used to automatically acquire the focus distance of the camera 100 after focusing. The autofocus unit 400 is used to perform corresponding focusing operations on the camera 100 under adjustment commands to enable autofocus functionality. The robotic arm 20 is connected to the photography module 10 at its end and is used to adjust the pose of the photography module 10 and, under adjustment commands, drive the photography module 10 according to a specified pattern. The camera 100 is used for motion tracking and shooting. The user input device 30 is used to set the effective point sorting, input trajectory planning parameters, and view the captured images. The control module 40 is communicatively connected to the camera module 10, the robotic arm 20, and the user input device 30, and is used to control the camera module 10 and the robotic arm 20. Specifically, the control module 40 includes a storage unit 41, a calculation unit 42, and a control unit 43. The storage unit 41 is used to store the pose information, focus information, focus distance of the camera 100, and the sorting and trajectory planning parameters input by the user input device 30. The calculation unit 42 is used to calculate the planned trajectory based on the information stored in the storage unit 41. The control unit 43 is used to issue adjustment commands to the camera module 10 and the robotic arm 20 according to the planned trajectory.

[0044] This embodiment also provides a method for generating a planned trajectory. Figure 6 This is a schematic diagram of the first process of the trajectory generation method provided in an embodiment of the present invention. Figure 6 As shown, the method for generating the planned trajectory includes the following steps:

[0045] S602 determines multiple valid points, including the pose information, focus information and focus distance of the camera 100.

[0046] First, multiple desired shooting positions are determined, and the posture, focus information, and focus distance of the camera 100 at each desired shooting position are adjusted or set to obtain the desired target and desired clarity at that desired shooting position. This determines the pose information, focus information, and focus distance of the camera 100 at each effective point. Specifically, the pose of the camera 100 can be manually adjusted by using the manual focus unit 200 to adjust the focus ring 110 and focus ring 120 of the camera 100. The ranging unit 300 detects the distance between the origin of the camera 100 coordinates and the object being photographed after adjustment as the focus distance. Then, the adjusted pose information, focus information, and focus distance are fed back to the control module 40 and stored in its storage unit 41. Alternatively, the pose information, focus information, and focus distance can be directly set in the user input device 30 and stored in the storage unit 41 of the control module 40.

[0047] S604 sets the sorting and trajectory planning parameters for each valid point.

[0048] The sequence of multiple valid points is sorted, and trajectory planning parameters such as speed and curve connection shape are set when the planned trajectory passes through the corresponding valid points. This determines the speed and direction of the video captured by camera 100 as it passes through the valid points, thereby achieving the desired speed and continuity of the video captured by camera 100 when passing through the valid points. Specifically, camera 100 can be manually dragged to each valid point in the desired order, and the order of each valid point can be fed back to control module 40 and stored in its storage unit 41; alternatively, the sorting and trajectory planning parameters of each valid point can be directly set in user input device 30 and stored in storage unit 41.

[0049] S606 calculates the planned trajectory based on multiple valid points and their order, as well as trajectory planning parameters.

[0050] The planned trajectory needs to pass through each effective point in sequence according to the order of multiple effective points. When reaching an effective point, the pose, focus and focus distance of the camera 100, as well as the speed and direction when passing through the effective point, must meet the parameters determined in S602 and S604. The calculation unit 42 of the control module 40 calculates according to the order of the effective points and the related pose, focus, focus distance, speed and direction to obtain a planned trajectory with a smooth transition and passing through each effective point in sequence. At the same time, the pose, focus, focus distance, speed and direction when passing through the effective points meet the requirements and is stored in the storage unit 41.

[0051] The trajectory generation method provided in this application can offline set the pose information, focus information, focus distance, sorting, and trajectory planning parameters of multiple effective points in a robotic arm photography system. Then, it calculates a smooth transition trajectory that satisfies the relevant parameters of each effective point. The planned trajectory exhibits stable and smooth focus tracking without soft impacts, allowing the robotic arm photography system to repeatedly shoot according to the planned trajectory to obtain the desired continuous footage. This ensures the repeatability and stability of multiple shooting sessions, thereby improving the convenience of video shooting and editing. While ensuring video shooting quality, it effectively reduces time costs and increases shooting efficiency. Furthermore, by setting different trajectory planning parameters, it can obtain a variety of planned trajectories, such as elliptical arc connections, straight line connections, and spline curve connections, which can be selected according to actual needs, thus improving its practicality.

[0052] The robotic arm photography system provided in this application can not only execute the above-mentioned trajectory generation method to calculate a stable and highly repeatable trajectory, but also perform motion photography according to the trajectory. Specifically, after obtaining the trajectory, the robotic arm photography system can be started to perform motion photography according to the trajectory. The control unit 43 of the control module 40 sends corresponding adjustment commands to the robotic arm 20 and the autofocus unit 400 according to the relevant parameters of the trajectory stored in the storage unit 41. Under the adjustment commands, the robotic arm 20 drives the photography module 10 to move and perform pose adjustment. The autofocus unit 400 adjusts the position and focus of the camera module 10. Under the command, the focus ring 110 and focus ring 120 data of the camera 100, as well as the focus distance, are adjusted so that the robotic arm 20 and the camera module 10 can stably pass through each effective point along the planned trajectory and capture the desired shooting scene at the effective point, thereby obtaining the desired shooting video. When repeated shooting is required, it is only necessary to control the robotic arm camera system to move and shoot again according to the planned trajectory. The operation is convenient, the operation is stable and the repeatability is high, thereby improving the convenience of video shooting and editing. While ensuring the quality of video shooting, it effectively reduces its time cost and improves its shooting efficiency.

[0053] Preferably, in this embodiment, the acceleration of the planned trajectory changes continuously. The planned trajectory calculated by the calculation unit 42 based on the information stored in the storage unit 41 not only satisfies the sorting of each effective point, pose information, focus information, focus distance, and trajectory planning parameters, but also ensures that the acceleration of the camera 100's continuous motion changes continuously. Correspondingly, the speed of the camera 100 also changes continuously, thereby greatly improving the stability and smoothness of the camera 100's movement according to the planned trajectory, reducing the occurrence of camera shake, and correspondingly improving the smoothness of the captured image, reducing the occurrence of image shake that affects the shooting quality.

[0054] In this embodiment, step S602, which determines multiple valid points including the pose information, focus information, and focus distance of the camera 100, includes: controlling the robotic arm to enter a zero-force drag mode; dragging and adjusting the pose, focus ring parameters, focus ring parameters, and focus distance of the camera 100 to capture the desired image at the desired pose point; using the pose information of the camera 100 at the desired pose point as the pose information of the camera 100 in the valid points, the focus ring parameters and focus ring parameters as the focus information of the camera 100 in the valid points, and the focus distance as the focus distance of the camera 100 in the valid points.

[0055] This is one specific form of offline determination of valid points. The robotic arm has a zero-force drag mode. In the zero-force drag mode, the robotic arm 20 can maintain its current state when it is not subjected to external force, and can adjust its posture according to the traction force when it is subjected to external force. When determining valid points, the robotic arm 20 is first controlled to enter the zero-force drag mode. Then, the camera module 10 or the end of the robotic arm 20 is manually pulled to make the camera 100 reach the desired posture point. The posture, focus ring parameters, zoom ring parameters, focus distance, etc. of the camera 100 are adjusted. When the image displayed in the camera 100 reaches the desired value, the state of the camera 100 is determined as one of the valid points and the relevant information of the valid point is stored in the storage unit 41 of the control module 40. Then, the camera 100 is manually pulled to the next desired posture point and adjusted until the desired image is displayed in the camera 100. The state of the camera 100 is determined as the next valid point and the relevant information is stored in the storage unit 41. This process is repeated to obtain multiple valid points. This offline method for determining valid locations can adjust various parameters based on the actual captured footage to obtain the desired image, thereby improving the convenience of determining valid locations and ensuring the effectiveness of capturing valid locations. This, in turn, ensures the effectiveness of the planned trajectory and reduces the possibility of repeated shooting. Valid locations can be sorted directly according to the order in which they were acquired, or multiple valid locations can be sorted using the user input device 30.

[0056] Of course, in addition to using the manual traction photography module 10 or the robotic arm 20 to determine multiple effective points, the pose information, focus information, focus distance, sorting of effective points, and trajectory planning parameters of each desired pose point can be directly input into the user input device 30. The control module 40 controls the robotic arm and the photography module 10 to perform corresponding operations, and adjusts the parameters of the desired pose point according to the shooting screen displayed in the user input device 30 until the desired image of the desired pose point is obtained. The current parameters of the camera 100 are then used as the corresponding parameters of the effective points. This process is repeated to determine multiple effective points.

[0057] In this embodiment, after calculating the planned trajectory, the calculated planned trajectory is used as the current planned trajectory, and the robotic arm photography system is controlled to enter the online adjustment mode. The online adjustment mode includes: controlling the robotic arm in the robotic arm photography system to drive the camera 100 to move and shoot according to the current planned trajectory; adjusting the effective points and their order and trajectory planning parameters for subsequent motion shooting; and calculating the subsequent planned trajectory based on the adjusted effective points and their order and trajectory planning parameters; using the subsequent planned trajectory as the current planned trajectory again, and controlling the robotic arm photography system to enter the online adjustment mode again; repeating this process until the motion shooting is completed, and using the actual motion shooting trajectory of the camera 100 as the final planned trajectory. After calculating the planned trajectory based on the information of each effective point, it is used as the current planned trajectory. The robotic arm and camera 100 in the robotic arm photography system are controlled to move and shoot according to the current planned trajectory. During the shooting process, the user can adjust the subsequent trajectory of the current planned trajectory at any time according to the shooting screen displayed on the camera 100 or the user input device 30. Specifically, the effective points can be redefined, the order of each effective point can be set, and the trajectory planning parameters can be adjusted. The calculation unit 42 calculates the subsequent planned trajectory that smoothly transitions with the initial planned trajectory based on the relevant information of the adjusted effective points, thus completing an online adjustment. After the subsequent planned trajectory is calculated, it is used as the robotic arm camera. The robotic arm camera system smoothly changes its current planned trajectory from the original planned trajectory to the subsequent planned trajectory to continue shooting, and enters a new round of online adjustment mode. During the shooting process, the subsequent trajectory of the current planned trajectory can be adjusted again according to the shooting image. The calculation unit 42 calculates the new subsequent planned trajectory again based on the relevant information of the effective points after the adjustment, thus completing another online adjustment. This process is repeated, and the planned trajectory of the robotic arm camera 100 system for subsequent shooting is continuously adjusted according to the current shooting image until the entire shooting is completed, and the actual shooting trajectory of the camera 100 is taken as the final planned trajectory.

[0058] This online trajectory generation method can adjust the subsequent shooting trajectory of camera 100 in real time based on the current shooting image, thereby improving the flexibility and accuracy of trajectory determination and the degree of user's subjective creativity, and ultimately achieving the desired shooting effect. Specifically, the method for smoothly transitioning the current planned trajectory to the subsequent planned trajectory can be as follows: first, calculate and generate smooth second-order geometrically continuous path parameters based on the parameters in the current planned trajectory; then, plan a velocity curve based on the current velocity parameters, the desired velocity, and the planned path length; finally, interpolate the above-mentioned second-order geometrically continuous path in real time according to the velocity curve pattern to obtain the control pose and velocity parameters for real-time control output.

[0059] Of course, once the robotic arm photography system enters the online adjustment mode, when the captured image meets the desired result, the user can choose not to make any adjustments based on the actual situation. In this case, the robotic arm photography system will complete the motion capture according to the initial planned trajectory and use that initial planned trajectory as the final planned trajectory.

[0060] Optionally, in this embodiment, after calculating the subsequent planned trajectory, it is first determined whether the movement and shooting of the robotic arm 20 and the camera 100 according to the subsequent planned trajectory is within the safe movement and shooting range; if so, the subsequent planned trajectory is used as the current planned trajectory again, and the robotic arm camera system is controlled to re-enter the online adjustment mode; if not, the movement and shooting continues according to the original planned trajectory. During the movement and shooting of the robotic arm camera system according to the current planned trajectory, after adjusting the subsequent movement and shooting and calculating the subsequent planned trajectory, the control module 40 first simulates and determines whether there is a collision hazard or a sudden change in speed that prevents the robotic arm and camera 100 from moving and shooting according to the subsequent planned trajectory. If there is no collision hazard and the lane change of the planned trajectory can be smoothly achieved, it indicates that the subsequent planned trajectory is within the safe movement and shooting range, and the subsequent planned trajectory is used as the current planned trajectory again, and the online adjustment mode is entered again for adjustment; if there is a collision hazard or the lane change of the planned trajectory cannot be smoothly achieved, it indicates that the subsequent planned trajectory is outside the safe movement and shooting range, and the robotic arm and camera 100 are controlled to continue moving and shooting according to the current planned trajectory.

[0061] The above settings can effectively improve the safety of motion shooting by the robotic arm photography system and reduce the occurrence of dangerous situations during motion shooting.

[0062] Specifically, in this embodiment, the robotic arm 20 is equipped with a force sensor, which is communicatively connected to the control module 40. The force sensor detects the force acting on the robotic arm and feeds it back to the control module 40. The control module 40 determines and controls the start and stop of the robotic arm based on the received force feedback. Correspondingly, the trajectory generation method further includes the following steps: during the movement of the robotic arm driving the camera module 10, it is determined whether the force acting on the robotic arm exceeds a preset load threshold. If so, it is determined that a collision has occurred between the robotic arm and the camera module 10, and the robotic arm is stopped. If not, it is determined that no collision has occurred between the robotic arm and the camera module 10, and the robotic arm continues its current movement. When the robotic arm moves the camera module 10 under the drag of the user or the control of the control module 40, the force sensor detects the force on the robotic arm in real time and feeds back the force signal to the control module 40. The control module 40 determines whether the force on the robotic arm is greater than the preset load threshold based on the received force feedback. If so, it indicates that the robotic arm 20 and the camera module 10 have been impacted by an external force, and the control module 40 stops the robotic arm to reduce further collision damage caused by the continued movement of the robotic arm, thereby improving the safety of the robotic arm camera system. If not, it indicates that the robotic arm and the camera module 10 have not been impacted by the outer wall and are in a safe movement state, and the control module 40 maintains the current movement state to ensure normal operation of the robotic arm.

[0063] Specifically, the force sensor can be a joint torque sensor. When the camera module 10 or any part of the robotic arm collides, the collision force can be fed back through the force on the joint of the robotic arm. The force state of the robotic arm camera system can be detected by a small number of force sensors, and the accuracy of collision state detection can be ensured.

[0064] Figure 7 This is a schematic diagram of the second process of the planning trajectory generation method provided in an embodiment of the present invention. Figure 7 As shown, the method for generating the planned trajectory includes the following steps:

[0065] The S702 controls the robotic arm to enter zero-force drag mode.

[0066] S704 drags the photography module 10 and adjusts the pose, focus ring parameters, focus ring parameters and focus distance of the camera 100 to capture the desired image at the desired pose point. The pose information of the camera 100 at the desired pose point is used as the pose information of the camera 100 in the effective point, the focus ring parameters and focus ring parameters are used as the focus information of the camera 100 in the effective point, and the focus distance is used as the focus distance of the camera 100 in the effective point.

[0067] S706 sets the sorting and trajectory planning parameters for each valid point;

[0068] S708 calculates the planned trajectory based on multiple valid points and their order, as well as trajectory planning parameters.

[0069] Figure 8 This is a schematic diagram illustrating the online adjustment process after calculating the planned trajectory in the trajectory generation method provided in this embodiment of the invention. Figure 8 The illustrated online adjustment process diagram includes:

[0070] S801 uses the calculated planned trajectory as the current planned trajectory.

[0071] The S802 controller puts the robotic arm photography system into online adjustment mode.

[0072] In the S803 control system for robotic arm photography, the robotic arm drives the camera 100 to move and capture images according to the current planned trajectory. It then adjusts the effective points, their order, and trajectory planning parameters for subsequent motion capture, and calculates the subsequent planned trajectory based on the adjusted effective points, their order, and trajectory planning parameters.

[0073] S804 determines whether the robotic arm and camera 100 are within a safe shooting range when moving and shooting according to the subsequently planned trajectory. If yes, proceed to step S805; otherwise, proceed to step S806.

[0074] S805 uses the subsequent planned trajectory as the current planned trajectory again. Then proceed to step S802.

[0075] The S806 continued to move and shoot along the original planned trajectory.

[0076] Figure 9 This is a partial flowchart illustrating the trajectory generation method provided in an embodiment of the present invention. Figure 9 As shown, the method for generating the planned trajectory includes the following steps:

[0077] The S902 robotic arm drives the camera module 10 to move.

[0078] S904 determines whether the force on the robotic arm exceeds a preset load threshold. If yes, proceed to step S906; otherwise, proceed to step S908.

[0079] S906 determines that the robotic arm or camera module 10 has collided and controls the robotic arm to shut down.

[0080] S908 determines that the robotic arm and the camera module 10 have not collided and controls the robotic arm to continue its current movement.

[0081] Specifically, in this embodiment, as Figure 1 and Figure 2As shown, the photography module 10 also includes a frame 500 connected to the end of the robotic arm. The camera 100 is installed inside the frame 500, and the focusing ring 110 and focusing ring 120 of the camera 100 are both fitted with adjusting gear rings. The autofocus unit 400 includes two focusing motors 410 located in the frame 500. The drive ends of the two focusing motors 410 are each equipped with a drive gear 420, and the two drive gears 420 mesh with the two adjusting gear rings in a one-to-one correspondence. This is one specific form of the autofocus unit 400 in the photography module 10. The frame 500 is connected to the end of the robotic arm and is used to support the camera 100 and the focusing motor 410. The focusing motor 410 is communicatively connected to the control module 40. When the robotic arm photography system moves and shoots according to the planned trajectory, the control unit 43 controls the start / stop status and operating speed of the two focusing motors 410 according to the focusing information of the camera 100 in the planned trajectory. The focusing motor 410 drives the drive gear 420 to rotate, and drives the focus ring 110 or the focus ring 120 to rotate through the meshing of the drive gear 420 with the corresponding adjustment gear ring, thereby realizing the automatic adjustment of focusing information such as focus amount and focus amount.

[0082] Optionally, in this embodiment, as Figure 2 As shown, the manual focus unit 200 may include a first knob 210 and a second knob 220 disposed on the frame 500 and connected to the camera 100. The first knob 210 is used to adjust the focus of the camera 100, and the second knob 220 is used to adjust the focus of the camera 100. Similarly, the frame 500 may also be provided with a plurality of buttons 230 for inputting switching signals of the camera 100.

[0083] In this embodiment, as Figures 2-4As shown, the frame 500 is rectangular and at least one side of the frame beam is provided with a carrier 520. The imaging module 10 also includes an adjustment seat 600. The adjustment seat 600 includes a first adjustment seat body 610, a second adjustment seat body 620 and a follower seat body 630. The first adjustment seat body 610 is slidably connected to the carrier 520 along the length direction of the corresponding frame beam. The second adjustment seat body 620 is slidably connected to the first adjustment seat body 610 along the front-back direction. The follower seat body 630 is located at the front end of the second adjustment seat body 620 and is provided with two forward-extending mounting arms 640. Two focusing motors 410 are installed one-to-one on the two mounting arms 640. This is one specific form of mounting the camera 100 on the carrier frame 500. The camera 100 is fixedly connected to the second adjustment seat 620. The position of the camera 100 in the left-right or up-down direction can be adjusted by moving the first adjustment seat 610 relative to the carrier 520 along the length direction of the corresponding frame beam. The position of the camera 100 in the front-back direction can be adjusted by moving the second adjustment seat 620 relative to the first adjustment seat 610, thereby improving the positional accuracy of the camera 100 mounted on the end of the robotic arm. The follower seat 630 is fixedly connected to the second adjustment seat 620, thus fixing the relative positions of the camera 100 and the follower seat 630. Both move synchronously with the second adjustment seat 620, thereby ensuring the positional correspondence between the focusing motor 410 and the focusing ring 110 and the focusing ring 120 during the position adjustment of the camera 100, and correspondingly ensuring the effective focusing of the focusing motor on the focusing ring 110 and the focusing ring 120. Specifically, the focusing motor 410 is provided with an adjustment clamp 411, and the mounting arm 640 is inserted into the adjustment clamp 411 of the corresponding focusing motor 410 and locked by a locking member. During installation, the locking member can be loosened or removed, and the mounting arm 640 can be... The adjusting clamp 411 is movable and can move back and forth along the length of the mounting arm 640 to adjust the front and back position of the focusing motor 410 so that the drive gear 420 at its drive end corresponds to the adjusting gear ring outside the focus ring 110 or focus ring 120 of the camera 100. Simultaneously, the adjusting clamp 411 can be rotated circumferentially to adjust the circumferential distance between the drive gear 420 and the corresponding adjusting gear ring, enabling them to mesh. After adjustment, the adjusting clamp 411 is locked in the mounting arm 640 by a locking member to ensure the stability of the focusing motor 410's adjustment of the focus ring 110 and the focusing ring. Therefore, the adjusting seat 600 can be adapted to different models of camera 100 and can ensure the effective focusing of the focusing motor 410 with the camera 100.

[0084] Specifically, aluminum tubing can be used for mounting arm 640.

[0085] Preferably, the first adjusting seat 610 or the carrier 520 is provided with a first locking member. When the first locking member is in the adjusting position, the first adjusting seat 610 can move and adjust relative to the carrier 520; when the first locking member is in the locked position, the first adjusting seat 610 is locked to the carrier 520 to ensure the positional stability of the first adjusting seat 610 connected to the carrier 520. Similarly, as Figure 4 As shown, the first adjusting seat 610 or the second adjusting seat 620 is provided with a second locking member 650. When the second locking member 650 is in the adjusting position, the second adjusting seat 620 can move back and forth relative to the first adjusting seat 610 for adjustment. When the second locking member 650 is in the locked position, the second adjusting seat 620 is locked to the first adjusting seat 610 to ensure the positional stability of the second adjusting seat 620 connected to the first adjusting seat 610, and correspondingly ensure the positional stability of the camera 100 connected to the frame 500, thereby ensuring stable shooting by the camera 100 during motion shooting. Specifically, both the first locking member and the second locking member 650 can be selected as locking bolts.

[0086] Specifically, such as Figure 3 and Figure 4 As shown, there can be three carriers 520, which are located on the left side frame beam, right side frame beam and lower side frame beam of the frame 500 respectively. There is one adjusting seat 600, which can be installed on one of the carriers 520 as needed.

[0087] Optionally, in this embodiment, as Figures 2-4 As shown, a handle 510 can also be provided on the frame 500. The user can apply force to the frame 500 by holding the handle 510 to drag the camera module 10 and manually adjust its position.

[0088] In this embodiment, the control module 40 may include a main controller and an auxiliary controller 44 communicatively connected to the main controller. The auxiliary controller 44 is communicatively connected to the photography module 10, and the main controller is communicatively connected to the robotic arm 20 and the user input device 30. Specifically, the main controller is communicatively connected to the robotic arm 20, the user input device 30, and the manual focus unit 200 of the photography module 10, and is used to receive corresponding feedback information, calculate the planned trajectory based on the feedback information, and issue adjustment commands; such as Figures 2-4As shown, the auxiliary controller 44 can be a microcontroller mounted on the carrier frame 500. This auxiliary controller 44 is communicatively connected to the range sensor and the two focusing motors 410. It is used to adjust the range sensor and the two focusing motors 410 according to the adjustment commands of the main controller, and to feed back the relevant signals from the range sensor and the focusing motors 410 to the main controller. Thus, the main controller and the auxiliary controller 44 assist each other in monitoring and controlling the robotic arm photography system, reducing the computational load of a single controller, thereby ensuring the computational and control speed of the control module 40 and ensuring its normal operation.

[0089] Specifically, the auxiliary controller 44 is also used to control the power supply status of the output power supply to the camera 100; the bottom of the robotic arm may be provided with a movable base 50, the main controller is installed in the movable base 50, and the power lines, signal lines, image data transmission lines, etc. in the robotic arm photography system can be wound around the robotic arm and connected between the main controller and the auxiliary controller 44.

[0090] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0091] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a planned trajectory, characterized in that, Includes the following steps: Multiple effective points are determined, including the pose information, focus information and focus distance of the camera (100) in the photography module (10). Multiple desired shooting positions are determined and the pose, focus information and focus distance of the camera (100) at each desired shooting position are adjusted or set so as to obtain the desired target and desired clarity of the shooting image at the desired shooting position. The focus information is the focus ring parameter and the focus ring parameter. Set the sorting and trajectory planning parameters for each of the effective points, wherein the trajectory planning parameters include the speed and curve connection shape of the planned trajectory when passing through the corresponding effective points, so as to determine the speed and direction of the video captured by the camera (100) when passing through the corresponding effective points; The planned trajectory is calculated based on multiple effective points and their order, and trajectory planning parameters. The speed of the planned trajectory changes continuously. The planned trajectory is used as the current planned trajectory, and the online adjustment mode is entered. The online adjustment mode includes: controlling the robotic arm (20) to drive the camera (100) to move and shoot according to the current planned trajectory; the user adjusts the effective points and their order and trajectory planning parameters of subsequent motion shooting according to the shooting screen displayed in the camera (100) or user input device (30); and the calculation unit (42) calculates the subsequent planned trajectory that smoothly transitions with the current planned trajectory according to the adjusted effective points and their order and trajectory planning parameters. The subsequent planned trajectory is used as the current planned trajectory again, and the online adjustment mode is entered again; this is repeated until the motion shooting is completed, and the actual motion shooting trajectory of the camera (100) is used as the final planned trajectory; The photography module (10) includes a frame (500) connected to the end of the robotic arm (20), an adjustment seat (600), a camera (100) for shooting, and an autofocus unit (400). The frame (500) is rectangular and at least one side of the frame beam is provided with a seat (520). The adjustment seat (600) includes a first adjustment seat body (610), a second adjustment seat body (620), and a follower seat body (630). The first adjustment seat body (610) slides along the length direction of the corresponding frame beam to the seat (520). The second adjustment seat body (620) slides along the front-back direction to the first adjustment seat body (610). The follower seat body (630) is located on the second adjustment seat body (610). The front end of the adjustment base (620) and the follower base (630) are provided with two forward-extending mounting arms (640); the camera (100) is fixedly connected to the second adjustment base (620), and the focus ring (110) and focus ring (120) of the camera (100) are both fitted with adjustment gear rings; the autofocus unit (400) includes two focusing motors (410), the two focusing motors (410) are installed one-to-one on the two mounting arms (640), and the driving ends of the two focusing motors (410) are each equipped with a drive gear (420), and the two drive gears (420) mesh with the two adjustment gear rings one-to-one.

2. The method for generating a planning trajectory according to claim 1, characterized in that, The step of determining multiple effective points, including the pose information, focus information, and focus distance of the camera (100) in the photography module (10), determining multiple desired shooting positions, and adjusting or setting the pose, focus information, and focus distance of the camera (100) at each desired shooting position to obtain a shooting image with the desired target and desired clarity at the desired shooting position, includes: Control the robotic arm to enter zero-force drag mode; Drag and adjust the pose, focus ring parameters, focus ring parameters and focus distance of the camera (100) to capture the desired image at the desired pose point. The pose information of the camera (100) at the desired pose point is used as the pose information of the camera (100) in the effective point, the focus ring parameters and focus ring parameters are used as the focus information of the camera (100) in the effective point, and the focus distance is used as the focus distance of the camera (100) in the effective point.

3. The method for generating a planning trajectory according to claim 1, characterized in that, After calculating the subsequent planned trajectory, it is first determined whether the movement and shooting of the robotic arm (20) and the camera (100) according to the subsequent planned trajectory are within the safe movement and shooting range; If so, the subsequent planned trajectory will be used as the current planned trajectory again, and the online adjustment mode will be entered again. If not, continue shooting along the original planned trajectory.

4. The method for generating a planning trajectory according to claim 1, characterized in that, The method for generating the planned trajectory also includes the following steps: During the process of the robotic arm (20) driving the camera module (10) to move, it is determined whether the force on the robotic arm (20) is greater than the preset load threshold. If so, it is determined that the robotic arm (20) or the camera module (10) has collided, and the robotic arm (20) is shut down. If not, it is determined that the robotic arm (20) and the photography module (10) did not collide, and the robotic arm (20) is controlled to continue its current movement.

5. The method for generating a planning trajectory according to any one of claims 1-4, characterized in that, The acceleration of the planned trajectory changes continuously.

6. A robotic arm photography system, characterized in that, The robotic arm photography system, capable of executing the trajectory generation method according to any one of claims 1-5, comprises: robotic arm (20); The photography module (10) includes a frame (500) connected to the end of the robotic arm (20), an adjustment seat (600), a camera (100) for shooting, and an autofocus unit (400). The frame (500) is rectangular and at least one side of the frame beam is provided with a seat (520). The adjustment seat (600) includes a first adjustment seat body (610), a second adjustment seat body (620), and a follower seat body (630). The first adjustment seat body (610) slides along the length direction of the corresponding frame beam to the seat (520). The second adjustment seat body (620) slides along the front-back direction to the first adjustment seat body (610). The follower seat body (630) is located at the front end of the second adjustment seat body (620), and the follower seat body (630) is located at the front end of the second adjustment seat body (620). 0) It is provided with two forward-extending mounting arms (640); the camera (100) has a manual focus function and an automatic focus function according to the adjustment command. The camera (100) is fixedly connected to the second adjustment base (620), and the focus ring (110) and the focusing ring (120) of the camera (100) are both fitted with adjustment gear rings; the automatic focus unit (400) includes two focus motors (410), and the drive ends of the two focus motors (410) are each equipped with drive gears (420). The two drive gears (420) mesh with the two adjustment gear rings one by one; the photography module (10) can adjust its posture and move to shoot according to the planned trajectory under the drive of the robotic arm (20); User input device (30) is used for setting the effective point sorting, inputting trajectory planning parameters, and viewing the captured images; and, The control module (40) is communicatively connected to the photography module (10), the robotic arm (20) and the user input device (30), and is used to control the photography module (10) and the robotic arm (20).

7. The robotic arm photography system according to claim 6, characterized in that, The control module (40) includes a storage unit (41), a calculation unit (42), and a control unit (43). The storage unit (41) is used to store the pose information, focus information, focus distance of the camera (100), and sorting and trajectory planning parameters input by the user input device (30). The calculation unit (42) is used to calculate the planned trajectory based on the stored information in the storage unit (41). The control unit (43) is used to issue adjustment commands to the photography module (10) and the robotic arm (20) based on the planned trajectory.

8. The robotic arm photography system according to claim 6, characterized in that, The robotic arm is equipped with a force sensor, which is communicatively connected to the control module (40) to detect the force on the robotic arm (20) and feed it back to the control module (40); the control module (40) is used to make a judgment based on the received force feedback and control the start and stop of the robotic arm (20).

Images